Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/16—Constructional details or arrangements
  • G06F1/1613—Constructional details or arrangements for portable computers
  • G06F1/1626—Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/16—Constructional details or arrangements
  • G06F1/1613—Constructional details or arrangements for portable computers
  • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
  • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
  • G06F1/1686—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being an integrated camera
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
  • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
  • G06F3/0233—Character input methods
  • G06F3/0236—Character input methods using selection techniques to select from displayed items
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/0304—Detection arrangements using opto-electronic means
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
  • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
  • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
  • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
  • G06F3/0482—Interaction with lists of selectable items, e.g. menus
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
  • G06F3/0485—Scrolling or panning
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2200/00—Indexing scheme relating to G06F1/04 - G06F1/32
  • G06F2200/16—Indexing scheme relating to G06F1/16 - G06F1/18
  • G06F2200/163—Indexing scheme relating to constructional details of the computer
  • G06F2200/1637—Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/048—Indexing scheme relating to G06F3/048
  • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation

Definitions

• the present invention concerns an information processing system comprising an optoelectronic device, associated display means and display control electronics that either is physically a part of said optoelectronic device or physically separate therefrom, but connected therewith via a data link, wherein the optoelectronic device comprises an internal memory, a data processing system for operating the device, including input- and/or output processes, and an optical image recorder for capturing information from the environment surrounding said optoelectronic device.
• the present invention also concerns a method for operating an information processing system of this kind.
• buttons or other means of input have led to a situation where user control of the device is seriously slowed down, e.g. during text input and menu item selection.
• miniaturized multiple-choice buttons typically leads to increased operator fatigue and higher incidence of erroneous data entries:
• a common way to implement menu item selection is to employ a four-way button which has to be repeatedly operated in order to step through menu items that are typically arranged in a 1-dimensional list or a 2-dimensional array of symbols. As the lists and menus tend to increase in size with the increased functionality of mobile devices, the number of button operations required will increase in a highly undesired manner. ⁇ ,
• the small area available for a display screen implies that displayed features must either be very small and correspondingly difficult to read, or they must be so few as to severely limit the information content in the display.
• US patent application 2004/0141085 discloses a system that uses an orientation detecting mechanism (an accelerometer) to determine the orientation of an imaging device, e.g. a digital camera.
• the acquired orientation information is used to reconfigure the displayed image for increased user viewing convenience.
• the reconfiguration is explicitly effected on the captured image only. The general problem of using small displays for viewing and performing operations within large and content-rich images, diagrams, etc. is not addressed.
• a solution to the problem of providing input to a small portable device with limited space for keyboard is promoted by the firm Memsic Inc., of North Andover, MA in U.S.A.
• the portable device is equipped with a small accelerometer which senses tilting of the device. By tilting right/left and/or up/down and clicking on a button on the device, navigation is possible on a screen on the device, i.e. scrolling through lists, menu selection, map scrolling, gaming, etc.
• An obvious drawback of this approach is the need for an accelerometer in the portable device, where space and cost typically are at an extreme premium. Beyond this, input is sensitive to motion of the device involving changes in speed or direction, and presupposes a given attitude of the portable device relative to the direction of the gravity vector.
• a possible solution to the text input problem, described in US Pat. No. 5,818,437 is to assign multiple letters to each of the numerical buttons on the device, and use built-in dictionaries to find words that match the button sequence keyed by the user.
• Drawbacks of this solution include the need to scroll through several valid alternatives to find the correct word, and having to explicitly spell words, names and abbreviations that are not included in the dictionary.
• the optoelectronic device comprises hardware and software adapted for analyzing information captured by said optical image recorder with respect to temporal variations in the signals from the optical image recorder, said variations being caused by translational and/or rotational motion of said optoelectronic device relative to real or virtual objects, textures or scenery within the field of view of said optical image recorder, that said processing hardware and software are adapted for providing an output signal in scalar or vector form on the basis of the analyzed information, said output signal being indicative of the motion characteristics of said optoelectronic device relative to the real or virtual objects, textures or scenery within the field of view of said optical image recorder, that the display control electronics is connected with said optoelectronic device for receiving said output signal, and that said display control electronics is adapted for modifying or changing a displayed message or image in response to the characteristics of said output signal.
• a method according to the invention which is characterized by deriving at least one parameter from translational and/or rotational motion of said optoelectronic device or auxiliary equipment linked to same, relative to real or virtual objects, textures or scenery objects within the field of view of said optical image recorder, and applying said at least one parameter as input to control electronics in said optoelectronic device and to effect a navigation and/or input commands in said display means, partly or completely in response to said at least one parameter.
• figure 1 shows the main components in an information processing system according to the present invention
• figures 2a and 2b a first preferred embodiment where the display screen on a mobile phone constitutes part of a virtual display where the viewing field is enhanced by scrolling a window across a larger page
• figure 3 a second preferred embodiment involving a virtual keyboard
• figure 4 a third preferred embodiment involving menu item selection
• figures 5a and 5b an example of a variant of the third preferred embodiment
• figure 6 a fourth preferred embodiment where an optoelectronic device according to the invention functions as a computer mouse
• figures 7a and 7b show examples of variants of the fourth embodiment.
• the main components of an optoelectronic device as used in the information processing system of the invention is shown in fig. 1.
• the optoelectronic device comprises an optical image recorder, an image motion analyser linked with the former and used for analyzing recorded images and generating a continuously updated image output to a connected display.
• the optoelectronic device is implemented in a mobile telephone or a PDA equipped with a digital camera and a display screen.
• the optoelectronic device is termed a "phone" in these preferred embodiments. This is not meant to preclude that the optoelectronic device in question may be another type of mobile platform equipped with a digital camera and a display screen, e.g. a helmet-mounted or head-up device or system.
• FIG. 2 A first preferred embodiment is shown in figure 2.
• This embodiment provides a virtual display wherein large pages of text, images, web content, etc. that would otherwise defy normal criteria for readability on the phone display screen, are accessed in selected segments that are scrolled across the larger page simply by moving the phone in three dimensions.
• This scrolling is rendered intuitive by correlation between the virtual motion across the page to be viewed with the real motion of the phone.
• the latter is typically a right/left and up/down motion of the phone above or in front of a surface or object or scene that exhibits adequate structure or texture for the camera to register movement.
• the degree of zoom can be used to control the speed of scrolling for a given motion input, e.g. the more zoom, the slower the scrolling, and vice versa.
• Combined with the lateral motion of the phone optionally there may be an in/out motion relative to the surface or object which effects a zoom in/zoom out function on the displayed image of the large page.
• One may combine scrolling with the zoom function, such that fast scrolling reduces the zoom factor and vice versa.
• the zoom level then applied may be maintained until the zoom function is activated again, e.g. by the relative in/out motion of the phone.
• the absolute zoom level shall be user controllable
• An alternative variant of phone motion which is possible in cases where no suitable surface or object is available in reasonably close proximity to the phone, is to rotate the phone right/left and up/down, i.e. panning and tilting the camera to cause the observed scene to move across the camera chip surface.
• this can be perceived intuitively as being related to tilting of the observer's head to view different parts of the page.
• motion of the camera in a line of sight to the distant objects will cause the image to change very little, and other means of providing zooming commands must be used.
• zooming in/out in the virtual image can be achieved in several ways, namely a) by using the zoom function built into the camera, if such exists; b) by pressing or toggling a mechanical input key, switch or pressure- sensitive field on the phone; c) by moving the phone in a prescribed pattern, e.g. rapid sideways or up/down rotation for zoom in/zoom out; or d) by acoustic input, e.g. voice command.
• This threshold shall typically apply for the recorded speed of motion (translational or rotational), and the magnitude of the threshold may be selected or adjusted according to the type of usage and environment that is encountered in each application.
• any predictable motion pattern that shall not serve as input cue to the optoelectronic device may be compensated for in the signal processing, e.g. a constant linear motion due to steady translational motion of the optoelectronic device such as walking.
• the "dead band” would be centered around a certain speed value corresponding to the constant motion.
• the algorithm for making the system insensitive to motion that is not intended by the operator to provide meaningful motion input to the optical image recorder may also be set to reject very rapid accelerations or motions, e.g. by setting an upper limit to the legitimate sensitive range.
• the processing can easily be set to pick up a specific motion as an input cue, e.g. cyclic motion within a defined frequency band such as would occur in response to rapid wagging of the optoelectronic device sideways or up and down.
• the phone movement can also be used to scroll a virtual keyboard for text input, for instance to message systems like SMS.
• a single function key is pressed to select a letter, and the range of characters can be extended or various special functions implemented by using movement in the third dimension.
• writing via a virtual keyboard is performed without the need for a set of physical keys representing the alphabet or other symbols on the face of the phone. Instead, letters or symbols are selected on a virtual keyboard shown in the display, by a two- step process: First, moving the phone in a lateral or rotational motion causes a visual indicator function to move across the virtual keyboard.
• a "Select" command is given to the phone, by one of the following modalities, namely a) by pressing or toggling a mechanical input key, switch or pressure- sensitive field on the phone; b) by moving the phone in a prescribed pattern, i.e. laterally up/down or rotationally (pan/tilt) or along the optical axis of the phone's camera (in/out); or c) by acoustic input, e.g. voice command.
• the virtual keyboard may be larger than the display, causing only part of it to be visible within the field defined by the phone's display screen. This situation is depicted in figure 3, and in this case the virtual keyboard scrolls across the display, one keyboard letter or symbol being highlighted or framed at a time upon passing a selector field in e.g. the middle of the screen.
• the whole virtual keyboard is shown inside the display, and a cursor or highlighting field moves across the virtual keyboard when the phone is moved.
• Selection of upper or lower case characters or special symbols or functions is achieved by entering a command into the phone by one of the modalities a) to c) described earlier in this paragraph.
• the command may be of the "Caps shift” or “Caps lock” type, depending on the situation.
• a sweep across a virtual background field where the menu icons are laid out can be performed quickly and simply.
• Motion patterns and selection modes can be similar to those described for the virtual keyboard above.
• a typical phone has many menus. The user could pre-set which menus should be subject to motion controlled selection, and then the camera would automatically be activated when these menus are selected. Menus could be nested hierarchically, and motion control could be used to navigate into and within sub-menus.
• the phone camera is automatically activated to detect phone movement.
• the phone calls the presently highlighted contact. • If the contact has more than one phone number, the first dial-up or select command produces a pop-up window with all the phone numbers. Phone movement is used to highlight the desired number and the dial-up button is used for placing the call.
• This strategy enables fast selection of the desired contact in one sweeping phone movement. It enables more contacts on the screen than the usual text list layout, and is based on human recognition of images/graphical symbols rather than text, which is normally a faster search method.
• the present invention could also in a general fashion implement a so-called relative pointer device, for instance for controlling the cursor on a computer screen. However, the present invention shall allow the extension of relative pointer technology far beyond that offered by e.g.
• a conventional computer mouse This opens for a fourth preferred embodiment, the principle of which is shown in fig.6, which basically renders the use of the optoelectronic device as three-dimensional computer mouse or similar input device to a mobile or stationary information processing system. Cursor movement on the computer screen is controlled by moving the optoelectronic device in one or more of the translational, rotational or combined translational/rotational modes described in the other preferred embodiments above, while mouse clicks can be effected in several ways, e.g.
• the input from the optoelectronic device to the information processing system is the operator signature, e.g. in legal or economic transactions or in access control.
• the operator in this case moves the optoelectronic device as he would a pen to create the pattern, which may be his name signature or some other pre-defined identifying pattern.
• the motion need not take place against any supporting surface, but could be performed in the air in front of the operator ("writing in the air").
• One example of such processing would be to scale the size of the recorded signature to a predefined norm.
• an alternative to moving the phone in the open air shall be to equip the phone with additional physical hardware, e.g. a rounded object, e.g. ball-like, which may be attached to the phone at the side where the camera is located.
• additional physical hardware e.g. a rounded object, e.g. ball-like, which may be attached to the phone at the side where the camera is located.
• the phone may be rested on the desktop or any other object, while the three-dimensional motions are achieved by tilting the phone in the various directions, as allowed by the thickness of the attached, rounded object.
• the translation movements of the mouse now can be replaced by a rotational movement around any of three axes and over a fairly small angle.
• Communication with the computer could be via a wire or by one of several wireless means.
• Bluetooth communication would enable using the mobile electronic device for controlling presentations, including pointer control and the possibility for the user to stand up and move around.
• An important advantage of this embodiment relative to the conventional computer mouse is that contact with or close proximity to a surface is not needed.
• it is possible to provide input commands linked to a motion in the third dimension either by increasing or reducing the distance from the camera of the mobile electronic device to an object in front of it or by employing the camera zoom function. This opens up opportunities for 3-D cursor control, e.g. in CAD software or in the display/manipulation of 3 dimensional virtual or real-space objects.
• the optoelectronic device can be equipped with a microphone, such as is the case when the optoelectronic device is a mobile telephone with camera. This makes it possible to insert audio files into the computer. Likewise, by means of a speaker or earphone attached to the optoelectronic device, it becomes possible to listen to audio files that have been located in the computer.
• relative motion is created by the optoelectronic device being stationary or nearly so.
• the relative motion is then created by moving an object, e.g. the operator's hand (or finger), in front of the optical image recorder of the optoelectronic device.
• This variant can be implemented in various ways, as shown in figs 7a and 7b.
• Figure 7a shows a person carrying an optoelectronic device according to the invention with a camera viewing out in front of the person. By moving his hands in front of the camera, this causes a cursor to move on the screen of a computer as shown in fig.7a, or on a screen in the optoelectronic device itself.
• Mouse clicks can be done in several ways, e.g.
• the person holds a contrast-enhancing or recognition aiding implement in his hand, e.g. a pen or pencil with a marker spot of high optical contrast on it, optionally in a color selected in an optimized spectral region, to simplify processing of the image.
• a contrast-enhancing or recognition aiding implement e.g. a pen or pencil with a marker spot of high optical contrast on it, optionally in a color selected in an optimized spectral region, to simplify processing of the image.
• Fingers or hands of the operator can be marked with a spot or symbol of high optical contrast, e.g. to show the positions of the fingertips.
• variants of this fourth embodiment exemplified in figures 7a and 7b can be used to effect handwritten input into the computer, the optoelectronic device or both, including name or other types of signatures for legal and transaction purposes or access control.
• Image analysis provides further possibilities for more complex and hence more compact or efficient input coding: For example, two fingers in contact with each other side by side may carry a different message from the same two fingers splayed away from each other.
• a simple example of how this is when writing to virtual keyboards (cf. below), in which case lower case symbols can be written with a single finger, upper case with two fingers close together.
• detection of more than one motion vector in the field of view of the optical sensing device of the optoelectronic device can be employed as input.
• An example of this is the use of two hands or fingers that are moved independently, but in coordinated fashion.
• the principle scheme in figure 7b can be used to provide keyboard input via a "virtual keyboard”:
• a keyboard may be printed on the table surface or on a flat sheet lying on the table, and the camera records the position of the operator's fingers on the keyboard.
• Image analysis software is used to determine which key is indicated, e.g. by analyzing the obscuration of keyboard by the hands and fingers. In cases where fingertips are tagged for enhanced visibility, the tag location relative to the keyboard is analyzed.
• the actual tapping to indicate selection of an indicated key may be achieved by one of the modalities mentioned above in connection with mouse clicking, or by providing a motion cue to the imaging software. An example of the latter would be to set an activation threshold related to the dwell time on a given key, i.e. the finger must rest a time exceeding a predetermined minimum for a tapping to be registered.
• a variant of a virtual keyboard would be to project a keyboard as an image onto the writing surface. Fingertip tags with high contrast could in this case be rendered especially useful by making them fluoresce in response to the light used in projecting the keyboard or another light source.
• virtual touch screen Another type of virtual keyboard can be established under a wider concept which may be termed a "virtual touch screen”:
• the optoelectronic device is equipped with a camera which is directed towards the screen display of a computer.
• the operator places a pointer or wand, or one or more Fingers or hand(s) against the screen, thereby obscuring parts of the displayed image.
• Image analysis software determines which of the displayed image parts or regions that are indicated, e.g. by analysing the contours of the object held in front of the screen. This principle thus provides the features of a touch-sensitive screen, providing input opportunities for menu selection, etc, but without the need for technically complex physical sensors to record the touching action.
• writing may be performed in a manner similar to the case described above where writing was performed on a printed or projected keyboard.
• the virtual touch screen provides opportunities for more sophisticated input operations than simple two-dimensional coordinate point indication, by making the image analysis software able to discriminate between different motion- and shape patterns for the operator's fingers and hands in front of the screen.
• An example of this is to use the flat palm of the hand to mark or wipe larger parts of a document or scene, to effect copying, deletion or transfer.
• One vital part of any of the indicated embodiments is the processing of camera image sequences to obtain the output signal which indicates movement of the mobile electronic device relative to imaged objects or scenes.
• motion estimation which is a well established research field where the present major application is compression of digital video sequences.
• the MPEG video compression standards see e.g.: http://www.chiariglione.org/mpeg/ ) rely heavily on motion estimation.
• the present invention has an advantage over conventional video compression, in that only the camera motion need be estimated, which means characterization of the general movement of the background of the scene. The motion of smaller objects across the background can be disregarded, which lowers the demand for processing power compared to video compression.
• the camera image needs to be split into macroblocks, typically of the size 16x 16 pixels.
• the motion of each macroblock from one frame to another is then computed, resulting in a motion vector (MV).
• MV motion vector
• MVF motion vector field
• Moving objects, mismatched macroblocks due to insufficient texture, etc. will show up as outliers in the MVF and can be disregarded through proper filtering.
• the implementation of MVF calculation in the present invention will be chosen in each particular embodiment according to the properties of the camera and other phone hardware, available processing power, etc.
• Block matching is probably the most common way to calculate the MVF (cf., e.g.: A. M. Telkap, Digital Video Processing, Prentice Hall, 1995; J. Watkinson, The Engineer 's Guide to Motion Compensation, Snell & Wilcox, 1994).
• each macroblock in the new frame is compared with shifted regions of the same size from the previous frame, and the shift which results in the minimum error is selected as the best MV for that macroblock.
• Other methods of camera motion estimation include phase correlation (cf., e.g.: J. Watkinson, The Engineer 's Guide to Motion Compensation, Snell & Wilcox, 1994; Y. Liang, Phase-Correlation Motion Estimation, Final project, Stanford University) and the utilization of MPEG motion vectors (cf., e.g.: M. PiIu, On Using Raw MPEG Motion Vectors To Determine Global Camera Motion, Hewlett-Packard Company, 1997).
• phase correlation cf., e.g.: J. Watkinson, The Engineer 's Guide to Motion Compensation, Snell & Wilcox, 1994; Y. Liang, Phase-Correlation Motion Estimation, Final project, Stanford University
• MPEG motion vectors cf., e.g.: M. PiIu,
• Power consumption is a potential concern in embodiments of the present invention where an imaging camera and/or a display screen are operated on a battery.
• the camera part of the system draws little power compared to an illuminated display screen.
• several power conservation strategies are possible, since imaging requirements are different from and generally less demanding than those in regular photography.
• the total number of available camera pixels need not be active in the primary photo-event or in subsequent signal processing steps.
• Frame rates may also be reduced, depending on the updating rate and response speed that is required in applications based on the present invention.
• several strategies are available to conserve power.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Computer Hardware Design (AREA)
• Multimedia (AREA)
• Position Input By Displaying (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=35057647

Family Applications (1)

Country Status (2)

Cited By (64)

Citations (5)

• 2004
  • 2004-09-27 NO NO20044073A patent/NO20044073D0/no unknown
• 2005
  • 2005-09-27 WO PCT/NO2005/000360 patent/WO2006036069A1/en active Application Filing

Patent Citations (6)

Also Published As

Similar Documents

Legal Events