WO2016098519A1 - 電子機器、電子機器の制御方法及びその制御プログラム - Google Patents
電子機器、電子機器の制御方法及びその制御プログラム Download PDFInfo
- Publication number
- WO2016098519A1 WO2016098519A1 PCT/JP2015/082495 JP2015082495W WO2016098519A1 WO 2016098519 A1 WO2016098519 A1 WO 2016098519A1 JP 2015082495 W JP2015082495 W JP 2015082495W WO 2016098519 A1 WO2016098519 A1 WO 2016098519A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mode
- proximity sensor
- movement
- output
- control
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2354/00—Aspects of interface with display user
Definitions
- the present invention relates to an electronic device, a method for controlling the electronic device, and a control program therefor.
- a typical mobile terminal has a touch panel screen that serves both as an image display and a user interface. By touching this screen, the user can make necessary inputs to display a desired image or input information. Can be performed.
- touch panel screen serves both as an image display and a user interface. By touching this screen, the user can make necessary inputs to display a desired image or input information. Can be performed.
- you want to operate the mobile device without touching the screen such as when the user's hand is wet or dirty, but how to perform input at that time is an issue Yes.
- Patent Document 1 discloses a mobile computing device including an infrared LED and an infrared proximity sensor. According to the technique of Patent Document 1, infrared light emitted from an infrared LED is reflected by a hand approaching a mobile computing device, and the reflected light is detected by an infrared proximity sensor to detect hand movement. It is possible to perform an input operation desired by the user such as swipe without contact by the movement (gesture) of the hand. Therefore, even if the user's hand is dirty, there is no risk of contaminating the mobile computing device.
- infrared light emitted from the infrared LED can be reflected by an object other than the user's hand and detected by the infrared detection sensor. Accordingly, there is a risk of causing a malfunction by erroneously recognizing the detection of such an object as the movement of the user's hand. In particular, it can be said that there is a high risk of false detection when a mobile computing device is placed near an object or an object crosses the front of an infrared detection sensor.
- setting a sleep mode in which the control device ignores the output from the infrared proximity sensor and suppressing false detection is one measure. However, if all the gesture operations are disabled due to the setting of the sleep mode, the sleep mode cannot be canceled without contact, and the mobile computing device is forced to be touched to cancel the mode. If your hands are dirty, there is a risk of defacement.
- the present invention has been made in view of the above circumstances, and provides an electronic device, an electronic device control method, and a control program thereof that can effectively suppress erroneous detection while realizing non-contact operation. With the goal.
- an electronic device reflecting one aspect of the present invention.
- a proximity sensor that detects an object and generates an output in a detection region in the proximity range; and At least one device controlled by a control signal;
- a control circuit for generating a control signal for controlling the at least one device, The control circuit performs a first control on the at least one device or the control circuit itself in response to an output of the proximity sensor corresponding to a first movement of the object relative to the proximity sensor.
- the control circuit changes from the second mode to the first mode based on an output of the proximity sensor corresponding to a second movement of the object relative to the proximity sensor, which is different from the first movement.
- the second movement is an operation in which the object stays in the detection area for a first time and then leaves the detection area within a second time.
- a method for controlling an electronic device reflecting one aspect of the present invention includes a proximity sensor that detects an object and generates an output in a detection region in the proximity range;
- An electronic device control method comprising at least one device controlled based on a control signal, A first mode for performing a first control on the at least one device or the control circuit itself in response to an output of the proximity sensor corresponding to a first movement of the object relative to the proximity sensor;
- the output of the proximity sensor corresponding to the first movement can be set by switching between a second mode in which the first control is skipped, Based on the output of the proximity sensor corresponding to the second movement of the object relative to the proximity sensor, which is different from the first movement, from the second mode to the first mode, or the first Switch from the first mode to the second mode,
- the second movement is an operation in which the object stays in the detection area for a first time and then leaves the detection area within a second time.
- a control program for a display device includes a proximity sensor that detects an object and generates an output in a detection region in the proximity range;
- An electronic device control program comprising: at least one device controlled based on a control signal; and a control circuit that generates a control signal for controlling the device based on an output of the proximity sensor;
- the control circuit controlled by the control program is A first mode for performing a first control on the at least one device or the control circuit itself in response to an output of the proximity sensor corresponding to a first movement of the object relative to the proximity sensor;
- a second mode in which the first control is skipped with respect to the output of the proximity sensor corresponding to the first movement;
- the second movement is an operation in which the object stays in the detection area for a
- an electronic device a control method for the electronic device, and a control program thereof that can effectively suppress erroneous detection while realizing a non-contact operation.
- HMD head mounted display
- FIG. 1 is a perspective view of an HMD 100 that is an electronic apparatus including a display unit according to the present embodiment.
- FIG. 2 is a front view of the HMD 100 according to the present embodiment.
- FIG. 3 is a view of the HMD 100 according to the present embodiment as viewed from above.
- the right side and the left side of the HMD 100 refer to the right side and the left side for the user wearing the HMD 100.
- the HMD 100 of the present embodiment has a frame 101 as a mounting member.
- a frame 101 that is U-shaped when viewed from above has a front part 101a to which two spectacle lenses 102 are attached, and side parts 101b and 101c extending rearward from both ends of the front part 101a.
- the two spectacle lenses 102 attached to the frame 101 may or may not have refractive power.
- the cylindrical main body 103 is fixed to the front part 101 a of the frame 101 on the upper part of the spectacle lens 102 on the right side (which may be on the left side according to the user's dominant eye).
- the main body 103 is provided with a display unit 104.
- a display control unit 104DR (see FIG. 6 described later) that controls display of the display unit 104 based on an instruction from the processor 121 described later is disposed. If necessary, a display unit may be arranged in front of both eyes.
- FIG. 4 is a schematic cross-sectional view showing the configuration of the display unit 104.
- the display unit 104 includes an image forming unit 104A and an image display unit 104B.
- the image forming unit 104A is incorporated in the main body unit 103, and includes a light source 104a, a unidirectional diffuser 104b, a condenser lens 104c, and a display element 104d.
- the image display unit 104B which is a so-called see-through type display member, is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1).
- the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1).
- the light source 104a has a function of illuminating the display element 104d.
- the light source 104a emits light having a predetermined wavelength width, whereby the image light obtained by illuminating the display element 104d can have a predetermined wavelength width, and the hologram optical element 104h transmits the image light. When diffracted, the image can be observed by the user over the entire observation angle of view at the position of the pupil B. Further, the peak wavelength for each color of the light source 104a is set in the vicinity of the peak wavelength of the diffraction efficiency of the hologram optical element 104h, so that the light use efficiency is improved.
- the light source 104a is composed of LEDs that emit RGB light, the cost of the light source 104a can be reduced, and a color image is displayed on the display element 104d when the display element 104d is illuminated. The color image can be visually recognized by the user.
- each of the RGB LED elements has a narrow emission wavelength width, the use of a plurality of such LED elements enables high color reproducibility and bright image display.
- the display element 104d displays an image by modulating the light emitted from the light source 104a in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels that serve as light transmitting regions in a matrix. Has been. Note that the display element 104d may be of a reflective type.
- the eyepiece prism 104f totally reflects the image light from the display element 104d incident through the base end face PL1 by the opposed parallel inner side face PL2 and outer side face PL3, and passes through the hologram optical element 104h to the user's pupil.
- external light is transmitted and guided to the user's pupil, and is composed of, for example, an acrylic resin together with the deflecting prism 104g.
- the eyepiece prism 104f and the deflection prism 104g are joined by an adhesive with the hologram optical element 104h sandwiched between inclined surfaces PL4 and PL5 inclined with respect to the inner surface PL2 and the outer surface PL3.
- the deflection prism 104g is joined to the eyepiece prism 104f, and becomes a substantially parallel flat plate integrated with the eyepiece prism 104f. By joining the deflecting prism 104g to the eyepiece prism 104f, it is possible to prevent distortion in the external image observed by the user through the display unit 104.
- the hologram optical element 104h diffracts and reflects the image light (light having a wavelength corresponding to the three primary colors) emitted from the display element 104d, guides it to the pupil B, enlarges the image displayed on the display element 104d, and enlarges the user's pupil. It is a volume phase type reflection hologram guided as a virtual image.
- the hologram optical element 104h has, for example, three wavelength ranges of 465 ⁇ 5 nm (B light), 521 ⁇ 5 nm (G light), and 634 ⁇ 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected).
- the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak
- the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half maximum of the diffraction efficiency peak. is there.
- the reflection-type hologram optical element 104h has high wavelength selectivity, and only diffracts and reflects light having a wavelength in the above-mentioned wavelength range (near the exposure wavelength).
- the hologram optical element 104h is transmitted, and a high external light transmittance can be realized.
- the light emitted from the light source 104a is diffused by the unidirectional diffusion plate 104b, condensed by the condenser lens 104c, and enters the display element 104d.
- the light incident on the display element 104d is modulated for each pixel based on the image data input from the display control unit 104DR, and is emitted as image light. Thereby, a color image is displayed on the display element 104d.
- Image light from the display element 104d enters the eyepiece prism 104f from its base end face PL1, is totally reflected a plurality of times by the inner side face PL2 and the outer side face PL3, and enters the hologram optical element 104h.
- the light incident on the hologram optical element 104h is reflected there, passes through the inner side surface PL2, and reaches the pupil B.
- the user can observe an enlarged virtual image of the image displayed on the display element 104d, and can visually recognize it as a screen formed on the image display unit 104B.
- the hologram optical element 104h constitutes a screen, or it can be considered that a screen is formed on the inner surface PL2.
- “screen” may refer to an image to be displayed.
- the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h transmit almost all of the external light, the user can observe an external field image (real image) through them. Therefore, the virtual image of the image displayed on the display element 104d is observed so as to overlap with a part of the external image. In this manner, the user of the HMD 100 can simultaneously observe the image provided from the display element 104d and the external image via the hologram optical element 104h. Note that when the display unit 104 is in the non-display state, the image display unit 104B is transparent, and only the external image can be observed.
- a display unit is configured by combining a light source, a liquid crystal display element, and an optical system.
- a self-luminous display element for example, an organic EL display
- Element for example, an organic EL display
- a transmissive organic EL display panel having transparency in a non-light emitting state may be used.
- the proximity sensor 105 disposed near the center of the frame 101 and the lens 106 a of the camera 106 disposed near the side are provided on the front of the main body 103 so as to face the front. It has been.
- the “proximity sensor” refers to a proximity range in front of the detection surface of the proximity sensor in order to detect that an object, for example, a part of a human body (such as a hand or a finger) is in proximity to the user's eyes.
- the signal is output by detecting whether or not it exists within the detection area.
- the proximity range may be set as appropriate according to the operator's characteristics and preferences. For example, the proximity range from the detection surface of the proximity sensor may be within a range of 200 mm. If the distance from the proximity sensor is 200 mm or less, the user can easily put the palm and fingers into and out of the user's field of view with the arm bent, so that the user can easily operate with gestures using the hands and fingers.
- the control circuit determines that the object exists in the proximity range based on a signal output from the proximity sensor when the object enters the detection area in the proximity range in front of the proximity sensor.
- an effective signal may be output from the proximity sensor to the control circuit.
- a passive proximity sensor has a detection unit that detects invisible light and electromagnetic waves emitted from an object when the object approaches.
- a passive proximity sensor there are a pyroelectric sensor that detects invisible light such as infrared rays emitted from an approaching human body, and a capacitance sensor that detects a change in electrostatic capacitance between the approaching human body and the like.
- the active proximity sensor includes an invisible light and sound wave projection unit, and a detection unit that receives the invisible light and sound wave reflected and returned from the object.
- Active proximity sensors include infrared sensors that project infrared rays and receive infrared rays reflected by objects, laser sensors that project laser beams and receive laser beams reflected by objects, and project ultrasonic waves. Then, there is an ultrasonic sensor that receives ultrasonic waves reflected by an object. Note that a passive proximity sensor does not need to project energy toward an object, and thus is excellent in low power consumption. Active proximity sensors can improve detection reliability. For example, even if the user is wearing gloves that do not transmit detection light emitted from the human body such as infrared light, the movement of the user's hand Can be detected. A plurality of types of proximity sensors may be used in combination.
- Proximity sensors are generally smaller and cheaper than cameras, and consume less power. In addition, it is difficult for the proximity sensor to perform complicated detection such as detection of the shape of the object, but it can determine the approach and separation of the object from the detection area, so that the user can pass the hand or finger or the palm of the hand. By holding it over, the HMD can be operated, and complicated image processing required for gesture recognition by analysis of the image captured by the camera is also unnecessary.
- FIG. 5 is an enlarged view of the proximity sensor 105 used in the present embodiment as viewed from the front.
- the proximity sensor 105 includes a light receiving unit 105 a that receives invisible light such as infrared light emitted from a human body as detection light.
- the light receiving unit 105a forms light receiving areas RA to RD arranged in 2 rows and 2 columns, and when receiving invisible light, signals corresponding to the light receiving areas RA to RD are individually output from the light receiving areas RA to RD. It has become.
- each of the light receiving areas RA to RD changes in intensity according to the distance from the light receiving unit 105a to the object, and the intensity increases as the distance decreases. If it is a pyroelectric sensor that detects infrared light emitted from the human body, it is difficult to falsely detect an object other than the exposed human body. For example, when working in a narrow space, the false detection is effectively prevented. There is an advantage that you can.
- the right sub-body portion 107 is attached to the right side portion 101b of the frame 101
- the left sub-body portion 108 is attached to the left side portion 101c of the frame 101.
- the right sub-main body portion 107 and the left sub-main body portion 108 have an elongated plate shape, and have elongated protrusions 107a and 108a on the inner side, respectively.
- the right sub-body portion 107 is attached to the frame 101 in a positioned state
- the elongated protrusion 108 a is attached to the side of the frame 101.
- the left sub-main body portion 108 is attached to the frame 101 in a positioned state.
- a geomagnetic sensor 109 for detecting geomagnetism
- a gyro and acceleration sensor 110 for detecting geomagnetism
- a speaker (or earphone) 111A and a microphone 111B are provided in the left sub-main body section 108.
- the main main body 103 and the right sub main body 107 are connected so as to be able to transmit signals through a wiring HS
- the main main body 103 and the left sub main body 108 are connected so as to be able to transmit signals through a wiring (not shown). Yes.
- FIG. 1 As schematically illustrated in FIG.
- the right sub-main body 107 is connected to the control unit CTU via a cord CD extending from the rear end.
- a 6-axis sensor in which a gyro and an acceleration sensor are integrated may be used.
- the HMD can be operated by sound based on an output signal generated from the microphone 111B according to the input sound.
- the main main body 103 and the left sub main body 108 may be configured to be wirelessly connected.
- FIG. 6 is a block diagram of main circuits of the HMD 100.
- the control unit CTU generates a control signal for the display unit 104 and each device device (to be described later) (to be referred to as a functional device), and thereby a processor 121 as a control circuit that controls each functional device, and an operation unit 122.
- a communication unit 124 for exchanging data with the outside
- a ROM 125 for storing programs and the like
- RAM 126 for storing image data and the like.
- Power supply circuit 128 and battery 127, and a storage device 129 such as an SSD or a flash memory.
- the processor 121 can use an application processor used in a smartphone or the like, but the type of the processor 121 is not limited. For example, if an application processor has built-in hardware necessary for image processing such as GPU and Codec as a standard, it can be said that the processor is suitable for a small HMD.
- the processor 121 receives a signal from the proximity sensor 105.
- the processor 121 controls image display on the display unit 104 via the display control unit 104DR.
- the processor 121 receives power from the battery 127 via the power supply circuit 128, operates according to a program stored in at least one of the ROM 124 and the storage device 129, and operates according to an operation input such as power-on from the operation unit 122 according to an operation input such as power-on. Can be input and stored in the RAM 126 and communicated with the outside via the communication unit 123 as necessary. Furthermore, as will be described later, the processor 121 executes image control according to the output from the proximity sensor 105, so that the user can perform screen control of the HMD 100 by gesture operation using a hand or a finger.
- FIG. 7 is a front view when the user US wears the HMD 100 of the present embodiment.
- FIG. 8 is a side view (a) and a top view (b) when the user US wears the HMD 100, and shows it together with the user's hand.
- FIG. 9 is a diagram showing an image visually recognized by the user US through the see-through type image display unit 104B.
- the gesture operation is an operation in which at least the hand HD or finger of the user US enters or leaves the detection area of the proximity sensor 105 and can be detected by the processor 121 of the HMD 100 via the proximity sensor 105.
- the screen 104i of the image display unit 104B is arranged so as to overlap the effective visual field EV of the user's eye facing the image display unit 104B (here, positioned in the effective visual field EV).
- the detection area SA of the proximity sensor 105 is in the visual field of the user's eye facing the image display unit 104B.
- the detection area SA is located within the stable focus field of the user's eye or inside the visual field (within about 90 ° horizontal and within about 70 ° vertical), and more preferably located inside the stable focus field.
- the proximity sensor 105 may be installed with its arrangement and orientation adjusted so as to overlap with the effective visual field EV or the inner visual field (horizontal within about 30 °, vertical within about 20 °).
- FIG. 9 shows an example in which the detection area SA overlaps the screen 104i.
- the detection region SA of the proximity sensor 105 is set within the visual field of the eye of the user US while the user US is wearing the frame 101 that is a mounting member on the head, the screen is displayed through the screen 104i. While observing the hand HD, the approach and withdrawal of the hand to the detection area SA of the proximity sensor 105 can be reliably recognized without accompanying eye movement.
- the gesture operation can be reliably performed while recognizing the detection area SA. .
- the gesture operation can be performed more reliably. If the detection area SA overlaps the screen 104i, the gesture operation can be performed more reliably.
- the proximity sensor has a plurality of light receiving regions as in the present embodiment, the entire plurality of light receiving regions are regarded as one light receiving unit, and the maximum detection range of the light receiving unit is regarded as a detection region.
- the detection area SA of the proximity sensor 105 is set to overlap the screen 104i, an image indicating the detection area SA is displayed on the screen 104i (for example, the range of the area SA is displayed by a solid line). ), The user can surely recognize the detection area SA, so that the operation by the gesture can be performed more reliably.
- the light receiving unit 105a determines that no gesture operation has been performed. To do.
- the light receiving unit 105a detects the invisible light emitted from the hand HD, and the proximity sensor 105 based on the invisible light is detected.
- the processor 121 determines that a gesture operation has been performed.
- the gesture operation is performed by the user's hand HD.
- the gesture operation may be performed by the user using a pointing device made of a material that can emit invisible light. May be performed.
- the light receiving unit 105a has the light receiving regions RA to RD arranged in 2 rows and 2 columns (see FIG. 5). Therefore, when the user US moves the hand HD closer to the front of the HMD 100 from either the left, right, up, or down directions, the output timings of signals detected in the light receiving areas RA to RD are different.
- 10 and 11 are examples of signal waveforms of the light receiving areas RA to RD, where the vertical axis represents the signal intensity of the light receiving areas RA to RD and the horizontal axis represents the time.
- the vertical axis represents the signal intensity of the light receiving areas RA to RD
- the horizontal axis represents the time.
- the light receiving areas RA and RC first receive invisible light. Therefore, as shown in FIG. 10, first, the signals in the light receiving areas RA and RC rise, the signals in the light receiving areas RB and RD rise later, and the signals in the light receiving areas RA and RC fall, and then the light receiving areas RB, The RD signal falls.
- the processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the right to the left. Accordingly, the processor 121 controls the display control unit 104DR to change the display so that the page is turned from the image G1 to the image G2, for example, in accordance with the movement of the hand HD as shown in FIG. be able to.
- the signals of the light receiving areas RA and RB rise first, the signals of the light receiving areas RC and RD rise after a delay, and the signals of the light receiving areas RA and RB further fall, and then the signals of the light receiving areas RC and RD. Is falling.
- the processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the top to the bottom.
- the presence and movement of the hand HD can be reliably detected by using the proximity sensor and positioning the detection region within the visual field of the user's eye facing the image display unit.
- the user US can view the hand operation and the operation in conjunction with each other, and can realize an intuitive operation.
- the proximity sensor is small and consumes less power than the camera, the continuous operation time of the HMD 100 can be extended.
- the proximity sensor 105 has a characteristic that the output increases or decreases in accordance with the distance between the hand HD and the proximity sensor 105 in the detection area SA. As shown in FIG. There is a case where the output of the sensor starts to increase from zero level almost simultaneously at time T1, and gradually increases until time T7. In this case, it is determined that an irrelevant object has approached the HMD 100 instead of starting gesture recognition using the hand or the pointing tool. Details will be described later.
- the execution of the gesture detection process by the processor 121 or the energization to the proximity sensor is stopped to detect the gesture operation. It may be temporarily interrupted.
- the processor 121 is responsive to the output of the proximity sensor 105 corresponding to the first movement of the object relative to the proximity sensor 105 (ie, the movement of the user's hand in front of the proximity sensor 105).
- a first mode for executing a first control on at least one functional device or the processor 121 itself can be executed. Further, the processor 121 can execute a second mode in which the first control is skipped with respect to the output of the proximity sensor 105 corresponding to the first movement.
- the processor 121 is set to the first mode, when the user wearing the HMD 100 moves the hand in front of the face and performs a predetermined first movement, the proximity sensor 105 responds to the movement of the hand.
- the processor 121 executes the first control on at least one functional unit or the processor 121 itself according to the output.
- the first movement of the object is, for example, a swipe operation by hand. However, this movement is different from the second movement described later.
- the first control is, for example, screen display control such as page turning according to the first movement by the user's hand such as the swipe operation. If the animation is displayed so that the previous and next page images slide in accordance with the swipe operation, the user can move the hand HD in front of the display screen of the display unit 104 to link the screen display and the operation. Can be visually recognized. Therefore, an easy-to-use user interface can be provided. Further, the processor 121 switches from the second mode to the first mode based on the output of the proximity sensor 105 corresponding to the second movement of the object relative to the proximity sensor 105, which is different from the first movement.
- screen display control such as page turning according to the first movement by the user's hand such as the swipe operation. If the animation is displayed so that the previous and next page images slide in accordance with the swipe operation, the user can move the hand HD in front of the display screen of the display unit 104 to link the screen display and the operation. Can be visually recognized. Therefore, an easy-to-use user interface can be provided. Further, the processor 121 switches
- the second movement is an operation in which the object stays in the detection area of the first time proximity sensor 105 and then leaves the detection area within the second time from there.
- this is a movement in which the user holds the hand in front of the HMD for a certain period of time and then removes the hand from the hand.
- This operation is performed when the object other than the user or the HMD enters the detection area SA, or to pass through the detection area or from a distance toward the detection area in order to distinguish it from a gesture operation for performing a screen operation.
- the movement is more complicated than the movement that approaches.
- the processor 121 When the processor 121 has set the second mode, even if the proximity sensor 105 generates an output corresponding to the first movement, the processor 121 skips the first control. When the user remembers the second movement, the second mode can be switched to the first mode without contact. Thereby, when control by gesture input is required, input by the proximity sensor 105 can be performed, and control of the functional device and the processor 121 itself can be performed according to the first movement.
- the first control that is skipped in the second mode does not include a display that indicates to the user that the setting of the first mode has been skipped. That is, even if they are common in terms of screen control, they are not skipped in the second mode if screen control different from that performed in accordance with the first movement in the first mode is performed.
- the processor 121 when the proximity sensor 105 has a characteristic that the output increases or decreases according to the distance between the hand HD and the proximity sensor 105 in the detection area SA, the processor 121 outputs the output of the proximity sensor 105. For example, when it is detected that the distance between the hand HD and the proximity sensor 105 in the detection area SA is less than 50 mm, for example, outside the range of the reference level (50 to 200 mm), the risk of false detection is reduced. Therefore, the first control corresponding to the first movement of the user's hand, which will be described later, may not be performed. At the same time, a warning mark indicating that the detected position of the hand HD is too close may be displayed on the screen 104i.
- the processor 121 executes screen display control according to a control program stored in at least one of the ROM 124 and the storage device 129.
- the direction in which the hand HD enters the detection area of the proximity sensor 105 is not limited (that is, the first direction is all directions).
- the first mode is set when the hand HD is released in the downward direction as the direction 2. That is, this series of movements is the second movement.
- the processor 121 performs screen display control, which is predetermined first control, according to the output signal of the proximity sensor 105 corresponding to the first movement.
- the proximity sensor 105 when the second mode is set, the proximity sensor 105 outputs a signal indicating that the object remains in the detection region in an inactive state for 3 seconds or more by, for example, placing the HMD 100 near the wall.
- the processor 121 does not switch to the first mode. Further, even when a signal corresponding to the first movement is generated, the screen display control as the first control is not performed. Thereby, it is possible to prevent a malfunction that the first mode is started against the user's intention.
- the second mode not all screen control is prohibited, but screen control different from the screen control according to the first movement by the user's hand in the first mode, for example, the first mode
- the screen control for displaying an icon for notifying the user that the mode setting has not been accepted can be executed without skipping.
- the images G1 and G2 are displayed on the screen 104i, but the display of the images G1 and G2 is arbitrary and is not necessarily displayed.
- step S90 the processor 121 first determines in step S90 whether an NG flag, which will be described later, is currently ON.
- the NG flag is turned ON when 3 seconds or more have passed while the object has entered the detection range of the proximity sensor 105. If the NG flag is ON, it is confirmed that the detection light (invisible light here) is not detected (Yes in step S92), and the NG flag is turned OFF in step S94. If the detection light is continuously detected in step S92, the process returns to step S90.
- step S96 the processor 121 waits until the light receiving unit 105a detects the detection light. If the detection light is detected, in step S98, the output of all the light reception sensors is gradually increased as shown in FIG. Determine if it is rising. If Yes in step S98, it is determined that the instruction is not an instruction by the user, and the process proceeds to step S123 described later. If the determination in step S98 is No, the process proceeds to step S102, and the approach direction of the hand HD is determined as follows based on the timing of the signals in the areas RA to RD. (1) Signal rise in areas RA and RC first, followed by signal rise in areas RB and RD: The hand HD has entered in the direction from right to left.
- step S103 the processor 121 stores the determined approach direction of the hand HD.
- step S104 the processor 121 determines whether or not the first mode is set, and when it is determined that the mode is released (that is, the second mode is set), the process continues.
- step S115 the built-in first timer is reset and started.
- step S116 a clock-shaped icon IC1 is displayed in the center of the screen 104i as shown in FIG. The user who has visually recognized the icon IC1 on the screen 104i can detect that the hand HD has entered the detection area of the proximity sensor 105 and start measuring the holding time. The icon IC1 may not be displayed.
- step S117 the processor 121 determines whether or not the time measured by the first timer is 1 second (first time) or more. If the time measured by the first timer is less than 1 second, the flow returns to step S117 unless the light receiving unit 105a determines that the detected light is not detected in step S118.
- step S117 When the processor 121 determines in step S117 that the time measured by the first timer has reached 1 second or more, it indicates that the light receiving unit 105a continues to detect the detection light, and thus the hand HD is detected by the proximity sensor 105. It can be inferred that it remained in the detection area for 1 second or more. Therefore, the processor 121 stops the timing of the first timer, and then proceeds to step S119. As shown in FIG. 13B, the processor 121 displays a hand-shaped icon IC2 on the screen 104i instead of the icon IC1. Display in the center (first screen display). By visually recognizing the icon IC2, it can be seen that the user US has held the hand HD in the detection area for 1 second or more and has cleared the first stage for the first mode setting operation.
- FIG. 14 is a time chart showing the state of signal detection.
- the processor 121 detects the signal of the proximity sensor 105 at intervals of 20 milliseconds, for example.
- the light receiving sensor detects from the light receiving sensor at time T3. Since the output is turned on, the determination of Yes is made in step S117 described above.
- the processor 121 resets and starts the second timer built in step S120, and waits until the light receiving unit 105a detects no detection light in step S121. Then, it is determined whether or not the time measured by the second timer has exceeded 3 seconds (second time).
- the second timer may be the first timer that stops timing. At least one of the first time and the second time can be arbitrarily adjusted between 0.5 and 5 seconds, for example, by an operation from the operation unit 122.
- the processor 121 receives an instruction from the user, and based on this instruction, at least one of the first time and the second time can be adjusted, so that the burden on the user holding the hand is reduced and the user's preference is reduced. Can be adapted to At least one of the first and second times may be a fixed value.
- step S122 If it is determined in step S122 that the time measured by the second timer is 3 seconds or less, it can be inferred that the hand HD has left the detection area of the proximity sensor 105 within 3 seconds, so the processor 121 determines in step S124 that The direction of detachment of the hand HD is determined from the signal timings in the areas RA to RD as follows. (5) Signal fall in areas RA and RC first, followed by signal fall in areas RB and RD: Hand HD left in the direction from right to left. (6) Signal fall in areas RB and RD first, followed by signal fall in areas RA and RC: Hand HD left in the direction from left to right.
- step S125 the processor 121 determines whether or not the direction of detaching the hand HD is downward. If it is determined that the hand HD is in the downward direction, the processor 121 determines that the user US wants to set the first mode, and sets the first mode in step S126. Further, as shown in FIG. 13C, instead of the icon IC2, an icon IC3 with a circle over the hand is displayed in the center of the screen 104i (second screen display different from the first screen display). ). By visually recognizing the icon IC3, the user US knows that the setting of the first mode has been accepted. Thereafter, as shown in FIG.
- the processor 121 displays a small icon IC4 having a hand shape indicating that the first mode is being set instead of the icon IC3 on the upper right of the screen 104i. Thereafter, the flow moves to step S113. Since the user can recognize that the first mode has been set by moving from the screen in FIG. 13B to the screen in FIG. 13D, the icon IC3 shown in FIG. 13C is displayed. May be omitted.
- step S125 if it is determined in step S125 that the direction of detachment of the hand HD is a direction other than the downward direction, there is a risk of erroneous detection, so the processor 121 shifts the flow to step S123 and turns on the NG flag. .
- the processor 121 shifts the flow to step S123 and turns on the NG flag. .
- FIG. 13 (e) instead of the icon IC2, an icon IC5 in which a cross is placed on the shape of the hand is displayed at the center of the screen 104i.
- the icon IC5 is erased after being displayed for a predetermined time (for example, 2 seconds). Thereafter, the flow moves to step S101.
- step S122 If it is determined in step S122 that the time measured by the second timer exceeds 3 seconds (that is, as shown in FIG. 14C), the hand is moved from the detection area SA at time T6 after time T5. In the case of separation), since the time until the separation is too long, the processor 121 determines that it is not the second movement by the user and moves the flow to step S123 without determining the direction of separation of the hand. Accordingly, when the HMD 100 is placed near the wall or another object is placed in front of the HMD 100 placed on the desk, the first mode is set, and the HMD 100 starts gesture recognition unintentionally. Is prevented.
- the light receiving unit 105a emits the detection light in step S118. If not detected (as shown in FIG. 14A, when the hand leaves the detection area SA at time T2 before time T3), the processor 121 shifts the flow to step S123.
- step S98 the process also proceeds to step S123. Therefore, from time T1 to time T3 after the first timer times 1 second and from time T3 to time T5 when the second timer times 3 seconds. In the meantime, even if the outputs of all the light receiving sensors gradually decrease and change to zero level, the first mode is not set. For example, assuming that the user wearing the HMD 100 for the inspection work approaches the inspection object, it is considered that all the light receiving sensors of the proximity sensor 105 are likely to increase the output gradually from the zero level at the same time. For this reason, it is possible to more reliably reduce the occurrence of malfunctions by determining that the approach of an object that gradually increases the output of all sensors from the zero level is not due to the behavior of the user's hand.
- step S104 determines that the gesture operation determined in step S102 has been performed for screen display control. Therefore, in step S105, the processor 121 waits until the light receiving unit 105a detects no detection light. If the detection is not detected, the processor 121 determines the gesture operation in accordance with the hand approach direction stored in step S103. Therefore, in step S106, the direction of detachment of the hand HD is determined from the timing of the signals in the areas RA to RD as follows. (9) Signal fall in areas RA and RC first, followed by signal fall in areas RB and RD: Hand HD left in the direction from right to left.
- step S107 the processor 121 determines whether or not the entering direction and the leaving direction of the hand HD match. If the entering direction and the leaving direction of the hand HD do not match, there is a risk of erroneous detection of the gesture operation. In this example, the gesture operation is not detected and the process proceeds to step S113. Note that a case where the entering direction and the leaving direction of the hand HD are different may be detected and used for another control.
- the processor 121 determines a gesture operation in the subsequent step S108. Specifically, when the processor 121 determines that the gesture operation (1) + (9) has been performed continuously, the screen moves from right to left (turns or scrolls) in step S109. In this manner, the display unit 104 is controlled via the display control unit 104DR. When the processor 121 determines that the gesture operation (2) + (10) has been continuously performed, the processor 121 displays the display via the display control unit 104DR so that the screen moves from left to right in step S110. The unit 104 is controlled.
- step S111 the processor 121 displays the display via the display control unit 104DR so that the screen moves from top to bottom.
- the unit 104 is controlled.
- step S112 the processor 121 displays the display via the display control unit 104DR so that the screen moves from the bottom to the top.
- the unit 104 is controlled. Specific screen display control will be described later with reference to FIG.
- the direction in which the user's hand enters or leaves the detection area is determined according to the timing at which the plurality of light receiving areas detect the user's hand, and the direction according to the determined direction is determined. Since the screen display is controlled, variations in control according to the signal from the proximity sensor can be increased.
- the user US can turn the page and scroll the image in the same direction as the movement of the hand in synchronization with the movement of the hand HD entering and leaving the space in front of the eyes while visually checking the movement through the image display unit 104B. It is easy to use because it can be operated intuitively.
- step S113 the processor 121 returns the flow to step S101 and continues the control unless a signal for instructing the cancellation of the first mode is input.
- step S113 if it is determined in step S113 that the user US has input a signal instructing to cancel the first mode, the processor 121 deletes the icon IC4 in step S114 and then cancels the first mode.
- the cancellation of the first mode (that is, switching from the first mode to the second mode) may be performed by an operation similar to the operation for setting the first mode. Further, after switching from the second mode to the first mode, control may be performed so as to switch from the first mode to the second mode when a predetermined period of time has elapsed since the first motion was last detected. .
- gesture operations are likely to be carried out continuously, it usually waits for detection of the next gesture operation. However, if the user US forgets the stop operation, it is consumed if the non-operation time is long. You may make it stop detection of gesture operation for electric power reduction.
- an object irrelevant to the user enters the detection area, or the object enters the detection area when the user unconsciously approaches another object. It may be difficult for a user wearing the HMD to notice that he / she is lying. For this reason, it can be said that the control of the present embodiment described above is particularly effective for preventing erroneous detection.
- FIG. 16 shows a time chart of the second embodiment.
- the processor 121 detects the signal of the proximity sensor 105 at a relatively long interval of 200 milliseconds before the first mode setting (that is, during the second mode setting).
- the configuration is the same as that of the first embodiment.
- the interval between the output signals from the proximity sensor 105 in the second mode is set longer than that in the first mode, thereby realizing the return to the normal state by the gesture operation while reducing the power consumption. be able to.
- the processor 121 operates the first timer, but thereafter detects the signal of the proximity sensor 105 at intervals of 200 milliseconds. To do.
- the detection of holding the hand HD is not particularly problematic even if it is performed at intervals of 200 milliseconds.
- since the first mode can be switched by the gesture operation, complicated control such as periodically returning from the power saving mode and starting the program for gesture recognition becomes unnecessary. Therefore, it is possible to prevent the power consumption from increasing.
- the processor 121 narrows the interval for detecting the signal from the proximity sensor 105 to an interval of 20 milliseconds. This is because if the detection interval is long, the light receiving timings in the plurality of light receiving regions in the proximity sensor 105 are the same, and there is a possibility that the direction in which the hand HD is detached cannot be detected.
- the processor 121 determines that the first mode should be set after the second timer is stopped at time T5
- the processor 121 detects the signal of the proximity sensor 105 so that the gesture operation can be accurately detected.
- the interval to be maintained is the 20 millisecond interval (see FIG. 16).
- the processor 121 determines that the first mode setting has not been set
- the interval for detecting the signal of the proximity sensor 105 is returned to the 200 millisecond interval to save energy. While waiting for new detection.
- the first mode is set when the hand that has stayed in the detection region for a predetermined time is released downward, regardless of the direction of entry of the hand.
- FIG. the first mode is set by allowing the hand HD to enter from the downward direction as the first direction, staying in the detection region for a predetermined time, and then detaching downward as the second direction.
- This mode may be set when the leaving direction after staying in the detection area for a predetermined time is perpendicular to the approaching direction.
- the direction in which the hand enters and leaves can be determined arbitrarily.
- the second movement switches from the second mode in which screen display control such as page feed is skipped to the first mode in which the screen control is executed.
- it also serves as a return from the low power consumption state to the normal state.
- the present invention is not limited to these embodiments.
- a specific image may be selected and determined from a plurality of images by the second movement.
- images G1 to G3 as display candidates are displayed side by side on the screen 104i, and an icon IC4 indicating the setting of the first mode is displayed.
- Page can be turned by gesture operation with the hand passing through the detection area of the proximity sensor, but the image is inadvertently enlarged so that the image cannot be selected and enlarged. To prevent.
- an icon IC2 is displayed as shown in FIG.
- an icon IC3 indicating that the gesture operation has been recognized is displayed as shown in FIG. Determines that the user desires to select the middle image G2, and displays the image G2 in an enlarged manner as shown in FIG.
- the display returns to the normal size display.
- the icon IC5 indicating that the image could not be selected is displayed as shown in FIG. Is displayed.
- the icon IC4 indicating the setting of the first mode is preferably not displayed when other icons are displayed to avoid confusion.
- the type of icon is not limited, and the user may be notified of the state by text or voice instead of the icon.
- the first control executed by the first movement in the first mode is described as screen display control such as page turning.
- the present invention is not limited to this, and the first control may be switching of an operation mode of an electronic device or adjustment of a function as described below.
- the operation mode of the HMD 100 is switched by the gesture operation as the first movement, and the screen display is controlled according to each operation mode. You may go.
- the transition between the screens is recognized as a swipe operation in which the hand moves up, down, left, and right, and is switched by sliding in accordance with the swipe operation.
- the hand HD up and down it is preferable to move the palm and the elbow of the user while keeping the palm and the elbow horizontally, or to move the palm up and down with the elbow as a fulcrum in order to avoid erroneous detection.
- the home screen G11 is displayed on the display unit 104.
- additional information such as the current date, temperature, and humidity is displayed at the top of the screen.
- a gesture operation is performed by moving the hand HD from the bottom to the top, a non-display mode in which no screen display is performed is entered, and the display unit 104 is switched to the non-display screen G01.
- the user US can observe only the external image through the display unit 104. From this screen display (that is, during the execution of the non-display mode), when the gesture operation is performed by moving the hand HD from top to bottom, the non-display mode is canceled and the display unit 104 is switched to the home screen G11.
- the gesture operation is performed by moving the hand HD from the right to the left from the display of the home screen G11
- the music display mode is displayed and the music title display screen G10 is displayed on the display unit 104.
- the title, author, and the like flowing from the speaker or earphone are displayed at the top of the screen, and the volume control VOL is displayed on the left side of the screen.
- the volume of the song to be played increases one by one, and conversely the hand HD from the top to the bottom.
- the volume decreases by one tick and the display changes with it.
- the display unit 104 is switched to the home screen G11.
- the display mode 104 is displayed and the imaging field angle display screen G12 is displayed on the display unit 104.
- the user US can capture the subject within the rectangular frame displayed on the imaging angle-of-view display screen G12 with the camera 106 as a still image or a moving image.
- the imaging may be started / finished in accordance with the switching of the screen, the imaging may be started / terminated after the standby time has elapsed from that point, or the predetermined hand movement detection described above may be performed.
- the imaging may be started / finished by operating the operation unit or by voice. From this screen display, when the hand HD is moved from right to left and a gesture operation is performed, the display unit 104 is switched to the home screen G11.
- the setting mode is displayed and the setting display screen G21 is displayed on the display unit 104.
- the display unit 104 switches to another setting display screen G20, or the hand HD is moved from the left to the right.
- the display unit 104 switches to another setting display screen G22, and each different setting can be performed using the operation unit 122, for example.
- the display unit 104 is switched to the home screen G11. In addition, you may perform the movement to the vertical direction or horizontal direction for selecting a candidate item from one display screen (menu screen) containing a some selection item using the detection of gesture operation.
- each screen is merely an example, and arbitrary items may be added or replaced from various screens including images associated with other functions. Moreover, what screen should be switched according to the moving direction of the hand, how to set the animation at the time of switching may be set as appropriate according to the preference of the user.
- a pyroelectric sensor which is a kind of passive sensor, is used as the proximity sensor.
- a light emitting unit that emits invisible light such as infrared light
- a light emitting unit that emits light.
- An active sensor having a light receiving unit that detects invisible light reflected by an object may be used.
- the use of an active sensor has an advantage that the gesture operation can be detected even when the user is wearing gloves that do not transmit detection light such as infrared rays.
- this invention is applicable not only to HMD but portable terminals, such as a smart phone.
- the direction of leaving the detection area is the specific direction (downward) regardless of the direction of entry into the detection area, and the time remaining in the detection area is a predetermined time (1 to 3 seconds). In this case, it is determined that the movement is the second movement.
- the direction of entering the detection area is a specific direction (upward) and the direction of leaving is another specific direction (downward) and the time remaining in the detection area is a predetermined time.
- the second movement is determined.
- the present invention is not limited to these, and various combinations are possible depending on the combination of the time remaining in the detection region and the first direction and the second direction. In this case, if the user is separated in a direction opposite to the direction in which the user has entered, or is separated in a direction that intersects the entered direction, the user can easily learn the operation content, and can prevent an erroneous operation.
- the first control is not limited to the above-described screen display control, electronic device operation mode switching, function adjustment, or the like, but after selecting an application and starting the application, or after selecting a menu.
- the control may be an operation that executes an operation defined in the menu.
- the mode in which the setting and release of the first mode (setting in the second mode) are performed based on the output from the proximity sensor described above and the mode in which the other mode (for example, the operation unit 122) is performed may be switched. .
- HMD 101 Frame 101a Front part 101b Side part 101c Side part 101d Long hole 101e Long hole 102 Eyeglass lens 103 Main body part 104 Display 104A Image forming part 104B Image display part 104DR Display control part 104a Light source 104b Unidirectional diffuser 104c Condensing lens 104d Display element 104f Eyepiece prism 104g Deflection prism 104h Hologram optical element 104i Screen 105 Proximity sensor 105a Light receiving part 106 Camera 106a Lens 107 Right sub-main part 107a Protrusion 108 Left sub-main part 108a Protrusion 109 Acceleration sensor 110 Gyro 111A Speaker (or earphone) 111B Microphone 121 Processor 122 Operation unit 123 GPS reception unit 124 Communication unit 125 ROM 126 RAM 127 Battery 129 Storage device 200 Smartphone CD code CTU Control unit HD Hand HS Wiring IC1 to IC5 Icon PL1 Base end surface PL2 Inner surface PL3 Outer surface PL4 Inc
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Optics & Photonics (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
Description
近接範囲にある検出領域において、物体を検出して出力を生成する近接センサと、
制御信号によって制御される少なくとも一つのデバイスと、
前記少なくとも一つのデバイスを制御するための制御信号を生成する制御回路と、を備え、
前記制御回路は、前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を実行可能であり、
前記制御回路は、前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作であるものである。
前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を切り替えて設定可能であり、
前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作であるものである。
前記制御プログラムによって制御された前記制御回路は、
前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を実行可能であり、
前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作であるものである。
以下に、本発明の第1実施形態を、図面を参照して説明する。図1は、本実施の形態にかかる、ディスプレイユニットを含む電子機器であるHMD100の斜視図である。図2は、本実施の形態にかかるHMD100を正面から見た図である。図3は、本実施の形態にかかるHMD100を上方から見た図である。以下、HMD100の右側及左側とは、HMD100を装着したユーザーにとっての右側及び左側をいうものとする。
(1)先に領域RA,RCの信号立ち上がり、次いで領域RB,RDの信号立ち上がり:手HDは右から左へ向かう方向に進入した。
(2)先に領域RB,RDの信号立ち上がり、次いで領域RA,RCの信号立ち上がり:手HDは左から右へ向かう方向に進入した。
(3)先に領域RA,RBの信号立ち上がり、次いで領域RC,RDの信号立ち上がり:手HDは上から下へ向かう方向に進入した。
(4)先に領域RC,RDの信号立ち上がり、次いで領域RA,RBの信号立ち上がり:手HDは下から上へ向かう方向に進入した。
(5)先に領域RA,RCの信号立ち下がり、次いで領域RB,RDの信号立ち下がり:手HDは右から左へ向かう方向に離脱した。
(6)先に領域RB,RDの信号立ち下がり、次いで領域RA,RCの信号立ち下がり:手HDは左から右へ向かう方向に離脱した。
(7)先に領域RA,RBの信号立ち下がり、次いで領域RC,RDの信号立ち下がり:手HDは上から下へ向かう方向に離脱した。
(8)先に領域RC,RDの信号立ち下がり、次いで領域RA,RBの信号立ち下がり:手HDは下から上へ向かう方向に離脱した。
(9)先に領域RA,RCの信号立ち下がり、次いで領域RB,RDの信号立ち下がり:手HDは右から左へ向かう方向に離脱した。
(10)先に領域RB,RDの信号立ち下がり、次いで領域RA,RCの信号立ち下がり:手HDは左から右へ向かう方向に離脱した。
(11)先に領域RA,RBの信号立ち下がり、次いで領域RC,RDの信号立ち下がり:手HDは上から下へ向かう方向に離脱した。
(12)先に領域RC,RDの信号立ち下がり、次いで領域RA,RBの信号立ち下がり:手HDは下から上へ向かう方向に離脱した。
図16に、第2実施形態のタイムチャートを示す。本実施形態では、図16に示すように、プロセッサ121は、第1モード設定前(つまり、第2モード設定中)においては、比較的長い200ミリ秒間隔で近接センサ105の信号を検出する。構成は第1実施形態と同様の構成である。第2モード設定中の信号検出の間隔を第1実施形態よりも長くすることにより、低消費電力化を図ることができる。特に、近接センサ105がアクティブタイプである場合、赤外線等の投射回数を減少できるので消費電力低減の効果が高い。ここで、第2モードにおける近接センサ105からの出力信号の間隔を、第1モードにおけるそれよりも長くしておくことで、低消費電力化を図りつつジェスチャー操作による通常状態への復帰を実現することができる。時刻T1で、近接センサ105からの信号により、検出領域への手HDの進入を検出すると、プロセッサ121は,第1タイマを動作させるが、その後も200ミリ秒間隔で近接センサ105の信号を検出する。手HDの保持の検出は、200ミリ秒間隔で行っても特に問題はない。このように、ジェスチャー操作により、第1モードへの切り替えも行うことができるので、定期的に節電モードから復帰してジェスチャー認識のためのプログラムを起動させるといった複雑な制御が不要になり、意図せず電力消費が大きくなるのを防ぐことができる。
以上の各実施の形態では、手の進入方向は問わず、検出領域に決まった時間留まった手が下方向に離脱した場合に、第1モードの設定を行うこととしたが、例えば図17に示すように、手HDを第1の方向としての下方向から進入させ、検出領域に所定の時間留まった後、第2の方向としての下方向に離脱させることで、第1モードを設定するようにしても良い。検出領域に所定の時間留まった後の離脱方向が、進入方向に対して垂直になる場合にこのモードを設定するようにしてもよい。このように、モード設定を行うための手の進入方向と離脱方向とを予め決めておくことで、誤検出の恐れがより低減される。手の進入方向と離脱方向は任意に決めることが出来る。
101 フレーム
101a 前方部
101b 側部
101c 側部
101d 長孔
101e 長孔
102 眼鏡レンズ
103 主本体部
104 ディスプレイ
104A 画像形成部
104B 画像表示部
104DR 表示制御部
104a 光源
104b 一方向拡散板
104c 集光レンズ
104d 表示素子
104f 接眼プリズム
104g 偏向プリズム
104h ホログラム光学素子
104i 画面
105 近接センサ
105a 受光部
106 カメラ
106a レンズ
107 右副本体部
107a 突起
108 左副本体部
108a 突起
109 加速度センサ
110 ジャイロ
111A スピーカ(又はイヤホン)
111B マイク
121 プロセッサ
122 操作部
123 GPS受信部
124 通信部
125 ROM
126 RAM
127 電池
129 ストレージデバイス
200 スマートフォン
CD コード
CTU 制御ユニット
HD 手
HS 配線
IC1~IC5 アイコン
PL1 基端面
PL2 内側面
PL3 外側面
PL4 傾斜面
PL4,PL5 傾斜面
RA-RD 受光領域
SA 検出領域
EV 有効視野
US ユーザー
Claims (15)
- 近接範囲にある検出領域において、物体を検出して出力を生成する近接センサと、
制御信号によって制御される少なくとも一つのデバイスと、
前記少なくとも一つのデバイスを制御するための制御信号を生成する制御回路と、を備え、
前記制御回路は、前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を実行可能であり、
前記制御回路は、前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作である、電子機器。 - 前記制御回路は、前記物体の前記第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードに切り替える、請求項1に記載の電子機器。
- 前記近接センサは、前記物体が前記検出領域内に進入する方向及び前記検出領域から離脱する方向を検出し、前記制御回路は、前記近接センサの出力に基づいて、前記物体の少なくとも一部が、第1の方向から前記検出領域に進入し、前記第1の時間以上留まった後、それから前記第2の時間以内に前記検出領域から第2の方向に離脱したと判断したときは、前記第2モードから前記第1モードへ切り替える、請求項1に記載の電子機器。
- 前記制御回路は、前記第2モードから前記第1モードに切り替えた後、最後に前記第1の動きを検出したときから一定時間経過すると、前記第1モードから前記第2モードへ切り替える、請求項2又は3に記載の電子機器。
- 前記近接センサは、前記物体が前記検出領域内に進入する方向及び前記検出領域から離脱する方向を検出し、前記制御回路は、前記第1モードにおいて、前記物体の少なくとも一部が前記検出領域に進入する前記第1の方向と、前記検出領域から離脱する前記第2の方向との組み合わせに基づいて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する、請求項3又は4に記載の電子機器。
- 前記近接センサは、前記検出領域内における前記物体と前記近接センサとの距離に応じた出力を生成し、前記制御回路は、前記近接センサの出力に基づいて、前記検出領域内において前記物体が前記近接センサに徐々に近づいていると判断したときは、前記第1モードから前記第2モードへの切り替え、又は、前記第2モードから前記第1モードへの切り替えをスキップする、請求項1~5のいずれかに記載の電子機器。
- 前記デバイスとして、画像を表示できる画面を備えた表示部材を有するディスプレイユニットを備え、
前記制御回路は、前記第1モードにおいて、前記第1の制御として、前記ディスプレイユニットの画面表示に関して規定された制御を実行し、前記第2モードにおいて、前記ディスプレイユニットの画面表示に関して規定された制御をスキップする、請求項1~6のいずれかに記載の電子機器。 - 前記制御回路は、前記近接センサの出力に基づいて、前記物体の少なくとも一部が前記検出領域内に前記第1の時間以上留まったと判断したときは、前記ディスプレイユニットの画面に第1の表示を行わせ、その後、前記第2の時間以内に前記検出領域から離脱したと判断したときは、前記ディスプレイユニットの画面に前記第1の表示とは異なる第2の表示を行わせる、請求項7に記載の電子機器。
- ユーザーの頭部に装着され、前記ユーザーの少なくとも一方の眼前に、前記表示部材が位置するように前記ディスプレイユニットを保持する装着部材を有する、請求項7又は8のいずれかに記載の電子機器。
- 前記ディスプレイユニットの表示部材はシースルー型である、請求項9に記載の電子機器。
- 前記近接センサは、物体から放射又は反射される不可視光を検出する受光部を有する、請求項1~10のいずれかに記載の電子機器。
- 前記制御回路は、ユーザーの指示を受け付けて、前記指示に基づいて前記第1の時間と前記第2の時間の少なくとも一方を調整可能である、請求項1~11のいずれかに記載の電子機器。
- ユーザーの手、指、またはユーザーの意思で動かされる指示具の動きを検出して、前記少なくとも一つのデバイス又は前記制御回路についての制御を行う、請求項1~12のいずれかに記載の電子機器。
- 近接範囲にある検出領域において、物体を検出して出力を生成する近接センサと、制御信号に基づいて制御される少なくとも一つのデバイスとを備える電子機器の制御方法であって、
前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を切り替えて設定可能であり、
前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作である、電子機器の制御方法。 - 近接範囲にある検出領域において、物体を検出して出力を生成する近接センサと、制御信号に基づいて制御される少なくとも一つのデバイスと、前記近接センサの出力に基づいて、前記デバイスを制御する制御信号を生成する制御回路と、を備える電子機器の制御プログラムであって、
前記制御プログラムによって制御された前記制御回路は、
前記近接センサに対する相対的な前記物体の第1の動きに対応する前記近接センサの出力に応じて、前記少なくとも一つのデバイス又は前記制御回路自身についての第1の制御を実行する第1モードと、前記第1の動きに対応する前記近接センサの出力に対して、前記第1の制御をスキップする第2モードと、を実行可能であり、
前記第1の動きとは異なる、前記近接センサに対する相対的な前記物体の第2の動きに対応する前記近接センサの出力に基づいて、前記第2モードから前記第1モードへ、又は、前記第1モードから前記第2モードへ切り替え、
前記第2の動きは、前記物体が第1の時間前記検出領域に留まった後、そこから第2の時間以内に前記検出領域から離脱する動作である、電子機器の制御プログラム。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/535,987 US10310624B2 (en) | 2014-12-17 | 2015-11-19 | Electronic apparatus, method for controlling electronic apparatus, and control program for the same |
JP2016564745A JP6617974B2 (ja) | 2014-12-17 | 2015-11-19 | 電子機器、電子機器の制御方法及びその制御プログラム |
EP15869727.6A EP3236335A4 (en) | 2014-12-17 | 2015-11-19 | Electronic instrument, method of controlling electronic instrument, and control program for same |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014255140 | 2014-12-17 | ||
JP2014-255140 | 2014-12-17 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016098519A1 true WO2016098519A1 (ja) | 2016-06-23 |
Family
ID=56126417
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/082495 WO2016098519A1 (ja) | 2014-12-17 | 2015-11-19 | 電子機器、電子機器の制御方法及びその制御プログラム |
Country Status (4)
Country | Link |
---|---|
US (1) | US10310624B2 (ja) |
EP (1) | EP3236335A4 (ja) |
JP (1) | JP6617974B2 (ja) |
WO (1) | WO2016098519A1 (ja) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018018288A (ja) * | 2016-07-27 | 2018-02-01 | 京セラ株式会社 | 電子機器 |
JP2018181351A (ja) * | 2018-04-24 | 2018-11-15 | 京セラ株式会社 | 電子機器 |
JP2019528478A (ja) * | 2016-08-23 | 2019-10-10 | ロベルト・ボッシュ・ゲゼルシャフト・ミト・ベシュレンクテル・ハフツングRobert Bosch Gmbh | 非接触制御式のプロジェクタ |
US10606361B2 (en) | 2016-07-27 | 2020-03-31 | Kyocera Corporation | Electronic device, control method, and non-transitory computer-readable recording medium |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9619036B2 (en) * | 2012-05-11 | 2017-04-11 | Comcast Cable Communications, Llc | System and methods for controlling a user experience |
JP6582419B2 (ja) * | 2015-01-27 | 2019-10-02 | セイコーエプソン株式会社 | 頭部装着型表示装置、頭部装着型表示装置の制御方法、および、コンピュータープログラム |
AU2020407551A1 (en) * | 2019-12-17 | 2022-08-11 | Johnson & Johnson Surgical Vision, Inc. | System and methods for a cassette capture mechanism for phacoemulsification surgical applications |
US12032803B2 (en) * | 2020-09-23 | 2024-07-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with three-dimensional environments |
US11562528B2 (en) | 2020-09-25 | 2023-01-24 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with three-dimensional environments |
US11954242B2 (en) | 2021-01-04 | 2024-04-09 | Apple Inc. | Devices, methods, and graphical user interfaces for interacting with three-dimensional environments |
US12026297B2 (en) | 2021-01-07 | 2024-07-02 | Samsung Electronics Co., Ltd. | Wearable electronic device and input structure using motion sensor in the same |
CN113064805A (zh) * | 2021-03-29 | 2021-07-02 | 联想(北京)有限公司 | 一种电子设备的控制方法以及控制装置 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011221672A (ja) * | 2010-04-06 | 2011-11-04 | Sony Corp | 画像処理装置、画像処理方法、プログラム、および電子装置 |
JP2012163611A (ja) * | 2011-02-03 | 2012-08-30 | Nec Casio Mobile Communications Ltd | 処理装置、処理方法およびプログラム |
JP2014062426A (ja) * | 2012-09-24 | 2014-04-10 | Panasonic Corp | 衛生洗浄装置 |
JP2014075759A (ja) * | 2012-10-05 | 2014-04-24 | Sharp Corp | 情報処理装置、およびプログラム |
JP2014531662A (ja) * | 2011-09-19 | 2014-11-27 | アイサイト モバイル テクノロジーズ リミテッド | 拡張現実システムのためのタッチフリーインターフェース |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8933876B2 (en) * | 2010-12-13 | 2015-01-13 | Apple Inc. | Three dimensional user interface session control |
US20110310005A1 (en) | 2010-06-17 | 2011-12-22 | Qualcomm Incorporated | Methods and apparatus for contactless gesture recognition |
US9448635B2 (en) * | 2012-04-16 | 2016-09-20 | Qualcomm Incorporated | Rapid gesture re-engagement |
JP2014033144A (ja) * | 2012-08-06 | 2014-02-20 | Fujitsu Ltd | 基板モジュール、電子装置、及び、電子装置の製造方法 |
US20140282280A1 (en) | 2013-03-14 | 2014-09-18 | Samsung Electronics Co., Ltd. | Gesture detection based on time difference of movements |
-
2015
- 2015-11-19 EP EP15869727.6A patent/EP3236335A4/en not_active Withdrawn
- 2015-11-19 JP JP2016564745A patent/JP6617974B2/ja not_active Expired - Fee Related
- 2015-11-19 WO PCT/JP2015/082495 patent/WO2016098519A1/ja active Application Filing
- 2015-11-19 US US15/535,987 patent/US10310624B2/en not_active Expired - Fee Related
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011221672A (ja) * | 2010-04-06 | 2011-11-04 | Sony Corp | 画像処理装置、画像処理方法、プログラム、および電子装置 |
JP2012163611A (ja) * | 2011-02-03 | 2012-08-30 | Nec Casio Mobile Communications Ltd | 処理装置、処理方法およびプログラム |
JP2014531662A (ja) * | 2011-09-19 | 2014-11-27 | アイサイト モバイル テクノロジーズ リミテッド | 拡張現実システムのためのタッチフリーインターフェース |
JP2014062426A (ja) * | 2012-09-24 | 2014-04-10 | Panasonic Corp | 衛生洗浄装置 |
JP2014075759A (ja) * | 2012-10-05 | 2014-04-24 | Sharp Corp | 情報処理装置、およびプログラム |
Non-Patent Citations (1)
Title |
---|
See also references of EP3236335A4 * |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2018018288A (ja) * | 2016-07-27 | 2018-02-01 | 京セラ株式会社 | 電子機器 |
US10606361B2 (en) | 2016-07-27 | 2020-03-31 | Kyocera Corporation | Electronic device, control method, and non-transitory computer-readable recording medium |
US10613638B2 (en) | 2016-07-27 | 2020-04-07 | Kyocera Corporation | Electronic device |
JP2019528478A (ja) * | 2016-08-23 | 2019-10-10 | ロベルト・ボッシュ・ゲゼルシャフト・ミト・ベシュレンクテル・ハフツングRobert Bosch Gmbh | 非接触制御式のプロジェクタ |
US10795455B2 (en) | 2016-08-23 | 2020-10-06 | Robert Bosch Gmbh | Projector having a contact-free control |
JP2018181351A (ja) * | 2018-04-24 | 2018-11-15 | 京セラ株式会社 | 電子機器 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2016098519A1 (ja) | 2017-09-28 |
US20180292906A1 (en) | 2018-10-11 |
JP6617974B2 (ja) | 2019-12-11 |
US10310624B2 (en) | 2019-06-04 |
EP3236335A4 (en) | 2018-07-25 |
EP3236335A1 (en) | 2017-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6617974B2 (ja) | 電子機器、電子機器の制御方法及びその制御プログラム | |
JP5957875B2 (ja) | ヘッドマウントディスプレイ | |
TWI498771B (zh) | 可辨識手勢動作的眼鏡 | |
WO2013088725A1 (en) | Head-mounted display and information display apparatus | |
EP2579145A2 (en) | Accessory to improve user experience with an electronic display | |
WO2015096335A1 (zh) | 交互识别***以及显示装置 | |
KR20130053377A (ko) | 텍스트 입력장치와 포인터 위치정보 입력장치가 구비된 복합 휴먼 인터페이스 장치 | |
JP6116336B2 (ja) | カメラ装置及びそれを備えた無線通信端末 | |
KR101268209B1 (ko) | 포인터 위치정보 입력부와 포인터 실행 명령부를 갖는 휴먼 인터페이스 장치 | |
WO2016052061A1 (ja) | ヘッドマウントディスプレイ | |
JP6122355B2 (ja) | 携帯端末装置 | |
CN106155604B (zh) | 显示处理方法和电子设备 | |
JP2006155244A (ja) | 情報表示装置 | |
WO2016072271A1 (ja) | 表示装置、表示装置の制御方法及びその制御プログラム | |
EP3276448B1 (en) | Wearable electronic device, gesture detection method for wearable electronic device, and gesture detection program for wearable electronic device | |
CN106155605B (zh) | 显示处理方法和电子设备 | |
JP2014154074A (ja) | 入力装置、頭部装着型表示装置および入力装置の制御方法 | |
WO2017094557A1 (ja) | 電子機器及びヘッドマウントディスプレイ | |
WO2017065050A1 (ja) | 入力装置、電子機器、電子機器の入力方法及びその入力プログラム | |
WO2017065051A1 (ja) | 入力装置、電子機器、電子機器の入力方法及びその入力プログラム | |
KR101350140B1 (ko) | 부화면을 갖는 휴대단말기, 부화면 장치 및 복수화면 제어방법 | |
JP6176368B2 (ja) | ヘッドマウントディスプレイ及び情報表示装置 | |
TWI492099B (zh) | 可辨識手勢動作的眼鏡 | |
WO2017104525A1 (ja) | 入力装置、電子機器及びヘッドマウントディスプレイ | |
WO2018092674A1 (ja) | 表示装置及びジェスチャー入力方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15869727 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016564745 Country of ref document: JP Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2015869727 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15535987 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |