EP1056611A1 - Procede pour commander ou manoeuvrer des systemes par informations d'image et procede d'acquisition d'informations d'image - Google Patents
Procede pour commander ou manoeuvrer des systemes par informations d'image et procede d'acquisition d'informations d'imageInfo
- Publication number
- EP1056611A1 EP1056611A1 EP99913073A EP99913073A EP1056611A1 EP 1056611 A1 EP1056611 A1 EP 1056611A1 EP 99913073 A EP99913073 A EP 99913073A EP 99913073 A EP99913073 A EP 99913073A EP 1056611 A1 EP1056611 A1 EP 1056611A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- image information
- eye
- eyes
- glasses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 239000011521 glass Substances 0.000 claims abstract description 49
- 210000001525 retina Anatomy 0.000 claims abstract description 26
- 230000005693 optoelectronics Effects 0.000 claims abstract description 24
- 230000001960 triggered effect Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000008569 process Effects 0.000 claims abstract description 7
- 230000006870 function Effects 0.000 claims description 31
- 230000002207 retinal effect Effects 0.000 claims description 14
- 230000011514 reflex Effects 0.000 claims description 13
- 210000000873 fovea centralis Anatomy 0.000 claims description 8
- 238000001454 recorded image Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 8
- 230000002596 correlated effect Effects 0.000 claims description 7
- 230000001276 controlling effect Effects 0.000 claims description 6
- 230000003213 activating effect Effects 0.000 claims description 4
- 230000002123 temporal effect Effects 0.000 claims description 2
- 230000004256 retinal image Effects 0.000 abstract description 2
- 230000005540 biological transmission Effects 0.000 description 7
- 230000004438 eyesight Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000004308 accommodation Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 210000001747 pupil Anatomy 0.000 description 3
- 230000004397 blinking Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000004304 visual acuity Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000004566 IR spectroscopy Methods 0.000 description 1
- 208000029091 Refraction disease Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000004430 ametropia Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000002559 palpation Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 208000014733 refractive error Diseases 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
- B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
- B60R21/015—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting the presence or position of passengers, passenger seats or child seats, and the related safety parameters therefor, e.g. speed or timing of airbag inflation in relation to occupant position or seat belt use
- B60R21/01512—Passenger detection systems
- B60R21/01552—Passenger detection systems detecting position of specific human body parts, e.g. face, eyes or hands
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B62—LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
- B62D—MOTOR VEHICLES; TRAILERS
- B62D6/00—Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
- B60K2360/20—Optical features of instruments
- B60K2360/21—Optical features of instruments using cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K31/00—Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator
- B60K31/0058—Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator responsive to externally generated signalling
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/011—Head-up displays characterised by optical features comprising device for correcting geometrical aberrations, distortion
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B2027/0192—Supplementary details
- G02B2027/0198—System for aligning or maintaining alignment of an image in a predetermined direction
Definitions
- the present invention relates to a method for controlling or operating systems using image information and a method for acquiring image information.
- control element When operating a control element must be searched optically or haptically, and the control element must be detected by optical fixation with the eyes and / or palpation of the shape.
- Control element on the other hand.
- a functional check can be carried out by feedback via functional lighting.
- image information is obtained by self-sufficient, person-independent image recording devices such as cameras or scanners. These devices require alignment with the help of special facilities such as camera viewfinder or insertion template.
- the viewing angle of the original image (object) or the recording format is determined by the lens used and the original image.
- a geometrical change of the picture requires a change of the lenses or the device with too large formats.
- Many of these devices are portable due to their power supply and weight.
- a separate device version is required for each device function.
- partially transparent glasses are described, with the aid of which the retinal reflex image of the eye is recorded electronically with different brightness of the surroundings by means of a rear-view mirror.
- This device which is described in various variants, enables the image seen directly by the eye to be scanned serially from the retina of the eye using a two-axis scanner and further processed by a computer.
- an illumination device preferably with lasers
- the previously recorded and processed image can be superimposed physiologically without delay on the original image over the same light path in the opposite direction and after repeated reflection on the inside of the glasses.
- the visual impression is to be improved by modifying the image and projecting it back into the eye.
- the object of the present invention is to provide a method for controlling or operating systems which simplifies the operation of the systems and relieves the respective operator. In particular when driving vehicles, the driver's workload is supposed to increase traffic safety.
- the method according to the invention for controlling or operating systems comprises the steps:
- the image patterns of the fovea centralis of both eyes are advantageously detected and compared with a stored pattern, the function being triggered by activating an operating element if the pattern matches.
- the system to be operated or a control element can be fixed with the eye or eyes to operate it. The triggering can take place, for example, by an acoustic command or by fixing the object for a predetermined time.
- Functions of a vehicle are preferably controlled by the control signals, e.g. Brakes, steering, lights, or they are used to trigger warning signals.
- the image information can be correlated with current driving data and / or vehicle data. Relative assignments of geometric image information of both eyes and / or temporal changes in the image information of both eyes are preferably evaluated.
- an infrared lighting system can be activated to improve vision, while the retinal reflex image is changed and projected back into the eye synchronously.
- the image information obtained can e.g. can be used to adapt the lighting or the driving lights to the respective line of sight.
- Image information obtained can also be temporarily stored in a memory or vehicle data memory for the reconstruction of subjective scenes or traffic situations.
- a method for acquiring image information is created, with the steps:
- Detecting an object or a scene with the human eye through optoelectronic glasses and recording the retinal reflex image by means of a scanning process Feeding the captured retinal reflex image to an image processing and storage system; and triggering a recording or copying function in order to record, copy and / or transmit the object seen by the eye.
- an auxiliary pattern is preferably projected into the eye in order to provide a cutout frame or to correct the recorded image.
- the auxiliary pattern can also be invisible to the eye and with the help of stored data on the curvature of the retina, the image can be rectified.
- the desired function is triggered by optically detecting the operating function of a device with the eyes at a suitable point and activating a single operating element.
- the pattern reflected by the retina in the area of the fovea centralis of both eyes is evaluated, in particular by comparing the pattern with stored patterns while checking for congruence. If the two patterns match the stored pattern, the process can be switched for many functions by activating a single control element.
- Such a fixation is also necessary in known methods if a system, e.g. using a mouse.
- the fixation happens on the mouse pointer.
- the desired icon for operation must be searched for and found on a screen, which also requires fixation. Only when the mouse pointer and icon can be fixed at the same time, operation via an additional control element is possible. For many people, this leads to the above
- fixation with the eyes is used to activate, for example, known objects.
- This can take place, for example, in which an intelligent operating system provided with an operating element is located at a preferred location.
- the operator looks at and fixes the system, For example, a lamp whose pattern is saved, he can turn it on, for example, with one control element when it is not lit, or turn it off when it is lit.
- a television can be switched on or off in the same way, for example.
- individual functions or operating elements can now also be represented graphically on a monitor, where they are then also fixed by the eye and can be triggered or switched using the single operating element.
- This control element can e.g. by an acoustic call (e.g. "switch on”, “switch off”, “weaken”, “amplify”) and a corresponding voice recognition in the device that is being used.
- the monitor which can be made small, is then the only control element of the device or system and can simultaneously offer various functions to choose from.
- the image of the operating element or monitor viewed by the operator's eyes and depicted on the retina is thus recorded by the retina, electronically stored and compared by the computer with known patterns. This is done with the opto-electronic glasses that are described in the aforementioned DE 196 31 414 A1 and DE 197 28 890 A1.
- the opto-electronic glasses are used to determine the exact fixation of the eyes either purely passively via the recording of the image in the area of the sharpest vision or with the aid of a marking reflected in the projection unit of the opto-electronic glasses to be the center of attention.
- the control element can be identified, a comparison of the image with stored patterns of the control elements in the computer is possible.
- the patterns of the existing control elements are transmitted by the device or system itself to the operator via IR or radio.
- the known glasses are designed so that the image of the surroundings transmitted through the glasses on the retina of the eye is recorded serially by means of the rear-view mirror on the inside of the glasses with the aid of a scanner with a fixed sampling frequency and a sensitive photo detector. The serial image signal is then sent to the image processing computer.
- part of the image acquisition cycle can be used to locate and track the pupil opening and to center the axis of symmetry of the image scan on it. This allows the image acquisition to follow the fast and slow movements of the eye particularly well. The alignment of the two eyes against each other can thus be measured and the images of both eyes can be correlated.
- the image information obtained from the retinal reflex image of the eye by means of the opto-electronic glasses is evaluated with a computer and used to obtain control signals.
- stationary image templates such as books, newspaper and documentation texts and printed images can also be recorded and used with the aid of the optoelectronic glasses.
- the system to be operated is a recording, storage, copying and / or transmission system.
- the image information is taken from the same point of view as the viewer sees it, directly from his eye (retina) and processed in the computer, stored, copied or distributed via electronic networks (fax, e-mail, Internet).
- an improved or supplemented image of the original or of auxiliary patterns and stencils can be superimposed on the original image with the aid of a laser projection device.
- the optoelectronic glasses are designed such that the image of the surroundings transmitted through the glasses on the retina of the eye is reflected by the rear-view mirror on the inside of the glasses or their glasses is recorded serially using a scanner with a fixed sampling frequency and a sensitive photo detector.
- the serial image signal is then passed to the image processing computer.
- part of the image acquisition cycle is used to locate and track the pupil opening and to center the axis of symmetry of the image acquisition on it. This happens in both eyes at the same time. In this way, the alignment of the two eyes against each other can be measured and the images of both eyes can be correlated.
- Eye and head movements which every person carries out when viewing and reading, due to the small instantaneous focus range of the eye (approx. 1 °) and the limited image recording speed, are not necessary when taking pictures with the opto-electronic glasses.
- the shooting conditions such as the distance to the original and the shooting angle, will vary depending on the application.
- the shortest possible recording time corresponds to the frame rate of the scanner.
- the captured image can be processed in the computer using a typical video standard. Since the images are geometrically distorted due to the curvature of the retina and the different viewing angles, it will be necessary to convert the image to a specific format after the image has been taken. Since the equalization of the images without optical aids means a relatively high computing effort, the invention advantageously provides that a geometric auxiliary pattern (e.g. a regular, dense network) is preferably projected onto the retina in the invisible infrared radiation range with the laser projector during or between two image recordings . A simplified equalization in the image computer is then made possible by the mathematical comparison of this internal auxiliary image with the external image.
- a geometric auxiliary pattern e.g. a regular, dense network
- FIG. 1 shows schematically the known opto-electronic glasses and their interaction with the human visual apparatus
- FIG. 2 shows the processing of the signals in a block diagram corresponding to the above-mentioned publication
- FIG. 3 schematically shows the view from a vehicle onto a tight curve
- FIG. 4 schematically, the view from a vehicle onto a wide curve
- Figure 5 schematically shows the view of a straight course of the road when driving normally
- FIG. 6 schematically shows a straight course of the road with the vehicle traveling far to the right
- Figure 7 schematically shows the view of two vehicles in front with the left eye
- Figure 8 schematically shows the view of two vehicles in front with the right eye.
- Figure 1 shows a schematic image of the known glasses when they are put on.
- the beam path for the recording of the retinal reflex via reflection of the beam axes on the inside of the glasses, as well as the visual field of the eye in transmission through the glasses is indicated.
- the individual optical, electronic and mechanical components such as mirrors, photo receivers, preamplifiers and scanners can today be used in a similar way to e.g. modern hearing aids are very miniaturized so that putting on the glasses, similar to sunglasses, means no disability for the operator or driver.
- the data transmission from the glasses to the computer or to a user-specific microprocessor or other electronic devices, e.g. of the vehicle, can be carried out via fiber optic cable or IR data transmission. Details can be found in the publication mentioned at the beginning.
- Figure 2 shows the beam path of the opto-electronic glasses, for example for the colors red, green and blue with the individual components and groups in the receiving and projection channel. 10
- the glasses are placed on an observer.
- the BG glasses serve as beam splitters.
- transmission for the light from the outside world, and in reflection, as an imaging surface of the light backscattered by the retina NH through the eye AA, which is generated with the aid of further imaging elements and a biaxial scanner for horizontal (HSS) and vertical deflection (VSS) is fed to a photo receiver in a receiving unit.
- the beam path is designed in such a way that the extension of the line of sight from the detector through the glasses always ends in the absorbing layer in a radiation sink SS.
- the extension of the line of sight of the eye through the glasses runs to the outside world AW.
- the simplest method of beam splitting on BG glasses is the use of 50% transmitting and 50% reflecting mirror glasses. Active, electronically controllable mirrors that switch from complete transmission to complete reflection in the two scanning cycles can also be used.
- the glasses can also be designed as a transparent lens which is attached to the head of the operator and placed in front of the eye.
- the eye AA forms parallel or almost parallel bundles of rays on the retina NH from the outside world AW.
- the fulcrum of the rays at different viewing angles to the outside world is in the pupil of the eye AP.
- the invention is based on a largely identical beam path for the left and right eyes.
- the invention provides that either the lenses of their lenses are individually adjusted by correspondingly different designs of the curvature of the outside and the inside, or that contact lenses are worn.
- the curvature of the inside and outside of the BG glasses is identical.
- the light scattered back from the eye by each individual pixel of the retina is in the same way an almost parallel bundle of rays which runs the same way as the incident light in the opposite direction to the inside of the partially reflective spectacle lens BG.
- the curvature of this surface is designed so that 1 1
- auxiliary mirror HS collimates the beams again and images them so that they run over the common pivot point on the axis of a horizontal scanner mirror HSS.
- a vertical deflection is carried out by a second scanner mirror VSS.
- the beam path between the projection and the receiving channel is separated with the help of a switching mirror SUS.
- a focusing device FE is arranged in the beam path of the projection unit behind the beam switch SUS and adjusts the size of the laser image spot and the scanned spot on reception via a visual field diaphragm GFB on the retina NH.
- the retinal reflex with the use of dichroic filters (DFR, DFG and DFB) and three separate detectors (PMR, PMG and PMB) in e.g. three color channels is divided so that a largely unadulterated color image can be recorded.
- DFR dichroic filters
- PMR photosensitive detector
- PMG three separate detectors
- PMB the beams of up to three lasers in the red, green and blue spectral range (LR, LG, LB) are combined with dichroic beam splitters after the separate image modulation of each color (MR, MG, MB) on a common axis.
- the optical signal with dichroic filters (DFR, DFG and DKB) in the receiving channel is broken down into the three color components by the three photo receivers, preferably photomultipliers (PMR, PMG and PMB), into the three basic colors and measured separately. Because of the weak light signals, mainly photon counting methods will be used.
- a marking e.g. a crosshair or a circle
- a single color will be necessary, with a corresponding reduction of the three color channels of the laser to only one color channel.
- a simple button in various embodiments can be used as a triggering or operating element.
- a voice command can also be used, provided that voice recognition is available on the device to be operated.
- the system can also be triggered by fixing it over a certain period of time.
- the transmission of the other data from the image processing computer of the optoelectronic glasses to the device or system can be carried out acoustically, via radio or via IR connections.
- the eye When capturing image information according to the invention using optoelectronic glasses, the eye is fixed on the original during the recording. Since the picture templates are stationary, a lower picture frame frequency can be used under certain circumstances than with the usual video and television standards. Multiple images can also be integrated. Both ways can be used for signal amplification in dim lighting.
- a rough check of the recorded image is possible from time to time in the glasses, similar to modern video cameras.
- the easiest way to achieve this is to place a dark shutter on the outside of the glasses, which blocks the view to the outside during this time.
- the stored image can then be viewed via the reflection on the inside of the glasses against the black background of the closure.
- Equalization is performed using stored data on the curvature of the retina.
- Data for the rectification can also be obtained by simultaneously mirroring an auxiliary pattern, which is preferably invisible to the retina, into the eye.
- the recorded image can be processed stereoscopically in two channels. Furthermore, information lying at least partially in the infrared range can be processed further.
- the recorded image can be used according to the invention for the production of photographic still images, motion picture recordings, free-format scanner recordings, for telescopic reproduction and / or recording, or for the production of copies.
- a variable detail frame is projected into the eye to position the image recording.
- FIGS. 3 and 4 show, using a sketch of a road course, how differently narrow and wide curves appear in perspective to the driver's eye. they show 14
- FIGS. 5 and 6 show a sketch of a straight course of the road in the first case that the vehicle is driving normally in the middle of the right lane and in the second case that it is driving too far to the right in the right lane. Both can be recognized by changing the angle and position of the marking lines of the road relative to the windshield frame. In this case, the position of the vehicle in the lane is derived by evaluating the image of the course of the road in relation to the image of the windshield frame.
- FIGS. 7 and 8 show the perspective change in the position of the images of two cars in front in the driver's right and left eyes. Both eyes fix the bus in the background (circle). The car in the foreground is to the right or left of the center. The computer can derive the relative position and distance to the two vehicles from this image jump.
- the information obtained can be used for a variety of control signals.
- the determined image information of the eyes can e.g. with current driving data, geometry data of the vehicle, or also with stored image data, such as Traffic signs are correlated.
- driving systems such as brakes activated, the steering hardened, the driving lights or fog lights switched on and off or adapted to the viewing direction.
- Any existing IR lighting system can be activated to improve the fog light values.
- the visual acuity can be improved locally in the event of poor accommodation for reading vehicle instruments.
- the image information obtained from both eyes can be used in connection with an activation control element for operating vehicle functions.
- the image information obtained can also be temporarily stored in a vehicle data memory for the reconstruction of subjective and objective traffic situations.
- the airbag can be deactivated, for example, when an illegal eye position is detected, which suggests an unsuitable sitting position of the driver.
- the blinking duty cycle is expediently used to determine irregularities such as sleep or stress.
- a method for operating systems by optically capturing the associated control element or object, the analysis of the image pattern of the fovea centralis of both eyes using opto-electronic glasses and an associated image processing and storage system triggering the assigned function by means of a single suitable control element .
- the single control element can trigger several different ones
- the operating system can be spatially separated from the systems to be operated. Fixing the too 16
- operating system can be recognized with the eyes and used to operate them.
- the fixing of a discrete control element, symbol or a discrete shape can also be recognized with the eyes and used to operate it.
- the opto-electronic glasses can additionally project a pattern onto the retina and, if the pattern matches the image of the control element, the operation can be triggered.
- the operating system consisting of the opto-electronic glasses and the single control element, can be movable. Data is preferably transmitted by radio, IR or ultrasound to the actual operating system, e.g. a PC.
- a desired icon can be fixed and triggered using the single control element.
- the recognized control element, icon, etc. can also be fixed over a predetermined time and triggered after the time has elapsed. It can also be triggered by any acoustic command.
- the systems can be operated via a decentralized arrangement of the operating function in the system, the operation being triggered by a coded signal.
- Group 2 Functions of mobile systems are controlled here.
- image information obtained from the retinal reflex image of the eye is evaluated by means of the optoelectronic glasses using a computer and used to obtain control signals.
- Geometric image information and / or brightness information can be evaluated, and relative assignments of geometric image information of both eyes can be evaluated. Changes in the image information of both eyes over time can also be evaluated.
- the determined image information of the eyes can be correlated with current driving data, or they can be correlated with geometry data of a vehicle.
- the determined image information of the eyes can also be stored with stored image data, such as Traffic signs are compared.
- the image information obtained can be used, for example, as follows: a) to trigger optical, acoustic or haptic warning signals, 17
- the evaluation of the retinal image can thus be used to activate vehicle functions, and at the same time, image processing can take place as required and a synchronous rear projection of the improved image into the eye can be carried out.
- the fog light and, if necessary, the rear fog light can be switched by the control.
- Visual acuity can also be improved locally in the event of poor accommodation for reading vehicle instruments.
- an existing IR lighting system can be activated to improve the fog light values.
- the image information obtained from both eyes is preferably used in conjunction with an activation control element for operating vehicle functions.
- the image information obtained can be temporarily stored in a vehicle data memory for the reconstruction of subjective and objective traffic situations. They can also be used to deactivate the airbag when an illegal eye position is detected.
- the blinking duty cycle can be used to determine irregularities, such as sleep or stress, and it can be determined whether the same geometrical image information is located in the fovea centralis in both eyes in order to recognize irregularities.
- image information which is preferably stationary, is acquired by clicking on it with the human eye, the reflection image of the retina of the eyes being recorded in a scanning process by means of the opto-electronic glasses and being fed to an image processing and storage system.
- the recording can also be carried out by simultaneously mirroring an auxiliary pattern that is invisible to the retina and comparing it with the eye.
- the recorded image is preferably checked by rear projection into the eye.
- the captured image is e.g. subject to color adjustment (white balance) and / or optionally processed in monochrome or color. It can also be processed stereoscopically in two channels, and information in the infrared range can be processed at least partially.
- the recorded image can be used, for example, for the production of photographic still images, motion picture recordings, free-format scanner recordings, for telescopic reproduction and / or recording, and for the production of copies.
- a variable cutout frame is preferably projected into the eye and an improved image can be projected onto the retina in synchronism with the recorded image.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Mechanical Engineering (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Combustion & Propulsion (AREA)
- Transportation (AREA)
- Chemical & Material Sciences (AREA)
- Eye Examination Apparatus (AREA)
- Position Input By Displaying (AREA)
Abstract
L'invention concerne un procédé permettant de commander ou de manoeuvrer des systèmes au moyen d'informations d'image, selon lequel un objet ou une scène est perçue par l'oeil humain à travers une lunette optoélectronique et l'image rétinienne réfléchie est captée pour permettre l'obtention d'informations d'image. Les informations d'image sont évaluées et comparées avec des données mémorisées. Ensuite, des signaux de commande servant au déclenchement d'une fonction d'un système sont produits. Le système peut être un système mobile, par exemple un véhicule, et peut comprendre également des sous-systèmes de ce véhicule. D'autres systèmes peuvent également être commandés, le déclenchement de la fonction associée se faisant au moyen d'un seul élément de commande approprié. Les informations d'image sont captées selon un processus de balayage de l'image réfléchie par la rétine et amenées à un système de traitement d'image et de mémorisation. Il est également possible, dans un système, de déclencher un processus de prise de vue ou de copie, pour effectuer une prise de vue, une copie et/ou une transmission de l'objet perçu par l'oeil, en tant qu'image.
Applications Claiming Priority (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE19807271A DE19807271A1 (de) | 1998-02-20 | 1998-02-20 | Verfahren zur Erfassung und Umsetzung von ortsfesten Bildinformationen |
DE19807065 | 1998-02-20 | ||
DE1998107081 DE19807081C2 (de) | 1998-02-20 | 1998-02-20 | Verfahren zum Bedienen von Systemen durch optische Erfassung eines Bedienelements oder Gegenstands |
DE19807271 | 1998-02-20 | ||
DE19807081 | 1998-02-20 | ||
DE19807065A DE19807065C2 (de) | 1998-02-20 | 1998-02-20 | Verfahren zur Steuerung von Funktionen mobiler Systeme |
PCT/DE1999/000421 WO1999042315A1 (fr) | 1998-02-20 | 1999-02-16 | Procede pour commander ou manoeuvrer des systemes par informations d'image et procede d'acquisition d'informations d'image |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1056611A1 true EP1056611A1 (fr) | 2000-12-06 |
Family
ID=27218157
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP99913073A Withdrawn EP1056611A1 (fr) | 1998-02-20 | 1999-02-16 | Procede pour commander ou manoeuvrer des systemes par informations d'image et procede d'acquisition d'informations d'image |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP1056611A1 (fr) |
JP (1) | JP2002503862A (fr) |
WO (1) | WO1999042315A1 (fr) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ATE357680T1 (de) | 2000-10-07 | 2007-04-15 | David Dickerson | Informationssystem und verfahren zur zurverfügungstellen von informationen unter verwendung eines holographischen elements |
WO2002097511A1 (fr) * | 2001-05-22 | 2002-12-05 | Physoptics Opto-Electronic Gmbh | Procede et dispositif permettant d'adapter un systeme optique a la ligne de visee de l'oeil humain |
EP1840627B1 (fr) | 2000-10-07 | 2012-12-12 | metaio GmbH | Dispositif et procédé destinés à la détermination de l'orientation d'un oeil |
EP1405122B1 (fr) | 2000-10-07 | 2007-07-18 | David Dickerson | Systeme permettant de determiner l'orientation d'un oeil |
EP1405121A1 (fr) * | 2000-10-07 | 2004-04-07 | Dickerson, David | Systeme d'information avec saisie de l'image de l'environnement sur la retine |
DE10103922A1 (de) | 2001-01-30 | 2002-08-01 | Physoptics Opto Electronic Gmb | Interaktives Datensicht- und Bediensystem |
DE102007049710A1 (de) * | 2007-10-17 | 2009-04-23 | Robert Bosch Gmbh | Visuelles Auslösen von Vorgängen in einem Kraftfahrzeug |
US20120169582A1 (en) * | 2011-01-05 | 2012-07-05 | Visteon Global Technologies | System ready switch for eye tracking human machine interaction control system |
DE102012005886B4 (de) * | 2012-03-23 | 2023-02-16 | Audi Ag | Verfahren zur Bedienung einer Bedienvorrichtung eines Kraftfahrzeugs |
KR101535593B1 (ko) * | 2013-02-05 | 2015-07-09 | 상명대학교서울산학협력단 | 근적외선 조명 및 카메라를 이용한 전방 객체 인식 방법 및 장치 |
JP6295995B2 (ja) * | 2015-04-28 | 2018-03-20 | 京セラドキュメントソリューションズ株式会社 | 情報処理装置、画像処理装置へのジョブ指示方法 |
CN104924907B (zh) * | 2015-06-19 | 2018-09-14 | 宇龙计算机通信科技(深圳)有限公司 | 一种调节车速的方法及装置 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4811226A (en) * | 1980-09-30 | 1989-03-07 | Toyota Jidosha Kogyo Kabushiki Kaisha | Optimum angle adjusting apparatus for vehicle equipments |
US5060062A (en) * | 1990-05-02 | 1991-10-22 | Dotson Robert S | Electronic vision aid device |
WO1993014454A1 (fr) * | 1992-01-10 | 1993-07-22 | Foster-Miller, Inc. | Interface de donnees integree a commande sensorielle |
DE19631414A1 (de) * | 1996-08-05 | 1998-02-19 | Daimler Benz Ag | Vorrichtung zur Aufnahme des Netzhautreflexbildes und Überlagerung von Zusatzbildern im Auge |
DE19728890A1 (de) * | 1997-07-07 | 1999-02-04 | Daimler Benz Ag | Verfahren zur Verbesserung des optischen Wahrnehmungsvermögens durch Modifikation des Netzhautbildes |
-
1999
- 1999-02-16 EP EP99913073A patent/EP1056611A1/fr not_active Withdrawn
- 1999-02-16 JP JP2000532296A patent/JP2002503862A/ja not_active Withdrawn
- 1999-02-16 WO PCT/DE1999/000421 patent/WO1999042315A1/fr not_active Application Discontinuation
Non-Patent Citations (1)
Title |
---|
See references of WO9942315A1 * |
Also Published As
Publication number | Publication date |
---|---|
JP2002503862A (ja) | 2002-02-05 |
WO1999042315A1 (fr) | 1999-08-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE112018001655B4 (de) | Anzeigevorrichtung und sich bewegender Körper mit der Anzeigevorrichtung | |
DE102020105652B4 (de) | Verfolgungssystem mit infrarot-kamera | |
EP3475755B1 (fr) | Dispositif d'affichage pour projeter un image virtuel dans le champ de vision d'un utilisateur | |
EP1226486B1 (fr) | Dispositif de presentation d'informations dans un vehicule | |
DE60208894T2 (de) | Regensensor mit stereoscopischem abbildungssystem | |
WO2014005972A1 (fr) | Système de caméra permettant de détecter la position d'un conducteur d'un véhicule à moteur | |
EP0917661A2 (fr) | Dispositif pour l'acquisition de l'image reflexe retinienne et la superposition d'images complementaires dans l'oeil | |
DE102016201567A1 (de) | Projektionsvorrichtung für eine Datenbrille, Verfahren zum Darstellen von Bildinformationen mittels einer Projektionsvorrichtung und Steuergerät | |
EP1056611A1 (fr) | Procede pour commander ou manoeuvrer des systemes par informations d'image et procede d'acquisition d'informations d'image | |
DE19802220A1 (de) | Anzeigevorrichtung | |
DE102015205167A1 (de) | Verfahren und Steuergerät zum Betreiben eines autostereoskopischen Sichtfeldanzeigegeräts für ein Fahrzeug | |
EP2732334B1 (fr) | Dispositif et procédé pour acquerir des identifiants biométriques | |
DE4417762C1 (de) | Vorrichtung und Verfahren zum Projizieren und Empfangen von Bildinformation mittels eines Bildschirms | |
DE102019004692B3 (de) | Vorrichtung und Verfahren zur Ermittlung von Bilddaten der Augen, von Augenpositionen und/oder einer Blickrichtung eines Fahrzeugnutzers in einem Fahrzeug | |
WO1999003013A1 (fr) | Procede pour l'amelioration du pouvoir de perception optique par modification de l'image retinale | |
DE102019106258A1 (de) | Insassenüberwachungsvorrichtung | |
WO2016046124A1 (fr) | Dispositif d'affichage pouvant être monté sur la tête d'un utilisateur et procédé de commande de ce dispositif d'affichage | |
DE102021104528A1 (de) | Optisches System für eine virtuelle Netzhautanzeige und Verfahren zum Projizieren von Bildinhalten auf eine Netzhaut | |
DE102021200026A1 (de) | Datenbrille und Verfahren zum Erzeugen eines virtuellen Bilds mit einem Rahmen unter Verwendung einer Datenbrille | |
DE19807081C2 (de) | Verfahren zum Bedienen von Systemen durch optische Erfassung eines Bedienelements oder Gegenstands | |
WO2020165033A1 (fr) | Dispositif de projection pour des lunettes à réalité augmentée, procédé pour représenter des informations d'image au moyen d'un dispositif de projection et appareil de commande | |
DE102019132460A1 (de) | Einstellen eines Rückspiegels an einem Fahrzeug | |
DE102018219420A1 (de) | Verfahren und Vorrichtung zur Bestimmung einer Entfernung von zumindest einem Objekt in einem Umfeld eines Fahrzeugs unter Verwendung einer Lasereinheit und einer Kameraeinheit | |
DE19807271A1 (de) | Verfahren zur Erfassung und Umsetzung von ortsfesten Bildinformationen | |
DE19807065C2 (de) | Verfahren zur Steuerung von Funktionen mobiler Systeme |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20000802 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): DE FR GB SE |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: EBERL, HEINRICH A. |
|
17Q | First examination report despatched |
Effective date: 20020823 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20030103 |