CN108337876A - Blind-guiding method, device and guide equipment - Google Patents
Blind-guiding method, device and guide equipment Download PDFInfo
- Publication number
- CN108337876A CN108337876A CN201780003288.XA CN201780003288A CN108337876A CN 108337876 A CN108337876 A CN 108337876A CN 201780003288 A CN201780003288 A CN 201780003288A CN 108337876 A CN108337876 A CN 108337876A
- Authority
- CN
- China
- Prior art keywords
- current direction
- depth
- prompt
- pixel
- earphone body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
- A61H2201/01—Constructive details
- A61H2201/0173—Means for preventing injuries
- A61H2201/0184—Means for preventing injuries by raising an alarm
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Epidemiology (AREA)
- Pain & Pain Management (AREA)
- Physical Education & Sports Medicine (AREA)
- Rehabilitation Therapy (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Traffic Control Systems (AREA)
Abstract
The present embodiments relate to a kind of blind-guiding method, device and guide equipment.The method includes：Obtain depth image；Based on the depth image determine can traffic areas, according to it is described can traffic areas determine candidate current direction；Prompt the candidate current direction；The distance of barrier on the candidate current direction is measured by distance measuring sensor, if the distance is more than pre-determined distance threshold values, the candidate current direction is the direction that can pass through；Can pass through direction described in prompt.For the embodiment of the present invention by depth image technological incorporation distance detection sensor, detection accuracy is high, by intuitively indicating current direction, the safety trip of guiding dysopia personage to dysopia personage.
Description
Technical field
The present embodiments relate to intelligent blind-guiding technical fields, such as are related to a kind of blind-guiding method, device and guide equipment.
Background technology
In recent years, the self-service trip of auxiliary dysopia personage is got growing concern for, with electronics, computer technology
Development, intelligent, efficient, light electronics guide ancillary equipment becomes main research object, and electronics guide ancillary equipment has
Help improve blind person's independent living ability, quality of making the life better.
Currently, common electronics guide ancillary equipment has electronics blind-guiding stick, wearable guide equipment etc., wearable guide is set
For due to the convenience in its use, increasingly welcome by dysopia personage.Wearable guide equipment, which mostly uses greatly, to be taken the photograph
As the perception of head progress environment, the detection of ground obstacle is carried out using image processing techniques, is generally divided into and is regarded based on monocular
The method of feel and method based on binocular vision.
During studying the prior art, inventor has found that at least there are the following problems in the related technology：Based on monocular
The image processing algorithm of vision is difficult the distance of determining barrier；Method based on binocular vision uses Feature Points Matching mode energy
It enough determines obstacle distance, but is difficult to find reliable characteristic point under low texture, bloom photograph or low-light conditions, cause to hinder
Object distance is hindered to calculate error larger.The error that distance calculates leads to false alarm or fails to report police, influences dysopia personage's
Safety trip.
Invention content
One purpose of the embodiment of the present invention is to provide a kind of new blind-guiding method, device and guide equipment, can help to regard
Feel that impedient people is gone on a journey safely.
In a first aspect, an embodiment of the present invention provides a kind of blind-guiding method, the blind-guiding method is applied to guide equipment, institute
The method of stating includes：
Obtain depth image；
Based on the depth image determine can traffic areas, according to it is described can traffic areas determine candidate current direction；
Prompt the candidate current direction；
The distance that barrier on the candidate current direction is measured by distance measuring sensor, presets if the distance is more than
Apart from threshold values, then the candidate current direction is the direction that can pass through；
Can pass through direction described in prompt.
Second aspect, the embodiment of the present invention additionally provide a kind of blind-guide device, and the blind-guide device is applied to guide equipment,
Described device includes：
Image collection module, for obtaining depth image；
The current direction determining mould of candidate, for based on depth image determination can traffic areas, led to according to described
Row region determines candidate current direction；
Current direction determining mould, for by distance measuring sensor measure barrier on the candidate current direction away from
From if the distance is more than pre-determined distance threshold values, the candidate current direction is the direction that can pass through；
Reminding module, for prompting the candidate current direction, and the direction that can pass through described in prompt.
The third aspect, the embodiment of the present invention additionally provide a kind of guide equipment, including：
Show screen；
Depth camera device, for obtaining depth image；
Distance measuring sensor, the distance for measuring barrier；
Controller, the controller include：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by described at least one
A processor executes, so that at least one processor is able to carry out above-mentioned method.
Blind-guiding method, device and guide equipment provided in an embodiment of the present invention, by obtaining in front of dysopia personage
Depth image, and to the depth image carry out analysis obtain depth image in can traffic areas, by can traffic areas obtain
The current direction of candidate.Detection in conjunction with distance measuring sensor to obstacle distance on candidate current direction, determines final pass through
Direction, and the direction of passing through is prompted to dysopia personage.The embodiment of the present invention is melted by depth image technology
Distance detection sensor is closed, detection accuracy is high, by intuitively indicating current direction, guiding vision barrier to dysopia personage
Hinder the safety trip of personage.
Description of the drawings
One or more embodiments are illustrated by the picture in corresponding attached drawing, these exemplary theorys
The bright restriction not constituted to embodiment, the element with same reference numbers label is expressed as similar element in attached drawing, removes
Non- to have special statement, composition does not limit the figure in attached drawing.
Fig. 1 is the application scenarios schematic diagram of blind-guiding method and device of the present invention；
Fig. 2 is the flow chart of one embodiment of blind-guiding method of the present invention；
Fig. 3 be blind-guiding method of the present invention one embodiment in part steps flow chart；
Fig. 4 be blind-guiding method of the present invention one embodiment in visual cues pass through direction exemplary plot；
Fig. 5 is the structural schematic diagram of one embodiment of blind-guide device of the present invention；
Fig. 6 is the structural schematic diagram of one embodiment of blind-guide device of the present invention；
Fig. 7 is the hardware architecture diagram of guide equipment provided in an embodiment of the present invention；
Fig. 8 is the hardware architecture diagram of guide equipment provided in an embodiment of the present invention；And
Fig. 9 is the hardware architecture diagram of glasses for guiding blind provided in an embodiment of the present invention.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
The every other embodiment obtained without creative efforts, shall fall within the protection scope of the present invention.
Blind-guiding method and device provided in an embodiment of the present invention are suitable for application scenarios shown in FIG. 1.Shown in Fig. 1
In application scenarios, including guide equipment 10 and user 20, wherein user 20 can be dysopsia people scholar, such as amblyopia or
Total blindness crowd.Guide equipment 10 can be that wearable guide equipment, guide equipment 10 such as the guide helmet, glasses for guiding blind are used for
Help the trip of user 20.Guide equipment 10 includes depth camera device for obtaining depth image and for measuring barrier
The distance measuring sensor of distance can also include display screen, earphone, and the wireless communication module etc. to earphone transmission audio.
Wherein, such as binocular depth camera, the depth camera head based on structure light, distance measuring sensor can be adopted depth camera device
With carrying out the sensor of ranging using sound wave such as ultrasonic distance-measuring sensor, wireless communication module such as bluetooth communication mould
Block.
Guide equipment 10 obtains the depth image in 20 front of user, and carries out analysis to the depth image and obtain depth image
In can traffic areas, by can traffic areas obtain candidate current direction.In conjunction with distance measuring sensor on the current direction of candidate
The detection of obstacle distance determines final direction of passing through, and the direction of passing through is prompted to user 20.
An embodiment of the present invention provides a kind of blind-guiding method, the blind-guiding method can be executed by the guide equipment 10 in Fig. 1,
As shown in Fig. 2, the blind-guiding method includes：
101：Obtain depth image.
Wherein, the depth image can be obtained by the depth camera device being arranged in guide equipment 10, the depth
The distance that the pixel in image includes the nearest barrier of each point in depth camera device to scene in face of user 20 is spent, i.e., deeply
Angle value.
102：Based on the depth image determine can traffic areas, according to it is described can traffic areas determine candidate passage side
To.
Obtained in face of user 20 based on the depth image does not have the region of barrier in scene, you can traffic areas, root
According to this can traffic areas determine the direction of travel of user 20.Specifically, in some embodiments of the blind-guiding method, the base
In the depth image determine can traffic areas, including：
Confirm and whether there is one or more snippets continuous pixel in the depth image, in the continuous pixel
The depth of each pixel is more than or equal to predetermined depth threshold values, and the corresponding physical width of the continuous pixel is more than
Or it is equal to preset secured physical width threshold values, and if there is the continuous pixel, then the continuous pixel pair
The region answered is can traffic areas.
Preset predetermined depth threshold value T1 (corresponding obstacle distance threshold value) and secured physical width threshold value W (corresponding people
Safe harbor width), one-row pixels point can be selected from depth image, and using the row pixel as input, in the row
One section of continuous pixel is searched in pixel, if the depth value of each pixel is all higher than T1, while the image in this section
The width of the corresponding physical space of section is more than or equal to W, then it is assumed that this section of image corresponds to one can traffic areas.I.e. if with
The distance of domain of the existence in face of family 20, the barrier in the region is all remote enough, and the width in the region is sufficiently wide, then the region
It may be considered transitable region.It whether there is above-mentioned continuous pixel in the depth image specifically, confirming, packet
It includes：
1021：One-row pixels point is selected from the depth image.
In actual use, above-mentioned one-row pixels point is usually selected from the lower part of depth image, because this part is closer
Face, the range information of the barrier measured can accurately more reflect the actual conditions of barrier.
1022：The one-row pixels point is scanned for, is therefrom selected a little as first end point, according to the first end
The pixel coordinate and depth value of point obtain the corresponding three dimensional physical coordinate of the first end point, obtain and the three dimensional physical coordinate
There is only the physical coordinates points that horizontal distance and horizontal distance are the Safe width threshold values, and the physical coordinates point is converted into
Corresponding pixel on the depth image, and using the pixel as the second endpoint.
To the one-row pixels point, can be scanned for from one of row pixel endpoint to another direction
(such as from the left side to the right search for or from the right to the left side search for), can also from midpoint respectively to the direction of two endpoints into
Row search.
For in the way of being searched for the right from the left side, using the endpoint of left end as first end point, it is assumed that the pixel of the point
Coordinate be p0 (u, v), depth value depth, can according to the internal reference matrix of the pixel coordinate and depth value and depth camera
To obtain three dimensional physical coordinate (X, Y, Z) of this under depth camera coordinate system.It can be obtained according to Safe width threshold values W
Another boundary point (X+W, Y, Z) of safety zone (can traffic areas), which is mapped on depth image, is obtained
Second endpoint p1 (u+delt, v).
1023：If the pixel depth between the first end point and second endpoint is all higher than the predetermined depth
Threshold values, then the corresponding region of pixel fragment that the first end point and second endpoint determine be can traffic areas, otherwise, from institute
State between first end point and second endpoint first depth be less than the predetermined depth threshold value pixel start to continue into
Row search.
Whether the point-by-point corresponding depth of pixel judged between p0 and p1 is more than predetermined depth threshold value T1, if so, recognizing
Between p0-p1 for can traffic areas otherwise abandon the region, and from the pixel for being unsatisfactory for predetermined depth threshold condition
Subsequent point continues to execute step 1022, until the row pixel search finishes.
Specifically, according to it is described can traffic areas determine candidate current direction, including：
If it is described can traffic areas be one, it is determined that it is described can the corresponding direction in traffic areas be candidate passage side
To；
If it is described can traffic areas be at least two, it is determined that steering angle it is minimum can the corresponding direction in traffic areas be
The candidate current direction.
I.e. if it is determined that can traffic areas be one, then using the corresponding direction in the region as candidate passage direction, such as
Fruit determine can traffic areas be multiple, then from all corresponding directions in region that can choose steering angle minimum in traffic areas
As the current direction of candidate.
103：Prompt the candidate current direction.
104：The distance that barrier on the candidate current direction is measured by distance measuring sensor, if the distance is more than
Pre-determined distance threshold values, then the candidate current direction is the direction that can pass through.
In actual use, after candidate current direction determines, guide equipment 10 can prompt the candidate passage side to user 20
To user 20 needs to be gone on the current direction of the candidate according to prompt.Then distance measuring sensor measures barrier in this direction
Distance.If barrier is at a distance sufficiently large, it is more than pre-determined distance threshold value, then can pass before showing, if obstacle distance
Less than pre-determined distance threshold value, then show impassabitity, needs to search for current direction again through depth image.
Under certain transparent scenes, for example glass door or French window, the laser that depth camera is sent out can be worn
Such saturating transparent substance, causes range measurement to generate mistake.Obstacle distance is measured in conjunction with distance measuring sensor, it can be to avoid this kind of
Mistake excludes accidentally to survey caused by depth camera self-defect.
105：Can pass through direction described in prompt.
In 103 and 105, prompt current direction that can also pass through the side of auditory tone cues by way of visual cues
Formula.The mode of visual cues includes：
Vision enhancement image is obtained based on the depth image, shows the vision enhancement image, and increase in the vision
The current direction is prompted in strong image.
Wherein, AR technologies may be used in vision enhancement image, please refer to Fig. 4, are generated with double according to monocular depth image
Visually poor left and right view.In practical applications, first identifier and second can be highlighted in the vision enhancement image
Mark is to prompt current direction, such as by first identifier includes in second identifier, for prompting direction of advance；By first identifier
It is shown on the left of second identifier, turns left for prompting, include on the right side of second identifier, for prompting to the right by first identifier
Turn, hide first identifier, only show second identifier, to prompt front not can traffic areas, user 20 needs to rotate left and right
It can traffic areas to find.As shown in figure 4, highlighted annulus and highlighted box can be respectively adopted in first identifier and second identifier
It indicates.
In the occasion that user 20 is amblyopia crowd, the mode of above-mentioned visual cues may be used, be total blindness people in user 20
The occasion of group, may be used the mode of auditory tone cues, and certainly, user 20 is the occasion of amblyopia crowd, can also be carried using sound
The mode shown, or in such a way that visual cues and auditory tone cues combine.Wherein, voice prompt method includes：
Utilize audio prompt passage direction.Specifically, user can be prompted by playing audio in one's ear in user 20.Example
Such as, to the left and right ear of user 20 simultaneously with first frequency play audio, to prompt direction of advance, to user 20 left ear with
Second frequency plays audio, is turned left with prompt, plays audio with second frequency to the auris dextra of user 20, is turned right with prompt,
To user 20 left and right ear simultaneously with second frequency play audio, with prompt front can not traffic areas, need user a left side
Right rotation can traffic areas to find.Wherein, the frequency values of first frequency are different from the frequency values of second frequency, can be by second
The frequency values value of frequency be more than first frequency frequency values, such as advance when or so ear play slowly drop drop sound, turn left or
Person plays rapid drop drop sound in left ear or auris dextra when turning right, front not can traffic areas when, in left and right, ear plays simultaneously
Rapid drop drop sound.Optionally, the crank degree turned right or turned left can match with the frequency values of second frequency, example
When such as needing big angle rotary, very rapid drop drop sound is played, when small angle rotation, play general rapid drop drop sound.
In other embodiments, audio can not also be played in the left ear or auris dextra of user 20 to prompt to advance,
Playing audio prompt in left ear and/or auris dextra can not advance, and 20 left-right rotary of user is needed to transfer searching direction of advance.I.e. if ear
While there is audio broadcasting, then user 20 cannot advance, and direction of advance is found in rotation to needs to the left or to the right, and when in one's ear, no audio is broadcast
When putting, then when front direction is direction of advance, user 20 can advance.
The embodiment of the present invention analyzes the depth image by obtaining the depth image in front of dysopia personage
Obtain in depth image can traffic areas, by can traffic areas obtain candidate current direction.In conjunction with distance measuring sensor to waiting
Gate the detection of obstacle distance on line direction, determine final direction of passing through, and will described in the direction that can pass through hinder to vision
Personage is hindered to prompt.For the embodiment of the present invention by depth image technological incorporation distance detection sensor, detection accuracy is high, leads to
It crosses and intuitively indicates current direction, the safety trip of guiding dysopia personage to dysopia personage.
Correspondingly, the embodiment of the present invention additionally provides a kind of blind-guide device, the blind-guide device is used for guide shown in FIG. 1
Equipment 10, as shown in figure 5, the blind-guide device 200 includes：
Image collection module 201, for obtaining depth image；
The current direction determining mould 202 of candidate, for based on depth image determination can traffic areas, according to it is described can
Traffic areas determines candidate current direction；
Current direction determining mould 203, for measuring barrier on the candidate current direction by distance measuring sensor
Distance, if the distance is more than pre-determined distance threshold values, the candidate current direction is the direction that can pass through；
Reminding module 204, for prompting the candidate current direction, and the direction that can pass through described in prompt.
The embodiment of the present invention analyzes the depth image by obtaining the depth image in front of dysopia personage
Obtain in depth image can traffic areas, by can traffic areas obtain candidate current direction.In conjunction with distance measuring sensor to waiting
Gate the detection of obstacle distance on line direction, determine final direction of passing through, and will described in the direction that can pass through hinder to vision
Personage is hindered to prompt.For the embodiment of the present invention by depth image technological incorporation distance detection sensor, detection accuracy is high, leads to
It crosses and intuitively indicates current direction, the safety trip of guiding dysopia personage to dysopia personage.
Optionally, in some embodiments of described device, Fig. 6 is please referred to, the reminding module 204 includes：
First prompting submodule 2041 shows the vision for obtaining vision enhancement image based on the depth image
Enhance image, and prompts current direction in the vision enhancement image；
And/or
Second prompting submodule 2042, for utilizing audio prompt passage direction.
Optionally, in some embodiments of described device, first prompting submodule 2041 is specifically used for：
First identifier and second identifier are highlighted in the vision enhancement image；
Include in the second identifier, for prompting when front direction to be current direction by the first identifier；
Include being turned left on the left of the second identifier for prompting by the first identifier；
Include being turned right on the right side of the second identifier for prompting by the first identifier；
It hides the first identifier and shows the second identifier, for the direction that can not pass through in front of prompting.
Optionally, in some embodiments of described device, second prompting submodule 2042 is specifically used for：
Audio is played to the first earphone body and the second earphone body, while with first frequency, with prompt when front direction is to pass through
Direction, the first earphone body are used to play audio to the left ear of user, and the second earphone body is used for the right side of user
Ear plays audio；
To the first earphone body, audio is played with second frequency, is turned left with prompt；
To the second earphone body, audio is played with second frequency, is turned right with prompt；
Audio is played to the first earphone body and the second earphone body, while with second frequency, to prompt front nothing
Can pass through direction.
Optionally, in some embodiments of described device, second prompting submodule is specifically used for：
Audio is played to the first earphone body and/or the second earphone body, the non-through line direction of front direction is worked as with prompt；
Stop playing audio to the first earphone body and the second earphone body, with prompt when front direction is passage side
To.
Optionally, in some embodiments of described device, the frequency values of the second frequency are more than the first frequency
Frequency values, the frequency values of the second frequency with turn left or right-handed crank degree matches.
Optionally, in some embodiments of described device, Fig. 6, the candidate current direction determining mould 202 are please referred to
Including the first candidate current direction determination sub-module 2021, it is used for：
Confirm and whether there is one or more snippets continuous pixel in the depth image, in the continuous pixel
The depth of each pixel is more than or equal to predetermined depth threshold values, and the corresponding physical width of the continuous pixel is more than
Or it is equal to preset secured physical width threshold values, and if there is the continuous pixel, then the continuous pixel pair
The region answered is can traffic areas.
Optionally, in some embodiments of described device, the described first candidate current direction determination sub-module 2021 has
Body is used for：
One-row pixels point is selected from the depth image；
The one-row pixels point is scanned for, is therefrom selected a little as first end point, according to the first end point
Pixel coordinate and depth value obtain the corresponding three dimensional physical coordinate of the first end point, and acquisition is only deposited with the three dimensional physical coordinate
In the physical coordinates point that water surface distance and horizontal distance are the Safe width threshold values, the physical coordinates point is converted into described
Corresponding pixel on depth image, and using the pixel as the second endpoint；
If the pixel depth between the first end point and second endpoint is all higher than the predetermined depth threshold values,
The corresponding region of pixel fragment that then first end point and second endpoint determine be can traffic areas, otherwise, from described the
Between end point and second endpoint first depth be less than the pixel of the predetermined depth threshold value subsequent point start after
It is continuous to scan for.
Optionally, in some embodiments of described device, the candidate current direction determining mould 202 includes the second time
Line direction determination sub-module 2022 is gated, is used for：
If it is described can traffic areas be one, it is determined that it is described can the corresponding direction in traffic areas be candidate passage side
To；
If it is described can traffic areas be at least two, it is determined that steering angle it is minimum can the corresponding direction in traffic areas be
The candidate current direction.
It should be noted that above-mentioned blind-guide device can perform the blind-guiding method that the embodiment of the present invention is provided, has execution
The corresponding function module of method and advantageous effect.The technical detail of detailed description not in blind-guide device embodiment, reference can be made to this
The blind-guiding method that inventive embodiments are provided.
Fig. 7 is the hardware architecture diagram of guide equipment 10 provided in an embodiment of the present invention, as shown in fig. 7, the guide is set
Standby 10 include：
Show screen 11, the depth camera device 12 for obtaining depth image, the ranging sensing for detecting obstacle distance
Device 13 and controller 14.Wherein, depth camera device 12 such as binocular depth camera, the depth camera based on structure light
Deng distance measuring sensor 13 is such as ultrasonic distance-measuring sensor.
Controller 14 includes one or more processors 141 and memory 142, is with a processor 141 in Fig. 7
Example.
Processor 141 can be connected with memory 142 by bus or other modes, to be connected by bus in Fig. 7
For.
Memory 142 is used as a kind of non-volatile computer readable storage medium storing program for executing, can be used for storing non-volatile software journey
Sequence, non-volatile computer executable program and module, as the corresponding program instruction of blind-guiding method in the embodiment of the present invention/
Module is (for example, attached image collection module shown in fig. 5 201, candidate current direction determining mould 202, current direction determining mould
203 and reminding module 204).Processor 141 by operation be stored in non-volatile software program in memory 142, instruction with
And module, the various function application to execute server and data processing, that is, realize the guide side of above method embodiment
Method.
Memory 142 may include storing program area and storage data field, wherein storing program area can store operation system
System, the required application program of at least one function；Storage data field can be stored uses created number according to blind-guide device
According to etc..In addition, memory 142 may include high-speed random access memory, can also include nonvolatile memory, such as extremely
A few disk memory, flush memory device or other non-volatile solid state memory parts.In some embodiments, memory
142 it is optional include the memory remotely located relative to processor 141, these remote memories can be by network connection to leading
Device for blind person.The example of above-mentioned network includes but not limited to internet, intranet, LAN, mobile radio communication and combinations thereof.
One or more of modules are stored in the memory 142, when by one or more of processors
When 141 execution, the blind-guiding method in above-mentioned any means embodiment is executed, for example, executing the method step in Fig. 2 described above
Rapid 101 to step 105, the method and step 1021 in Fig. 3 to step 1023；Realize module in module 201-204, Fig. 6 in Fig. 5
The function of 201-204, submodule 2021-2022, submodule 2041-2042.
Guide equipment 10 can be optional to realize the function of playing audio in one's ear in user 20 by jack peripheral earphone
, in some embodiments of the guide equipment 10, as shown in figure 8, the guide equipment 10 can be with 15 He of earphone integrated
Wireless communication module 16.For wireless communication module 16 for playing out audio transmission to earphone 15, earphone 15 includes the first ear
Body and the second earphone body, the first earphone body are used to play audio to the left ear of user 20, and the second earphone body is used for
Audio is played to the auris dextra of user 20.Wherein, wireless communication module 16 can be bluetooth communication module, and earphone 15 can be that bone passes
Lead bluetooth headset.
Wherein, guide equipment 10 can be that wearable guide equipment, the Fig. 9 such as the guide helmet, glasses for guiding blind show guide
Equipment 10 is the occasion of glasses for guiding blind.Glasses for guiding blind includes the first eyeglass 18a and the second eyeglass 18b, the first display screen 11a settings
On the first eyeglass 18a, second display screen 11b is arranged on the second eyeglass 18b.Depth camera device 12 includes first camera section
Part 12a and second camera section part 12b, first camera section part 12a are located at the side of the first eyeglass 18a, second camera section part 12b
In the side of the second eyeglass 18b.Wherein, first camera section part 12a can be the emission element of depth camera device, the second camera shooting
Component 12b can be the receiving part of depth camera device.Distance measuring sensor 13 is located at the first eyeglass 18a and the second eyeglass 18b
Top, across the first eyeglass 18a and the second eyeglass 18b.Controller 14 is set in any one temple, wireless communication module
16 are also set in any one temple, and power supply can also be arranged in temple and power for entire glasses for guiding blind.Earphone 15 includes first
Earphone body 151 and the second earphone body 152.
The said goods can perform the method that the embodiment of the present invention is provided, and has the corresponding function module of execution method and has
Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to the method that the embodiment of the present invention is provided.
An embodiment of the present invention provides a kind of non-volatile computer readable storage medium storing program for executing, the computer-readable storage mediums
Matter is stored with computer executable instructions, which is executed by one or more processors, such as in Fig. 7
One processor 141 may make said one or multiple processors to can perform the blind-guiding method in above-mentioned any means embodiment,
For example, execute method and step 101 in Fig. 2 described above to step 105, the method and step 1021 in Fig. 3 to step 1023；
Realize the work(of module 201-204, submodule 2021-2022, submodule 2041-2042 in module 201-204, Fig. 6 in Fig. 5
Energy.
The apparatus embodiments described above are merely exemplary, wherein the unit illustrated as separating component can
It is physically separated with being or may not be, the component shown as unit may or may not be physics list
Member, you can be located at a place, or may be distributed over multiple network units.It can be selected according to the actual needs
In some or all of module achieve the purpose of the solution of this embodiment.
Through the above description of the embodiments, those of ordinary skill in the art can be understood that each embodiment
The mode of general hardware platform can be added to realize by software, naturally it is also possible to pass through hardware.Those of ordinary skill in the art can
With understand all or part of flow realized in above-described embodiment method be can be instructed by computer program it is relevant hard
Part is completed, and the program can be stored in a computer read/write memory medium, the program is when being executed, it may include as above
State the flow of the embodiment of each method.Wherein, the storage medium can be magnetic disc, CD, read-only memory (Read-
Only Memory, ROM) or random access memory (RandomAccessMemory, RAM) etc..
Finally it should be noted that：The above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；At this
It under the thinking of invention, can also be combined between the technical characteristic in above example or different embodiment, step can be with
It is realized with random order, and there are many other variations of different aspect present invention as described above, for simplicity, they do not have
Have and is provided in details；Although the present invention is described in detail referring to the foregoing embodiments, the ordinary skill people of this field
Member should understand that：It still can be with technical scheme described in the above embodiments is modified, or to which part skill
Art feature carries out equivalent replacement；And these modifications or replacements, each reality of the present invention that it does not separate the essence of the corresponding technical solution
Apply the range of a technical solution.
Claims (23)
1. a kind of blind-guiding method, the blind-guiding method is applied to guide equipment, which is characterized in that the method includes：
Obtain depth image；
Based on the depth image determine can traffic areas, according to it is described can traffic areas determine candidate current direction；
Prompt the candidate current direction；
The distance that barrier on the candidate current direction is measured by distance measuring sensor, if the distance is more than pre-determined distance
Threshold values, then the candidate current direction is the direction that can pass through；
Can pass through direction described in prompt.
2. according to the method described in claim 1, it is characterized in that, the current direction of prompt, including：
Vision enhancement image is obtained based on the depth image, shows the vision enhancement image, and in the vision enhancement figure
The current direction is prompted as in；
And/or
Utilize current direction described in audio prompt.
3. according to the method described in claim 2, it is characterized in that, described prompt the passage in the vision enhancement image
Direction, including：
First identifier and second identifier are highlighted in the vision enhancement image；
Include in the second identifier, for prompting when front direction to be current direction by the first identifier；
Include being turned left on the left of the second identifier for prompting by the first identifier；
Include being turned right on the right side of the second identifier for prompting by the first identifier；
It hides the first identifier and shows the second identifier, for the direction that can not pass through in front of prompting.
4. according to the method described in claim 2, it is characterized in that, described using current direction described in audio prompt, including：
Audio is played to the first earphone body and the second earphone body, while with first frequency, it is current direction to work as front direction with prompt,
The first earphone body is used to play audio to the left ear of user, and the second earphone body is used to play to the auris dextra of user
Audio；
To the first earphone body, audio is played with second frequency, is turned left with prompt；
To the second earphone body, audio is played with second frequency, is turned right with prompt；
Audio is played to the first earphone body and the second earphone body, while with second frequency, can not be led to prompt front
Line direction.
5. according to the method described in claim 2, it is characterized in that, described using current direction described in audio prompt, including：
Audio is played to the first earphone body and/or the second earphone body, with prompt when front direction is non-through line direction；
Stop playing audio to the first earphone body and the second earphone body, with prompt when front direction is current direction.
6. according to the method described in claim 4, it is characterized in that, the frequency values of the second frequency are more than the first frequency
Frequency values, the frequency values of the second frequency with turn left or right-handed crank degree matches.
7. according to the method described in claim 1, it is characterized in that, it is described based on the depth image determine can traffic areas,
Including：
Confirm and whether there is one or more snippets continuous pixel in the depth image, each of in the continuous pixel
The depth of pixel be more than or equal to predetermined depth threshold values, and the corresponding physical width of the continuous pixel be more than or
Equal to preset secured physical width threshold values, if there is the continuous pixel, then the continuous pixel is corresponding
Region is can traffic areas.
8. the method according to the description of claim 7 is characterized in that it is described confirm in the depth image with the presence or absence of one section or
The depth of the continuous pixel of multistage, each pixel in the continuous pixel is more than or equal to predetermined depth valve
Value, and the corresponding physical width of the continuous pixel is more than or equal to preset secured physical width threshold values, including：
One-row pixels point is selected from the depth image；
The one-row pixels point is scanned for, is therefrom selected a little as first end point, according to the pixel of the first end point
Coordinate and depth value obtain the corresponding three dimensional physical coordinate of the first end point, and there is only water with the three dimensional physical coordinate for acquisition
Flat distance and the physical coordinates point that horizontal distance is the Safe width threshold values, the depth is converted by the physical coordinates point
Corresponding pixel on image, and using the pixel as the second endpoint；
If the pixel depth between the first end point and second endpoint is all higher than the predetermined depth threshold values, institute
State the corresponding region of pixel fragment that first end point and second endpoint determine be can traffic areas, otherwise, from the first end
Between point and second endpoint first depth be less than the pixel of the predetermined depth threshold value subsequent point start to continue into
Row search.
9. according to the method described in claim 1, it is characterized in that, can the candidate passage side of traffic areas determination described in the basis
To, including：
If it is described can traffic areas be one, it is determined that it is described can the corresponding direction in traffic areas be candidate current direction；
If it is described can traffic areas be at least two, it is determined that steering angle it is minimum can the corresponding direction in traffic areas be described
The current direction of candidate.
10. a kind of blind-guide device, the blind-guide device is applied to guide equipment, which is characterized in that described device includes：
Image collection module, for obtaining depth image；
The current direction determining mould of candidate, for based on depth image determination can traffic areas, according to it is described can FOH
Domain determines candidate current direction；
Current direction determining mould, the distance for measuring barrier on the candidate current direction by distance measuring sensor, such as
Distance described in fruit is more than pre-determined distance threshold values, then the candidate current direction is the direction that can pass through；
Reminding module, for prompting the candidate current direction, and the direction that can pass through described in prompt.
11. device according to claim 10, which is characterized in that the reminding module includes：
First prompting submodule, for based on depth image acquisition vision enhancement image, showing the vision enhancement image,
And current direction is prompted in the vision enhancement image；
And/or
Second prompting submodule, for utilizing audio prompt passage direction.
12. according to the devices described in claim 11, which is characterized in that first prompting submodule is specifically used for：
First identifier and second identifier are highlighted in the vision enhancement image；
Include in the second identifier, for prompting when front direction to be current direction by the first identifier；
Include being turned left on the left of the second identifier for prompting by the first identifier；
Include being turned right on the right side of the second identifier for prompting by the first identifier；
It hides the first identifier and shows the second identifier, for the direction that can not pass through in front of prompting.
13. according to the devices described in claim 11, which is characterized in that second prompting submodule is specifically used for：
Audio is played to the first earphone body and the second earphone body, while with first frequency, it is current direction to work as front direction with prompt,
The first earphone body is used to play audio to the left ear of user, and the second earphone body is used to play to the auris dextra of user
Audio；
To the first earphone body, audio is played with second frequency, is turned left with prompt；
To the second earphone body, audio is played with second frequency, is turned right with prompt；
Audio is played to the first earphone body and the second earphone body, while with second frequency, can not be led to prompt front
Line direction.
14. according to the devices described in claim 11, which is characterized in that second prompting submodule is specifically used for：
Audio is played to the first earphone body and/or the second earphone body, the non-through line direction of front direction is worked as with prompt；
Stop playing audio to the first earphone body and the second earphone body, with prompt when front direction is current direction.
15. device according to claim 13, which is characterized in that the frequency values of the second frequency are more than first frequency
The frequency values of rate, the frequency values of the second frequency with turn left or right-handed crank degree matches.
16. device according to claim 10, which is characterized in that the candidate current direction determining mould includes the first time
Line direction determination sub-module is gated, is used for：
Confirm and whether there is one or more snippets continuous pixel in the depth image, each of in the continuous pixel
The depth of pixel be more than or equal to predetermined depth threshold values, and the corresponding physical width of the continuous pixel be more than or
Equal to preset secured physical width threshold values, if there is the continuous pixel, then the continuous pixel is corresponding
Region is can traffic areas.
17. device according to claim 16, which is characterized in that the described first candidate current direction determination sub-module is specific
For：
One-row pixels point is selected from the depth image；
The one-row pixels point is scanned for, is therefrom selected a little as first end point, according to the pixel of the first end point
Coordinate and depth value obtain the corresponding three dimensional physical coordinate of the first end point, and there is only water with the three dimensional physical coordinate for acquisition
Identity distance from and horizontal distance be the Safe width threshold values physical coordinates point, the physical coordinates point is converted into the depth
Corresponding pixel on image, and using the pixel as the second endpoint；
If the pixel depth between the first end point and second endpoint is all higher than the predetermined depth threshold values, institute
State the corresponding region of pixel fragment that first end point and second endpoint determine be can traffic areas, otherwise, from the first end
Between point and second endpoint first depth be less than the pixel of the predetermined depth threshold value subsequent point start to continue into
Row search.
18. device according to claim 10, which is characterized in that the candidate current direction determining mould includes the second time
Line direction determination sub-module is gated, is used for：
If it is described can traffic areas be one, it is determined that it is described can the corresponding direction in traffic areas be candidate current direction；
If it is described can traffic areas be at least two, it is determined that steering angle it is minimum can the corresponding direction in traffic areas be described
The current direction of candidate.
19. a kind of guide equipment, which is characterized in that including：
Show screen；
Depth camera device, for obtaining depth image；
Distance measuring sensor, the distance for measuring barrier；
Controller, the controller include：
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and described instruction is by least one place
It manages device to execute, so that at least one processor is able to carry out claim 1-9 any one of them methods.
20. guide equipment according to claim 19, which is characterized in that the guide equipment further includes earphone and channel radio
Interrogate module, the wireless communication module for audio transmission to be played out to earphone, the earphone include the first earphone body and
Second earphone body, the first earphone body are used to play audio to the left ear of user, and the second earphone body is used for using
The auris dextra of person plays audio.
21. the guide equipment according to claim 19 or 20, which is characterized in that the guide equipment is glasses for guiding blind, institute
It includes the first eyeglass and the second eyeglass to state glasses for guiding blind；
The display screen includes the first display screen and second display screen, and first display screen is arranged in first eyeglass
On, the second display screen is arranged on second eyeglass；
The depth camera device includes first camera section part and second camera section part, and the first camera section part is located at described the
The side of one eyeglass, the second camera section part are located at the side of second eyeglass；
The distance measuring sensor is located at the top of first eyeglass and second eyeglass, across first eyeglass and described
Second eyeglass；
The controller is set in any one temple.
22. a kind of non-volatile computer readable storage medium storing program for executing, which is characterized in that the computer-readable recording medium storage has
Computer executable instructions make the guide equipment hold when the computer executable instructions are executed by guide equipment
Row claim 1-9 any one of them methods.
23. a kind of computer program product, which is characterized in that the computer program product includes being stored in non-volatile calculating
Computer program on machine readable storage medium storing program for executing, the computer program include program instruction, when described program is instructed by guide
When equipment executes, the guide equipment perform claim is made to require 1-9 any one of them methods.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2017/115002 WO2019109301A1 (en) | 2017-12-07 | 2017-12-07 | Blind guidance method and apparatus, and blind guidance device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108337876A true CN108337876A (en) | 2018-07-27 |
Family
ID=62924215
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201780003288.XA Pending CN108337876A (en) | 2017-12-07 | 2017-12-07 | Blind-guiding method, device and guide equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108337876A (en) |
| WO (1) | WO2019109301A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110470307A (en) * | 2019-08-28 | 2019-11-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of visually impaired patient navigation system and method |
| CN112640447A (en) * | 2018-08-30 | 2021-04-09 | 韦奥机器人股份有限公司 | Depth sensing computer vision system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104055657A (en) * | 2014-06-18 | 2014-09-24 | 浙江师范大学 | Blind guide crutch based on Kinect and realization method of blind guide crutch |
| CN104287946A (en) * | 2014-10-24 | 2015-01-21 | 中国科学院计算技术研究所 | Device and method for prompting blind persons to avoid obstacles |
| CN104899869A (en) * | 2015-05-14 | 2015-09-09 | 浙江大学 | Plane and barrier detection method based on RGB-D camera and attitude sensor |
| CN106109188A (en) * | 2015-05-08 | 2016-11-16 | 丁麒木 | Active hand push guide dolly |
| CN106855411A (en) * | 2017-01-10 | 2017-06-16 | 深圳市极思维智能科技有限公司 | A kind of robot and its method that map is built with depth camera and obstacle avoidance system |
| CN107007437A (en) * | 2017-03-31 | 2017-08-04 | 北京邮电大学 | Interactive blind person's householder method and equipment |
| CN107225570A (en) * | 2017-04-20 | 2017-10-03 | 深圳前海勇艺达机器人有限公司 | The barrier-avoiding method and device of intelligent robot |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2629737B1 (en) * | 2010-09-24 | 2016-07-13 | Heptagon Micro Optics Pte. Ltd. | White cane with integrated electronic travel aid using 3d tof sensor |
| CN106214437B (en) * | 2016-07-22 | 2018-05-29 | 杭州视氪科技有限公司 | A kind of intelligent blind auxiliary eyeglasses |
-
2017
- 2017-12-07 CN CN201780003288.XA patent/CN108337876A/en active Pending
- 2017-12-07 WO PCT/CN2017/115002 patent/WO2019109301A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104055657A (en) * | 2014-06-18 | 2014-09-24 | 浙江师范大学 | Blind guide crutch based on Kinect and realization method of blind guide crutch |
| CN104287946A (en) * | 2014-10-24 | 2015-01-21 | 中国科学院计算技术研究所 | Device and method for prompting blind persons to avoid obstacles |
| CN106109188A (en) * | 2015-05-08 | 2016-11-16 | 丁麒木 | Active hand push guide dolly |
| CN104899869A (en) * | 2015-05-14 | 2015-09-09 | 浙江大学 | Plane and barrier detection method based on RGB-D camera and attitude sensor |
| CN106855411A (en) * | 2017-01-10 | 2017-06-16 | 深圳市极思维智能科技有限公司 | A kind of robot and its method that map is built with depth camera and obstacle avoidance system |
| CN107007437A (en) * | 2017-03-31 | 2017-08-04 | 北京邮电大学 | Interactive blind person's householder method and equipment |
| CN107225570A (en) * | 2017-04-20 | 2017-10-03 | 深圳前海勇艺达机器人有限公司 | The barrier-avoiding method and device of intelligent robot |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112640447A (en) * | 2018-08-30 | 2021-04-09 | 韦奥机器人股份有限公司 | Depth sensing computer vision system |
| CN112640447B (en) * | 2018-08-30 | 2022-04-29 | 韦奥机器人股份有限公司 | Depth sensing computer vision system |
| CN110470307A (en) * | 2019-08-28 | 2019-11-19 | 中国科学院长春光学精密机械与物理研究所 | A kind of visually impaired patient navigation system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019109301A1 (en) | 2019-06-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11620835B2 (en) | Obstacle recognition method and apparatus, storage medium, and electronic device | |
| CN105554385B (en) | A kind of remote multi-modal biological characteristic recognition methods and its system | |
| CN104537292B (en) | The method and system detected for the electronic deception of biological characteristic validation | |
| US10438069B2 (en) | Method and apparatus for detecting abnormal situation, and non-transitory computer-readable recording medium | |
| KR101765970B1 (en) | Voice-body identity correlation | |
| EP3156754A1 (en) | Dart game device having multiple cameras and computer program stored in computer-readable medium | |
| CN104036488B (en) | Binocular vision-based human body posture and action research method | |
| CN104287946B (en) | Blind person's avoidance suggestion device and method | |
| US20170024874A1 (en) | Method and apparatus for detecting abnormal situation | |
| CN107748869A (en) | 3D face identity authentications and device | |
| US20130177296A1 (en) | Generating metadata for user experiences | |
| US11468789B2 (en) | Electroencephalogram—controlled video input and auditory display blind guiding apparatus and method | |
| KR102129698B1 (en) | Automatic fish counting system | |
| EP4033458A2 (en) | Method and apparatus of face anti-spoofing, device, storage medium, and computer program product | |
| JP2000137792A (en) | Eye part detecting device | |
| CN106162144A (en) | A kind of visual pattern processing equipment, system and intelligent machine for overnight sight | |
| CN108337876A (en) | Blind-guiding method, device and guide equipment | |
| US20200242335A1 (en) | Information processing apparatus, information processing method, and recording medium | |
| JP2005165778A (en) | Head mounted display device and its control method | |
| KR20220044897A (en) | Wearable device, smart guide method and device, guide system, storage medium | |
| CN113678141A (en) | Stereophonic device for blind and visually impaired persons | |
| CN106991376A (en) | With reference to the side face verification method and device and electronic installation of depth information | |
| CN112700568B (en) | Identity authentication method, equipment and computer readable storage medium | |
| CN111243070B (en) | Virtual reality presenting method, system and device based on 5G communication | |
| JP5360115B2 (en) | Head mounted display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180727 |
|
| RJ01 | Rejection of invention patent application after publication |