CN106056534A - Obstruction perspective method and device based on smart glasses - Google Patents
Obstruction perspective method and device based on smart glasses Download PDFInfo
- Publication number
- CN106056534A CN106056534A CN201610373513.6A CN201610373513A CN106056534A CN 106056534 A CN106056534 A CN 106056534A CN 201610373513 A CN201610373513 A CN 201610373513A CN 106056534 A CN106056534 A CN 106056534A
- Authority
- CN
- China
- Prior art keywords
- image
- shelter
- intelligent glasses
- mentioned
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 99
- 239000004984 smart glass Substances 0.000 title abstract 3
- 239000011521 glass Substances 0.000 claims description 211
- 230000000007 visual effect Effects 0.000 claims description 190
- 239000011159 matrix material Substances 0.000 claims description 38
- 230000009466 transformation Effects 0.000 claims description 31
- 238000001514 detection method Methods 0.000 claims description 27
- 238000000605 extraction Methods 0.000 claims description 23
- 239000000284 extract Substances 0.000 claims description 19
- 238000004422 calculation algorithm Methods 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 238000003384 imaging method Methods 0.000 claims description 11
- 230000000694 effects Effects 0.000 claims description 9
- 230000008859 change Effects 0.000 description 28
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 17
- 239000000463 material Substances 0.000 description 14
- 210000003128 head Anatomy 0.000 description 13
- 238000012545 processing Methods 0.000 description 13
- 238000012216 screening Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000000903 blocking effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000010339 dilation Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002594 fluoroscopy Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005304 joining Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000004017 vitrification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The present invention provides an obstruction perspective method and device based on smart glasses. The method comprises: collecting a first image in a user's view and identifying the obstruction image in the first image through a smart glasses; collecting a second image in the user's view through at least one image collection device; and employing the part corresponding to the obstruction image in the second image to take place of the obstruction image in the first image, and jointing and generating an unscreened image in the user's view. The part corresponding to the obstruction image in the second image is employed to take place of the obstruction image in the first image so as to realize the perspective obstruction.
Description
Technical field
The present invention relates to human-computer interaction technique field, particularly relate to a kind of shelter perspective method based on intelligent glasses and
Device.
Background technology
In the presence of having shelter, tending to form blind area, observer cannot obtain shelter and stop
The visual field of part, thus easily cause variety of problems.Such as, in various driving cabins the column of windshield to form the visual field dead
Angle, car door part also can block the visual field of observer.In recent years, due to the development of intelligent glasses, seeing through objects is possibly realized.So
And, current intelligent glasses can only be used for the internal structure of seeing through objects, it is impossible to has an X-rayed whole object, such as shelter, and
The image of internal structure being known a priori by object is needed could object to be had an X-rayed.
Summary of the invention
The present invention provides a kind of shelter perspective method based on intelligent glasses and device, with solve existing intelligent glasses without
The problem of method perspective shelter.
The present invention provides a kind of shelter perspective method based on intelligent glasses, including: used by intelligent glasses collection
First image in the person visual field also identifies the shelter image in described first image；Made by least one image capture device collection
Second image in the user visual field；Utilize the first image described in the partial replacement of corresponding described shelter image in described second image
In shelter image, splicing generate the user visual field unobstructed image.
In one embodiment, utilize the first figure described in the partial replacement of corresponding described shelter image in described second image
Shelter image in Xiang, splicing generates the unobstructed image in the user visual field, including: except shelter from described first image
Image section outside image and described second image extract multiple fisrt feature point and multiple second feature point respectively, and described
Two characteristic points and described fisrt feature point one_to_one corresponding；According to all described fisrt feature points and all described second feature points,
It is calculated the image transformation matrix being converted to user perspective by the imaged viewing angle of described image capture device；Utilize described figure
As transformation matrix carries out image conversion to described second image；Described second image after image is converted blocks described in correspondence
Described unobstructed image is generated together with the part of object image described first image mosaic after removing described shelter image.
In one embodiment, the image section in addition to shelter image from described first image and described second figure
As extracting multiple fisrt feature point and multiple second feature point respectively, including: by feature extraction algorithm from described first image
In the image section in addition to shelter image and described second image extract multiple fisrt feature point and multiple second respectively
Characteristic point.
In one embodiment, remove the setting pixel wide district at the edge of described first image after described shelter image
Territory comprises the described shelter image of part.
In one embodiment, by image convert after described second image in corresponding described shelter image part with
Remove before described first image mosaic after described shelter image generates described unobstructed image together, also include: root
According to imaging angle and the spatial positional information of described image capture device, corresponding institute in described second image after image is converted
The stitching position between the first image after stating the part of shelter image and removing described shelter image carries out initial alignment.
In one embodiment, by the shelter image in the first image described in intelligent glasses identification, including: according to blocking
Pictorial symbolization in the constant attribute of thing or shelter, identifies the shelter image in described first image.
In one embodiment, the method also includes: when the picture in the user visual field changes, described intelligent glasses
Updating described first image, the second image described in described image capture device Resurvey is to update described second image, and base
Described first image after updating and described second image after renewal regenerate described unobstructed image.
In one embodiment, the method also includes: when the visual angle of user changes, and described intelligent glasses is according to institute
The detection data stating the detection data of the gyroscope on intelligent glasses and the gyroscope of a setting position determine the need for updating
Described first image；If so, described first image is updated by described intelligent glasses, and based on described first image after updating
Regenerate described unobstructed image.
In one embodiment, also include: after the visual angle of user changes, use another image capture device again
Gather described second image, and described second image based on Resurvey regenerates described unobstructed image.
In one embodiment, described shelter regards relative to the imaging of the deflection of user with described image capture device
The difference at angle is less than 90 degree.
In one embodiment, the number of described image capture device is multiple, is made by multiple image capture device collections
Second image in the user visual field, including: the 3rd image in the user visual field is gathered by each described image capture device；By each institute
State the 3rd image mosaic and form described second image of have no occluder together.
The present invention also provides for a kind of shelter arrangement for perspective based on intelligent glasses, including: shelter image acquisition unit,
For being gathered first image in the user visual field by intelligent glasses and identifying the shelter image in described first image；Replace
Image acquisition unit, for gathering second image in the user visual field by least one image capture device；Unobstructed image is raw
Become unit, for utilizing blocking in the first image described in the partial replacement of corresponding described shelter image in described second image
Object image, splicing generates the unobstructed image in the user visual field.
In one embodiment, described unobstructed image generation unit includes: feature point extraction module, for from described first
In image, image section and described second image in addition to shelter image extract multiple fisrt feature point and multiple respectively
Two characteristic points, described second feature point and described fisrt feature point one_to_one corresponding；Transformation matrix generation module, for according to all
Described fisrt feature point and all described second feature points, be calculated and be converted to by the imaged viewing angle of described image capture device
The image transformation matrix of user perspective；Image transform module, is used for utilizing described image transformation matrix to described second image
Carry out image conversion；Unobstructed image generation module, blocks described in correspondence in described second image after being converted by image
Described unobstructed image is generated together with the part of object image described first image mosaic after removing described shelter image.
In one embodiment, described feature point extraction module includes: feature extraction algorithm module, is used for passing through feature extraction
The algorithm image section in addition to shelter image from described first image and described second image extract multiple respectively
One characteristic point and multiple second feature point.
In one embodiment, described unobstructed image generation module is additionally operable to perform: after removing described shelter image
The setting pixel wide region at the edge of described first image comprises the described shelter image of part.
In one embodiment, described unobstructed image generation unit also includes: initial alignment module, for according to described figure
As imaging angle and the spatial positional information of collecting device, corresponding described shelter in described second image after image is converted
Stitching position between the part of image and the first image after removing described shelter image carries out initial alignment.
In one embodiment, described shelter image acquisition unit includes: shelter image collection module, for according to screening
Pictorial symbolization in the constant attribute of block material or shelter, identifies the shelter image in described first image.
In one embodiment, this device also includes: the first image update unit, sends out for the picture in the user visual field
During changing, described intelligent glasses updates described first image, the second image described in described image capture device Resurvey with
Update described second image, and regenerate described based on described second image after described first image after updating and renewal
Unobstructed image.
In one embodiment, this device also includes: the second image update unit, for changing at the visual angle of user
Time, described intelligent glasses is according to the detection data of the gyroscope on described intelligent glasses and the detection of the gyroscope of a setting position
Data determine the need for updating described first image；Unobstructed image update unit, for if so, by described intelligent glasses
Update described first image, and regenerate described unobstructed image based on described first image after updating.
In one embodiment, this device also includes: the 3rd image update unit, for changing at the visual angle of user
After, use the second image described in another image capture device Resurvey, and described second image based on Resurvey is again
Generate described unobstructed image.
In one embodiment, described replacement image acquisition unit is additionally operable to perform: described shelter is relative to user
Deflection is less than 90 degree with the difference of the imaged viewing angle of described image capture device.
In one embodiment, the number of described image capture device is multiple, and described replacement image acquisition unit includes: the
Three image collection module, for gathering the 3rd image in the user visual field by each described image capture device；Second image obtains
Delivery block, for forming described second image of have no occluder together by each described 3rd image mosaic.
The shelter perspective method based on intelligent glasses of the embodiment of the present invention and device, utilize the photographic head of intelligent glasses
The image in the user visual field collected, obtains position and the image of shelter in the visual field by multiple methods analyst identification,
Utilize the image in the visual field that external image capture device collection is blocked, and the external detector collection of part that will be blocked
To image be replaced, then carry out registrating, splicing by replacement image and user field-of-view image, consequently, it is possible to use intelligence
Glasses can see, observing shelter when, the image being blocked by obstructions such that it is able to produces the effect of perspective shelter
Really, the dead angle caused by this shelter in can effectively removing the user visual field with this.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
In having technology to describe, the required accompanying drawing used is briefly described, it should be apparent that, the accompanying drawing in describing below is only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to
Other accompanying drawing is obtained according to these accompanying drawings.In the accompanying drawings:
Fig. 1 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of the embodiment of the present invention；
Fig. 2 is the method flow schematic diagram of the unobstructed image that splicing generates the user visual field in one embodiment of the invention；
Fig. 3 is the method flow signal of the unobstructed image that splicing generates the user visual field in another embodiment of the present invention
Figure；
Fig. 4 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of further embodiment of this invention；
Fig. 5 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of yet another embodiment of the invention；
Fig. 6 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of another embodiment of the present invention；
Fig. 7 is the side of the second image being gathered the user visual field in one embodiment of the invention by multiple image capture devices
Method schematic flow sheet；
Fig. 8 is the schematic flow sheet of the method that splicing obtains the second image in one embodiment of the invention；
Fig. 9 is the schematic diagram of the first image that intelligent glasses gathers in one embodiment of the invention；
Figure 10 be in one embodiment of the invention image capture device collect comprise shelter image portion shown in corresponding diagram 9
The schematic diagram of the second image divided；
Figure 11 is the schematic diagram of the have no occluder image according to image mosaic shown in Fig. 9 and Figure 10；
Figure 12 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of one embodiment of the invention；
Figure 13 is the structural representation of unobstructed image generation unit in one embodiment of the invention；
Figure 14 is the structural representation of unobstructed image generation unit in one embodiment of the invention；
Figure 15 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of another embodiment of the present invention；
Figure 16 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of another embodiment of the present invention；
Figure 17 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of further embodiment of this invention；
Figure 18 is the structural representation replacing image acquisition unit of one embodiment of the invention.
Detailed description of the invention
For making the purpose of the embodiment of the present invention, technical scheme and advantage clearer, below in conjunction with the accompanying drawings to this
Bright embodiment is described in further details.Here, the schematic description and description of the present invention is used for explaining the present invention, but also
Not as a limitation of the invention.In the embodiment of the present invention, the order of each step is only used for the mistake that the method that schematically illustrates is implemented
Journey, sequence of steps can make the appropriate adjustments as required, and this is not construed as limiting by the present invention.
The shelter perspective method based on intelligent glasses of the present invention, when wearing intelligent glasses, creatively
By auxiliary detector, such as external camera, and utilize this auxiliary detector collection be blocked part image, in other words should
Detector can cover the blind area that user causes due to blocking of shelter, the part that then will be blocked and detection
The image that device collects splices, thus obtains the image of blind area.Object Vitrification management, energy will be blocked in the visual field
Enough reach to eliminate the purpose of blind area.
Fig. 1 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of the embodiment of the present invention.Such as Fig. 1 institute
Show, the shelter perspective method based on intelligent glasses of the embodiment of the present invention, it may include step:
S110: gather first image in the user visual field by intelligent glasses and identify the shelter in above-mentioned first image
Image；
S120: gathered second image in the user visual field by least one image capture device；
S130: utilize the screening in above-mentioned first image of partial replacement of corresponding above-mentioned shelter image in above-mentioned second image
Block material image, splicing generates the unobstructed image in the user visual field.
In above-mentioned steps S110, this intelligent glasses can be various different intelligent glasses, and it is typically provided image
Collecting device, such as photographic head.Above-mentioned user typically refers to wear the user of above-mentioned intelligent glasses, due to the figure of intelligent glasses
As collecting device position close with the distance of position of human eye, so user it can be seen that image, intelligent glasses also may be used
To see.Shelter can make the generation blind area, the visual field of user, and the part user being blocked by obstructions cannot be observed, such as
The column of vehicle window can block the partial visual field of occupant.
In one embodiment, the image that user sees with both intelligent glasses can be approximately considered identical, can be ignored it
Therebetween image simple deformation, can simplify image processing process with this.
In another embodiment, both image capture devices of the user visual field i.e. nature visual field and intelligent glasses are seen
The difference of image can be corrected or revise by position relationship based on human eye with the image capture device of intelligent glasses, can with this
To improve the verity of intelligent glasses image on display screen, user can be promoted and experience.
In one embodiment, the user visual field i.e. nature visual field and the seen figure of both image capture devices of intelligent glasses
The difference of picture, it is possible to use twin camera carries out the method for camera calibration and is corrected.Specifically, position of human eye can be placed
One video camera, by this camera acquisition to scaling board image and intelligent glasses camera collection to scaling board image carry out
Contrast, obtains the transformation matrix T of an image.By all images collected through intelligent glasses all warp before being shown to human eye
Cross the conversion of matrix T, collect at position of human eye such that it is able to obtain an approximation and show image.
This first image can be by the image capture device collection on this intelligent glasses.Shelter in this first image
Image can carry out graphical analysis acquisition by the processor in this intelligent glasses or the extra processor arranged, and i.e. may determine that
Whether shelter is there is in the user visual field.Can there is multiple shelter in the user visual field, above-mentioned shelter image is permissible
It is multiple.In one embodiment, certain identification can be carried out by the shelter that the photographic head of intelligent glasses is photographed, from
And judging that the image using which part is replaced, the process using intelligent glasses identification can also be by intelligent glasses self
Whether gyroscope apparatus auxiliary reference, exist shelter in can identifying the visual field with the photographic head of this intelligent glasses, and use
Corresponding image is replaced.Owing to the blind area in the whole visual field can be covered by image collecting device, so shelter periphery
Image and the image being blocked can effectively splice, so can preferably judge the position at place, the visual field and make
It is replaced with corresponding image.
In one embodiment, above-mentioned intelligent glasses can be can Clairvoyant type intelligent glasses.This can Clairvoyant type intelligent glasses
Display screen can allow nature light to penetrate, and can also be able to see with this while ensureing user viewing intelligent glasses display image
To the natural reality visual field.The image produced in above-mentioned intelligent glasses can superpose with the target image in the true visual field, uses
Shelter image in the true visual field of image covering part processed in intelligent glasses, can reach to have an X-rayed shelter with this
Effect.The image being blocked can be obtained by one or more image acquisition device, and wherein, at least 1 image is adopted
Acquisition means can not be blocked by obstructions.
In above-mentioned steps S120, image capture device can be the various equipment that can gather image, such as camera, takes the photograph
As head, infrared image detection device etc..Above-mentioned image capture device may be mounted at various diverse location, is hidden as long as can photograph
The image in the user visual field that block material blocks, in other words, needs to cover the position in the visual field being blocked by obstructions, i.e. covers
Blind area to user, it is possible to collecting the image of the part that is blocked, the most above-mentioned image capture device is arranged on roof
The position at edge, the blind area of car window upright post can be covered.
It is preferred that above-mentioned image capture device is arranged on the back side of above-mentioned shelter, can ensure that with this thing that is blocked hides
The image in the user visual field of gear necessarily can be collected.
In one embodiment, utilize an above-mentioned image capture device the to photograph visual field that all shelters are blocked
During image, an image capture device can be only used to gather above-mentioned second image.In another embodiment, use multiple figure simultaneously
Shoot the image of the different piece in the visual field that shelter is blocked, the gathered figure of different images collecting device as collecting device simultaneously
As sliceable together, generate and comprise the image in all or whole the blocked visual field of shelter.
In one embodiment, the position of above-mentioned image capture device can be relatively fixed with the position of shelter, can with this
To greatly reduce the operand of image procossing, image processing effect can be improved.
In above-mentioned steps S130, in the second image, the part of corresponding above-mentioned shelter image can refer to that shelter is blocked
The image in the user visual field.Such as, car window upright post shelters from the stop-light in the user visual field, then right in the second image
Should the part of above-mentioned shelter image can be the image of stop-light.This unobstructed image understands the figure after perspective shelter
Picture.The image that above-mentioned image capture device gathers can process by transport processor in several ways, such as, pass through wireless or have
Line mode, this processor can be the processor on this intelligent glasses, or the extra processor arranged.Image capture device
The image in the user visual field can being blocked by obstructions with Real-time Collection, and screen display can be shown by real-time update intelligent glasses
Unobstructed image.
In the embodiment of the present invention, included the figure in the user visual field being blocked by obstructions by image capture device collection
Picture, it may be unnecessary to be known a priori by the image being blocked by obstructions.By utilizing corresponding shelter image in above-mentioned second image
Above-mentioned first image of partial replacement in shelter image, and splice generate the user visual field unobstructed image, Ke Yishi
The now perspective whole object of shelter.Consequently, it is possible to can overcome existing intelligent glasses can only for seeing through objects internal structure, and
The shortcoming that can not have an X-rayed whole object, it is also possible to break through existing intelligent glasses and need to be known a priori by the office of internal structure of body image
Sex-limited.
Fig. 2 is the method flow schematic diagram of the unobstructed image that splicing generates the user visual field in one embodiment of the invention.
As in figure 2 it is shown, in above-mentioned steps S130, utilize the partial replacement of corresponding above-mentioned shelter image in above-mentioned second image above-mentioned
Shelter image in first image, splicing generates the method for the unobstructed image in the user visual field, it may include step:
S131: image section and above-mentioned second image in addition to shelter image extract respectively from above-mentioned first image
Multiple fisrt feature points and multiple second feature point, above-mentioned second feature point and above-mentioned fisrt feature point one_to_one corresponding；
S132: according to all above-mentioned fisrt feature points and all above-mentioned second feature points, be calculated and adopted by above-mentioned image
The imaged viewing angle of collection equipment is converted to the image transformation matrix of user perspective；
S133: utilize above-mentioned image transformation matrix that above-mentioned second image is carried out image conversion；
S134: the part of corresponding above-mentioned shelter image and the above-mentioned screening of removal in above-mentioned second image after image is converted
Above-mentioned first image mosaic after block material image generates above-mentioned unobstructed image together.
In above-mentioned steps S131, above-mentioned fisrt feature point and second feature point can be can reflection in respective image
The point of shape, such as the summit of the object in image or flex point, can be obtained by the graded of the intensity of this point.Specifically will
Which point in image, as characteristic point, can depend on the needs.Above-mentioned second feature point and above-mentioned fisrt feature point one_to_one corresponding,
Can refer to that above-mentioned second feature point and above-mentioned fisrt feature point are the point in the user visual field on image corresponding to same point picture.
The quantity of characteristic point the most more, image mosaic the most accurate, in an embodiment, paired characteristic point (fisrt feature point and
Two characteristic points occur in pairs) at least 5 right, this is conducive to the correct and stable of image transformation matrix result.
In one embodiment, all first in the image section in addition to shelter image can be extracted in the first image
Characteristic point, can extract all second feature points in the second image, and the characteristic vector of recordable individual features point and feature
Value, can be carried out by all fisrt feature points extracted and all second feature point foundation characteristic vector and eigenvalue further
Join, and can be used for calculating above-mentioned image transformation matrix by fisrt feature point and the second feature point that the match is successful, the most initially extract
Fisrt feature point and second feature point quantity can be different, such as second feature point quantity is more than fisrt feature point quantity.
In above-mentioned steps S132～S133, above-mentioned second image is that the imaged viewing angle from above-mentioned image capture device is carried out
Shooting, above-mentioned first image is to shoot from user perspective, during this two visual angles difference, above-mentioned second image and above-mentioned first image
In common images part typically have relative deformation.Utilize above-mentioned image transformation matrix that this second image is carried out image change
Change, user perspective can be converted to by the imaged viewing angle as collecting device by this second image, anamorphose problem can be eliminated.
In above-mentioned steps S134, above-mentioned second image after can being converted by above-mentioned first image of contrast and image,
Obtain the part of corresponding above-mentioned shelter image in above-mentioned second image after image conversion；Remove the screening in above-mentioned first image
Block material image；Afterwards, the part of corresponding above-mentioned shelter image and removal shelter in the second image after then image being converted
The first image mosaic after image is together.
Feature extraction algorithm can have a variety of, such as, picture carries out repeatedly virtualization and processes, by the picture after virtualization every time
Carry out difference with figure before virtualization and obtain a vector matrix.So can stack in a cube of formation 3*3*3 and search.In
The value that heart point is the biggest or less than the point of periphery is just characterized a little.Record characteristic vector and this feature of characteristic point simultaneously
Point in the gradient of all directions as the eigenvalue of this feature point.If the eigenvalue of two characteristic points is the most identical with characteristic vector,
Simultaneously with it is believed that the two characteristic point is same point the most mutually correspondence.The matrix of this transformation matrix 3*3.By
Feature point correspondence solves equation the matrix asking minimum variance can obtain this 3*3.
In the present embodiment, by above-mentioned image transformation matrix, above-mentioned second image is carried out image conversion, generate in splicing
Unobstructed image, with this, (intelligent glasses becomes to be possible to prevent the angle in imaged viewing angle and the user visual field of image capture device
As visual angle) inconsistent in the case of, the imaging of the part that is blocked is directly perceived not, causes the confusion in the user visual field, thus affects
The problem of the normal visual field of user.
The figure in addition to shelter image in one embodiment, in above-mentioned steps S131, from above-mentioned first image
As part and above-mentioned second image extract multiple fisrt feature point and the method for multiple second feature point, detailed description of the invention respectively
May is that by the feature extraction algorithm image section in addition to shelter image and above-mentioned second from above-mentioned first image
Image extracts multiple fisrt feature point and multiple second feature point respectively.
Features described above extraction algorithm can extract flex point, flex point can refer to periphery other pixel change differ greatly
Point.For example, if there is a vertical limit on image, it is clear that the right and left on limit all with the pixel of the line at place, limit
Change greatly, but this edge can not become characteristic point i.e. flex point, because limit is a line, a line has too many point, nothing
Method accurately judges that wherein which point can be as flex point.The most such as, if two limits are intersected, then the most necessarily there is an intersection point,
And this intersection point compared with the pixel of its periphery in any direction difference the biggest.And this intersection point is unique one
Individual, it can't produce multiple flex points along limit, so, this intersection point can be chosen as characteristic point, namely flex point.Thus
Understanding, characteristic point can be typically the position of the wedge angle of objects in images.Features described above point includes above-mentioned fisrt feature point and
Two characteristic points.
In one embodiment, in above-mentioned steps S134, remove the limit of above-mentioned first image after above-mentioned shelter image
The setting pixel wide region of edge comprises the above-mentioned shelter image of part.In the present embodiment, the image section of shelter is gone
Remove, retain the marginal portion being blocked simultaneously, image mosaic quality can be improved as the foundation of image mosaic.Getting rid of
During shield portions, non-shield portions can be retained when, when the splicing of image, and the portion that non-shield portions connects with shield portions
Partial image can do certain process display when makes display seem more natural.The value of the marginal portion of this reservation can make
By fixing pixel, or width can be determined according to the difference between the pixel spliced mutually.In one embodiment, above-mentioned edge
The width of part, can be so that smoothing of show of image with this no less than 5 pixels.
In another embodiment, the width of above-mentioned marginal portion can be 0～5 pixel, or can be by the limit of shelter
Edge carries out dilation procedure, and the shelter image making the shelter image removed ratio actual can be larger, with this, can make spliced
Image is more natural and smoother.
Fig. 3 is the method flow signal of the unobstructed image that splicing generates the user visual field in another embodiment of the present invention
Figure.As it is shown on figure 3, the splicing shown in Fig. 2 generates the method for unobstructed image in the user visual field, upper after image is converted
State the part of corresponding above-mentioned shelter image in the second image and remove above-mentioned first image mosaic after above-mentioned shelter image
Before generating above-mentioned unobstructed image (step S134), may further comprise the step of: together
S135: according to imaging angle and the spatial positional information of above-mentioned image capture device, above-mentioned after image is converted
Splicing between the part of corresponding above-mentioned shelter image and the first image after removing above-mentioned shelter image in second image
Position carries out initial alignment.In the present embodiment, spell as image with reference to the information such as angles and positions of external image capture device
The initial positioning information connect can accelerate the speed of image mosaic.
In above-mentioned steps S135, above-mentioned image capture device is the most relatively-stationary with the position of shelter, quilt
There is certain relativeness in the image that the image blocked can collect with image capture device.This spatial positional information can be
Image capture device and the relative tertiary location of shelter, now, when find shelter position after, can be by this space
The information of position is recorded, and with this understanding, may be considered and do not occur relatively to transport between shelter and image capture device
Move, it is possible to navigate to the positional information of shelter and utilize this positional information to carry out image mosaic, such as, certain angle
Shelter is there is, then the information that the image capture device on this angle direction collects can be used to replacement and blocks on direction
Object image.Consequently, it is possible to the operation using intelligent glasses to do image mosaic when, splicing needs search corresponding spy
The scope of the image levying point (fisrt feature point and corresponding second feature point) can be greatly reduced, such that it is able to improve at image
Reason speed.In one embodiment, the gyroscope of intelligent glasses itself can be utilized to provide the information in direction etc. simultaneously, can be in advance
Know the angular relationship that shelter and intelligent glasses user exist, say, that whether user is observed towards the direction of shelter
, whether this can be as showing fluoroscopy images or splice an auxiliary foundation of budget.In one embodiment, can deposit simultaneously
In the case of multiple image capture devices, the relative position between multiple image capture devices can be fixed, and carries out one with this
After the image mosaic of secondary multiple image capture device, it is only necessary to retain image transformation matrix and be applied to new image,
Without image is carried out feature extraction and finds the operation of character pair point every time, image procossing speed can be improved with this
Degree.
In one embodiment, in above-mentioned steps S110, by the shelter in above-mentioned first image of intelligent glasses identification
The method of image, detailed description of the invention may is that the pictorial symbolization on the constant attribute according to shelter or shelter, in identification
State the shelter image in the first image.
Specifically, such as can be according to the borderline region identification shelter image at pictorial symbolization place.First image is at image
Processing procedure can be extracted the border of image, can be so a lot of region by whole first image division, constant attribute or
The region at pictorial symbolization place can be identified as shelter, and the image in this region can be replaced.
In the present embodiment, this pictorial symbolization can be the most again the uniquely tagged arranged on shelter, in order to identify this screening
Block material.Identify that the method for the shelter image in the first image can include the color of shelter, pattern, relative position or feature
Mark etc..Color can refer to the self color relative constancy of shelter, can gathering the color of shelter by a period of time
Limit so object to this color with certain scope can give necessarily to weight in as the judgement of shelter.Pattern and spy
Levy mark essence identical, may refer to meet specific pattern feature as identification, can be by the outline identification at place for blocking after identification
Thing.Position relatively uses the relativeness between mode record shelter and the user of gyroscope thus applies and to screening
In the identification of block material.Shelter is known the algorithm that method for distinguishing can be object identification.
Blocked the feature of object by intelligent glasses record, or on shelter, use pictorial symbolization, or by note
The locus of record shelter, such as by intelligent glasses gyroscope record shelter and intelligent glasses angulation, thus can
This shelter is identified when intelligent glasses is towards this angle.Blind area and shelter itself typically less change, logical
Cross attribute to shelter this feature more constant own to reach shelter must be identified, the standard of shelter identification can be improved
Exactness.Such as, the color of shelter is more fixing, changes hardly, when natural visual field change, shelter self
The feature identification such as image also not change calibrate shelter or blind area.Shelter part in the natural visual field can be recorded
Come and be removed.
In one embodiment, in the visual field, do not find shelter, can be without carrying out the replacement operation of image, it is possible to use intelligence
Pattern captured by glasses photographic head.If through mode intelligent glasses can also use the method not shown directly allow user
Directly observing through eyeglass and do not show, user can be immediately seen the nature visual field through intelligent glasses, these Brilliant Eyes
Mirror can be the intelligent glasses of through mode, such as, be similar to the intelligent glasses of ***glass and hololens.
Fig. 4 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of further embodiment of this invention.Such as Fig. 4
Shown in, the shelter perspective method based on intelligent glasses shown in Fig. 1, may further comprise the step of:
S140: when the picture in the user visual field changes, above-mentioned intelligent glasses updates above-mentioned first image, described
Second image described in image capture device Resurvey to update described second image, and based on update after above-mentioned first image
Above-mentioned unobstructed image is regenerated with described second image after updating.
In the present embodiment, when the natural visual field being blocked changes when, such as user head is subjected to displacement, can
Adjust with the gyro data according to intelligent glasses self and update above-mentioned unobstructed image.Specifically, above-mentioned first can be updated
Image and above-mentioned second image, and based on the second image after the first image after updating and renewal, utilize above-mentioned steps S120
～S130 regenerates unobstructed image.Consequently, it is possible to can realize when the picture in the user visual field changes without hiding
The real-time update of gear image.In one embodiment, above-mentioned image capture device can Real-time Collection the second image, can when needed profit
With the second image of Resurvey to the second image before updating.
In one embodiment, when the visual angle of intelligent glasses user changes when, can first judge New view is
No there is shelter, repeat above-mentioned steps necessary.Judge the method for shelter in the visual field it may be that first, use image to adopt
Collection equipment carries out the most complete splicing to image, forms a width complete image, using this image as template, with reference to intelligent glasses
The gyroscope information of the image photographed and intelligent glasses self, selects the partial image region in template as visual field window,
Visual field window that contrast images collecting device collects and the image that intelligent glasses self photographs, it is judged that shelter.Use
The image that image capture device collects, is replaced shielded image.
Fig. 5 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of yet another embodiment of the invention.Such as Fig. 5
Shown in, the shelter perspective method based on intelligent glasses shown in Fig. 1, may further comprise the step of:
S150: when the visual angle of user changes, above-mentioned intelligent glasses is according to the gyroscope on above-mentioned intelligent glasses
Detection data and the detection data of gyroscope of a setting position determine the need for updating above-mentioned first image；
S160: if so, update above-mentioned first image by above-mentioned intelligent glasses, and based on above-mentioned first image after updating
Regenerate above-mentioned unobstructed image.
In above-mentioned steps S150, setting position can be each position outside intelligent glasses, such as on car.According to intelligence
The detection data of the detection data of the gyroscope on glasses and the gyroscope of this setting position can determine in various situations to be made
The visual angle change situation of user, such as, determine whether user perspective changes during car travels.
In above-mentioned steps S160, when the gyro of the detection data according to the gyroscope on intelligent glasses He this setting position
When the detection data of instrument judge the visual angle change of user more than a set angle, this first figure can be updated by intelligent glasses
Picture, and generate unobstructed image based on the first image after updating, can be when user perspective changes with this, the most more
New unobstructed image.
In other embodiments, if according to the detection data of the gyroscope on intelligent glasses and the gyroscope of this setting position
Detection data judge that the visual angle change of user, less than above-mentioned set angle, can be approximately considered user perspective and become
Change, then can not update the first image.In one embodiment, above-mentioned image capture device can be video camera, at this point it is possible in real time
Gather the second image, and can above-mentioned second image of real-time update, the second image can be always by Resurvey, and can be optionally
For generating above-mentioned have no occluder image.
In the present embodiment, the second image can be with Resurvey, it is also possible to not Resurvey, as long as still can wrap in the second image
Include the user visual field that the shelter in the first image after being updated is blocked.
Fig. 6 is the schematic flow sheet of the shelter perspective method based on intelligent glasses of another embodiment of the present invention.Such as Fig. 6
Shown in, the shelter perspective method based on intelligent glasses shown in Fig. 1, may further comprise the step of:
S170: after the visual angle of user changes, uses above-mentioned second figure of another image capture device Resurvey
Picture, and above-mentioned second image based on Resurvey regenerates above-mentioned unobstructed image.
In the present embodiment, this another image capture device be change relative to the visual angle of user before for gather
State the image capture device for the second image, can be another in above-mentioned multiple image capture device.This another image
Collecting device can collect the image in the user visual field that shelter in the new visual angle of user is blocked.This another image
Collecting device and above-mentioned image capture device can be different on imaging angle and/or position.According to the difference of user perspective,
Different image capture devices is used to gather the second image, it is ensured that all can have an X-rayed under more middle user perspectives and block
Thing.
In one embodiment, the shelter perspective method based on intelligent glasses shown in Fig. 1, above-mentioned shelter is relative to making
The deflection of user is less than 90 degree with the difference of the imaged viewing angle of above-mentioned image capture device.
In the present embodiment, shelter is relative to the difference of the deflection of user Yu the imaged viewing angle of above-mentioned image capture device
Less than 90 degree, can overcome due to the difference between three-dimensional actual space and the image expressing two-dimensional signal, improve perspective
The more accuracy of result.In other embodiments, shelter becomes relative to the deflection of user and above-mentioned image capture device
As the difference at visual angle can be other angles, such as, can determine according to the collectable angle of visual field of image capture device, such as, can be
The 1/2 of the camera lens visual angle of image capture device.In one embodiment, this subtense angle can be 30 degree and less, can obtain with this
More satisfactory image.In another embodiment, image capture device can use 180 degree of wide-angle lens, and now this subtense angle can connect
Closely reach 90 degree, the image of full shot can be obtained by the image mosaic that multiple camera lenses gather.
Fig. 7 is the side of the second image being gathered the user visual field in one embodiment of the invention by multiple image capture devices
Method schematic flow sheet.The number of above-mentioned image capture device is multiple, as it is shown in fig. 7, gathered by multiple image capture devices
The method of second image in the user visual field, it may include step:
S121: gathered the 3rd image in the user visual field by each above-mentioned image capture device；
S122: each above-mentioned 3rd image mosaic is formed together above-mentioned second image of have no occluder.
In above-mentioned steps S121, the visual angle of each above-mentioned image capture device and/or position can each portion identical, can adopt
Collect different above-mentioned 3rd images.
In above-mentioned steps S122, by each 3rd image mosaic to be formed the second image of have no occluder together, can
When an image capture device can not gather the field-of-view image that complete shelter blocks, each image capture device is adopted
The field-of-view image that the shelter of the part of collection blocks is stitched together, and constitutes the field-of-view image that complete shelter blocks, thus
For replacing the shelter image in above-mentioned first image.
In another embodiment, above-mentioned second image threeth image above-mentioned than being spliced to form has broader regarding
Wild image, when the picture in the user visual field changes and/or user perspective changes, if above-mentioned second figure
As still being able to include the image in the visual field that shelter blocks, above-mentioned second image can be updated, if the first image becomes
Change, the image of the first image correspondence position after directly can finding in this second image and updating, can improve with this
Image processing speed.
Fig. 8 is the schematic flow sheet of the method that splicing obtains the second image in one embodiment of the invention.As shown in Figure 8, exist
In above-mentioned steps S122, the method that each above-mentioned 3rd image mosaic is formed together above-mentioned second image of have no occluder, can
Including step:
S1221: the image section in addition to shelter image extracts multiple 3rd spy respectively from each above-mentioned 3rd image
Levy a little, third feature point one_to_one corresponding in different above-mentioned 3rd images；
S1222: according to third feature point in different above-mentioned 3rd images, be calculated by the one-tenth of other image capture devices
Image transition matrix as imaged viewing angle that visual angle effect is one of them image capture device；
S1223: the 3rd image utilizing above-mentioned image transition matrix to be gathered other image capture devices above-mentioned is carried out
Image converts；
S1224: the 3rd image that the 3rd image after being converted by image gathers with one of them image capture device above-mentioned
Splicing, to obtain the image of whole above-mentioned shelter image in corresponding above-mentioned first image.
In the present embodiment, concrete methods of realizing can be similar with the joining method shown in Fig. 2, such as in above-mentioned steps S1221
In, feature point extraction algorithm can be used to extract above-mentioned third feature point.In above-mentioned steps S1224, available each to be spliced
The partial replacement shelter image of the corresponding shelter image in image, cut out, be spliced to form in corresponding above-mentioned first image whole
The image of individual shelter image.
In one specific embodiment, perspective shelter method can include step:
(1) intelligent glasses is used shelter image to be identified, by the identification to target object extrinsic calibration, it is judged that
Go out to need the shelter being had an X-rayed, or the direction by the gyroscope record shelter of intelligent glasses self, or by right
The identification of shelter self reaches to calibrate the purpose of shelter.
(2) external detector is used to collect the image being blocked.
(3) character pair point is found at external image with in there is the natural visual field of shelter.
(4) image transformation matrix and transformed external detector image are obtained.
(5) by image overlay to the former image that there is shelter.
(6) use new images superposition or replace shelter parts of images.
(7) when the natural visual field changes, repeating above operation, on the thing that makes to be blocked, the image of superposition is according to being blocked
Content do not stop to change.
(8), when the intelligent glasses user visual field changes, shelter is first judged whether, if there is no shelter
The most do not carry out image replacement, then repeat (1)-(7) operation if there is shelter.
(9) use whole detector gained image to carry out spliced image as template, will scheme captured by intelligent glasses
Picture and intelligent glasses gyroscope information data are searched in template as window, and the correspondence image obtaining being blocked by obstructions completes
Replace.
The part that the method for the present embodiment, blind area and shelter shelter from is probably real-time change, this part number
According to having real-time；A kind of pivoting results having an X-rayed intelligent glasses and perspective method thereof is the internal structure of object, and this is interior
Portion's structure is the most relatively-stationary, the most also will not can change over time without real-time.The present invention can apply
Have the place of bigger harm in traffic, military blind area, simultaneously because can the object that blocks of real-time perspective, thus ensure
Result show accurately and available.
Fig. 9 is the schematic diagram of the first image that intelligent glasses gathers in one embodiment of the invention.The figure that intelligent glasses is seen
As the image being to see in the natural visual field of user can be approximately considered.As it is shown in figure 9, the first image that intelligent glasses gathers
Shelter image 301 and unshielding object image 302 is comprised in 300.The blind area (corresponding shelter image 301) in the user visual field exists
The right side in the visual field.In other embodiments, blind area be likely to the visual field left side, above, lower section, periphery around or center etc..
In the first image 300 in part i.e. unshielding object image 302 in addition to shelter image 301, fisrt feature can be selected
Point A1, B1, C1, D1, E1, F1, those characteristic points are flex point or intersection point.
Figure 10 be in one embodiment of the invention image capture device collect comprise shelter image portion shown in corresponding diagram 9
The schematic diagram of the second image divided.As shown in Figure 10, the second image 400 collected both had comprised shelter image in corresponding diagram 9
The image of 301, also comprises the image of the most corresponding above-mentioned unshielding object image 302.Second image 400 can find
Second feature point A2, B2, C2, D2, E2, F2 of corresponding fisrt feature point A1, B1, C1, D1, E1, F1.In second image 400
Five-pointed star image has deformed relative to the five-pointed star image shown in Fig. 9, and the second feature point C2 of corresponding fisrt feature point C1
Position has been blocked.As long as the shelter image in the second image 400 does not include shelter image in corresponding first image 300
The part of 301.
The second feature point A2 in fisrt feature point A1, B1, C1, D1, E1, F1 and Figure 10 in Fig. 9, B2, C2, D2, E2,
F2 might not overlap well, after Figure 10 is carried out image conversion, can make fisrt feature point A1, B1, C1, D1, E1, F1
Coincide together well with second feature point A2, B2, C2, D2, E2, F2, in order to realize image mosaic, make spliced image
Meet the natural visual field of human eye.
In other embodiments, it is possible to use multiple image capture devices carry out the collection of image, thus ensure being blocked
The visual field is completely covered with the natural visual field.Can be in the way of using image mosaic between the image that different images collecting device obtains
Image is connected together and obtains above-mentioned second image 400.
Figure 11 is the schematic diagram of the have no occluder image according to image mosaic shown in Fig. 9 and Figure 10.As shown in figure 11,
Utilize the shelter in partial replacement first image 300 of corresponding shelter image 301 in the second image 400 after image conversion
Image 301, can obtain have no occluder image 500, wherein replaces parts of images 501 with dotted line depending on going out.Image mosaic is got up shape
Becoming a width complete image, or can also be shown by the image the being blocked by obstructions virtual image of use, the visual field is dead
Angle can be removed by this means.The image mosaic of the part that is blocked finally collected by image capture device is to the existing visual field
The perspective to shelter is completed on the image of scope.
The shelter perspective method based on intelligent glasses of the embodiment of the present invention, utilizes the camera collection of intelligent glasses to arrive
The image in the user visual field, obtained position and the image of shelter in the visual field by multiple methods analyst identifying, outside utilization
The image in the visual field that the image capture device collection put is blocked, and the figure that the part being blocked is collected with external detector
As being replaced, then carry out registrating, splicing by replacement image and user field-of-view image, consequently, it is possible to use intelligent glasses energy
Enough can see, observing shelter when, the image being blocked by obstructions such that it is able to produce the effect of perspective shelter, with
This dead angle caused by this shelter in can effectively removing the user visual field.
Based on the inventive concept identical with the shelter perspective method based on intelligent glasses shown in Fig. 1, the application implements
Example additionally provides a kind of shelter arrangement for perspective based on intelligent glasses, as example below is above-mentioned.Owing to this is based on Brilliant Eyes
The principle that the shelter arrangement for perspective of mirror solves problem is similar to shelter perspective method based on intelligent glasses, therefore this based on
The enforcement of the shelter arrangement for perspective of intelligent glasses may refer to the enforcement of shelter perspective method based on intelligent glasses, repeats
Part repeats no more.
Figure 12 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of one embodiment of the invention.Such as Figure 12
Shown in, the shelter arrangement for perspective based on intelligent glasses of the embodiment of the present invention, it may include: shelter image acquisition unit 210,
Replacing image acquisition unit 220 and unobstructed image generation unit 230, above-mentioned each unit is linked in sequence.
Shelter image acquisition unit 210 for gathering first image in the user visual field and identifying by intelligent glasses
State the shelter image in the first image.
Replace image acquisition unit 220 for being gathered second figure in the user visual field by least one image capture device
Picture.
Unobstructed image generation unit 230 is for utilizing the part of corresponding above-mentioned shelter image in above-mentioned second image to replace
Changing the shelter image stated in the first image, splicing generates the unobstructed image in the user visual field.
In shelter image acquisition unit 210, this intelligent glasses can be various different intelligent glasses, general on it
It is provided with image capture device, such as photographic head.Above-mentioned user typically refers to wear the user of above-mentioned intelligent glasses, due to intelligence
The image capture device position of glasses is close with the distance of position of human eye, thus user it can be seen that image, intelligence
Glasses it can also be seen that.Shelter can make the generation blind area, the visual field of user, and the part user being blocked by obstructions cannot be seen
Observing, the column of such as vehicle window can block the partial visual field of occupant.
In one embodiment, the image that user sees with both intelligent glasses can be approximately considered identical, can be ignored it
Therebetween image simple deformation, can simplify image processing process with this.
In another embodiment, both image capture devices of the user visual field i.e. nature visual field and intelligent glasses are seen
The difference of image can be corrected or revise by position relationship based on human eye with the image capture device of intelligent glasses, can with this
To improve the verity of intelligent glasses image on display screen, user can be promoted and experience.
In one embodiment, the user visual field i.e. nature visual field and the seen figure of both image capture devices of intelligent glasses
The difference of picture, it is possible to use twin camera carries out the method for camera calibration and is corrected.Specifically, position of human eye can be placed
One video camera, by this camera acquisition to scaling board image and intelligent glasses camera collection to scaling board image carry out
Contrast, obtains the transformation matrix T of an image.By all images collected through intelligent glasses all warp before being shown to human eye
Cross the conversion of matrix T, collect at position of human eye such that it is able to obtain an approximation and show image.
This first image can be by the image capture device collection on this intelligent glasses.Shelter in this first image
Image can carry out graphical analysis acquisition by the processor in this intelligent glasses or the extra processor arranged, and i.e. may determine that
Whether shelter is there is in the user visual field.Can there is multiple shelter in the user visual field, above-mentioned shelter image is permissible
It is multiple.In one embodiment, certain identification can be carried out by the shelter that the photographic head of intelligent glasses is photographed, from
And judging that the image using which part is replaced, the process using intelligent glasses identification can also be by intelligent glasses self
Whether gyroscope apparatus auxiliary reference, exist shelter in can identifying the visual field with the photographic head of this intelligent glasses, and use
Corresponding image is replaced.Owing to the blind area in the whole visual field can be covered by image collecting device, so shelter periphery
Image and the image being blocked can effectively splice, so can preferably judge the position at place, the visual field and make
It is replaced with corresponding image.
In one embodiment, above-mentioned intelligent glasses can be can Clairvoyant type intelligent glasses.This can Clairvoyant type intelligent glasses
Display screen can allow nature light to penetrate, and can also be able to see with this while ensureing user viewing intelligent glasses display image
To the natural reality visual field.The image produced in above-mentioned intelligent glasses can superpose with the target image in the true visual field, uses
Shelter image in the true visual field of image covering part processed in intelligent glasses, can reach to have an X-rayed shelter with this
Effect.The image being blocked can be obtained by one or more image acquisition device, and wherein, at least 1 image is adopted
Acquisition means can not be blocked by obstructions.
In replacing image acquisition unit 220, image capture device can be the various equipment that can gather image, such as
Camera, photographic head, infrared image detection device etc..Above-mentioned image capture device may be mounted at various diverse location, as long as can clap
Take the photograph the image in the user visual field being blocked by obstructions, in other words, need to cover the position in the visual field being blocked by obstructions
Put, i.e. cover the blind area of user, it is possible to collecting the image of the part that is blocked, the most above-mentioned image capture device sets
Put the position at edge at roof, the blind area of car window upright post can be covered.
It is preferred that above-mentioned image capture device is arranged on the back side of above-mentioned shelter, can ensure that with this thing that is blocked hides
The image in the user visual field of gear necessarily can be collected.
In one embodiment, utilize an above-mentioned image capture device the to photograph visual field that all shelters are blocked
During image, an image capture device can be only used to gather above-mentioned second image.In another embodiment, use multiple figure simultaneously
Shoot the image of the different piece in the visual field that shelter is blocked, the gathered figure of different images collecting device as collecting device simultaneously
As sliceable together, generate and comprise the image in all or whole the blocked visual field of shelter.
In one embodiment, the position of above-mentioned image capture device can be relatively fixed with the position of shelter, can with this
To greatly reduce the operand of image procossing, image processing effect can be improved.
In unobstructed image generation unit 230, in the second image, the part of corresponding above-mentioned shelter image can refer to hide
The image in the user visual field that block material is blocked.Such as, car window upright post shelters from the stop-light in the user visual field, then and
In two images, the part of corresponding above-mentioned shelter image can be the image of stop-light.This unobstructed image understands perspective and hides
Image after block material.The image that above-mentioned image capture device gathers can process by transport processor in several ways, the most logical
Crossing wirelessly or non-wirelessly mode, this processor can be the processor on this intelligent glasses, or the extra processor arranged.Figure
As the image in the user visual field that collecting device can be blocked by obstructions with Real-time Collection, and can show by real-time update intelligent glasses
The unobstructed image of screen display.
In the embodiment of the present invention, included the figure in the user visual field being blocked by obstructions by image capture device collection
Picture, it may be unnecessary to be known a priori by the image being blocked by obstructions.By utilizing corresponding shelter image in above-mentioned second image
Above-mentioned first image of partial replacement in shelter image, and splice generate the user visual field unobstructed image, Ke Yishi
The now perspective whole object of shelter.Consequently, it is possible to can overcome existing intelligent glasses can only for seeing through objects internal structure, and
The shortcoming that can not have an X-rayed whole object, it is also possible to break through existing intelligent glasses and need to be known a priori by the office of internal structure of body image
Sex-limited.
Figure 13 is the structural representation of unobstructed image generation unit in one embodiment of the invention.As shown in figure 13, above-mentioned
Unobstructed image generation unit 230 comprises the steps that feature point extraction module 231, transformation matrix generation module 232, image convert mould
Block 233 and unobstructed image generation module 234, above-mentioned each sequence of modules connects.
Feature point extraction module 231 is for image section in addition to shelter image from above-mentioned first image and above-mentioned
Second image extracts multiple fisrt feature point and multiple second feature point, above-mentioned second feature point and above-mentioned fisrt feature point respectively
One_to_one corresponding.
Transformation matrix generation module 232 is for according to all above-mentioned fisrt feature points and all above-mentioned second feature points, meter
Calculate and obtain being converted to by the imaged viewing angle of above-mentioned image capture device the image transformation matrix of user perspective.
Image transform module 233 is used for utilizing above-mentioned image transformation matrix that above-mentioned second image is carried out image conversion.
Corresponding above-mentioned shelter figure in the unobstructed image generation module 234 above-mentioned second image after image is converted
Above-mentioned unobstructed image is generated together with the part of picture above-mentioned first image mosaic after removing above-mentioned shelter image.
In feature point extraction module 231, above-mentioned fisrt feature point and second feature point can be can in respective image
The point of reflection shape, such as the summit of the object in image or flex point, can be obtained by the graded of the intensity of this point.
Which, specifically using the point in image as characteristic point, can depend on the needs.Above-mentioned second feature point and above-mentioned fisrt feature point one
One is corresponding, can refer to that above-mentioned second feature point and above-mentioned fisrt feature point are the image in the user visual field corresponding to same point picture
On point.The quantity of characteristic point the most more, image mosaic the most accurate, in an embodiment, paired characteristic point (fisrt feature
Point and second feature point occur in pairs) at least 5 right, this is conducive to the correct and stable of image transformation matrix result.
In one embodiment, all first in the image section in addition to shelter image can be extracted in the first image
Characteristic point, can extract all second feature points in the second image, and the characteristic vector of recordable individual features point and feature
Value, can be carried out by all fisrt feature points extracted and all second feature point foundation characteristic vector and eigenvalue further
Join, and can be used for calculating above-mentioned image transformation matrix by fisrt feature point and the second feature point that the match is successful, the most initially extract
Fisrt feature point and second feature point quantity can be different, such as second feature point quantity is more than fisrt feature point quantity.
In transformation matrix generation module 232 and image transform module 233, above-mentioned second image is from above-mentioned image acquisition
The imaged viewing angle of equipment shoots, and above-mentioned first image is to shoot from user perspective, during this two visual angles difference, and above-mentioned second
Common images part in image and above-mentioned first image typically has relative deformation.Utilize above-mentioned image transformation matrix to this
Two images carry out image conversion, can be converted to user perspective by as the imaged viewing angle of collecting device by this second image, permissible
Eliminate anamorphose problem.
In unobstructed image generation module 234, above-mentioned after can converting by contrasting above-mentioned first image and image
Second image, obtains the part of corresponding above-mentioned shelter image in above-mentioned second image after image conversion；Remove above-mentioned first
Shelter image in image；Afterwards, then by image convert after the second image in corresponding above-mentioned shelter image part with
Remove the first image mosaic after shelter image together.
In the present embodiment, by above-mentioned image transformation matrix, above-mentioned second image is carried out image conversion, generate in splicing
Unobstructed image, with this, (intelligent glasses becomes to be possible to prevent the angle in imaged viewing angle and the user visual field of image capture device
As visual angle) inconsistent in the case of, the imaging of the part that is blocked is directly perceived not, causes the confusion in the user visual field, thus affects
The problem of the normal visual field of user.
In one embodiment, features described above point extraction module 231 comprises the steps that feature extraction algorithm module 2311.
Feature extraction algorithm module 2311 for by feature extraction algorithm from above-mentioned first image except shelter figure
Image section and above-mentioned second image outside Xiang extract multiple fisrt feature point and multiple second feature point respectively.
Features described above extraction algorithm can extract flex point, flex point can refer to periphery other pixel change differ greatly
Point.For example, if there is a vertical limit on image, it is clear that the right and left on limit all with the pixel of the line at place, limit
Change greatly, but this edge can not become characteristic point i.e. flex point, because limit is a line, a line has too many point, nothing
Method accurately judges that wherein which point can be as flex point.The most such as, if two limits are intersected, then the most necessarily there is an intersection point,
And this intersection point compared with the pixel of its periphery in any direction difference the biggest.And this intersection point is unique one
Individual, it can't produce multiple flex points along limit, so, this intersection point can be chosen as characteristic point, namely flex point.Thus
Understanding, characteristic point can be typically the position of the wedge angle of objects in images.Features described above point includes above-mentioned fisrt feature point and
Two characteristic points.
In one embodiment, above-mentioned unobstructed image generation module 234 is additionally operable to perform: after removing above-mentioned shelter image
The setting pixel wide region at edge of above-mentioned first image comprise the above-mentioned shelter image of part.
In the present embodiment, the image section of shelter is removed, retains the marginal portion being blocked simultaneously, can be as figure
As the foundation of splicing, improve image mosaic quality.When getting rid of shield portions, non-shield portions can be retained when, at figure
During the splicing of picture, the parts of images that non-shield portions connects with shield portions can do certain process display when and make display
Seem more natural.The value of the marginal portion of this reservation can use fixing pixel, or can according to the pixel spliced mutually it
Between difference determine width.In one embodiment, the width of above-mentioned marginal portion be no less than 5 pixels, with this can so that
It is smooth that image shows.
In another embodiment, the width of above-mentioned marginal portion can be 0～5 pixel, or can be by the limit of shelter
Edge carries out dilation procedure, and the shelter image making the shelter image removed ratio actual can be larger, with this, can make spliced
Image is more natural and smoother.
Figure 14 is the structural representation of unobstructed image generation unit in one embodiment of the invention.As shown in figure 14, above-mentioned
Unobstructed image generation unit 230 may also include that initial alignment module 235, is connected to above-mentioned image transform module 233 and above-mentioned
Between unobstructed image generation module 234.
Initial alignment module 235 is for the imaging angle according to above-mentioned image capture device and spatial positional information, to figure
As in above-mentioned second image after conversion the part of corresponding above-mentioned shelter image with remove first after above-mentioned shelter image
Stitching position between image carries out initial alignment.
In initial alignment module 235, above-mentioned image capture device is the most relatively-stationary with the position of shelter,
There is certain relativeness in the image that the image being blocked can collect with image capture device.This spatial positional information is permissible
The relative tertiary location of image capture device and shelter, now, when find shelter position after, can be the most empty by this
Between the information of position record, with this understanding, may be considered between shelter and image capture device and do not occur relatively
Motion, it is possible to navigate to the positional information of shelter and utilize this positional information to carry out image mosaic, such as, certain angle
There is shelter on direction in degree, then the information that the image capture device on this angle direction collects can be used to replace and hides
Block material image.Consequently, it is possible to the operation using intelligent glasses to do image mosaic when, splicing needs search correspondence
The scope of the image of characteristic point (fisrt feature point and corresponding second feature point) can be greatly reduced, such that it is able to improve image
Processing speed.In one embodiment, the gyroscope of intelligent glasses itself can be utilized to provide the information in direction etc. simultaneously, can be pre-
First know the angular relationship that shelter and intelligent glasses user exist, say, that whether user is seen towards the direction of shelter
Examining, whether this can be as showing fluoroscopy images or splice an auxiliary foundation of budget.In one embodiment, can be simultaneously
In the case of there is multiple image capture device, the relative position between multiple image capture devices can be fixed, and carries out with this
After the image mosaic of the most multiple image capture devices, it is only necessary to retain image transformation matrix and be applied to new image i.e.
Can, it is not necessary to image will be carried out feature extraction every time and find the operation of character pair point, image procossing can be improved with this
Speed.
In one embodiment, above-mentioned shelter image acquisition unit 210 comprises the steps that shelter image collection module 211.
The shelter image collection module 211 pictorial symbolization in the constant attribute according to shelter or shelter, knows
Shelter image in the most above-mentioned first image.
Specifically, such as can be according to the borderline region identification shelter image at pictorial symbolization place.First image is at image
Processing procedure can be extracted the border of image, can be so a lot of region by whole first image division, constant attribute or
The region at pictorial symbolization place can be identified as shelter, and the image in this region can be replaced.
In the present embodiment, this pictorial symbolization can be the most again the uniquely tagged arranged on shelter, in order to identify this screening
Block material.Identify that the method for the shelter image in the first image can include the color of shelter, pattern, relative position or feature
Mark etc..Color can refer to the self color relative constancy of shelter, can gathering the color of shelter by a period of time
Limit so object to this color with certain scope can give necessarily to weight in as the judgement of shelter.Pattern and spy
Levy mark essence identical, may refer to meet specific pattern feature as identification, can be by the outline identification at place for blocking after identification
Thing.Position relatively uses the relativeness between mode record shelter and the user of gyroscope thus applies and to screening
In the identification of block material.Shelter is known the algorithm that method for distinguishing can be object identification.
Blocked the feature of object by intelligent glasses record, or on shelter, use pictorial symbolization, or by note
The locus of record shelter, such as by intelligent glasses gyroscope record shelter and intelligent glasses angulation, thus can
This shelter is identified when intelligent glasses is towards this angle.Blind area and shelter itself typically less change, logical
Cross attribute to shelter this feature more constant own to reach shelter must be identified, the standard of shelter identification can be improved
Exactness.Such as, the color of shelter is more fixing, changes hardly, when natural visual field change, shelter self
The feature identification such as image also not change calibrate shelter or blind area.Shelter part in the natural visual field can be recorded
Come and be removed.
In one embodiment, in the visual field, do not find shelter, can be without carrying out the replacement operation of image, it is possible to use intelligence
Pattern captured by glasses photographic head.If through mode intelligent glasses can also use the method not shown directly allow user
Directly observing through eyeglass and do not show, user can be immediately seen the nature visual field through intelligent glasses, these Brilliant Eyes
Mirror can be the intelligent glasses of through mode, such as, be similar to the intelligent glasses of ***glass and hololens.
Figure 15 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of another embodiment of the present invention.Such as figure
Shown in 15, the shelter arrangement for perspective based on intelligent glasses shown in Figure 12, may also include that the first image update unit 240, with
Above-mentioned unobstructed image generation unit 230 connects.
First image update unit 240 is when the picture in the user visual field changes, and above-mentioned intelligent glasses is more
The most above-mentioned first image, the second image described in described image capture device Resurvey to update described second image, and based on
Above-mentioned first image after renewal and described second image after renewal regenerate above-mentioned unobstructed image.
In the present embodiment, when the natural visual field being blocked changes when, such as user head is subjected to displacement, can
Adjust with the gyro data according to intelligent glasses self and update above-mentioned unobstructed image.Specifically, above-mentioned first can be updated
Image and above-mentioned second image, and regenerate the second figure after unobstructed image and renewal based on the first image after updating
Picture.Consequently, it is possible to the real-time update of unobstructed image can be realized when the picture in the user visual field changes.One reality
Execute in example, above-mentioned image capture device can Real-time Collection the second image, the second image pair of Resurvey can be utilized when needed
The second image before renewal.
Figure 16 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of another embodiment of the present invention.Such as figure
Shown in 16, the shelter arrangement for perspective based on intelligent glasses shown in Figure 12, may also include that the second image update unit 250 He
Unobstructed image update unit 260, the two is connected with each other, and the second image update unit 250 and above-mentioned unobstructed image generate single
Unit 230 connects.
Second image update unit 250 is for when the visual angle of user changes, and above-mentioned intelligent glasses is according to above-mentioned
The detection data of the detection data of the gyroscope on intelligent glasses and the gyroscope of a setting position determine the need for updating
State the first image.
Unobstructed image update unit 260 is used for if so, by above-mentioned intelligent glasses above-mentioned first image of renewal, and based on
Above-mentioned first image after renewal regenerates above-mentioned unobstructed image.
In the second image update unit 250, setting position can be each position outside intelligent glasses, such as on car.
The detection data of the gyroscope of the detection data according to the gyroscope on intelligent glasses and this setting position can be in various situations
The lower visual angle change situation determining user, such as, determine whether user perspective changes during car travels.
In unobstructed image update unit 260, when the detection data according to the gyroscope on intelligent glasses and this setting
When the detection data of the gyroscope of position judge the visual angle change of user more than a set angle, can be updated by intelligent glasses
This first image, and generate unobstructed image based on the first image after updating, can change at user perspective with this
Time, the unobstructed image of real-time update.
In other embodiments, if according to the detection data of the gyroscope on intelligent glasses and the gyroscope of this setting position
Detection data judge that the visual angle change of user, less than above-mentioned set angle, can be approximately considered user perspective and become
Change, then can not update the first image.
In the present embodiment, the second image can be with Resurvey, it is also possible to not Resurvey, as long as still can wrap in the second image
Include the user visual field that the shelter in the first image after being updated is blocked.In one embodiment, above-mentioned image is adopted
Collection equipment can be video camera, at this point it is possible to Real-time Collection the second image, and can above-mentioned second image of real-time update, the second figure
Picture can be always by Resurvey, and may be optionally used for generating above-mentioned have no occluder image.
Figure 17 is the structural representation of the shelter arrangement for perspective based on intelligent glasses of further embodiment of this invention.Such as figure
Shown in 17, the shelter arrangement for perspective based on intelligent glasses shown in Figure 12, may also include that the 3rd image update unit 270, with
Above-mentioned unobstructed image generation unit 230 connects.
3rd image update unit 270, for after the visual angle of user changes, uses another image capture device
Above-mentioned second image of Resurvey, and above-mentioned second image based on Resurvey regenerates above-mentioned unobstructed image.
In the present embodiment, this another image capture device be change relative to the visual angle of user before for gather
State the image capture device for the second image, can be another in above-mentioned multiple image capture device.This another image
Collecting device can collect the image in the user visual field that shelter in the new visual angle of user is blocked.This another image
Collecting device and above-mentioned image capture device can be different on imaging angle and/or position.According to the difference of user perspective,
Different image capture devices is used to gather the second image, it is ensured that the visual angle of user perspective shelter is wider.
In one embodiment, above-mentioned replacement image acquisition unit 220 is additionally operable to perform: above-mentioned shelter is relative to user
The difference 90 degree of deflection and the imaged viewing angle of above-mentioned image capture device.
In the present embodiment, shelter is relative to the difference of the deflection of user Yu the imaged viewing angle of above-mentioned image capture device
Less than 90 degree, can overcome due to the difference between three-dimensional actual space and the image expressing two-dimensional signal, improve perspective
The more accuracy of result.In other embodiments, shelter becomes relative to the deflection of user and above-mentioned image capture device
As the difference at visual angle can be other angles, such as, can determine according to the collectable angle of visual field of image capture device, such as, can be
The 1/2 of the camera lens visual angle of image capture device.In one embodiment, this subtense angle can be 30 degree and less, can obtain with this
More satisfactory image.In another embodiment, image capture device can use 180 degree of wide-angle lens, and now this subtense angle can connect
Closely reach 90 degree, the image of full shot can be obtained by the image mosaic that multiple camera lenses gather.
Figure 18 is the structural representation replacing image acquisition unit of one embodiment of the invention.As shown in figure 18, above-mentioned figure
As the number of collecting device is multiple, above-mentioned replacement image acquisition unit 220 comprises the steps that the 3rd image collection module 221 and
Two image collection module 222, the two is connected with each other.
3rd image collection module 221 for gathering the 3rd figure in the user visual field by each above-mentioned image capture device
Picture.
Second image collection module 222 for forming above-mentioned the of have no occluder together by each above-mentioned 3rd image mosaic
Two images.
In the 3rd image collection module 221, the visual angle of each above-mentioned image capture device and/or position can each portion phases
With, different above-mentioned 3rd images can be gathered.
In the second image collection module 222, by each 3rd image mosaic is formed the second of have no occluder together
Image, can adopt each image when an image capture device can not gather the field-of-view image that complete shelter blocks
The field-of-view image that the shelter of the part of collection equipment collection blocks is stitched together, and constitutes the cyclogram that complete shelter blocks
Picture, thus for replacing the shelter image in above-mentioned first image.
In another embodiment, above-mentioned second image threeth image above-mentioned than being spliced to form has broader regarding
Wild image, when the picture in the user visual field changes and/or user perspective changes, if above-mentioned second figure
As still being able to include the image in the visual field that shelter blocks, above-mentioned second image can be updated, if the first image becomes
Change, the image of the first image correspondence position after directly can finding in this second image and updating, can improve with this
Image processing speed.
The shelter arrangement for perspective based on intelligent glasses of the embodiment of the present invention, utilizes the camera collection of intelligent glasses to arrive
The image in the user visual field, obtained position and the image of shelter in the visual field by multiple methods analyst identifying, outside utilization
The image in the visual field that the image capture device collection put is blocked, and the figure that the part being blocked is collected with external detector
As being replaced, then carry out registrating, splicing by replacement image and user field-of-view image, consequently, it is possible to use intelligent glasses energy
Enough can see, observing shelter when, the image being blocked by obstructions such that it is able to produce the effect of perspective shelter, with
This dead angle caused by this shelter in can effectively removing the user visual field.
In the description of this specification, reference term " embodiment ", " specific embodiment ", " some enforcements
Example ", " such as ", " example ", the description of " concrete example " or " some examples " etc. mean to combine this embodiment or example describes
Specific features, structure, material or feature are contained at least one embodiment or the example of the present invention.In this manual,
The schematic representation of above-mentioned term is not necessarily referring to identical embodiment or example.And, the specific features of description, knot
Structure, material or feature can combine in any one or more embodiments or example in an appropriate manner.
Those skilled in the art are it should be appreciated that embodiments of the invention can be provided as method, system or computer program
Product.Therefore, the reality in terms of the present invention can use complete hardware embodiment, complete software implementation or combine software and hardware
Execute the form of example.And, the present invention can use at one or more computers wherein including computer usable program code
The upper computer program product implemented of usable storage medium (including but not limited to disk memory, CD-ROM, optical memory etc.)
The form of product.
The present invention is with reference to method, equipment (system) and the flow process of computer program according to embodiments of the present invention
Figure and/or block diagram describe.It should be understood that can the most first-class by computer program instructions flowchart and/or block diagram
Flow process in journey and/or square frame and flow chart and/or block diagram and/or the combination of square frame.These computer programs can be provided
Instruction arrives the processor of general purpose computer, special-purpose computer, Embedded Processor or other programmable data processing device to produce
A raw machine so that the instruction performed by the processor of computer or other programmable data processing device is produced for real
The device of the function specified in one flow process of flow chart or multiple flow process and/or one square frame of block diagram or multiple square frame now.
These computer program instructions may be alternatively stored in and computer or other programmable data processing device can be guided with spy
Determine in the computer-readable memory that mode works so that the instruction being stored in this computer-readable memory produces and includes referring to
Make the manufacture of device, this command device realize at one flow process of flow chart or multiple flow process and/or one square frame of block diagram or
The function specified in multiple square frames.
These computer program instructions also can be loaded in computer or other programmable data processing device so that at meter
Perform sequence of operations step on calculation machine or other programmable devices to produce computer implemented process, thus at computer or
The instruction performed on other programmable devices provides for realizing at one flow process of flow chart or multiple flow process and/or block diagram one
The step of the function specified in individual square frame or multiple square frame.
Particular embodiments described above, has been carried out the purpose of the present invention, technical scheme and beneficial effect the most in detail
Describe in detail bright, be it should be understood that the specific embodiment that the foregoing is only the present invention, the guarantor being not intended to limit the present invention
Protect scope, all within the spirit and principles in the present invention, any modification, equivalent substitution and improvement etc. done, should be included in this
Within the protection domain of invention.
Claims (22)
1. a shelter perspective method based on intelligent glasses, it is characterised in that including:
Gather first image in the user visual field by intelligent glasses and identify the shelter image in described first image；
Second image in the user visual field is gathered by least one image capture device；
Utilize the shelter image in the first image described in the partial replacement of corresponding described shelter image in described second image,
Splicing generates the unobstructed image in the user visual field.
2. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that utilize described second figure
Shelter image in first image described in the partial replacement of corresponding described shelter image in Xiang, splicing generates the user visual field
Unobstructed image, including:
From described first image, image section and described second image in addition to shelter image extract multiple first respectively
Characteristic point and multiple second feature point, described second feature point and described fisrt feature point one_to_one corresponding；
According to all described fisrt feature points and all described second feature points, it is calculated by the one-tenth of described image capture device
As the image transformation matrix that visual angle effect is user perspective；
Utilize described image transformation matrix that described second image is carried out image conversion；
The part of corresponding described shelter image and the described shelter image of removal in described second image after image is converted
After described first image mosaic generate described unobstructed image together.
3. shelter perspective method based on intelligent glasses as claimed in claim 2, it is characterised in that from described first image
In the image section in addition to shelter image and described second image extract multiple fisrt feature point and multiple second respectively
Characteristic point, including:
By the feature extraction algorithm image section in addition to shelter image from described first image and described second figure
As extracting multiple fisrt feature point and multiple second feature point respectively.
4. shelter perspective method based on intelligent glasses as claimed in claim 2, it is characterised in that
The setting pixel wide region at the edge removing described first image after described shelter image comprises the described of part
Shelter image.
5. shelter perspective method based on intelligent glasses as claimed in claim 2, it is characterised in that after image is converted
Described second image in corresponding described shelter image part with remove described first image after described shelter image
It is stitched together before the described unobstructed image of generation, also includes:
Imaging angle according to described image capture device and spatial positional information, in described second image after converting image
At the beginning of stitching position between the part of corresponding described shelter image and the first image after removing described shelter image is carried out
Begin to position.
6. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that known by intelligent glasses
Shelter image in the most described first image, including:
Pictorial symbolization in constant attribute according to shelter or shelter, identifies the shelter image in described first image.
7. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that also include:
When picture in the user visual field changes, described intelligent glasses updates described first image, described image acquisition
Second image described in equipment Resurvey to update described second image, and based on update after described first image and update after
Described second image regenerate described unobstructed image.
8. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that also include:
When the visual angle of user changes, described intelligent glasses is according to the detection data of the gyroscope on described intelligent glasses
Determine the need for updating described first image with the detection data of the gyroscope of a setting position；
If so, update described first image by described intelligent glasses, and regenerate based on described first image after updating
Described unobstructed image.
9. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that also include:
After the visual angle of user changes, use the second image described in another image capture device Resurvey, and based on
Described second image of Resurvey regenerates described unobstructed image.
10. shelter perspective method based on intelligent glasses as claimed in claim 1, it is characterised in that described shelter phase
With the difference of the imaged viewing angle of described image capture device, 90 degree are less than for the deflection of user.
11. shelter perspective methods based on intelligent glasses as claimed in claim 1, it is characterised in that described image acquisition
The number of equipment is multiple, is gathered second image in the user visual field by multiple image capture devices, including:
The 3rd image in the user visual field is gathered by each described image capture device；
Each described 3rd image mosaic is formed together described second image of have no occluder.
12. 1 kinds of shelter arrangement for perspective based on intelligent glasses, it is characterised in that including:
Shelter image acquisition unit, for gathering first image in the user visual field and identifying described first by intelligent glasses
Shelter image in image；
Replace image acquisition unit, for being gathered second image in the user visual field by least one image capture device；
Unobstructed image generation unit, for utilizing in described second image described in the partial replacement of corresponding described shelter image
Shelter image in first image, splicing generates the unobstructed image in the user visual field.
13. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that described unobstructed figure
As signal generating unit includes:
Feature point extraction module, is used for the image section in addition to shelter image and described second figure from described first image
As extracting multiple fisrt feature point and multiple second feature point respectively, described second feature point and described fisrt feature point one a pair
Should；
Transformation matrix generation module, for according to all described fisrt feature points and all described second feature points, is calculated
The image transformation matrix of user perspective is converted to by the imaged viewing angle of described image capture device；
Image transform module, is used for utilizing described image transformation matrix that described second image is carried out image conversion；
Unobstructed image generation module, the portion of corresponding described shelter image in described second image after image is converted
Described unobstructed image is generated together with dividing described first image mosaic after removing described shelter image.
14. shelter arrangement for perspective based on intelligent glasses as claimed in claim 13, it is characterised in that described characteristic point carries
Delivery block includes:
Feature extraction algorithm module, for by feature extraction algorithm from described first image in addition to shelter image
Image section and described second image extract multiple fisrt feature point and multiple second feature point respectively.
15. shelter arrangement for perspective based on intelligent glasses as claimed in claim 13, it is characterised in that described unobstructed figure
As generation module is additionally operable to perform:
The setting pixel wide region at the edge removing described first image after described shelter image comprises the described of part
Shelter image.
16. shelter arrangement for perspective based on intelligent glasses as claimed in claim 13, it is characterised in that described unobstructed figure
As signal generating unit also includes:
Initial alignment module, for the imaging angle according to described image capture device and spatial positional information, converts image
After described second image in corresponding described shelter image part with remove the first image after described shelter image it
Between stitching position carry out initial alignment.
17. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that described shelter figure
As acquiring unit includes:
Shelter image collection module, the pictorial symbolization in the constant attribute according to shelter or shelter, identify described
Shelter image in first image.
18. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that also include:
First image update unit, when the picture in the user visual field changes, described intelligent glasses updates described
First image, the second image described in described image capture device Resurvey to update described second image, and based on updating after
Described first image and update after described second image regenerate described unobstructed image.
19. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that also include:
Second image update unit, for when the visual angle of user changes, described intelligent glasses is according to described Brilliant Eyes
The detection data of the detection data of the gyroscope on mirror and the gyroscope of a setting position determine the need for updating described first
Image；
Unobstructed image update unit, for if so, by described intelligent glasses update described first image, and based on update after
Described first image regenerate described unobstructed image.
20. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that also include:
3rd image update unit, for after the visual angle of user changes, uses another image capture device again to adopt
Collect described second image, and described second image based on Resurvey regenerates described unobstructed image.
21. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that described replacement image
Acquiring unit is additionally operable to perform:
Described shelter is less than 90 degree relative to the difference of the deflection of user with the imaged viewing angle of described image capture device.
22. shelter arrangement for perspective based on intelligent glasses as claimed in claim 12, it is characterised in that described image acquisition
The number of equipment is multiple, and described replacement image acquisition unit includes:
3rd image collection module, for gathering the 3rd image in the user visual field by each described image capture device；
Second image collection module, for forming described second figure of have no occluder together by each described 3rd image mosaic
Picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610373513.6A CN106056534B (en) | 2016-05-31 | 2016-05-31 | Intelligent glasses-based method and device for perspective of shelters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610373513.6A CN106056534B (en) | 2016-05-31 | 2016-05-31 | Intelligent glasses-based method and device for perspective of shelters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106056534A true CN106056534A (en) | 2016-10-26 |
| CN106056534B CN106056534B (en) | 2022-03-18 |
Family
ID=57172226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610373513.6A Active CN106056534B (en) | 2016-05-31 | 2016-05-31 | Intelligent glasses-based method and device for perspective of shelters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106056534B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106791393A (en) * | 2016-12-20 | 2017-05-31 | 维沃移动通信有限公司 | A kind of image pickup method and mobile terminal |
| CN107360354A (en) * | 2017-07-31 | 2017-11-17 | 广东欧珀移动通信有限公司 | Photographic method, device, mobile terminal and computer-readable recording medium |
| WO2017206042A1 (en) * | 2016-05-31 | 2017-12-07 | 中国科学院深圳先进技术研究院 | Method and apparatus for seeing through obstruction using smart glasses |
| CN109374636A (en) * | 2018-11-29 | 2019-02-22 | 成都铁安科技有限责任公司 | Pantograph image acquisition system, detection system and detection method |
| CN109587410A (en) * | 2018-12-12 | 2019-04-05 | 北京北电科林电子有限公司 | Panoptic visualization safety defense monitoring system |
| CN109934076A (en) * | 2017-12-19 | 2019-06-25 | 广州汽车集团股份有限公司 | Generation method, device, system and the terminal device of the scene image of vision dead zone |
| CN110059641A (en) * | 2019-04-23 | 2019-07-26 | 重庆工商大学 | Depth birds recognizer based on more preset points |
| CN110390299A (en) * | 2019-07-24 | 2019-10-29 | 江西师范大学 | A kind of the cylinder back side visible system and method for view-based access control model enhancing |
| CN110661978A (en) * | 2019-10-29 | 2020-01-07 | 维沃移动通信有限公司 | Photographing method and electronic equipment |
| CN110738696A (en) * | 2019-08-27 | 2020-01-31 | 中国科学院大学 | Driving blind area perspective video generation method and driving blind area view perspective system |
| CN112896047A (en) * | 2021-03-03 | 2021-06-04 | 上海三一重机股份有限公司 | Auxiliary image generation method and device for working machine and working machine |
| CN113272177A (en) * | 2018-11-15 | 2021-08-17 | 法雷奥开关和传感器有限责任公司 | Method, computer program product, mobile communication device and communication system for providing visual information about at least a part of an environment |
| CN113534281A (en) * | 2020-04-14 | 2021-10-22 | 深圳市博利凌科技有限公司 | Scanner and method for sensing position of hidden object behind surface of object |
| WO2022040988A1 (en) * | 2020-08-26 | 2022-03-03 | 深圳市大疆创新科技有限公司 | Image processing method and apparatus, and movable platform |
| WO2022161261A1 (en) * | 2021-01-28 | 2022-08-04 | 维沃移动通信有限公司 | Image display method and apparatus, and electronic device |
| WO2023029122A1 (en) * | 2021-08-30 | 2023-03-09 | 歌尔股份有限公司 | Method and apparatus for restoring image occlusion area, device, and storage medium |
| TWI810840B (en) * | 2022-03-09 | 2023-08-01 | 圓展科技股份有限公司 | Method for transparentizing target object and image processing system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101589328A (en) * | 2007-11-20 | 2009-11-25 | 松下电器产业株式会社 | Image display device and display packing thereof, program, integrated circuit, glasses type head mounted display, automobile, binoculars and console display |
| CN103761721A (en) * | 2013-12-30 | 2014-04-30 | 西北工业大学 | Stereoscopic vision fast image stitching method suitable for space tether-robot |
| CN104057882A (en) * | 2013-03-22 | 2014-09-24 | 通用汽车环球科技运作有限责任公司 | System For Viewing A Curb In A Front Region On The Basis Of Two Cameras |
| US9197877B2 (en) * | 2011-11-22 | 2015-11-24 | Universitat De Valéncia | Smart pseudoscopic-to-orthoscopic conversion (SPOC) protocol for three-dimensional (3D) display |
| CN105205796A (en) * | 2014-06-30 | 2015-12-30 | 华为技术有限公司 | Wide-area image acquisition method and apparatus |
| CN105511081A (en) * | 2016-02-03 | 2016-04-20 | 深圳纳德光学有限公司 | Head-wearing type two-eye displaying device |
-
2016
- 2016-05-31 CN CN201610373513.6A patent/CN106056534B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101589328A (en) * | 2007-11-20 | 2009-11-25 | 松下电器产业株式会社 | Image display device and display packing thereof, program, integrated circuit, glasses type head mounted display, automobile, binoculars and console display |
| US9197877B2 (en) * | 2011-11-22 | 2015-11-24 | Universitat De Valéncia | Smart pseudoscopic-to-orthoscopic conversion (SPOC) protocol for three-dimensional (3D) display |
| CN104057882A (en) * | 2013-03-22 | 2014-09-24 | 通用汽车环球科技运作有限责任公司 | System For Viewing A Curb In A Front Region On The Basis Of Two Cameras |
| CN103761721A (en) * | 2013-12-30 | 2014-04-30 | 西北工业大学 | Stereoscopic vision fast image stitching method suitable for space tether-robot |
| CN105205796A (en) * | 2014-06-30 | 2015-12-30 | 华为技术有限公司 | Wide-area image acquisition method and apparatus |
| CN105511081A (en) * | 2016-02-03 | 2016-04-20 | 深圳纳德光学有限公司 | Head-wearing type two-eye displaying device |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017206042A1 (en) * | 2016-05-31 | 2017-12-07 | 中国科学院深圳先进技术研究院 | Method and apparatus for seeing through obstruction using smart glasses |
| US10607414B2 (en) | 2016-05-31 | 2020-03-31 | Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences | Method and device for seeing through an obstruction based on smart glasses, and computer-readable storage medium |
| CN106791393A (en) * | 2016-12-20 | 2017-05-31 | 维沃移动通信有限公司 | A kind of image pickup method and mobile terminal |
| CN106791393B (en) * | 2016-12-20 | 2019-05-17 | 维沃移动通信有限公司 | A kind of image pickup method and mobile terminal |
| CN107360354A (en) * | 2017-07-31 | 2017-11-17 | 广东欧珀移动通信有限公司 | Photographic method, device, mobile terminal and computer-readable recording medium |
| CN109934076A (en) * | 2017-12-19 | 2019-06-25 | 广州汽车集团股份有限公司 | Generation method, device, system and the terminal device of the scene image of vision dead zone |
| CN113272177B (en) * | 2018-11-15 | 2024-01-26 | 法雷奥开关和传感器有限责任公司 | Method, computer program product, mobile communication device and communication system for providing visual information about at least a part of an environment |
| CN113272177A (en) * | 2018-11-15 | 2021-08-17 | 法雷奥开关和传感器有限责任公司 | Method, computer program product, mobile communication device and communication system for providing visual information about at least a part of an environment |
| CN109374636A (en) * | 2018-11-29 | 2019-02-22 | 成都铁安科技有限责任公司 | Pantograph image acquisition system, detection system and detection method |
| CN109374636B (en) * | 2018-11-29 | 2021-05-25 | 成都铁安科技有限责任公司 | Pantograph image acquisition system, detection system and detection method |
| CN109587410A (en) * | 2018-12-12 | 2019-04-05 | 北京北电科林电子有限公司 | Panoptic visualization safety defense monitoring system |
| CN110059641A (en) * | 2019-04-23 | 2019-07-26 | 重庆工商大学 | Depth birds recognizer based on more preset points |
| CN110390299A (en) * | 2019-07-24 | 2019-10-29 | 江西师范大学 | A kind of the cylinder back side visible system and method for view-based access control model enhancing |
| CN110738696A (en) * | 2019-08-27 | 2020-01-31 | 中国科学院大学 | Driving blind area perspective video generation method and driving blind area view perspective system |
| CN110738696B (en) * | 2019-08-27 | 2022-09-09 | 中国科学院大学 | Driving blind area perspective video generation method and driving blind area view perspective system |
| CN110661978A (en) * | 2019-10-29 | 2020-01-07 | 维沃移动通信有限公司 | Photographing method and electronic equipment |
| CN113534281A (en) * | 2020-04-14 | 2021-10-22 | 深圳市博利凌科技有限公司 | Scanner and method for sensing position of hidden object behind surface of object |
| WO2022040988A1 (en) * | 2020-08-26 | 2022-03-03 | 深圳市大疆创新科技有限公司 | Image processing method and apparatus, and movable platform |
| WO2022161261A1 (en) * | 2021-01-28 | 2022-08-04 | 维沃移动通信有限公司 | Image display method and apparatus, and electronic device |
| CN112896047A (en) * | 2021-03-03 | 2021-06-04 | 上海三一重机股份有限公司 | Auxiliary image generation method and device for working machine and working machine |
| WO2023029122A1 (en) * | 2021-08-30 | 2023-03-09 | 歌尔股份有限公司 | Method and apparatus for restoring image occlusion area, device, and storage medium |
| TWI810840B (en) * | 2022-03-09 | 2023-08-01 | 圓展科技股份有限公司 | Method for transparentizing target object and image processing system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106056534B (en) | 2022-03-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106056534A (en) | Obstruction perspective method and device based on smart glasses | |
| KR101994121B1 (en) | Create efficient canvas views from intermediate views | |
| EP2966863B1 (en) | Hmd calibration with direct geometric modeling | |
| JP6595726B2 (en) | Transition between binocular and monocular fields | |
| US10607414B2 (en) | Method and device for seeing through an obstruction based on smart glasses, and computer-readable storage medium | |
| JP4692526B2 (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| US20230419438A1 (en) | Extraction of standardized images from a single-view or multi-view capture | |
| JP4936491B2 (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| CN112085659A (en) | Panorama splicing and fusing method and system based on dome camera and storage medium | |
| CN109035307A (en) | Setting regions target tracking method and system based on natural light binocular vision | |
| CN107122754A (en) | Posture identification method and device | |
| CN110278366A (en) | A kind of panoramic picture weakening method, terminal and computer readable storage medium | |
| KR101395388B1 (en) | Apparatus and method for providing augmented reality | |
| US10992928B1 (en) | Calibration system for concurrent calibration of device sensors | |
| CN115223231A (en) | Sight direction detection method and device | |
| CN115810179A (en) | Human-vehicle visual perception information fusion method and system | |
| CN106791800A (en) | The quality diagnosis method and device of panoramic video | |
| CN109151446A (en) | Control method and electronic equipment | |
| JP2003101690A (en) | Image processing method, digital camera, and recording medium | |
| CN110723073B (en) | Automobile A column perspective method | |
| EP4322114A1 (en) | Projective bisector mirror | |
| CN108280802A (en) | Image acquiring method and device based on 3D imagings | |
| CN206615400U (en) | Panoramic view data collection system and device based on double panorama cameras | |
| CN115834858A (en) | Display method and device, head-mounted display equipment and storage medium | |
| KR20230029782A (en) | visual aid |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |