CN101639355A - Three dimensional plane extraction method - Google Patents
Three dimensional plane extraction method Download PDFInfo
- Publication number
- CN101639355A CN101639355A CN200910044192A CN200910044192A CN101639355A CN 101639355 A CN101639355 A CN 101639355A CN 200910044192 A CN200910044192 A CN 200910044192A CN 200910044192 A CN200910044192 A CN 200910044192A CN 101639355 A CN101639355 A CN 101639355A
- Authority
- CN
- China
- Prior art keywords
- overbar
- hypothesis
- plane
- centerdot
- kappa
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a three dimensional plane extraction method. The input of the technical scheme is that any method is used for obtaining a three dimensional line feature set and a sparse digitalelevation model data in a target scene, obtaining stereo image of a target scene from two different visual angles and obtaining all the inner parameters and the outer parameters of the stereoscopic camera as well as the line extracting result of the stereo image; the output of the technical scheme is the three dimensional plane with high positioning precision and favorable boundary integrity andaccuracy. In the treating process of the technical scheme, the positioning of the three dimensional plane and the extraction and separation of the plane zone boundary are divided into two stages whichare connected for independently finishing two tasks and reducing error impact between the two tasks, which causes the results of the two stages to be respectively optimal, so that the three dimensional plane extraction is optimal.
Description
Technical field
The present invention relates to Photogrammetry and Remote Sensing, computer vision field, relate in particular to a kind of extracting method of three-dimensional planar.
Background technology
The man-made target three-dimensional reconstruction has in fields such as city planning, disaster monitoring, communications facility construction very widely to be used, and for a long time, is the important topic that Photogrammetry and Remote Sensing, computer vision field study for a long period of time always.In the target three-dimensional reconstruction scheme of various polyhedral structures, it is one of them committed step that three-dimensional planar extracts.
At present, the method for extracting three-dimensional planar mainly contains four kinds: first kind, utilize intensive digital elevation model (Digital Elevation Model, the vehicle economy M) data that directly obtain, and obtain three-dimensional planar by match; Second kind, by obtaining the target area image under the different illumination conditions, the reflection characteristic model on combining target surface is set up a plurality of reflectograms, utilizes the photometry stereo vision method to calculate the surface direction of each object table millet cake, realizes the reconstruction to target surface; The third, utilize the method for linear feature perception marshalling, suppose the shape in known target surface plane zone, the linear feature that has extracted is carried out the marshalling of ad hoc rules, as profile marshalling at the rectangle plane zone, generate a plurality of hypothesises of plane mechanism, and then carry out the checking of the hypothesis of plane mechanism according to image or other priori; The 4th kind, the linear feature that extracts is mated, calculate corresponding 3 d-line, 3 d-line is carried out the coplane analysis by forward intersection, generate all possible hypothesis of plane mechanism and formation level border to form closed plane domain, utilize various evidences that these enclosed region are confirmed.Comparatively speaking, the shortcoming of first method is the increase along with the spot elevation packing density, and procurement cost can increase simultaneously, and for baroque target, if the general type that lacks plane quantity or object construction is as priori, all planes of extracting target surface are very difficult.The shortcoming of second method is to need accurately known a plurality of illumination conditions, thereby this method is more common in the resurfacing of the small size target under the simple photoenvironment, for the illumination condition complexity, background is various or the target surface reflection characteristic such as there are differences at various non-cooperation situations, this method is difficult to realize satisfactory results.The shortcoming of the third method is that the geometric configuration that can extract the plane is restricted.Than first three methods, the 4th kind of method applicability is stronger, and little to the dependence of altitude figures, even can not need the support of altitude figures, simultaneously, this method adopts a kind of bottom-up leaching process, and the three-dimensional planar that can solve various polygonal shapes extracts.
In the three-dimensional planar leaching process, the three-dimensional planar information that need obtain comprises two aspects: the one, and the three-dimensional planar locating information shows as the plane equation that the plane is opened at its place; The 2nd, the three-dimensional planar boundary information, the complete sum that shows as closed plane is the boundary line accurately.In existing the 4th kind of three-dimensional planar extractive technique, to extract these two kinds of information finishes simultaneously as a task, the bearing accuracy on plane can be subjected to the influence of boundary alignment error, and can be reduced because of the positional parameter error on plane by the boundary alignment precision that Plane intersects obtains.
Summary of the invention
The technical problem to be solved in the present invention is in the three-dimensional planar leaching process, the location of three-dimensional planar and the extraction on plane domain border are separated into two successive stages, independently finish this two tasks respectively, reduce the error effect between them, make the result in two stages reach optimum respectively, thereby realize the optimum that three-dimensional planar extracts.
The stereo-picture of the object scene that the input of technical solution of the present invention is 3 d-line feature set in the object scene that obtains with any means and sparse digital elevation model data, obtain from two different visual angles and obtain all inside and outside parameters of their stereoscopic camera, the straight line of stereoscopic image extracts the result, the output of technical scheme is bearing accuracy height, border integrality and the good three-dimensional planar of accuracy, and the purpose of technical application scheme is that the resurfacing work for follow-up polyhedral structure target lays the first stone.
Thinking of the present invention is: the three-dimensional planar leaching process is divided into two successive optimization step, first step is to obtain the hi-Fix information of the open three-dimensional planar in target surface place, implementation method is a kind of hypothesis substantive approach of cutting apart, idiographic flow is that the member's linear feature that utilizes the open three-dimensional planar of 3 d-line feature generation to suppose → utilize that three-dimensional planar is supposed is cut apart the flat blocks hypothesis that optimal planar piece hypothesis → merging optimum is supposed → chosen to the open a plurality of flat blocks of three-dimensional planar hypothesis formation, the open three-dimensional planar of generation high position precision; Second step is the border of formation level, based on the open three-dimensional planar that generates, the plane intersection line that obtains with Plane intersects, linear feature that extracts in the image and the heuristic knowledge relevant with object construction are as the foundation that generates planar boundary, be the complete border of open planar configuration, idiographic flow is the linear feature that extracts in the stereo-picture of the border that utilizes the member's linear feature and the interplanar intersection formation level on open plane → utilize and the heuristic rule planar boundary that replenishes disappearance → generate the closed hypothesis of plane mechanism by planar boundary, utilizes optimization method to select the optimum closed hypothesis of plane mechanism.
Technical scheme of the present invention is a kind of three dimensional plane extraction method, specifically comprises the steps:
Two width of cloth images of the different visual angles of known a certain scene are called left and right image, and all inside and outside parameters of absorbing the left and right camera of this two width of cloth image, and the coordinate of wherein left and right camera shooting center in world coordinate system is designated as (X respectively
O1, Y
O1, Z
O1) and (X
O2, Y
O2, Z
O2); Linear feature collection by extracting in this two width of cloth image is designated as H respectively
1And H
2N in known this scene
SBar 3 d-line feature is designated as S={L
i| i=1,2 ..., N
S, the arbitrary element L in this set
iRepresent a three-dimensional straight line segment, its two end points are respectively (X
I1, Y
I1, Z
I1) and (X
I2, Y
I2, Z
I2); Sparse dem data in known this scene is designated as the spot elevation set
The average error of element is λ in this set.
The first step is found the solution the open three-dimensional planar at target surface place
In (one) step, the three-dimensional planar hypothesis generates
The set angle threshold phi is approximate consistent if the angle of two 3 d-lines, is then judged their direction vector less than φ; Setpoint distance thresholding ε, if three dimensions o'clock to the distance of a 3 d-line or a three-dimensional planar less than ε, judge that then this three dimensions point is on this 3 d-line or on this three-dimensional planar.The value of two thresholdings is decided according to known 3 d-line features location precision.
The angular range of two adjacent boundary place straight lines on any one plane on target setting surface is
Minimum angle between any two adjacent planes of target surface is
Their value is relevant with the design feature of target surface.
In the 1st step, calculate the three-dimensional planar hypothesis
3 d-line feature among the pair set S makes up in twos, generates the three-dimensional planar hypothesis, and concrete steps are:
Optional two 3 d-line feature L
u∈ S and L
v∈ S, L
uEnd points be (X
U1, Y
U1, Z
U1) and (X
U2, Y
U2, Z
U2), L
vEnd points be (X
V1, Y
V1, Z
V1) and (X
V2, Y
V2, Z
V2), utilize their objective definition function f (a, b, c, d):
f(a,b,c,d)=(a·X
u1+b·Y
u1+c·Z
u1+d)
2+(a·X
u2+b·Y
u2+c·Z
u2+d)
2+(a·X
v1+b·Y
v1+c·Z
v1+d)
2+(a·X
v2+b·Y
v2+c·Z
v2+d)
2
Adopt the gradient descent method calculate f (a, b, c, when d) getting minimum value (c d), obtains by L for a, b
uAnd L
vThe three-dimensional planar hypothesis equation of appointment is aX+bY+cZ+d=0.
Suppose set omega={ P according to all three-dimensional planars that above-mentioned steps obtains
i| i=1,2 ..., N} is for any one three-dimensional planar hypothesis P wherein
i, plane equation is a
iX+b
iY+c
iZ+d
i=0, two 3 d-line features that are used to generate it are called P
iMember's linear feature.
In the 2nd step, the three-dimensional planar that merges coplane is supposed
For any one three-dimensional planar hypothesis P
i∈ Ω, the mid point of two member's linear feature mid point lines that calculates it is in plane P
iOn projection, be designated as (X
M i, Y
M i, Z
M i).
The 1st) step, judge the coplanar relation that three-dimensional planar is supposed
For any two three-dimensional planars hypothesis P
i∈ Ω and P
j∈ Ω is if the inequality group below satisfying is then judged P
iAnd P
jCoplane:
The 2nd) step, the three-dimensional planar hypothesis of search coplane
Generate a non-directed graph G, with each three-dimensional planar hypothesis P
i∈ Ω, then keeps between the node of their correspondences connecting, otherwise does not connect if any two three-dimensional planars hypothesis is a coplane as a node.
Calculate all very big groups of non-directed graph G, be designated as { Q
W| w=1,2 ..., N
Q, any one greatly rolls into a ball Q
wBe the subclass of Ω, be designated as
Any two three-dimensional planars hypothesis all is a coplane in the subclass.
The 3rd) step, the three-dimensional planar that merges coplane is supposed
Greatly roll into a ball Q for any one
w, merge all three-dimensional planar hypothesis that it comprises, obtain a new three-dimensional planar hypothesis P
w, its plane equation is designated as a
wX+b
wY+c
wZ+d
w=0, wherein:
Will
In all member's linear features of each three-dimensional planar hypothesis project to P
wOn, all projection straight line sections that obtain become P
wMember's linear feature.
Greatly roll into a ball the three-dimensional planar hypothesis set that obtains by all and be designated as Ω={ P
w| w=1,2 ..., N}, wherein, N=N
Q, three-dimensional planar is supposed P arbitrarily
wMember's linear feature set be designated as
Any one member's linear feature L
j wEnd points be designated as (X
J, 1 w, Y
J, 1 w, Z
J, 1 w) and (Z
J, 2 w, Y
J, 2 w, Z
J, 2 w).
In (two) step, the three-dimensional planar hypothesis confirms
In the 1st step, cut apart three-dimensional planar and be assumed to be the flat blocks hypothesis
For any one three-dimensional planar hypothesis P
i∈ Ω utilizes its member's linear feature M
iIt is cut apart, obtain some flat blocks hypothesis.Concrete grammar may further comprise the steps:
The 1st) step, member's linear feature of merging conllinear
For P
iAny two member's linear feature L
j i,
Calculate the angle of their place straight lines, be designated as μ, and L
j iMid point to L
k iThe distance of place straight line and L
k iMid point to L
j iThe distance of place straight line is designated as η respectively
1And η
2
If μ<φ and η
1<ε and μ
2<ε then judges L
j iAnd L
k iConllinear generates a new 3 d-line feature with them, and two end points of this 3 d-line feature are L
j iAnd L
k iAt a distance of farthest two end points, add new 3 d-line feature to set M
iIn, and from set M
iMiddle deletion L
j iAnd L
k i
The 2nd) step, calculate P
iThe intersection point of all member's linear features
For P
iAny two member's linear feature L
α i,
If the angle theta of their place straight lines satisfies
Then calculate L
α iAnd L
β iIntersection point or the intersection point of their extended line.
The set that the intersection point of all calculating is formed is designated as R
i
The 3rd) step, generate the non-directed graph that is used for the segmentation plane hypothesis
Generate a non-directed graph, be designated as G
i, its node is by two class dot generation: the one, and intersection point set R
iIn each element, the 2nd, for M
iIn any one member's linear feature, if do not have and M on the extended line outside its end points
iIn the intersection point of other member's linear feature, and this end points is not set R
iIn element, then this end points becomes figure G
iA node.
Non-directed graph G
iIn any two internodal annexations comprise two classes: the one, if the point of their correspondences on same member's linear feature or its extended line, and on the connection line segment of these two points not and figure G
iIn the corresponding point of other node, then their keep to connect; The 2nd, the annexation of structure, building method is as follows:
For any one member's linear feature L
α iIf its one or two end points is figure G
iNode, then handle in two kinds of situation:
First kind of situation, L
α iWith M
iIn any other member's linear feature all non-intersect
Select any member's linear feature
And β ≠ α specifies on its place straight line the G with figure
iIn two corresponding two points of node, these two points are positioned at L
β iThe mid point both sides and the connection line segment between them on not and figure G
iThe middle corresponding point of node is designated as (X with them
1, Y
1, Z
1) and (X
2, Y
2, Z
2), and calculate
Wherein, (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and (X
α, 2 i, Y
α, 2 i, Z
α, 2 i) be L
α iEnd points.For arbitrarily
And l ≠ α, l ≠ β calculate
Wherein, (X
L, 1 i, Y
L, 1 i, Z
L, 1 i) and (X
L, 2 i, Y
L, 2 i, Z
L, 2 i) be L
l iEnd points.If satisfy
Then calculate
If f
5F
6>0, tie point (X then
α, i 1, Y
α, 1 i, Z
α, 1 i) and (X
2, Y
2, Z
2) at figure G
iMiddle corresponding node, and point (X
α, 2 i, Y
α, 2 i, Z
α, 2 i) and (X
1, Y
1, Z
1) at figure G
iMiddle corresponding node.
Second kind of situation, L
α iWith M
iIn at least one other member's linear feature intersect
If (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and figure G
iA node corresponding, select member's linear feature L
α iOr on its extended line with (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) nearest intersection point, be designated as (X
3, Y
3, Z
3), suppose that this intersection point is L
α iWith member's linear feature L
β iIntersection point.Select L
β iOr on its extended line with figure G
iIn the corresponding point of node, if at it and point (X
3, Y
3, Z
3) between line on not and figure G
iThe middle corresponding point of node then is designated as (X with it
4, Y
4, Z
4), if such point has two, then respectively they are designated as (X
4, Y
4, Z
4) and (X
5, Y
5, Z
5).
If (X
4, Y
4, Z
4) be L
β iWith the intersection point of other member's linear feature, suppose that this member's linear feature is L
l i, and L
l iOr exist on its extended line and figure G
iThe middle corresponding point of node, it satisfies and point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) drop on L
β iThe same side, and with point (X
4, Y
4, Z
4) between line on not and figure G
iIn the corresponding point of node, then connect it and point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) at figure G
iMiddle corresponding node; Otherwise, tie point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and point (X
4, Y
4, Z
4) at figure G
iMiddle corresponding node.If (X
5, Y
5, Z
5) exist, then according to (X
4, Y
4, Z
4) same method handles.
If (X
α, 2 i, Y
α, 2 i, Z
α, 2 i) be figure G
iNode, method of then constructing annexation and (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) identical.
The 4th) step, cut apart generation flat blocks hypothesis
Search non-directed graph G
iIn all rings, for any one ring wherein, if the node number that belongs to same member's linear feature or its extended line that it comprises is no more than 2, then generate a flat blocks hypothesis with it, all nodes that it comprises are formed an orderly point set according to its annexation, represent the vertex set of this flat blocks hypothesis.
The set of being cut apart the flat blocks hypothesis that obtains by all planes among the Ω is designated as
Wherein, any one flat blocks hypothesis
Vertex set by it is represented, is designated as
Cut apart among the Ω and obtain
The equation of three-dimensional planar hypothesis be designated as
In the 2nd step, calculate the reliability measure that each flat blocks is supposed
Spot elevation set J and left and right image that the basis of flat blocks hypothesis reliability measure is known calculate each some projection in left and right image among the J, obtain two plane point sets, are designated as respectively
With
Wherein,
With
It is respectively the spot elevation among the J
Subpoint in left and right image.
The arbitrary plane piece is supposed
Calculate its reliability measure, comprise the steps:
Projection plane piece hypothesis
Vertex set to left and right image, remember any summit
Projection in left and right image is respectively
With
The order projection on connection summit in left and right image respectively according to each fixed point in the vertex set forms
Two polygon view fields.
For any spot elevation
If the projection in its left image
Be positioned at
In the view field in left image, and the projection in its right image
Be positioned at
In the view field in right image, then judge spot elevation
Quilt
Cover.
Quilt
The set that all spot elevations that cover are formed is designated as
The quantity of element is α in the set
w
The 2nd) step, calculate the elevation consistance and estimate
If
The spot elevation number α that covers
w>0, then calculate
The elevation consistance estimate for
Wherein,
If α
w=0, then
The 3rd) step, calculate the gray scale similarity measure
All images point is formed a point set in the view field in left image, is designated as K
w L, the gradation of image value of being had a few in them is formed an array, is designated as { h
Wt| t=1,2 ..., o
w.
Foundation
Three pairs of same places in left and right image
With
With
And
With
Set of computations K
w LThe middle same place of all elements point in right image, the set that these same places are formed is designated as K
w R, the gradation of image value of being had a few in them is obtained by interpolation algorithm, and these gray-scale values are formed another array, are designated as { h
Wt' | t=1,2 ..., o
w.Calculate
Gray scale similarity measure between the view field in left and right image is
Wherein,
The 4th) step, calculate
Reliability measure
In the 3rd step, search for reliable flat blocks hypothesis
Divide two steps to finish:
The 1st) step, the reliable flat blocks hypothesis that search has known spot elevation to cover
Set up a non-directed graph, for any one flat blocks hypothesis
If
Then generate a node of this non-directed graph with it, the property value of node is E
wFor any two nodes,, then connect them if all there is not lap in the view field of the flat blocks of their correspondences hypothesis in left and right image.
Calculate all very big groups of this non-directed graph, therefrom choose the set of all flat blocks hypothesis that the very big group of node attribute values sum maximum comprises, be designated as K
1, it represents that chlamydate spot elevation is as the reliable flat blocks hypothesis of weighing foundation.
The 2nd) step, the reliable flat blocks hypothesis that search does not have known spot elevation to cover
Set up a non-directed graph, for any one flat blocks hypothesis
If
And it and K
1In any one flat blocks hypothesis view field in left and right image all do not have lap, then use
Generate a node of this non-directed graph, the property value of node is E
wFor any two nodes,, then connect them if all there is not lap in the view field of the flat blocks of their correspondences hypothesis in left and right image.
Calculate all very big groups of this non-directed graph, therefrom choose the set of all flat blocks hypothesis that the very big group of node attribute values sum maximum comprises, be designated as K
2, the spot elevation that its expression does not have to cover is as weighing foundation and and K
1In all flat blocks suppose coexisting reliable flat blocks hypothesis.
With K
1∪ K
2All flat blocks hypothesis that comprise are as reliable flat blocks hypothesis.
In the 4th step, the reliable flat blocks that merges coplane is supposed
The 1st) step, judge the coplanar relation that flat blocks is supposed
For any two reliable flat blocks hypothesis
If following inequality is formed upright, then judge
With
Coplane:
The 2nd) step, the reliable flat blocks hypothesis of search coplane
Generate a non-directed graph, be designated as
Use K
1∪ K
2In each reliable flat blocks hypothesis generate a node, if any two reliable flat blocks hypothesis are coplanes, then connect the node of their correspondences.
Calculating chart
All very big groups, be designated as
Any one is greatly rolled into a ball
Be figure
A subclass, be designated as
Any two reliable flat blocks hypothesis all are coplanes in this subclass.
The 3rd) step, the reliable flat blocks that merges coplane is supposed
For any one very big group
Merge all reliable flat blocks hypothesis that it comprises, obtain an open plane
Plane equation is designated as
Wherein:
Will
In all summits of each reliable flat blocks hypothesis project to
On, connecting all subpoints in turn according to the order between the summit in the flat blocks hypothesis, all projection straight line sections that obtain become
Member's linear feature.
The open plane of all that obtain after the merging is designated as
Wherein,
The three-dimensional planar real border of their correspondences is unknown, open arbitrarily plane
Plane equation be
The set of its all member's linear features is designated as
Any one member's linear feature
End points be designated as
With
Second step, the border of formation level
In (one) step, utilize member's linear feature on open plane and plane intersection line formation level border to suppose
In the 1st step, generate the plane intersection line set
Judge that they can intersect, calculate their intersection equation, be designated as
The set of being made up of all plane intersection lines is designated as U.
In the 2nd step, judge the collinear relationship between plane intersection line and plane member's linear feature and revise member's linear feature set on open plane
For any open plane
In U, select it with
The intersection on any open plane, if its any member's linear feature
The angle of place straight line and this plane intersection line is less than φ, and
Mid point to the distance of plane intersection line less than ε, then judge
With this plane intersection line conllinear, and from open plane
Member's linear feature set
Middle deletion
By all with
The set formed of the plane intersection line of certain member's linear feature conllinear, be designated as
In the 3rd step, replenish the open plane that lacks
For any one open plane
Any one member's linear feature
Use a kind of method, generate a new open plane by it by the wall scroll 3 d-line generation hypothesis of plane mechanism,
Become its member's linear feature, add this plane to set
In.
The 4th step is at the open plane computations plane intersection line that replenishes
For any one open plane
If it is generated by the wall scroll 3 d-line, then utilize and identical method of the 1st step, calculate it and any other open plane
Intersection, and with this plane intersection line add to the set U in; If this plane intersection line with
The angle of any member's linear feature place straight line less than φ, and
Mid point to the distance of plane intersection line less than ε, then this plane intersection line is added to the straight line set
In, and from set
Member's linear feature of middle this conllinear of deletion.
In the 5th step, add the member linear feature of plane intersection line for open plane
For any plane intersection line among the set U, suppose that it is open plane
With
Intersection, then this plane intersection line is added to
With
Member's linear feature set
With
In go.
The end points of the member's linear feature that increases newly is uncertain at this moment.
In the 6th step, whether arbitrary plane intersection and other member's linear feature intersect in member's linear feature set on the open plane of judgement
For any open plane
Any one member's linear feature, if it belong to the set U, and it and
The angle of any other member's linear feature belong to scope
Judge that then these two member's linear features intersect.
The 7th step is by member's linear feature formation level border on open plane
Method may further comprise the steps:
The 1st) go on foot, split all member's linear feature to two subclass of the face of setting level
For any one open plane
Its all member's linear features are split into two subclass, and a subclass is
Be designated as U
1, another subclass is
Be designated as U
2
The 2nd) step, generate the non-directed graph set
Generate U
2All subclass.
Choose U
2Any one subclass and U
1Form new member's linear feature collection, be designated as U
0, generate a non-directed graph.This non-directed graph is with U
0In each member's linear feature as node, if member's linear feature of any two node correspondences intersects, then they are coupled together.
The 3rd) step, generate planar boundary
Its all Hamilton circles of search in each non-directed graph that previous step obtains, for each unduplicated circle,, calculate the intersection point of member's linear feature of adjacent node correspondence according to the order of connection between the node that wherein comprises, they as the summit, are generated a closed hypothesis of plane mechanism; If there is no the Hamilton circle is then searched for all Hamilton paths, for each unduplicated path, according to the order of connection between the node that wherein comprises, calculate the intersection point of member's linear feature of adjacent node correspondence, they as the summit, are generated a semi-open hypothesis of plane mechanism.
The set that all the closed hypothesises of plane mechanism that generate and the semi-open hypothesis of plane mechanism are formed be designated as Ω '=P '
k| k=1,2 ..., N ' }, any one element P ' wherein
kOrderly vertex set with it is represented, is designated as
If
Expression P '
kBe a closed hypothesis of plane mechanism, otherwise, expression P '
kIt is a semi-open hypothesis of plane mechanism.Any one closed or semi-open hypothesis of plane mechanism P '
kPlane equation be identical with the open plane that generates it, for corresponding with Ω ', this plane equation is rewritten as a '
kX+b '
kY+c '
kZ+d '
k=0.
In (two) step, utilize the disappearance border of extracting the linear feature and the additional semi-open hypothesis of plane mechanism of heuristic rule
For arbitrary plane hypothesis P '
k∈ Ω ' if it is semi-open, then replenishes the border of disappearance for it, at first utilize and extract linear feature H
1And H
2If, can search the linear feature that can be used for replenishing the disappearance border, then this step finishes, and generates one or more closed hypothesises of plane mechanism; Otherwise, utilize heuristic rule to replenish the border of disappearance, generate a closed hypothesis of plane mechanism.Method is as follows:
In the 1st step, utilize the linear feature that extracts in the image to replenish the disappearance border of the semi-open hypothesis of plane mechanism
The 1st) step, calculate P '
kThe region of disappearance border projection in left and right image
Projection P '
kTo left and right image, obtain point set respectively
With
Set the distance threshold h of an expression linear feature integrality
d, its expression, in piece image, if linear feature is not extract completely, the distance between the end points that is extracted to it of its real end points is less than h so
d
Specify P '
kThe zone at projection place in left image, disappearance border be a quadrilateral, be designated as R
k L, its summit is designated as g respectively
1, g
2, g
3And g
4, wherein,
g
3It is line segment
Extended line on to the some g
2Distance be h
dPoint, g
4Be line segment Tl
k 2Tl
k 1Extended line on to the some g
1Distance be h
dPoint, connecting them becomes R
k LFour edges g
1g
2, g
2g
3, g
3g
4And g
4g
1
Utilize similar method, calculate P '
kThe region R at projection place in right image, disappearance border
k R, its four summits are designated as g ' respectively
1, g '
2, g '
3And g '
4
The 2nd) in the step, utilize region R
k LAnd R
k RInterior H
1And H
2In linear feature replenish the disappearance border of the semi-open hypothesis of plane mechanism
For H
1In any one linear feature, if its two end points all drop on R
k LIn, and its extended line and line segment g
2g
3And g
4g
1Intersect, then connect two intersection points and obtain the straight line section, according to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate the three dimensions point of two end points correspondences of this straight-line segment, with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism.For H
2In any one linear feature, if its two end points all drop on R
k RIn, and its extended line and line segment g '
2G '
3And g '
4G '
1Intersect, calculate its same place of end points in left image, in the computing method of same place and the first step of the present invention the 3rd of the 2nd step in (two) step the) the same place computing method in step are identical, connecting two same places is the straight line section, if any disappearance border that searches in this straight-line segment and the left image is conllinear not, then calculate the extended line and the line segment g ' of this linear feature in right image
2G '
3And g '
4G '
1Intersection point, connect these two intersection points and obtain the straight line section, according to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate the three dimensions point of two end points correspondences of this straight-line segment, with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism.
The 2nd step is according to the disappearance border of the additional semi-open hypothesis of plane mechanism of heuristic rule
If respectively with T
k 2T
k 1With
Be shared border and P
k' two adjacent hypothesises of plane mechanism have all obtained complete border, according to the isometric rule of adjacent polygon plane common edge, at T
k 2T
k 1Or 1 Φ of appointment on its extended line
1, make line segment T
k 2Φ
1With with T
k 2T
k 1For the shared border of the adjacent plane on shared border hypothesis isometric,
Or 1 Φ of appointment on its extended line
2, make line segment
With with
For the shared border of the adjacent plane on shared border hypothesis isometric, line segment Φ
1Φ
2Become additional disappearance border.
If respectively with T
k 2T
k 1With
Be shared border and P '
kHave only a hypothesis of plane mechanism to obtain complete border in two adjacent hypothesises of plane mechanism, suppose that this adjacent plane is assumed to be P '
l, and T
k 2T
k 1Be P '
lWith P '
kShared border, then according to the shared border of adjacent polygon plane isometric rule, at T
k 2T
k 1Or specify on its extended line a bit, make line segment T
k 2Φ
1With P '
lShared border isometric,
Or 1 Φ of appointment on its extended line
2, make line segment
With T
k 2Φ
1Isometric, line segment Φ
1Φ
2Become additional disappearance border.
According to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate Φ
1And Φ
2Corresponding three dimensions point is with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism.
The shared border here is meant adjacent two boundary lines that polygon plane is owned together.
The closed hypothesis of plane mechanism among the Ω ' and the closed hypothesis of plane mechanism that is generated by the semi-open hypothesis of plane mechanism among the Ω ' merged becomes closed hypothesis of plane mechanism set, is designated as Ω "={ P "
i| i=1,2 ..., N " }, wherein, any closed hypothesis of plane mechanism P "
iBy n "
iThe bar boundary line is formed.
(3) goes on foot, and chooses the closed hypothesis of plane mechanism of global optimum
The 1st goes on foot, and calculates the reliability measure of the closed hypothesis of plane mechanism
For any one closed hypothesis of plane mechanism P "
i∈ Ω ", utilize formula one to calculate P "
iThe elevation consistance estimate, if P "
iIt is 0 that the known elevation that covers is counted, and then specifying its elevation consistance to estimate is 1/ (2 λ), utilizes formula three, calculates its reliability measure, is designated as E "
iAt P "
iOn the plane, place, specify next-door neighbour outside its every boundary line, with the rectangle plane zone of this boundary line as a long limit, the long length that equals the boundary line, wide half that equals to grow, the elevation consistance of utilizing formula one to calculate this rectangle plane zone is estimated, if it is 0 that the known elevation that it covers is counted, then specifying its elevation consistance to estimate is 1/ (2 λ), utilize formula three, calculate its reliability measure, note is E by the reliability measure in the rectangle plane zone of j bar boundary line correspondence "
I, j, calculate closed hypothesis of plane mechanism P according to following formula "
iThe reliability measure on corresponding true planar border, border be
In the 2nd step, find the solution the optimum closed hypothesis of plane mechanism
Generate a non-directed graph, each closed hypothesis of plane mechanism is a node to be designated as G in "; with set omega ", utilize formula four computation's reliability estimate as with the attribute of the corresponding node of the closed hypothesis of plane mechanism, if all there is not lap in the view field of the closed hypothesis of plane mechanism of any two node correspondences in left image and right image, then connect this two nodes, otherwise do not connect.Calculating chart G " all very big groups, therefrom choose all closed hypothesises of plane mechanism that the very big group of node attribute values sum maximum comprises, as the closed hypothesis of plane mechanism of optimum, promptly three-dimensional planar extracts the result.
Description of drawings
Fig. 1 is the synoptic diagram of three dimensional plane extraction method flow process of the present invention;
Fig. 2 is the synoptic diagram that the elevation consistance of calculating flat blocks hypothesis is estimated;
Fig. 3 is the synoptic diagram that replenishes the disappearance border of the semi-open hypothesis of plane mechanism according to heuristic rule.
Embodiment
Below in conjunction with accompanying drawing the present invention is further explained.
Fig. 1 is the synoptic diagram of the described three dimensional plane extraction method flow process of the inventive method; Technical scheme comprises: the first step, find the solution the open three-dimensional planar at target surface place, and this step was divided into for two steps again, (1) step, the three-dimensional planar hypothesis generates, and this step is divided into two sub-steps again, the 1st step, calculating three-dimensional planar hypothesis, in the 2nd step, the three-dimensional planar that merges coplane is supposed, (two) step, the three-dimensional planar hypothesis confirms, this step is divided into four sub-steps again, in the 1st step, cuts apart three-dimensional planar and is assumed to be the flat blocks hypothesis, the 2nd step, calculate the reliability measure of each flat blocks hypothesis, in the 3rd step, search for the most reliable flat blocks hypothesis, in the 4th step, the reliable flat blocks that merges coplane is supposed; Second step, the border of formation level, this step was divided into for three steps again, (1) step, utilize the member's linear feature and the plane intersection line formation level border hypothesis on open plane, this step is divided into seven sub-steps again, the 1st step, the set of generation plane intersection line, the 2nd step, judge the collinear relationship between plane intersection line and plane member's linear feature and revise member's linear feature set on open plane, the 3rd step, replenish the open plane of disappearance, the 4th step, at the open plane computations plane intersection line that replenishes, the 5th step, add the member linear feature of plane intersection line for open plane, in the 6th step, whether arbitrary plane intersection and other member's linear feature intersect in member's linear feature set on the open plane of judgement, the 7th step, by member's linear feature formation level border on open plane, in (two) step, utilize the disappearance border of extracting the linear feature and the additional semi-open hypothesis of plane mechanism of heuristic rule, this step is divided into two sub-steps again, the 1st step, utilize the linear feature that extracts in the image to replenish the disappearance border of the semi-open hypothesis of plane mechanism, the 2nd goes on foot, replenish the disappearance border of the semi-open hypothesis of plane mechanism according to heuristic rule, (3) goes on foot, and chooses the closed hypothesis of plane mechanism of global optimum, and this step is divided into two sub-steps again, the 1st step, calculate the reliability measure of the closed hypothesis of plane mechanism, in the 2nd step, find the solution the optimum closed hypothesis of plane mechanism.
Fig. 2 is the synoptic diagram that the elevation consistance of the 2nd step calculating flat blocks hypothesis during (two) of the described first step of the inventive method goes on foot is estimated: O
1And O
2Be camera center, the left and right sides, spot elevation
With O
1And O
2Line hand over flat blocks in a F respectively
1And F
2This method is calculated spot elevation respectively
With a F
1, F
2Distance, if these two distances are more little, represent this flat blocks hypothesis and spot elevation
Consistance good more.
Fig. 3 is the synoptic diagram that the 2nd step was utilized the disappearance border of the additional semi-open hypothesis of plane mechanism of heuristic rule in going on foot in (two) of described second step of the inventive method: suppose that the semi-open hypothesis of plane mechanism has two known borders, Fig. 3 a) in, A and B are the closed hypothesises of plane mechanism with integral edge, C is a semi-open hypothesis of plane mechanism, T
k 2T
k 1Be the common edge boundary line of A and C, T
k 2T
k 3Be the common edge boundary line of B and C, dotted line is the disappearance boundary line Φ that replenishes for plane C according to the isometric enlightening rule in common edge boundary line
1Φ
2Fig. 3 b) in, with Fig. 3 in a) different place be that B also is a semi-open hypothesis of plane mechanism, at first, utilizes the isometric rule in common edge boundary line of A and C, specifies an end points Φ of the disappearance boundary line of C
1,, specify another end points Φ of disappearance boundary line again according to the common edge boundary line of B and C and the isometric rule in common edge boundary line of A and C
2, connect them, generate the disappearance boundary line of semi-open hypothesis of plane mechanism C.
Describe the relevant details among the present invention below in detail.
First point calculates the projection of three dimensions point on three-dimensional planar
Projection three-dimensional spatial point described in the present invention is all adopted in the following method to the step of three-dimensional planar, and particular content is:
Suppose that three dimensions point is (X
0, Y
0, Z
0), the three-dimensional planar equation is aX+bY+cZ+d=0, then projection (the X of this three dimensions point on this three-dimensional planar
0', Y
0', Z
0'), wherein
Second point calculates the projection of three dimensions point on left and right image
Projection three-dimensional spatial point described in the present invention is all adopted in the following method to the step of known left and right image, and particular content is:
According to photogrammetric Theory, utilize known left and right image imaging parameter, can obtain the relevant collinearity equation group of every width of cloth image, can calculate the projection of three dimensions point in left and right image according to collinearity equation.
Obtain the method for collinearity equation group specifically referring to Wang Zhizhuo by the image imaging parameter, Principlesof photogrammetry with remote sensing, Press of Wuhan Technical University ofSurveying and Mapping, 1990,18-26.
Thirdly, calculate the projection of three-dimensional straight line segment on three-dimensional planar
Projection three-dimensional straight-line segment described in the present invention is all adopted in the following method to the step of three-dimensional planar, and particular content is:
According to the method for projection three-dimensional spatial point to three-dimensional planar, two end points of projection straight line section are to three-dimensional planar respectively, and connecting two subpoints that obtain is the straight line section, and this straight-line segment is exactly the projection of being asked.
The 4th point calculates the projection of three-dimensional straight line segment on left and right image
Projection three-dimensional straight-line segment described in the present invention is all adopted in the following method to the step of known left and right image, and particular content is:
According to the method for projection three-dimensional straight-line segment to left and right image, two end points of projection straight line section are to left and right image respectively, and connecting two subpoints that obtain in every width of cloth image is the straight line section, and these two straight-line segments are exactly the projection of being asked.
The 5th point, the gradient descent method is calculated plane equation
In (one) step of the first step of the present invention, in the 2nd step, adopt the gradient descent method to calculate by L
uAnd L
vThe method of the plane equation of appointment, particular content is:
Become known for generating two 3 d-line feature L on plane
u∈ S and L
v∈ S, end points is respectively (X
U1, Y
U1, Z
U1) and (X
U2, Y
U2, Z
U2) and (X
V1, Y
V1, Z
V1) and (X
V2, Y
V2, Z
V2), calculate by them and determine that the method for plane equation is made up of following three steps:
The 1st) go on foot, find the solution the initial value of plane equation coefficient
Calculation level (X
0, Y
0, Z
0) and two value D
u, D
v, wherein
Obtain L
uAnd L
vThe normalization direction vector be
With
And their cross product v
u* v
v=(α β γ).
Calculating χ=-(α X
0+ β Y
0+ λ Z
0), obtain a four-vector (α β γ χ); Calculate trivector (α ' β ' γ ')=(v
u+ v
v) * (α β γ), further calculate χ '=-(α ' X
0+ β ' Y
0+ γ ' Z
0) obtain another four-vector (α ' β ' γ ' χ ').
According to objective function
f(a,b,c,d)=(a·X
u1+b·Y
u1+c·Z
u1+d)
2+(a·X
u2+b·Y
u2+c·Z
u2+d)
2+(a·X
v1+b·Y
v1+c·Z
v1+d)
2+(a·X
v2+b·Y
v2+c·Z
v2+d)
2
If f (α β γ χ)>f (α ' β ' γ ' χ '), the initial value of plane equation coefficient
Be (α ' β ' γ ' χ '), otherwise be (α β γ χ).
The 2nd) step, utilize the gradient descent method to calculate plane equation coefficient
Objective function be f (a, b, c, d), the plane equation coefficient initial value is
The iteration convergence condition is
Utilize the gradient descent method to calculate optimum plane equation coefficient.
The computation process of gradient descent method is referring to " principle of optimality, the method and find the solution software " of Yang Mingsheng, Luo Changtong work, Beijing: Science Press,, 23-31 page or leaf in 2006.
The 3rd) in the step, obtain the hypothesis of plane mechanism
Calculate I=f (a
(k), b
(k), c
(k), d
(k)), if I>4 ε
2, then do not generate the hypothesis of plane mechanism; Otherwise obtain hypothesis of plane mechanism equation is a
(k)X+b
(k)Y+c
(k)Z+d
(k)=0.
The 6th point is greatly rolled into a ball the rapid solving method
All very big groups find the solution and all adopt a kind of method for solving of greatly rolling into a ball fast among the present invention, specifically referring to Tomita E, Tanaka A, Takahashia H.The worst-case time complexity for generating allmaximal cliques and computational experiments.Theoretical Computer Science, 2006,363:28-42.
The 7th point, foundation
Three pairs of same places in left and right image
With
With
And
With
Set of computations K
w LIn any some same place in right image
The 2nd step in (two) step of the first step of the present invention the 3rd) in the step, utilize
Three pairs of same places in left and right image
With
With
And
With
Calculate the same place of arbitrfary point in right image in the left projection polygon, particular content is:
Generating vector V and matrix M is respectively
[a
11?a
12?a
13?a
21?a
22?a
23]
T=M
-1V
The 8th point, the first step of the present invention (two) step the 2nd the step the 3rd) step in calculating K
w RThe interpolation algorithm of middle arbitrary image point gray-scale value adopts the bilinear interpolation algorithm.
The 9th point is by the method for the wall scroll 3 d-line generation hypothesis of plane mechanism
In the 3rd step in (one) step in second step of the present invention, utilize wall scroll 3 d-line feature to generate the method for three-dimensional planar hypothesis referring to a kind of method of utilizing many views picture to extract three-dimensional planar and rebuild buildings, be Baillard C, Zisserman A.A plane-sweep strategy for the 3D reconstruction ofbuildings from multiple images.ISPRS Journal of Photogrammetry and RemoteSensing, 2000,33 (B2): 56-62.
The 10th point, search Hamilton circle and Hamilton route method in non-directed graph
In the 7th step in (one) step in second step of the present invention, search Hamilton circle and Hamilton route method are with reference to the political affairs of model benefit in non-directed graph, Wang Yi, Gong's generation just waits " the graph theory guiding " of translating, Beijing: People's Telecon Publishing House, Gary Chartrand, Ping Zhang work .2007,122-136 page or leaf.
Claims (11)
1. three dimensional plane extraction method, the stereo-picture of 3 d-line feature set in the known target scene and sparse digital elevation model data, the object scene obtained from two different visual angles and obtain all inside and outside parameters of their stereoscopic camera, the straight line of stereoscopic image extracts the result, it is characterized in that, specifically comprise the steps:
The first step is obtained the hi-Fix information that three-dimensional planar is opened at the target surface place
Utilize the 3 d-line feature to generate open three-dimensional planar hypothesis, utilize member's linear feature of three-dimensional planar hypothesis to cut apart the open a plurality of flat blocks hypothesis of three-dimensional planar hypothesis formation, utilize a plurality of flat blocks hypothesis to choose optimal planar piece hypothesis, merge optimum flat blocks hypothesis at last, generate open three-dimensional planar;
Second step, the border of formation level
Based on the open three-dimensional planar that generates, utilize the member's linear feature and the interplanar intersection formation level border of open three-dimensional planar, utilize linear feature and the heuristic rule extracted in the stereo-picture to replenish the planar boundary of disappearance again, generate the closed hypothesis of plane mechanism by planar boundary, utilize optimization method to select the optimum closed hypothesis of plane mechanism.
2. three dimensional plane extraction method according to claim 1 is characterized in that:
Two width of cloth images of the different visual angles of known a certain scene are called left and right image, and all inside and outside parameters of absorbing the left and right camera of this two width of cloth image, and the coordinate of wherein left and right camera shooting center in world coordinate system is designated as (X respectively
O1, Y
O1, Z
O1) and (X
O2, Y
O2, Z
O2); Linear feature collection by extracting in this two width of cloth image is designated as H respectively
1And H
2N in known this scene
SBar 3 d-line feature is designated as S={L
i| i=1,2 ..., N
S, the arbitrary element L in this set
iRepresent a three-dimensional straight line segment, its two end points are respectively (X
I1, Y
I1, Z
I1) and (X
I2, Y
I2, Z
I2); Sparse dem data in known this scene is designated as the spot elevation set
The average error of element is λ in this set;
The first step is found the solution the open three-dimensional planar at target surface place
In (one) step, the three-dimensional planar hypothesis generates the set angle threshold phi; Setpoint distance thresholding ε;
The angular range of two adjacent boundary place straight lines on any one plane on target setting surface is
Minimum angle between any two adjacent planes of target surface is
In the 1st step, calculate the three-dimensional planar hypothesis
3 d-line feature among the pair set S makes up in twos, generates three-dimensional planar hypothesis set omega={ P
i| i=1,2 ..., N} is for any one three-dimensional planar hypothesis P wherein
i, plane equation is a
iX+b
iY+c
iZ+d
i=0, two 3 d-line features that are used to generate it are called P
iMember's linear feature;
In the 2nd step, the three-dimensional planar that merges coplane is supposed
The three-dimensional planar of all coplanes hypothesis merges among the pair set Ω, and the three-dimensional planar hypothesis set that obtains is designated as Ω={ P
w| w=1,2 ..., N}, three-dimensional planar hypothesis P arbitrarily
wMember's linear feature set be designated as
Any one member's linear feature L
j wEnd points be designated as (X
J, 1 w, Y
J, 1 w, Z
J, 1 w) and (X
J, 2 w, Y
J, 2 w, Z
J, 2 w);
In (two) step, the three-dimensional planar hypothesis confirms
In the 1st step, cut apart three-dimensional planar and be assumed to be the flat blocks hypothesis
For any one three-dimensional planar hypothesis P
i∈ Ω utilizes its member's linear feature M
iIt is cut apart, obtain some flat blocks hypothesis;
The set of being cut apart the flat blocks hypothesis that obtains by all planes among the Ω is designated as
Wherein, any one flat blocks hypothesis
Vertex set by it is represented, is designated as
Cut apart among the Ω and obtain
The equation of three-dimensional planar hypothesis be designated as
In the 2nd step, calculate the reliability measure that each flat blocks is supposed
Spot elevation set J and left and right image that the basis of flat blocks hypothesis reliability measure is known calculate each some projection in left and right image among the J, obtain two plane point sets, are designated as respectively
With
Wherein,
With
It is respectively the spot elevation among the J
Subpoint in left and right image;
The arbitrary plane piece is supposed
Calculate its reliability measure;
In the 3rd step, search for reliable flat blocks hypothesis;
In the 4th step, the reliable flat blocks that merges coplane is supposed;
The open plane of all that obtain after the merging is designated as
The three-dimensional planar real border of their correspondences is unknown, open arbitrarily plane
Plane equation be
The set of its all member's linear features is designated as
Any one member's linear feature
End points be designated as
With
Second step, the border of formation level
In (one) step, utilize member's linear feature on open plane and plane intersection line formation level border to suppose
In the 1st step, generate the plane intersection line set
Choose two open planes arbitrarily
If
Judge that they can intersect, calculate their intersection equation, be designated as
The set of being made up of all plane intersection lines is designated as U;
In the 2nd step, judge the collinear relationship between plane intersection line and plane member's linear feature and revise member's linear feature set on open plane
For any open plane
In U, select it with
The intersection on any open plane, if its any member's linear feature
The angle of place straight line and this plane intersection line is less than φ, and
Mid point to the distance of plane intersection line less than ε, then judge
With this plane intersection line conllinear, and from open plane
Member's linear feature set
Middle deletion
By all with
The set formed of the plane intersection line of certain member's linear feature conllinear, be designated as
In the 3rd step, replenish the open plane that lacks
For any one open plane
Any one member's linear feature
Use a kind of method, generate a new open plane by it by the wall scroll 3 d-line generation hypothesis of plane mechanism,
Become its member's linear feature, add this plane to set
In;
The 4th step is at the open plane computations plane intersection line that replenishes
For any one open plane
If it is generated by the wall scroll 3 d-line, then calculate it and any other open plane
Intersection, and with this plane intersection line add to the set U in; If this plane intersection line with
The angle of any member's linear feature place straight line less than φ, and
Mid point to the distance of plane intersection line less than ε, then this plane intersection line is added to the straight line set
In, and from set
Member's linear feature of middle this conllinear of deletion;
In the 5th step, add the member linear feature of plane intersection line for open plane
For any plane intersection line among the set U, suppose that it is open plane
With
Intersection, then this plane intersection line is added to
With
Member's linear feature set
With
In go;
In the 6th step, whether arbitrary plane intersection and other member's linear feature intersect in member's linear feature set on the open plane of judgement
For any open plane
Any one member's linear feature, if it belong to the set U, and it and
The angle of any other member's linear feature belong to scope
Judge that then these two member's linear features intersect;
The 7th step is by member's linear feature formation level border on open plane
The set that all the closed hypothesises of plane mechanism that generate and the semi-open hypothesis of plane mechanism are formed be designated as Ω '=P '
k| k=1,2 ..., N ' }, any one element P ' wherein
kOrderly vertex set with it is represented, is designated as
Plane equation be a '
kX+b '
kY+c '
kZ+d '
k=0;
In (two) step, utilize the disappearance border of extracting the linear feature and the additional semi-open hypothesis of plane mechanism of heuristic rule
For arbitrary plane hypothesis P '
k∈ Ω ' if it is semi-open, then replenishes the border of disappearance for it, at first utilize and extract linear feature H
1And H
2If, can search the linear feature that can be used for replenishing the disappearance border, then this step finishes, and generates one or more closed hypothesises of plane mechanism; Otherwise, utilize heuristic rule to replenish the border of disappearance, generate a closed hypothesis of plane mechanism;
(3) goes on foot, and chooses the closed hypothesis of plane mechanism of global optimum
The 1st goes on foot, and calculates the reliability measure of the closed hypothesis of plane mechanism;
In the 2nd step, find the solution the optimum closed hypothesis of plane mechanism
Generate a non-directed graph, each closed hypothesis of plane mechanism is a node to be designated as G in "; with set omega ", its reliability measure as with the attribute of the corresponding node of the closed hypothesis of plane mechanism, if all there is not lap in the view field of the closed hypothesis of plane mechanism of any two node correspondences in left image and right image, then connect this two nodes, otherwise do not connect; Calculating chart G " all very big groups, therefrom choose all closed hypothesises of plane mechanism that the very big group of node attribute values sum maximum comprises, extract the result as three-dimensional planar.
3. three dimensional plane extraction method according to claim 2 is characterized in that, the 3 d-line feature among the pair set S makes up in twos, generates the three-dimensional planar hypothesis, and concrete steps are:
Optional two 3 d-line feature L
u∈ S and L
v∈ S, L
uEnd points be (X
U1, Y
U1, Z
U1) and (X
U2, Y
U2, Z
U2), L
vEnd points be (X
V1, Y
V1, Z
V1) and (X
V2, Y
V2, Z
V2), utilize their objective definition function f (a, b, c, d):
f(a,b,c,d)=(a·X
u1+b·Y
u1+c·Z
u1+d)
2+(a·X
u2+b·Y
u2+c·Z
u2+d)
2
+(a·X
v1+b·Y
v1+c·Z
v1+d)
2+(a·X
v2+b·Y
v2+c·Z
v2+d)
2
Adopt the gradient descent method calculate f (a, b, c, when d) getting minimum value (c d), obtains by L for a, b
uAnd L
vThe three-dimensional planar hypothesis equation of appointment is aX+bY+cZ+d=0.
4. three dimensional plane extraction method according to claim 3 is characterized in that, the method that merges the three-dimensional planar hypothesis of coplane is:
For any one three-dimensional planar hypothesis P
i∈ Ω, the mid point of two member's linear feature mid point lines that calculates it is in plane P
iOn projection, be designated as (X
M i, Y
M i, Z
M i);
The 1st) step, judge the coplanar relation that three-dimensional planar is supposed
For any two three-dimensional planars hypothesis P
i∈ Ω and P
j∈ Ω is if the inequality group below satisfying is then judged P
iAnd P
jCoplane:
The 2nd) step, the three-dimensional planar hypothesis of search coplane
Generate a non-directed graph G, with each three-dimensional planar hypothesis P
i∈ Ω, then keeps between the node of their correspondences connecting, otherwise does not connect if any two three-dimensional planars hypothesis is a coplane as a node;
Calculate all very big groups of non-directed graph G, be designated as { Q
w| w=1,2 ..., N
Q, any one greatly rolls into a ball Q
wBe the subclass of Ω, be designated as
The 3rd) step, the three-dimensional planar that merges coplane is supposed
Greatly roll into a ball Q for any one
w, merge all three-dimensional planar hypothesis that it comprises, obtain a new three-dimensional planar hypothesis P
w, its plane equation is designated as a
wX+b
wY+c
wZ+d
w=0, wherein:
Will
In all member's linear features of each three-dimensional planar hypothesis project to P
wOn, all projection straight line sections that obtain become P
wMember's linear feature.
5. three dimensional plane extraction method according to claim 4 is characterized in that, for any one three-dimensional planar hypothesis P
i∈ Ω utilizes its member's linear feature M
iIt is cut apart, obtain some flat blocks hypothesis, concrete grammar may further comprise the steps:
The 1st) step, member's linear feature of merging conllinear
For P
iAny two member's linear feature L
j i,
Calculate the angle of their place straight lines, be designated as μ, and L
j iMid point to L
k iThe distance of place straight line and L
k iMid point to L
j iThe distance of place straight line is designated as η respectively
1And η
2
If μ<φ and η
1<ε and μ
2<ε then judges L
j iAnd L
k iConllinear generates a new 3 d-line feature with them, and two end points of this 3 d-line feature are L
j iAnd L
k iAt a distance of farthest two end points, add new 3 d-line feature to set M
iIn, and from set M
iMiddle deletion L
j iAnd L
k i
The 2nd) step, calculate P
iThe intersection point of all member's linear features
For P
iAny two member's linear feature L
α i,
If the angle theta of their place straight lines satisfies
Then calculate L
α iAnd L
β iIntersection point or the intersection point of their extended line;
The set that the intersection point of all calculating is formed is designated as R
i
The 3rd) step, generate the non-directed graph that is used for the segmentation plane hypothesis
Generate a non-directed graph, be designated as G
i, its node is by two class dot generation: the one, and intersection point set R
iIn each element, the 2nd, for M
iIn any one member's linear feature, if do not have and M on the extended line outside its end points
iIn the intersection point of other member's linear feature, and this end points is not set R
iIn element, then this end points becomes figure G
iA node;
Non-directed graph G
iIn any two internodal annexations comprise two classes: the one, if the point of their correspondences on same member's linear feature or its extended line, and on the connection line segment of these two points not and figure G
iIn the corresponding point of other node, then their keep to connect; The 2nd, the annexation of structure, building method is as follows:
For any one member's linear feature L
α iIf its one or two end points is figure G
iNode, then handle in two kinds of situation:
First kind of situation, L
α iWith M
iIn any other member's linear feature all non-intersect
Select any member's linear feature
And β ≠ α specifies on its place straight line the G with figure
iIn two corresponding two points of node, these two points are positioned at L
β iThe mid point both sides and the connection line segment between them on not and figure G
iThe middle corresponding point of node is designated as (X with them
1, Y
1, Z
1) and (X
2, Y
2, Z
2), and calculate
Wherein, (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and (X
α, 2 i, Y
α, 2 i, Z
α, 2 i) be L
α iEnd points; For arbitrarily
And l ≠ α, l ≠ β calculate
Wherein, (X
L, 1 i, Y
L, 1 i, Z
L, 1 i) and (X
L, 2 i, Y
L, 2 i, Z
L, 2 i) be L
l iEnd points; If satisfy
Then calculate
If f
5F
6>0, tie point (X then
α, 1 i, Y
α, 1 i, Z
α, 1 i) and (X
2, Y
2, Z
2) at figure G
iMiddle corresponding node, and point (X
α, 2 i, Y
α, 2 i, X
α, 2 i) and (X
1, Y
1, Z
1) at figure G
iMiddle corresponding node;
Second kind of situation, L
α iWith M
iIn at least one other member's linear feature intersect
If (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and figure G
iA node corresponding, select member's linear feature L
α iOr on its extended line with (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) nearest intersection point, be designated as (X
3, Y
3, Z
3), suppose that this intersection point is L
α iWith member's linear feature L
β iIntersection point; Select L
β iOr on its extended line with figure G
iIn the corresponding point of node, if at it and point (X
3, Y
3, Z
3) between line on not and figure G
iThe middle corresponding point of node then is designated as (X with it
4, Y
4, Z
4), if such point has two, then respectively they are designated as (X
4, Y
4, Z
4) and (X
5, Y
5, Z
5);
If (X
4, Y
4, Z
4) be L
β iWith the intersection point of other member's linear feature, suppose that this member's linear feature is L
l i, and L
l iOr exist on its extended line and figure G
iThe middle corresponding point of node, it satisfies and point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) drop on L
β iThe same side, and with point (X
4, Y
4, Z
4) between line on not and figure G
iIn the corresponding point of node, then connect it and point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) at figure G
iMiddle corresponding node; Otherwise, tie point (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) and point (X
4, Y
4, Z
4) at figure G
iMiddle corresponding node; If (X
5, Y
5, Z
5) exist, then according to (X
4, Y
4, Z
4) same method handles;
If (X
α, 2 i, Y
α, 2 i, Z
α, 2 i) be figure G
iNode, method of then constructing annexation and (X
α, 1 i, Y
α, 1 i, Z
α, 1 i) identical;
The 4th) step, cut apart generation flat blocks hypothesis
Search non-directed graph G
iIn all rings, for any one ring wherein, if the node number that belongs to same member's linear feature or its extended line that it comprises is no more than 2, then generate a flat blocks hypothesis with it, all nodes that it comprises are formed an orderly point set according to its annexation, represent the vertex set of this flat blocks hypothesis.
6. three dimensional plane extraction method according to claim 5 is characterized in that, the arbitrary plane piece is supposed
Calculate its reliability measure, comprise the steps:
Projection plane piece hypothesis
Vertex set to left and right image, remember any summit
Projection in left and right image is respectively
With
The order projection on connection summit in left and right image respectively according to each fixed point in the vertex set forms
Two polygon view fields;
For any spot elevation
If the projection in its left image
Be positioned at
In the view field in left image, and the projection in its right image
Be positioned at
In the view field in right image, then judge spot elevation
Quilt
Cover;
Quilt
The set that all spot elevations that cover are formed is designated as
The quantity of element is α in the set
w
The 2nd) step, calculate the elevation consistance and estimate
(formula one)
Wherein,
If α
w=0, then
The 3rd) step, calculate the gray scale similarity measure
All images point is formed a point set in the view field in left image, is designated as K
w L, the gradation of image value of being had a few in them is formed an array, is designated as { h
Wt| t=1,2 ..., o
w;
Foundation
Three pairs of same places in left and right image
With
With
And
With
Set of computations K
w LThe middle same place of all elements point in right image, the set that these same places are formed is designated as K
w R, the gradation of image value of being had a few in them is obtained by interpolation algorithm, and these gray-scale values are formed another array, are designated as { h
Wt' | t=1,2 ..., o
w; Calculate
Gray scale similarity measure between the view field in left and right image is
Wherein,
7. three dimensional plane extraction method according to claim 6 is characterized in that, the arbitrary plane piece is supposed
The step of searching for the most reliable flat blocks hypothesis is:
The 1st) step, the reliable flat blocks hypothesis that search has known spot elevation to cover
Set up a non-directed graph, for any one flat blocks hypothesis
If
Then generate a node of this non-directed graph with it, the property value of node is E
wFor any two nodes,, then connect them if all there is not lap in the view field of the flat blocks of their correspondences hypothesis in left and right image;
Calculate all very big groups of this non-directed graph, therefrom choose the set of all flat blocks hypothesis that the very big group of node attribute values sum maximum comprises, be designated as K
1, it represents that chlamydate spot elevation is as the reliable flat blocks hypothesis of weighing foundation;
The 2nd) step, the reliable flat blocks hypothesis that search does not have known spot elevation to cover
Set up a non-directed graph, for any one flat blocks hypothesis
If
And it and K
1In any one flat blocks hypothesis view field in left and right image all do not have lap, then use
Generate a node of this non-directed graph, the property value of node is E
wFor any two nodes,, then connect them if all there is not lap in the view field of the flat blocks of their correspondences hypothesis in left and right image;
Calculate all very big groups of this non-directed graph, therefrom choose the set of all flat blocks hypothesis that the very big group of node attribute values sum maximum comprises, be designated as K
2, the spot elevation that its expression does not have to cover is as weighing foundation and and K
1In all flat blocks suppose coexisting reliable flat blocks hypothesis;
With K
1∪ K
2All flat blocks hypothesis that comprise are as reliable flat blocks hypothesis.
8. three dimensional plane extraction method according to claim 7 is characterized in that, the method that merges the reliable flat blocks hypothesis of coplane is;
The 1st) step, judge the coplanar relation that flat blocks is supposed
For any two reliable flat blocks hypothesis
If following inequality is formed upright, then judge
With
Coplane:
The 2nd) step, the reliable flat blocks hypothesis of search coplane generates a non-directed graph, is designated as
Use K
1∪ K
2In each reliable flat blocks hypothesis generate a node, if any two reliable flat blocks hypothesis are coplanes, then connect the node of their correspondences;
Calculating chart
All very big groups, be designated as
Any one is greatly rolled into a ball
Be figure
A subclass, be designated as
The 3rd) step, the reliable flat blocks that merges coplane is supposed
For any one very big group
Merge all reliable flat blocks hypothesis that it comprises, obtain an open plane
Plane equation is designated as
Wherein:
9. three dimensional plane extraction method according to claim 8 is characterized in that, be may further comprise the steps by the method on member's linear feature formation level border on open plane:
The 1st) go on foot, split all member's linear feature to two subclass of the face of setting level
For any one open plane
Its all member's linear features are split into two subclass, and a subclass is
Be designated as U
1, another subclass is
Be designated as U
2
The 2nd) step, generate the non-directed graph set
Generate U
2All subclass;
Choose U
2Any one subclass and U
1Form new member's linear feature collection, be designated as U
0, generate a non-directed graph; This non-directed graph is with U
0In each member's linear feature as node, if member's linear feature of any two node correspondences intersects, then they are coupled together;
The 3rd) step, generate planar boundary
Its all Hamilton circles of search in each non-directed graph that previous step obtains, for each unduplicated circle,, calculate the intersection point of member's linear feature of adjacent node correspondence according to the order of connection between the node that wherein comprises, they as the summit, are generated a closed hypothesis of plane mechanism; If there is no the Hamilton circle is then searched for all Hamilton paths, for each unduplicated path, according to the order of connection between the node that wherein comprises, calculate the intersection point of member's linear feature of adjacent node correspondence, they as the summit, are generated a semi-open hypothesis of plane mechanism.
10. three dimensional plane extraction method according to claim 9 is characterized in that, utilizes the disappearance border of extracting the linear feature and the additional semi-open hypothesis of plane mechanism of heuristic rule, and method is as follows:
In the 1st step, utilize the linear feature that extracts in the image to replenish the disappearance border of the semi-open hypothesis of plane mechanism
The 1st) step, calculate P '
kThe projection in left and right image of disappearance border the region projection P '
kTo left and right image, obtain point set respectively
With
Set the distance threshold h of an expression linear feature integrality
d
Specify P '
kThe zone at projection place in left image, disappearance border be a quadrilateral, be designated as P
k L, its summit is designated as g respectively
1, g
2, g
3And g
4, wherein,
g
3It is line segment
Extended line on to the some g
2Distance be h
dPoint, g
4Be line segment Tl
k 2Tl
k 1Extended line on to the some g
1Distance be h
dPoint, connecting them becomes R
k LFour edges g
1g
2, g
2g
3, g
3g
4And g
4g
1
Utilize similar method, calculate P '
kThe region R at projection place in right image, disappearance border
k R, its four summits are designated as g respectively
1', g
2', g
3' and g
4';
The 2nd) in the step, utilize region R
k LAnd R
k RInterior H
1And H
2In linear feature replenish the disappearance border of the semi-open hypothesis of plane mechanism
For H
1In any one linear feature, if its two end points all drop on R
k LIn, and its extended line and line segment g
2g
3And g
4g
1Intersect, then connect two intersection points and obtain the straight line section, according to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate the three dimensions point of two end points correspondences of this straight-line segment, with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism; For H
2In any one linear feature, if its two end points all drop on R
k RIn, and its extended line and line segment g
2' g
3' and g
4' g
1' intersect, calculate its same place of end points in left image, connecting two same places is the straight line section, if any disappearance border that searches in this straight-line segment and left image conllinear not then calculates the extended line and the line segment g of this linear feature in right image
2' g
3' and g
4' g
1' intersection point, connect these two intersection points and obtain the straight line section, according to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate the three dimensions point of two end points correspondences of this straight-line segment, with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism;
The 2nd step is according to the disappearance border of the additional semi-open hypothesis of plane mechanism of heuristic rule
If respectively with T
k 2T
k 1With
For shared border, with P '
kTwo adjacent hypothesises of plane mechanism have all obtained complete border, according to the isometric rule of adjacent polygon plane common edge, at T
k 2T
k 1Or 1 Φ of appointment on its extended line
1, make line segment T
k 2Φ
1With with T
k 2T
k 1For the shared border of the adjacent plane on shared border hypothesis isometric,
Or 1 Φ of appointment on its extended line
2, make line segment
With with
For the shared border of the adjacent plane on shared border hypothesis isometric, line segment Φ
1Φ
2Become additional disappearance border;
If respectively with T
k 2T
k 1With
Be shared border and P '
kHave only a hypothesis of plane mechanism to obtain complete border in two adjacent hypothesises of plane mechanism, suppose that this adjacent plane is assumed to be P '
l, and T
k 2T
k 1Be P '
lWith P '
kShared border, then according to the shared border of adjacent polygon plane isometric rule, at T
k 2T
k 1Or specify on its extended line a bit, make line segment T
k 2Φ
1With P '
lShared border isometric,
Or 1 Φ of appointment on its extended line
2, make line segment
With T
k 2Φ
1Isometric, line segment Φ
1Φ
2Become additional disappearance border;
According to P '
kPlane equation and the collinearity equation in the photogrammetric Theory, calculate Φ
1And Φ
2Corresponding three dimensions point is with P '
kVertex set in remove T
k 1With
Outside all summits together, generate a closed hypothesis of plane mechanism;
The closed hypothesis of plane mechanism among the Ω ' and the closed hypothesis of plane mechanism merging that is generated by the semi-open hypothesis of plane mechanism among the Ω ' are become closed hypothesis of plane mechanism set, are designated as Ω "=P '
i| i=1,2 ..., N " }, wherein, any closed hypothesis of plane mechanism P "
iBy n "
iThe bar boundary line is formed.
11. three dimensional plane extraction method according to claim 10 is characterized in that, the method for calculating the reliability measure of the closed hypothesis of plane mechanism is:
For any one closed hypothesis of plane mechanism P "
i∈ Ω ", utilize formula one to calculate P "
iThe elevation consistance estimate, if P "
iIt is 0 that the known elevation that covers is counted, and then specifying its elevation consistance to estimate is 1/ (2 λ), utilizes formula three, calculates its reliability measure, is designated as E "
iAt P "
iOn the plane, place, specify next-door neighbour outside its every boundary line, with the rectangle plane zone of this boundary line as a long limit, the long length that equals the boundary line, wide half that equals to grow, the elevation consistance of utilizing formula one to calculate this rectangle plane zone is estimated, if it is 0 that the known elevation that it covers is counted, then specifying its elevation consistance to estimate is 1/ (2 λ), utilize formula three, calculate its reliability measure, note is E by the reliability measure in the rectangle plane zone of j bar boundary line correspondence
I, j", calculate closed hypothesis of plane mechanism P according to following formula "
iThe reliability measure on corresponding true planar border, border be
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009100441925A CN101639355B (en) | 2009-08-26 | 2009-08-26 | Three dimensional plane extraction method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009100441925A CN101639355B (en) | 2009-08-26 | 2009-08-26 | Three dimensional plane extraction method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101639355A true CN101639355A (en) | 2010-02-03 |
CN101639355B CN101639355B (en) | 2011-06-15 |
Family
ID=41614454
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2009100441925A Expired - Fee Related CN101639355B (en) | 2009-08-26 | 2009-08-26 | Three dimensional plane extraction method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101639355B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103411590A (en) * | 2013-09-02 | 2013-11-27 | 武汉大学 | Method and system for automatically determining installation position of camera in photogrammetric survey |
CN107464258A (en) * | 2017-07-28 | 2017-12-12 | 南京华捷艾米软件科技有限公司 | A kind of method that three-dimensional planar is extracted from image |
CN109859314A (en) * | 2019-03-12 | 2019-06-07 | 上海曼恒数字技术股份有限公司 | Three-dimensional rebuilding method, device, electronic equipment and storage medium |
CN112747734A (en) * | 2019-10-31 | 2021-05-04 | 深圳拓邦股份有限公司 | Environment map direction adjusting method, system and device |
CN117853313A (en) * | 2023-10-24 | 2024-04-09 | 广东创兴精密制造股份有限公司 | Method, device, equipment and medium for converting three-dimensional graph into two-dimensional graph |
-
2009
- 2009-08-26 CN CN2009100441925A patent/CN101639355B/en not_active Expired - Fee Related
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103411590A (en) * | 2013-09-02 | 2013-11-27 | 武汉大学 | Method and system for automatically determining installation position of camera in photogrammetric survey |
CN103411590B (en) * | 2013-09-02 | 2015-09-02 | 武汉大学 | Automatically the method and system of photogrammetric middle video camera installation position are determined |
CN107464258A (en) * | 2017-07-28 | 2017-12-12 | 南京华捷艾米软件科技有限公司 | A kind of method that three-dimensional planar is extracted from image |
CN109859314A (en) * | 2019-03-12 | 2019-06-07 | 上海曼恒数字技术股份有限公司 | Three-dimensional rebuilding method, device, electronic equipment and storage medium |
CN112747734A (en) * | 2019-10-31 | 2021-05-04 | 深圳拓邦股份有限公司 | Environment map direction adjusting method, system and device |
CN112747734B (en) * | 2019-10-31 | 2024-04-30 | 深圳拓邦股份有限公司 | Method, system and device for adjusting direction of environment map |
CN117853313A (en) * | 2023-10-24 | 2024-04-09 | 广东创兴精密制造股份有限公司 | Method, device, equipment and medium for converting three-dimensional graph into two-dimensional graph |
Also Published As
Publication number | Publication date |
---|---|
CN101639355B (en) | 2011-06-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN103278170B (en) | Based on mobile robot's cascade map creating method that remarkable scene point detects | |
CN102136155B (en) | Object elevation vectorization method and system based on three dimensional laser scanning | |
US12046003B2 (en) | Method and system for visual localization | |
US20160379366A1 (en) | Aligning 3d point clouds using loop closures | |
US20160012638A1 (en) | Image processing | |
CN106595659A (en) | Map merging method of unmanned aerial vehicle visual SLAM under city complex environment | |
JP2018522345A (en) | Method and apparatus for real-time mapping and localization | |
Xu et al. | Robust segmentation and localization of structural planes from photogrammetric point clouds in construction sites | |
Alidoost et al. | An image-based technique for 3D building reconstruction using multi-view UAV images | |
CN106570468A (en) | Method for reconstructing LiDAR original point cloud building contour | |
US10192004B2 (en) | Estimation of three-dimensional models of roofs from spatial two-dimensional graphs | |
Cosido et al. | Hybridization of convergent photogrammetry, computer vision, and artificial intelligence for digital documentation of cultural heritage-a case study: the magdalena palace | |
CN105956066B (en) | A kind of automatic identification method of fold geomorphic type | |
Tian et al. | Knowledge-based building reconstruction from terrestrial video sequences | |
Alcantarilla et al. | Large-scale dense 3D reconstruction from stereo imagery | |
CN101639355A (en) | Three dimensional plane extraction method | |
Xu et al. | A volumetric change detection framework using UAV oblique photogrammetry–a case study of ultra-high-resolution monitoring of progressive building collapse | |
Fritsch et al. | Modeling facade structures using point clouds from dense image matching | |
Yoo et al. | A vision-based indoor positioning systems utilizing computer aided design drawing | |
Rothermel et al. | Fast and robust generation of semantic urban terrain models from UAV video streams | |
Rosinol | Densifying sparse vio: a mesh-based approach using structural regularities | |
Huang et al. | Fast initialization method for monocular slam based on indoor model | |
Sun et al. | Window detection employing a global regularity level set from oblique unmanned aerial vehicle images and point clouds | |
Meixner et al. | Interpretation of 2D and 3D building details on facades and roofs | |
Li et al. | Lightweight 3D modeling of urban buildings from range data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
CF01 | Termination of patent right due to non-payment of annual fee | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20110615 Termination date: 20160826 |