CN110910387A - Point cloud building facade window extraction method based on significance analysis - Google Patents

Point cloud building facade window extraction method based on significance analysis Download PDF

Info

Publication number
CN110910387A
CN110910387A CN201910954155.1A CN201910954155A CN110910387A CN 110910387 A CN110910387 A CN 110910387A CN 201910954155 A CN201910954155 A CN 201910954155A CN 110910387 A CN110910387 A CN 110910387A
Authority
CN
China
Prior art keywords
point
points
facade
building
window
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910954155.1A
Other languages
Chinese (zh)
Other versions
CN110910387B (en
Inventor
郝雯
楚良
王映辉
石争浩
赵明华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Technology
Original Assignee
Xian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Technology filed Critical Xian University of Technology
Priority to CN201910954155.1A priority Critical patent/CN110910387B/en
Publication of CN110910387A publication Critical patent/CN110910387A/en
Application granted granted Critical
Publication of CN110910387B publication Critical patent/CN110910387B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G06V10/507—Summing image-intensity values; Histogram projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a building facade window extraction method based on significance analysis, which comprises the steps of 1, calculating the difference between the normal vector of each point and the main direction of a building facade to obtain global significance, and setting a threshold value to extract points with high global significance; step 2, calculating the difference of geometric features of each point and a local adjacent point in the point cloud of the facade of the building, and setting a threshold value to extract points with high local significance; step 3, combining the points with high significance extracted in the step 1 and the step 2, and removing the points closer to the highest point and the lowest point in the set to obtain a high-significance characteristic point set; step 4, longitudinally and transversely segmenting the high-significance characteristic point set, constructing a cumulative histogram of point cloud data in each segment, converting the cumulative histogram into a binary form, and determining a 1-0 transformation position as the longitudinal and transverse segmentation position of the window; and 5, correcting the segmentation result to finish the extraction of the window. The problem of inaccurate identification of the facade repetitive structure caused by poor quality of point cloud data in the prior art is solved.

Description

Point cloud building facade window extraction method based on significance analysis
Technical Field
The invention belongs to the technical field of interdisciplines combining computer vision and pattern recognition, and particularly relates to a point cloud building facade window extraction method based on significance analysis.
Background
Buildings are common objects in urban scenes, and windows are important constituent elements of building facades. Accurate detection and extraction of windows is an important research in computer vision. The method can be widely applied to multiple fields of building reconstruction, virtual tourism, entertainment and the like. People have strong identification ability and can quickly locate and identify windows on buildings, but how to make a computer automatically identify the windows of a facade like a human is still difficult.
Currently, existing window detection methods can be divided into two categories: an image-based window extraction method and a point cloud-based window extraction method.
① Window extraction method based on image
M ü ller first analyzes and corrects the wall image, divides the image into structural combinations with semantic information, then generates shape grammar according to the division result, completes the detection of the repeated structure in the image by using the shape grammar.
The acquisition of building images is susceptible to occlusion, reflection and illumination variations, and in addition, image-based window extraction methods often require that the detected images be in parallel view.
② Window extraction method based on point cloud data
When a building is scanned by the laser scanner, the glass cannot reflect the laser beam back to the scanner, and the point cloud data of the window obtained by scanning is very little, so that the cavity on the wall surface corresponds to the window. Zolanvari proposes a segmentation-based window detection method: firstly, extracting the wall surface of a building by using a random sampling consistency algorithm, then transversely segmenting the wall surface, projecting points in each segment to a locally defined x axis to obtain a plurality of lines, clustering the lines according to the distance, and detecting cavities on the wall surface by judging the intermediate distance of more than 2 times of the distance between the points along the line direction; then the wall surface is longitudinally divided and then the same treatment is carried out. Pu divides the extracted wall surface boundary points into inner contour points and outer contour points, and the inner contour points are used for extracting windows. Similar to Pu, Peethambaran extracts contour points of the wall surface by an alpha-shape method, removes peripheral contour points which are close to a convex shell of the wall surface, and the method assumes that a window or a door on the wall surface is aligned with a two-dimensional coordinate axis: firstly, calculating the number of weighted points from a boundary point to a scanning line closest to the boundary point along the x-axis direction, and drawing a waveform according to the number of the weighted points, wherein a local lowest point is a division point; then, the same operation is performed in the y-axis direction, and finally, the division of the window is completed. Aijazi projects the three-dimensional point cloud wall data to a two-dimensional space, a window is divided based on geometric information, the 2D projection before window extraction inevitably causes precision loss, and the method converts three-dimensional identification into two-dimensional identification, which causes loss of three-dimensional information and inaccurate identification. Friedman et al use fourier analysis to identify repetitive features and extract repetitive structures from the building point cloud by processing the boundary lines. Mesolongtis assumes that windows are arranged in a plurality of periodic structures and employs a voting scheme for window detection.
The existing window extraction method based on point cloud needs to segment buildings in advance or extract internal boundary points of wall surfaces, and the steps are susceptible to the influence of scanning data quality. Due to the shielding, noise interference and massive point cloud data in the scanning process, the existing method is not suitable for extracting the facade window of the complex building.
Disclosure of Invention
The invention aims to provide a building facade window extraction method based on significance analysis, and solves the problem that facade repetitive structure identification is inaccurate due to poor scanning quality of three-dimensional point cloud data in the prior art.
The invention adopts the technical scheme that a building facade window extraction method based on significance analysis is implemented according to the following steps:
step 1, calculating by comparing the difference between the normal vector of each point in the point cloud of the facade of the building and the main direction of the facade of the building to obtain the global significance of each point on the facade, and extracting the feature points with high global significance by setting a threshold;
step 2, calculating the geometric feature difference between each point and a local adjacent point in the point cloud of the facade of the building by using the fast point feature histogram descriptor to obtain the local significance of each point, and extracting feature points with high local significance by setting a threshold;
step 3, merging the characteristic point set obtained in the step 1 and the characteristic points with high local significance obtained in the step 2, and removing points which are close to the highest point and the lowest point in the merged characteristic point set to obtain a high-significance characteristic point set forming a window frame;
and 4, according to the set of the salient feature points obtained in the step 3, firstly, longitudinally segmenting the set of the salient feature points, constructing a cumulative histogram of point cloud data on each slice, converting the cumulative histogram into a binary form by setting a threshold value, wherein a 1-0 conversion position is a vertical segmentation position, and then, performing the same operation as the vertical direction on the horizontal direction of the set of the salient feature points obtained in the step 3 to determine a horizontal segmentation position.
And 5, correcting the segmentation result obtained in the step 4 according to the proximity relation between the windows and the structural similarity of the windows, namely finishing the extraction of the building facade windows.
The invention is also characterized in that:
the specific implementation steps of the step 1 are as follows:
step 1.1, calculating the main direction of the point cloud of the facade of the building by using a principal component analysis method;
step 1.2, calculating a normal vector of each point in the facade point cloud by using a principal component analysis method;
and 1.3, calculating the global significance of the points.
Step 1.1 comprises the following specific steps:
let p beiIs any point in the point cloud of the building facade, N is the total number of the midpoint of the point cloud of the building facade, and the third-order covariance matrix M of the point cloud of the building facade is as follows:
Figure BDA0002226711360000041
in the formula (1), the reaction mixture is,
Figure BDA0002226711360000047
is the average position of the points in the point cloud of the facade,
Figure BDA0002226711360000043
performing eigenvalue decomposition on a third-order covariance matrix M of the point cloud of the opposite surface through singular value decomposition to obtain an eigenvalue lambda of the third-order covariance matrix M3、λ2、λ1Wherein λ is3≥λ2≥λ1If > 0, the minimum eigenvalue λ is1Corresponding feature vector
Figure BDA0002226711360000044
The main direction of the point cloud of the facade of the building is obtained.
Step 1.2 the steps are:
finding out point cloud midpoint p of building facade point by using k-d treeiK adjacent points pj，pj∈{p1，p2，…，pkPoint piThird order covariance matrix of
Figure BDA0002226711360000045
Comprises the following steps:
Figure BDA0002226711360000046
in the formula (3), p' is a point piThe average position of the k neighboring points of (c),
Figure BDA0002226711360000051
by singular value decomposition of the point piThird order covariance matrix of
Figure BDA0002226711360000052
Performing eigenvalue decomposition to obtain a third-order covariance matrix
Figure BDA0002226711360000053
Characteristic value of
Figure BDA0002226711360000054
Wherein the content of the first and second substances,
Figure BDA0002226711360000055
then point piThe normal vector of (a) is a minimum eigenvalue
Figure BDA0002226711360000056
Corresponding feature vector
Figure BDA0002226711360000057
I.e. point piHas a normal vector of
Figure BDA0002226711360000058
Step 1.3 comprises the following steps:
step 1.3.1, calculating the point cloud midpoint p of the facade of the buildingiNormal vector of (1)
Figure BDA0002226711360000059
In the main direction of the facade of a building
Figure BDA00022267113600000510
Distance d ofi：
Figure BDA00022267113600000511
Will point piIs defined as the distance diMapping in hyperbolic tangent function space:
Figure BDA00022267113600000512
in the formula (6), lambda controls the steepness of the shape of the hyperbolic tangent function, and c is the geometric centroid of the hyperbolic tangent function;
according to the distance d in the point cloud of the facade of the buildingiDistribution range of (d)iDividing into T parts, counting the distance d between the normal direction of all points and the main direction of the building facadeiThe number of points falling within the range of T values, assuming NtRespectively representing the number of points falling in the T shares, and then the probability that a certain point in the point cloud of the facade of the building falls in the T-th component in the histogram
Figure BDA00022267113600000513
Comprises the following steps:
Figure BDA00022267113600000514
the geometric centroid c of the hyperbolic tangent function is:
Figure BDA0002226711360000061
in the formula (8), htTo correspond to the distance diThe middle value of the value range of the tth component of the distribution histogram, wherein T is 5, and lambda is 0.5;
step 1.3.2, extracting points with high global significance by setting a threshold α,
Figure BDA0002226711360000062
if S isglobal(pi) Greater than α, point piI.e., a high global saliency point, otherwise, the point is a non-saliency point, with the threshold α ranging from 0.5-0.6.
The specific implementation steps of the step 2 are as follows:
step 2.1, calculating a Fast Point Feature Histogram (FPFH) of each point in the point cloud of the facade of the building, and assuming piFinding out the midpoint p of the point cloud of the facade of the building by using a k-d treeiK adjacent points pj，pj∈{p1,p2,…pk}，piHas a normal vector of
Figure BDA0002226711360000063
pjHas a normal vector of
Figure BDA0002226711360000069
At point piThe local coordinate system u, v, w is defined above:
Figure BDA0002226711360000065
calculation of a normal vector using equation (11)
Figure BDA0002226711360000066
And
Figure BDA0002226711360000067
deviation α between, φ, θ:
Figure BDA0002226711360000068
in equation (11), α, φ, θ form a simple point feature histogram SPFH, point piThe fast point feature histogram FPFH can be calculated using equation (12):
Figure BDA0002226711360000071
in the formula (12), wjIs a point piAnd its neighboring point pjThe euclidean distance between;
for any two points p in the point cloud of the facade of the buildingiAnd pjGeometric dissimilarity thereof
Figure BDA0002226711360000076
Can be calculated according to equation (13):
Figure BDA0002226711360000072
step 2.2: p obtained according to step 2.1iAnd point pjGeometric dissimilarity of, point piAnd point pjLocal dissimilarity of (d)H(pi,pj) Calculating using equation (14):
Figure BDA0002226711360000075
in the formula (14), | | pi-pj| | is the distance between two points;
then, point piLocal significance of Slocal(pi) Comprises the following steps:
Figure BDA0002226711360000073
step 2.3, extracting points with high local significance in the point cloud in the building by setting a threshold value β:
Figure BDA0002226711360000074
if S islocal(pi) Greater than β, point piI.e., a high local saliency point, otherwise, this point is a non-saliency point, with β set to 0.9.
The specific steps of the step 3 are as follows: the points with high global significance obtained in the step 1 and the points obtained in the step 2 are combinedMerging the obtained points with high local significance to obtain a candidate high-significance point set, traversing the candidate high-significance point set to obtain a maximum z value zmaxAnd the minimum z value zmin，ziIs a point piRemove points closer to the maximum and minimum z values, i.e., if | zmax-zi< gamma or | zmin-ziIf [ gamma ] is less than [ gamma ], and [ gamma ] is set to 0.1-0.2, the point p is removediA highly significant set of points is obtained which constitutes the window frame.
The specific implementation steps of the step 4 are as follows:
step 4.1, traversing each point in the high salient point set obtained in step 3, and respectively calculating the maximum and minimum x and y values in the high salient point set, wherein x and y are the x and y coordinates of the point, namely xmax，ymax，xminAnd yminIf (x)max-xmin)<(ymax-ymin) If not, the point cloud of the building facade is segmented along the y axis, otherwise, the point cloud of the building facade is segmented along the x axis, and after the number L of the segments is determined, each segment s is obtained by segmentationi＝{s1,s2,s3…sLThe width of each slice is:
Figure BDA0002226711360000081
calculating the number of points in each slice:
Figure BDA0002226711360000082
in the formula (17), i is a slice siA subscript of (a);
the total number of points in each slice is:
Figure BDA0002226711360000083
drawing a histogram according to the number of the middle points of each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise, setting the slice as 0, and setting delta y as 0.1; the 1-0 transformation position in the histogram is the vertical cutting position of the window;
step 4.2, go through step 3 to obtainCalculating the maximum and minimum z values, namely z, of the high salient point set respectively for each point in the high salient point setmaxAnd zmin，ziFor the z coordinate of the point, carrying out segmentation along the z value, determining the number L of the segmentation, and obtaining each slice s by segmentationi＝{s1,s2,s3…sLThe width of each slice is:
Figure BDA0002226711360000084
calculating the number of points in each slice:
Figure BDA0002226711360000085
in the formula (19), i is a slice siThe subscript of (a) is,
the total number of points in each slice is:
Figure BDA0002226711360000091
drawing a histogram according to the number of the middle points of each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise, setting the threshold value epsilon as 0, and setting delta z as 0.1; the 1-0 transformation in the histogram is the horizontal cut point of the window.
The specific implementation steps of the step 5 are as follows:
step 5.1, calculating each initial window cluster C according to the segmentation result of the step 4 by using a formula (21)iCentral point of (2)
Figure BDA0002226711360000092
The position of (2):
Figure BDA0002226711360000093
wherein N iscFor the number of points in each initial cluster of windows, (x)j,yj,zj) Is CiThe coordinate value of any point;
each central point
Figure BDA0002226711360000094
Representing a window candidate obtained by the segmentation in the step 4, calculating each initial window cluster C respectivelyiThe x, y, z values of medium maximum and minimum, i.e. ximax,yimax,zimaxAnd ximin,yimin,ziminUsing | ximax-ximinL and | zimax-ziminCalculating the length and width of each window candidate, and removing short point cloud clusters;
step 5.2, recording four neighborhoods of the upper, lower, left and right of each window candidate according to the proximity relation, and combining two nodes of the two neighborhoods into a window when the two nodes of the neighborhoods and the window candidate have similar length or width or when the two nodes of the neighborhoods are in the height range of the window candidate if the neighborhood on one side of the window candidate has a plurality of window nodes; when the condition is not satisfied, the two window joints are not combined.
The invention has the beneficial effects that: according to the building facade window extraction method based on the significance analysis, the objectivity of building facade window extraction is better guaranteed based on the idea of the significance analysis, the segmentation result is optimized by utilizing the proximity relation among windows and the structural similarity of the windows, and the accuracy of building facade window extraction is better guaranteed.
Drawings
FIG. 1 is a data diagram of a facade point cloud in the method for extracting a facade window of a building according to the present invention;
FIG. 2 is a point extraction graph of high global significance in the point cloud of the facade of the building of FIG. 1 in accordance with the present invention;
FIG. 3 is a point extraction graph of high local saliency in the point cloud of the facade of the building of FIG. 1 in accordance with the present invention;
FIG. 4 is a candidate high saliency point map of a building facade window extraction method of the present invention incorporating high global saliency points and high local saliency points;
FIG. 5 is a schematic diagram of the longitudinal segmentation of high significance points in the building facade window extraction method of the present invention;
FIG. 6 is a schematic diagram of the cross-cut of a high significance point in the building facade window extraction method of the present invention;
FIG. 7 is a schematic view of neighborhood window merging in the building facade window extraction method of the present invention;
fig. 8 is a schematic diagram of neighborhood window non-merging in the building facade window extraction method of the invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention relates to a building facade window extraction method based on significance analysis, which adopts building facade point cloud data as shown in figure 1 and is implemented according to the following steps:
step 1, calculating by comparing the difference between the normal vector of each point in the point cloud of the building facade and the main direction of the building facade to obtain the global significance of each point on the facade, and extracting the feature points with high global significance by setting a threshold, specifically:
step 1.1, calculating the main direction of the point cloud of the facade of the building by using a principal component analysis method, which comprises the following steps:
let p beiIs any point in the point cloud of the building facade, N is the total number of the midpoint of the point cloud of the building facade, and the third-order covariance matrix M of the point cloud of the building facade is as follows:
Figure BDA0002226711360000111
in the formula (1), the reaction mixture is,
Figure BDA0002226711360000112
is the average position of the points in the point cloud of the facade,
Figure BDA0002226711360000113
performing eigenvalue decomposition on a third-order covariance matrix M of the point cloud of the opposite surface through singular value decomposition to obtain an eigenvalue lambda of the third-order covariance matrix M3、λ2、λ1Wherein λ is3≥λ2≥λ1If > 0, the minimum eigenvalue λ is1Corresponding feature vector
Figure BDA0002226711360000114
The method comprises the following steps of (1) obtaining a main direction of a point cloud of a facade of a building;
step 1.2, calculating a normal vector of each point in the facade point cloud by using a principal component analysis method, specifically comprising the following steps: finding out point cloud midpoint p of building facade point by using k-d treeiK adjacent points pj，pj∈{p1，p2，…pkPoint piThird order covariance matrix of
Figure BDA0002226711360000115
Comprises the following steps:
Figure BDA0002226711360000116
in the formula (3), p' is a point piThe average position of the k neighboring points of (c),
Figure BDA0002226711360000117
by singular value decomposition of the point piThird order covariance matrix of
Figure BDA0002226711360000118
Performing eigenvalue decomposition to obtain a third-order covariance matrix
Figure BDA0002226711360000119
Characteristic value of
Figure BDA00022267113600001110
Wherein the content of the first and second substances,
Figure BDA00022267113600001111
then point piThe normal vector of (a) is a minimum eigenvalue
Figure BDA00022267113600001112
Corresponding feature vector
Figure BDA00022267113600001113
I.e. point piHas a normal vector of
Figure BDA00022267113600001114
Step 1.3, calculating the global significance of the points, specifically:
step 1.3.1, calculating the point cloud midpoint p of the facade of the buildingiNormal vector of (1)
Figure BDA00022267113600001115
In the main direction of the facade of a building
Figure BDA00022267113600001116
Distance d ofi：
Figure BDA0002226711360000121
Will point piIs defined as the distance diMapping in hyperbolic tangent function space:
Figure BDA0002226711360000122
in the formula (6), lambda controls the steepness of the shape of the hyperbolic tangent function, and c is the geometric centroid of the hyperbolic tangent function;
according to the distance d in the point cloud of the facade of the buildingiDistribution range of (d)iDividing into T parts, counting the distance d between the normal direction of all points and the main direction of the building facadeiThe number of points falling within the range of T values, assuming NtRespectively representing the number of points falling in the T shares, and then the probability that a certain point in the point cloud of the facade of the building falls in the T-th component in the histogram
Figure BDA0002226711360000127
Comprises the following steps:
Figure BDA0002226711360000124
the geometric centroid c of the hyperbolic tangent function is:
Figure BDA0002226711360000125
in the formula (8), htTo correspond to the distance diThe middle value of the value range of the tth component of the distribution histogram, wherein T is 5, and lambda is 0.5;
step 1.3.2, extracting points with high global significance by setting a threshold α,
Figure BDA0002226711360000126
if S isglobal(pi) Greater than α, point piOtherwise, the point is an insignificant point, wherein the threshold α is in the range of 0.5-0.6, as shown in fig. 2, the high global significant point is extracted;
step 2, calculating the geometric feature difference between each point and a local adjacent point in the building facade point cloud by using the fast point feature histogram descriptor to obtain the local significance of each point, and extracting feature points with high local significance by setting a threshold, wherein the method specifically comprises the following steps:
step 2.1, calculating a Fast Point Feature Histogram (FPFH) of each point in the point cloud of the facade of the building, and assuming piFinding out the midpoint p of the point cloud of the facade of the building by using a k-d treeiK adjacent points pj，pj∈{p1,p2,…pk}，piHas a normal vector of
Figure BDA0002226711360000131
pjHas a normal vector of
Figure BDA0002226711360000132
At point piThe local coordinate system u, v, w is defined above:
Figure BDA0002226711360000133
calculation of a normal vector using equation (11)
Figure BDA0002226711360000134
And
Figure BDA0002226711360000135
deviation α between, φ, θ:
Figure BDA0002226711360000136
in equation (11), α, φ, θ form a simple point feature histogram SPFH, point piThe fast point feature histogram FPFH can be calculated using equation (12):
Figure BDA0002226711360000137
in the formula (12), wjIs a point piAnd its neighboring point pjThe euclidean distance between;
for any two points p in the point cloud of the facade of the buildingiAnd pjGeometric dissimilarity thereof
Figure BDA0002226711360000138
Can be calculated according to equation (13):
Figure BDA0002226711360000139
step 2.2: p obtained according to step 2.1iAnd point pjGeometric dissimilarity of, point piAnd point pjLocal dissimilarity of (d)H(pi,pj) Calculating using equation (14):
Figure BDA0002226711360000141
in the formula (14), | | pi-pj| | is the distance between two points;
then, point piLocal significance of Slocal(pi) Comprises the following steps:
Figure BDA0002226711360000142
step 2.3, extracting points with high local significance in the point cloud in the building by setting a threshold value β:
Figure BDA0002226711360000143
if S islocal(pi) Greater than β, point piOtherwise, the point is an insignificant point, wherein β is set to 0.9, which is the extracted high local significant point shown in fig. 3;
step 3, combining the points with high global significance obtained in the step 1 and the points with high local significance obtained in the step 2 to obtain a candidate high significant point set shown in fig. 4, traversing the candidate high significant point set to obtain a maximum z value zmaxAnd the minimum z value zmin，ziIs a point piRemove points closer to the maximum and minimum z values, i.e., if | zmax-zi< gamma or | zmin-ziIf [ gamma ] is less than [ gamma ], and [ gamma ] is set to 0.1-0.2, the point p is removediA highly significant set of points is obtained which constitutes the window frame.
Step 4, according to the significant feature point set obtained in the step 3, firstly, longitudinally segmenting the significant feature point set, constructing a cumulative histogram of point cloud data on each slice, and converting the cumulative histogram into a binary form by setting a threshold, wherein a 1-0 conversion position is a vertical segmentation position, and then performing the same operation with the vertical direction on the horizontal direction of the significant feature point set obtained in the step 3 to determine a horizontal segmentation position, specifically:
step 4.1, traversing each point in the high salient point set obtained in the step 3, and respectively calculating the most significant points in the high salient point setThe maximum and minimum x, y values, x and y being the x, y coordinates of the point, i.e. xmax，ymax，xminAnd yminIf (x)max-xmin)<(ymax-ymin) If not, segmenting the point cloud of the building facade along the y axis, otherwise, segmenting along the x axis, determining the number L of segments, and segmenting to obtain each segment si＝{s1,s2,s3…sLThe width of each slice is:
Figure BDA0002226711360000151
calculating the number of points in each slice:
Figure BDA0002226711360000152
in the formula (17), i is a slice siA subscript of (a);
the total number of points in each slice is:
Figure BDA0002226711360000153
drawing a histogram according to the number of the middle points of each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise setting the slice as 0, and setting delta y as 0.1, wherein a 1-0 transformation position in the histogram is a vertical cutting position of the window as shown in fig. 5;
step 4.2, traversing each point in the high salient point set obtained in the step 3, and respectively calculating the maximum and minimum z values in the high salient point set, namely zmaxAnd zmin，ziFor the z coordinate of the point, carrying out segmentation along the z value, determining the number L of the segmentation, and obtaining each slice s by segmentationi＝{s1,s2,s3…sLThe width of each slice is:
Figure BDA0002226711360000154
calculating the number of points in each slice:
Figure BDA0002226711360000155
in the formula (19), i is a slice siThe subscript of (a) is,
the total number of points in each slice is:
Figure BDA0002226711360000156
drawing a histogram according to the number of the middle points in each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise setting the slice as 0, and setting the delta z as 0.1, wherein the 1-0 transformation position in the histogram is the horizontal dividing point of the window as shown in fig. 6.
And 5, correcting the segmentation result obtained in the step 4 according to the proximity relation between the windows and the structural similarity of the windows, namely finishing the extraction of the building facade windows, which specifically comprises the following steps:
step 5.1, calculating each initial window cluster C according to the segmentation result of the step 4 by using a formula (21)iCentral point of (2)
Figure BDA0002226711360000161
The position of (2):
Figure BDA0002226711360000162
wherein N iscFor the number of points in each initial cluster of windows, (x)j,yj,zj) Is CiThe coordinate value of any point;
each central point
Figure BDA0002226711360000163
Representing a window candidate obtained by the segmentation in the step 4, calculating each initial window cluster C respectivelyiThe x, y, z values of medium maximum and minimum, i.e. ximax,yimax,zimaxAnd ximin,yimin,ziminUsing | ximax-ximinL and | zimax-ziminCalculating the length and width of each window candidate, and removing short point cloud clusters;
step 5.2, recording four neighborhoods of the upper, lower, left and right of each window candidate according to the proximity relation, and combining two nodes of the two neighborhoods into a window when the two nodes of the neighborhoods and the window candidate have similar length or width or when the two nodes of the neighborhoods are in the height range of the window candidate if the neighborhood on one side of the window candidate has a plurality of window nodes; when the condition is not satisfied, the two window joints are not combined.
As shown in fig. 7, for a window W1In the window W2In the window W3And window W4Respectively is a window W1Left neighborhood, upper neighborhood, lower neighborhood, window W5And window W6Is a window W1The right neighborhood of (c). Window W5And window W6And window W1Have a similar width, and the window W5The highest point and the window W6Is at the lowest position of the window W1So that the window W is5And window W6Should be incorporated into the same window.
As shown in FIG. 8, the window W8And window W9Is a window W7Right neighborhood of (W), window W8And window W9And window W7Different in length and width, and a window W8The highest point and the window W9Has already exceeded W7Height range of (1), thus W8And W9Cannot be combined into the same window.
According to the building facade window extraction method based on significance analysis, for the three-dimensional point cloud object, the significance region has strong uniqueness and invariance and is greatly different from surrounding regions. No matter the volume of the observed three-dimensional object is enlarged or reduced, the data density distribution is dense or sparse, and the salient region has strong invariance, so that the objectivity of the extraction of the building facade window is better ensured based on the idea of significance analysis, the segmentation result is optimized by utilizing the proximity relation between the windows and the structural similarity of the windows, and the accuracy of the extraction of the building facade window is better ensured.

Claims (9)

1. A building facade window extraction method based on significance analysis is characterized by comprising the following steps:
step 1, calculating by comparing the difference between the normal vector of each point in the point cloud of the facade of the building and the main direction of the facade of the building to obtain the global significance of each point on the facade, and extracting the feature points with high global significance by setting a threshold;
step 2, calculating the geometric feature difference between each point and a local adjacent point in the point cloud of the facade of the building by using the fast point feature histogram descriptor to obtain the local significance of each point, and extracting feature points with high local significance by setting a threshold;
step 3, merging the characteristic point set obtained in the step 1 and the characteristic points with high local significance obtained in the step 2, and removing points which are close to the highest point and the lowest point in the merged characteristic point set to obtain a high-significance characteristic point set forming a window frame;
step 4, according to the significant feature point set obtained in the step 3, firstly, longitudinally segmenting the significant feature point set, constructing a cumulative histogram of point cloud data on each slice, converting the cumulative histogram into a binary form by setting a threshold value, wherein a 1-0 conversion position is a vertical segmentation position, and then performing the same operation with the vertical direction on the horizontal direction of the significant feature point set obtained in the step 3 to determine a horizontal segmentation position;
and 5, correcting the segmentation result obtained in the step 4 according to the proximity relation between the windows and the structural similarity of the windows, namely finishing the extraction of the building facade windows.
2. The building facade window extraction method based on the significance analysis is characterized in that the step 1 is implemented by the following steps:
step 1.1, calculating the main direction of the point cloud of the facade of the building by using a principal component analysis method;
step 1.2, calculating a normal vector of each point in the facade point cloud by using a principal component analysis method;
and 1.3, calculating the global significance of the points.
3. The building facade window extraction method based on the significance analysis according to claim 2, wherein the step 1.1 comprises the following specific steps:
let p beiIs any point in the point cloud of the building facade, N is the total number of the midpoint of the point cloud of the building facade, and the third-order covariance matrix M of the point cloud of the building facade is as follows:
Figure FDA0002226711350000021
in the formula (1), the reaction mixture is,
Figure FDA0002226711350000022
is the average position of the points in the point cloud of the facade,
Figure FDA0002226711350000023
performing eigenvalue decomposition on a third-order covariance matrix M of the point cloud of the opposite surface through singular value decomposition to obtain an eigenvalue lambda of the third-order covariance matrix M3、λ2、λ1Wherein λ is3≥λ2≥λ1If > 0, the minimum eigenvalue λ is1Corresponding feature vector
Figure FDA0002226711350000024
The main direction of the point cloud of the facade of the building is obtained.
4. The building facade window extraction method based on the significance analysis according to claim 3, wherein the step 1.2 comprises the following specific steps:
finding out point cloud midpoint p of building facade point by using k-d treeiK adjacent points pj，pj∈{p1,p2,…pkPoint piThird order covariance matrix of
Figure FDA0002226711350000025
Comprises the following steps:
Figure FDA0002226711350000026
in the formula (3), p' is a point piThe average position of the k neighboring points of (c),
Figure FDA0002226711350000027
by singular value decomposition of the point piThird order covariance matrix of
Figure FDA0002226711350000028
Performing eigenvalue decomposition to obtain a third-order covariance matrix
Figure FDA0002226711350000029
Characteristic value of
Figure FDA00022267113500000210
Wherein the content of the first and second substances,
Figure FDA00022267113500000211
then point piThe normal vector of (a) is a minimum eigenvalue
Figure FDA00022267113500000212
Corresponding feature vector
Figure FDA00022267113500000213
I.e. point piHas a normal vector of
Figure FDA0002226711350000031
5. The building facade window extraction method based on the significance analysis according to claim 4, wherein the step 1.3 is as follows:
step 1.3.1, calculating the point cloud midpoint p of the facade of the buildingiNormal vector of (1)
Figure FDA0002226711350000032
In the main direction of the facade of a building
Figure FDA0002226711350000033
Distance d ofi：
Figure FDA0002226711350000034
Will point piIs defined as the distance diMapping in hyperbolic tangent function space:
Figure FDA0002226711350000035
in the formula (6), lambda is the degree of steepness for controlling the shape of the hyperbolic tangent function, and c is the geometric centroid of the hyperbolic tangent function;
according to the distance d in the point cloud of the facade of the buildingiDistribution range of (d)iDividing into T parts, counting the distance d between the normal direction of all points and the main direction of the building facadeiThe number of points falling within the range of T values, assuming NtRespectively representing the number of points falling in the T shares, and then the probability that a certain point in the point cloud of the facade of the building falls in the T-th component in the histogramComprises the following steps:
Figure FDA0002226711350000037
the geometric centroid c of the hyperbolic tangent function is:
Figure FDA0002226711350000038
in the formula (8), htTo correspond to the distance diThe middle value of the value range of the tth component of the distribution histogram, wherein T is 5, and lambda is 0.5;
step 1.3.2, extracting points with high global significance by setting a threshold α,
Figure FDA0002226711350000041
if S isglobal(pi) Greater than α, point piI.e., a high global saliency point, otherwise, the point is a non-saliency point, with the threshold α ranging from 0.5-0.6.
6. The building facade window extraction method based on the significance analysis according to claim 1, wherein the step 2 is implemented by the following steps:
step 2.1, calculating a Fast Point Feature Histogram (FPFH) of each point in the point cloud of the facade of the building, and assuming piFinding out the midpoint p of the point cloud of the facade of the building by using a k-d treeiK adjacent points pj，pj∈{p1,p2,…pk}，piHas a normal vector of
Figure FDA0002226711350000042
pjHas a normal vector of
Figure FDA0002226711350000043
At point piThe local coordinate system u, v, w is defined above:
Figure FDA0002226711350000044
calculation of a normal vector using equation (11)
Figure FDA0002226711350000045
And
Figure FDA0002226711350000046
deviation α between, φ, θ:
Figure FDA0002226711350000047
in equation (11), α, φ, θ form a simple point feature histogram SPFH, point piThe fast point feature histogram FPFH can be calculated using equation (12):
Figure FDA0002226711350000048
in the formula (12), wjIs a point piAnd its neighboring point pjThe euclidean distance between;
for any two points p in the point cloud of the facade of the buildingiAnd pjGeometric dissimilarity thereof
Figure FDA0002226711350000049
Can be calculated according to equation (13):
Figure FDA0002226711350000051
step 2.2: p obtained according to step 2.1iAnd point pjGeometric dissimilarity of, point piAnd point pjLocal dissimilarity of (d)H(pi,pj) Calculating using equation (14):
Figure FDA0002226711350000052
in the formula (14), | | pi-pj| | is the distance between two points;
then, point piLocal significance of Slocal(pi) Is composed of：
Figure FDA0002226711350000053
Step 2.3, extracting points with high local significance in the point cloud in the building by setting a threshold value β:
Figure FDA0002226711350000054
if S islocal(pi) Greater than β, point piI.e., a high local saliency point, otherwise, this point is a non-saliency point, with β set to 0.9.
7. The building facade window extraction method based on the significance analysis according to claim 1, wherein the concrete steps of the step 3 are as follows: combining the points with high global significance obtained in the step 1 and the points with high local significance obtained in the step 2 to obtain a candidate high significant point set, traversing the candidate high significant point set to obtain a maximum z value zmaxAnd the minimum z value zmin，ziIs a point piRemove points closer to the maximum and minimum z values, i.e., if | zmax-zi< gamma or | zmin-ziIf [ gamma ] is less than [ gamma ], and [ gamma ] is set to 0.1-0.2, the point p is removediA highly significant set of points is obtained which constitutes the window frame.
8. The building facade window extraction method based on the significance analysis according to claim 1 or 7, wherein the step 4 is implemented by the following steps:
step 4.1, traversing each point in the high salient point set obtained in step 3, and respectively calculating the maximum and minimum x and y values in the high salient point set, wherein x and y are the x and y coordinates of the point, namely xmax，ymax，xminAnd yminIf (x)max-xmin)<(ymax-ymin) Then the point cloud of the facade of the building is segmented along the y axis,otherwise, carrying out segmentation along the x axis, determining the number L of the segmentation, and then obtaining each slice s by segmentationi＝{s1,s2,s3…sLThe width of each slice is:
Figure FDA0002226711350000061
calculating the number of points in each slice:
Figure FDA0002226711350000062
in the formula (17), i is a slice siA subscript of (a);
the total number of points in each slice is:
Figure FDA0002226711350000063
drawing a histogram according to the number of the middle points of each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise, setting the slice as 0, and setting delta y as 0.1; the 1-0 transformation position in the histogram is the vertical cutting position of the window;
step 4.2, traversing each point in the high salient point set obtained in the step 3, and respectively calculating the maximum and minimum z values in the high salient point set, namely zmaxAnd zmin，ziFor the z coordinate of the point, carrying out segmentation along the z value, determining the number L of the segmentation, and obtaining each slice s by segmentationi＝{s1,s2,s3…sLThe width of each slice is:
Figure FDA0002226711350000064
calculating the number of points in each slice:
Figure FDA0002226711350000065
in the formula (19), i is a slice siThe subscript of (a) is,
the total number of points in each slice is:
Figure FDA0002226711350000071
drawing a histogram according to the number of the middle points of each slice, setting a threshold value epsilon, setting the slice with the threshold value larger than epsilon as 1, otherwise, setting the threshold value epsilon as 0, and setting delta z as 0.1; the 1-0 transformation in the histogram is the horizontal cut point of the window.
9. The building facade window extraction method based on the significance analysis according to claim 1, wherein the concrete implementation steps of the step 5 are as follows:
step 5.1, calculating each initial window cluster C according to the segmentation result of the step 4 by using a formula (21)iCentral point of (2)
Figure FDA0002226711350000072
The position of (2):
Figure FDA0002226711350000073
wherein N iscFor the number of points in each initial cluster of windows, (x)j,yj,zj) Is CiThe coordinate value of any point;
each central point
Figure FDA0002226711350000074
Representing a window candidate obtained by the segmentation in the step 4, calculating each initial window cluster C respectivelyiThe x, y, z values of medium maximum and minimum, i.e. ximax,yimax,zimaxAnd ximin,yimin,ziminUsing | ximax-ximinL and | zimax-ziminCalculating the length and width of each window candidate, and removing short point cloud clusters;
step 5.2, recording four neighborhoods of the upper, lower, left and right of each window candidate according to the proximity relation, and combining two nodes of the two neighborhoods into a window when the two nodes of the neighborhoods and the window candidate have similar length or width or when the two nodes of the neighborhoods are in the height range of the window candidate if the neighborhood on one side of the window candidate has a plurality of window nodes; when the condition is not satisfied, the two window joints are not combined.
CN201910954155.1A 2019-10-09 2019-10-09 Point cloud building facade window extraction method based on significance analysis Active CN110910387B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910954155.1A CN110910387B (en) 2019-10-09 2019-10-09 Point cloud building facade window extraction method based on significance analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910954155.1A CN110910387B (en) 2019-10-09 2019-10-09 Point cloud building facade window extraction method based on significance analysis

Publications (2)

Publication Number Publication Date
CN110910387A true CN110910387A (en) 2020-03-24
CN110910387B CN110910387B (en) 2022-03-04

Family

ID=69815359

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910954155.1A Active CN110910387B (en) 2019-10-09 2019-10-09 Point cloud building facade window extraction method based on significance analysis

Country Status (1)

Country Link
CN (1) CN110910387B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393467A (en) * 2021-06-21 2021-09-14 合肥工业大学 Point cloud video self-adaptive segmentation method based on mixed significance detection
CN114092697A (en) * 2021-11-09 2022-02-25 南京林业大学 Building facade semantic segmentation method with attention fused with global and local depth features
CN114092775A (en) * 2021-11-23 2022-02-25 中国人民解放军战略支援部队航天工程大学士官学校 Method, system and equipment for extracting window of supervision type building
CN114742968A (en) * 2022-06-13 2022-07-12 西南石油大学 Elevation map generation method based on building elevation point cloud
CN114882192A (en) * 2022-07-08 2022-08-09 浙江国遥地理信息技术有限公司 Building facade segmentation method and device, electronic equipment and storage medium
CN116563171A (en) * 2023-07-11 2023-08-08 深圳大学 Point cloud enhancement method and related equipment for building repeated structure

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090310867A1 (en) * 2008-06-12 2009-12-17 Bogdan Calin Mihai Matei Building segmentation for densely built urban regions using aerial lidar data
CN104392486A (en) * 2014-11-25 2015-03-04 西安理工大学 Point-cloud scene rebuilding method
CN104484668A (en) * 2015-01-19 2015-04-01 武汉大学 Unmanned aerial vehicle multi-overlapped-remote-sensing-image method for extracting building contour line
CN104700398A (en) * 2014-12-31 2015-06-10 西安理工大学 Point cloud scene object extracting method
CN106407925A (en) * 2016-09-09 2017-02-15 厦门大学 Automatic extracting method of laser scanning point cloud tree based on local interval maximal value
CN107146280A (en) * 2017-05-09 2017-09-08 西安理工大学 A kind of point cloud building method for reconstructing based on cutting
CN110047036A (en) * 2019-04-22 2019-07-23 重庆交通大学 Territorial laser scanning data building facade extracting method based on polar coordinates grid

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090310867A1 (en) * 2008-06-12 2009-12-17 Bogdan Calin Mihai Matei Building segmentation for densely built urban regions using aerial lidar data
CN104392486A (en) * 2014-11-25 2015-03-04 西安理工大学 Point-cloud scene rebuilding method
CN104700398A (en) * 2014-12-31 2015-06-10 西安理工大学 Point cloud scene object extracting method
CN104484668A (en) * 2015-01-19 2015-04-01 武汉大学 Unmanned aerial vehicle multi-overlapped-remote-sensing-image method for extracting building contour line
CN106407925A (en) * 2016-09-09 2017-02-15 厦门大学 Automatic extracting method of laser scanning point cloud tree based on local interval maximal value
CN107146280A (en) * 2017-05-09 2017-09-08 西安理工大学 A kind of point cloud building method for reconstructing based on cutting
CN110047036A (en) * 2019-04-22 2019-07-23 重庆交通大学 Territorial laser scanning data building facade extracting method based on polar coordinates grid

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HANYUN WANG ET AL.: ""Road Boundaries Detection Based on Local Normal Saliency From Mobile Laser Scanning Data"", 《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113393467A (en) * 2021-06-21 2021-09-14 合肥工业大学 Point cloud video self-adaptive segmentation method based on mixed significance detection
CN113393467B (en) * 2021-06-21 2022-10-28 合肥工业大学 Point cloud video self-adaptive segmentation method based on mixed significance detection
CN114092697A (en) * 2021-11-09 2022-02-25 南京林业大学 Building facade semantic segmentation method with attention fused with global and local depth features
CN114092775A (en) * 2021-11-23 2022-02-25 中国人民解放军战略支援部队航天工程大学士官学校 Method, system and equipment for extracting window of supervision type building
CN114742968A (en) * 2022-06-13 2022-07-12 西南石油大学 Elevation map generation method based on building elevation point cloud
CN114742968B (en) * 2022-06-13 2022-08-19 西南石油大学 Elevation map generation method based on building elevation point cloud
CN114882192A (en) * 2022-07-08 2022-08-09 浙江国遥地理信息技术有限公司 Building facade segmentation method and device, electronic equipment and storage medium
CN116563171A (en) * 2023-07-11 2023-08-08 深圳大学 Point cloud enhancement method and related equipment for building repeated structure
CN116563171B (en) * 2023-07-11 2023-11-28 深圳大学 Point cloud enhancement method and related equipment for building repeated structure

Also Published As

Publication number Publication date
CN110910387B (en) 2022-03-04

Similar Documents

Publication Publication Date Title
CN110910387B (en) Point cloud building facade window extraction method based on significance analysis
CN107146280B (en) Point cloud building reconstruction method based on segmentation
CN111681197B (en) Remote sensing image unsupervised change detection method based on Siamese network structure
CN108681737B (en) Method for extracting image features under complex illumination
CN112164145B (en) Method for rapidly extracting indoor three-dimensional line segment structure based on point cloud data
CN114387329B (en) Building contour progressive regularization method based on high-resolution remote sensing image
CN110473224B (en) Automatic RSF level set image segmentation method based on KL entropy
CN114463396B (en) Point cloud registration method utilizing plane shape and topological graph voting
CN114972743A (en) Radius expansion-based hierarchical single tree extraction method
CN112070787B (en) Aviation three-dimensional point cloud plane segmentation method based on opponent reasoning theory
CN112734816A (en) Heterogeneous image registration method based on CSS-Delaunay
CN109544578B (en) Three-dimensional fragment recombination restoration method based on maximum similarity matching
CN109977892B (en) Ship detection method based on local saliency features and CNN-SVM
CN109117841B (en) Scene text detection method based on stroke width transformation and convolutional neural network
CN116824379A (en) Laser point cloud building contour progressive optimization method based on multidimensional features
Friedman et al. Real Time Detection of Repeated Structures in Point Clouds of Urban Scenes.
CN114283199B (en) Dynamic scene-oriented dotted line fusion semantic SLAM method
CN116469092A (en) Self-adaptive Alpha Shapes contour extraction method based on DBSCAN algorithm
CN115205560A (en) Monocular camera-based prior map-assisted indoor positioning method
CN111626325B (en) Feature-based image matching method
CN115409954A (en) Dense point cloud map construction method based on ORB feature points
Xu et al. A novel method for wheel rim recognition
Burochin et al. An unsupervised hierarchical segmentation of a facade building image in elementary 2D-models
CN112949361A (en) Fingerprint identification method and device
Cai et al. A stereo matching algorithm based on color segments

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant