CN112037230B - Forest image segmentation method based on superpixels and hyper-metric profile map - Google Patents
Forest image segmentation method based on superpixels and hyper-metric profile map Download PDFInfo
- Publication number
- CN112037230B CN112037230B CN201910481471.1A CN201910481471A CN112037230B CN 112037230 B CN112037230 B CN 112037230B CN 201910481471 A CN201910481471 A CN 201910481471A CN 112037230 B CN112037230 B CN 112037230B
- Authority
- CN
- China
- Prior art keywords
- super
- segmentation
- pixel
- image
- dissimilarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000003709 image segmentation Methods 0.000 title claims abstract description 22
- 230000011218 segmentation Effects 0.000 claims abstract description 120
- 230000003595 spectral effect Effects 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 27
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 abstract description 46
- 230000000694 effects Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 17
- 239000003086 colorant Substances 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000004456 color vision Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a forest image segmentation method based on super pixels and a super metric profile map, which comprises the steps of firstly segmenting an image into super pixels by utilizing linear spectral clustering, and calculating dissimilarity between adjacent super pixel areas; then merging the areas with higher similarity from bottom to top, and simultaneously updating the edge weights in the excessive metric profile; and finally outputting the image segmented by the optimal weight threshold. A smaller weight threshold will produce over-segmentation, with segmentation results retaining only edges of higher significance as the threshold increases. The invention can set the segmentation threshold T autonomously, and can set the segmentation threshold T with different sizes according to actual needs so as to obtain a better segmentation result. Compared with other existing image segmentation algorithms, the method has the advantages of remarkably reduced operation complexity, higher algorithm speed, good segmentation effect, small dependence on initial parameters and the like, is very suitable for segmenting high-resolution unmanned aerial vehicle forest region aerial images, and has high popularization and application values.
Description
Technical Field
The invention relates to the field of image segmentation, in particular to a forest image segmentation method based on super pixels and a super metric profile map.
Background
The unmanned aerial vehicle as a low-altitude remote sensing tool has the advantages of low flight cost, simple and convenient operation, flexibility, free shooting time and the like, has remarkable results in the fields of geological exploration, environmental investigation, victim detection and the like, and has immeasurable development prospect. In order to realize intelligent analysis of image information captured by an unmanned aerial vehicle, research on a rapid and high-quality image segmentation algorithm is always a hotspot problem in the image field. The image segmentation refers to dividing an image into a plurality of mutually non-overlapping areas, so that the same area presents similarity and the adjacent areas are obviously different, and the method is a key step in image analysis, pattern recognition and computer vision.
Because unmanned aerial vehicle aerial images have high resolution, complex scenes and various application ranges, no very mature segmentation algorithm can be suitable for various unmanned aerial vehicle images nowadays.
Super-pixels are subregions in an image which have consistency and can maintain local structural features of the image, and the essence of a linear spectral clustering (linear spectral clustering, LSC) super-pixel generation algorithm is to map data in an original space to a high-dimensional space and then cluster the data. The algorithm first designs a kernel functionThe pixel features in the image pixel set V are mapped to a high-dimensional feature space, and then image segmentation is achieved using a weighted K-means clustering algorithm. The feature of the pixel point p in the image is denoted as (l) p ,α p ,β p ,x p ,y p ) L, α, β denote the color components L, a and b, respectively, of p in the CIELAB color space, L being the color brightness, the a-channel representing the position between red and green, and the b-channel representing the position between yellow and blue; x and y are the vertical and horizontal coordinates of p in the image plane. Kernel mapping from low-dimensional pixel space to high-dimensional feature space>The definition is as follows:
wherein,,
p and q are pixel points, parameter C c And C s Weights for controlling color and position information, respectively, C s ＝r×C c ，r＝0.075。
The hyper-metric profile is derived from Arbelaez and the like, and an image hierarchy segmentation method based on edge detection is provided, namely, a global edge probability-direction watershed transformation-hyper-metric profile (globalized probability of boundary, oriented watershed transform and ultrametric contour map, gPb-OWT-UCM).
The bottom-up image hierarchy segmentation is constructed by a region merging algorithm. Regarding the initial segmentation as a graph, the initial segmentation region is a node of the graph, connecting adjacent regions R 1 And R is R 2 Edge C of the graph, edge set c= { C }, region R 1 And R is R 2 The dissimilarity between the two is taken as the weight W (C) of the edge, and the weight set is W (C) = { W (C) }. The hierarchical segmentation is a process of sequencing edges through dissimilarity among regions and iteratively merging the most similar regions, and the specific algorithm steps are as follows:
input edge weight set W (C).
And outputting the segmentation result of each iteration.
Step1, selecting the edge C with the smallest weight in W (C) * ，
c * ＝arg min c∈C W(c)。
Step2. If R 1 ，R 2 Is quilt c * The two divided regions are then combined to r=r 1 ∪R 2 。
Step3 delete edge C from edge set C * 。
Step4, if C is an empty set, ending the iteration; otherwise, the edge set C and the weight set W (C) of the edge are updated and the step1 is turned.
Contours generated by the hierarchical segmentation bottom layer can keep boundaries with weaker strength, and meanwhile over-segmentation can be caused; while the profile of the upper layer is only sensitive to the stronger border, part of the necessary edge information may be missing. Considering that the weight of the currently deleted edge is minimum in the hierarchical segmentation process, the weight of all the rest edges must be larger than the weight of the previously deleted edge, so that an hyper-metric contour map with an index hierarchical structure can be constructed, the hierarchical segmentation of the image is understood as a set of different segmentation results under a plurality of scales, and the appropriate contour is obtained by selecting the scales as the optimal segmentation result.
Since the algorithm gPb-OWT-UCM combines multidimensional features (3-dimensional color features+17-dimensional texture features) in a plurality of dimensions in different directions when calculating edge intensity by gPb, the algorithm is obviously unsuitable for high-resolution unmanned aerial vehicle aerial images at the cost of huge computational complexity to obtain accurate segmentation results.
Disclosure of Invention
The invention provides a forest image segmentation method based on super pixels and a super metric profile map, which is used for segmenting an image obtained by aerial photography of an unmanned aerial vehicle.
In order to achieve the above purpose, the present invention provides a forest image segmentation method based on super pixels and super metric profile, which comprises the following steps:
s1: performing superpixel segmentation on an original image to generate a superpixel image comprising a plurality of superpixel areas, wherein the original image is M multiplied by N and is an RGB image;
s2: converting the super-pixel image from an RGB color space to an HSV color space, and equally dividing a hue component interval, a saturation component interval and a brightness component interval of the converted super-pixel image in the HSV color space into n subintervals;
s3: respectively counting and normalizing the number of pixels in each sub-interval of each super-pixel region in the tone component interval, the saturation component interval and the brightness component interval to obtain a normalized histogramWherein m=3×n, n is an integer greater than 1, i represents the i-th super pixel region,/-th super pixel region>Normalized values of the number of pixels in the 1 st to n th sections among the tone component sections, respectively representing the i-th super-pixel region,/>Normalized values of the number of pixels respectively representing the 1 st to n th sections of the ith super pixel region among the saturation component sections,/th>Pixel number normalization values respectively representing the 1 st to n th sections of the ith super pixel region in the brightness component sections;
s4: the dissimilarity D (R) between all adjacent two super-pixel regions is calculated according to i ,R j )：
Wherein R is i 、R j Representing two adjacent super pixel regions;
s5: according to the calculation result of the step S4, a dissimilarity sequence S is obtained according to the sequence from small dissimilarity to large dissimilarity, and each element in the dissimilarity sequence S is the dissimilarity between two adjacent super-pixel areas;
s6: initializing to generate a matrix U, wherein U=M×N and is a zero matrix;
s7: performing one-to-one correspondence between all pixel points located between two adjacent super pixel areas in the super pixel map and elements in the matrix U, and respectively assigning dissimilarities calculated in the step S4 to corresponding elements in the matrix U;
s8: the element U with the minimum value is taken out from the dissimilarity sequence S, two adjacent super-pixel areas corresponding to the element U are combined, the combination rule is that the element in the matrix U corresponding to each pixel point between the two adjacent super-pixel areas corresponding to the element U is updated to the element U, and meanwhile, the histogram of the combined super-pixel area R is calculated to beWhere a () represents the total number of pixels in the super pixel region;
s9: calculating dissimilarity between the merged superpixel region R and all the superpixel regions adjacent to the merged superpixel region R in the step S8, and updating the dissimilarity sequence S according to the dissimilarity;
s10: repeating the steps S8-S9 until the number of elements in the dissimilarity sequence S is zero;
s11: normalizing all elements in the matrix U to be between 0 and 1;
s12: selecting a segmentation threshold T, removing adjacent areas smaller than the segmentation threshold T from the super-pixel image obtained in the step S1, and reserving the adjacent areas larger than the segmentation threshold T to obtain a segmentation image containing a plurality of sub-areas;
s13: and (3) calculating the average color of each subarea in the segmentation map obtained in the step S12, and filling each average color in the corresponding subarea to obtain a final segmentation result map.
In one embodiment of the present invention, in step S1, the number of super pixel areas in the super pixel map is initialized to 100.
In an embodiment of the present invention, in step S1, a super-pixel map is generated by using a linear spectral clustering method.
In an embodiment of the present invention, the range of the hue component interval is between 0 and 255, the range of the saturation component interval is between 0 and 255, and the range of the brightness component interval is between 0 and 255.
In one embodiment of the invention, n is 25.
In an embodiment of the present invention, the normalization function used in step S11 is a sigmoid function.
The forest image segmentation method based on the super-pixel and the super-metric profile map can set the segmentation threshold T autonomously, and can set the segmentation thresholds T with different sizes according to actual needs so as to obtain a better segmentation result. Compared with other existing image segmentation algorithms, the method has the advantages of remarkably reduced operation complexity, higher algorithm speed, good segmentation effect, small dependence on initial parameters and the like, is very suitable for segmenting high-resolution unmanned aerial vehicle forest region aerial images, and has high popularization and application values.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is an original image of an embodiment of the present invention;
FIG. 2 is a super-pixel diagram of an embodiment of the present invention;
FIG. 3 is a diagram illustrating an image corresponding to a normalized matrix U according to an embodiment of the present invention;
fig. 4a is a graph of the segmentation result obtained when the segmentation threshold T is 0.5;
fig. 4b is a graph of the segmentation result obtained when the segmentation threshold T is 0.93;
fig. 4c is a graph of the segmentation result obtained when the segmentation threshold T is 0.99;
FIGS. 5 a-5 d are original images of a second embodiment of the present invention;
FIGS. 6a to 6d are graphs showing the results of manual segmentation according to a second embodiment of the present invention;
FIGS. 7 a-7 d are diagrams illustrating ISODATA segmentation results according to a second embodiment of the present invention;
FIGS. 8 a-8 d are graphs showing FCM segmentation results according to a second embodiment of the present invention;
FIGS. 9 a-9 d are graphs of the segmentation results of gPb-OWT-UCM in accordance with a second embodiment of the present invention;
FIGS. 10a to 10d are diagrams showing the segmentation result of the LSC-UCM according to the second embodiment of the present invention;
FIG. 11 is a graph showing the comparison of the error rates of the algorithms according to the second embodiment of the present invention;
FIG. 12 is a graph showing the average cross-correlation of algorithms according to a second embodiment of the present invention;
FIG. 13 is a gray contrast ratio diagram of each algorithm according to a second embodiment of the present invention;
FIG. 14 is a schematic diagram of a run-time comparison of algorithms according to a second embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without any inventive effort, are intended to be within the scope of the invention.
The invention provides a forest image segmentation method (LSC-UCM) based on super pixels and a super metric profile map, which comprises the steps of firstly utilizing linear spectral clustering to segment an image into super pixels and calculating dissimilarity between adjacent super pixel areas; then merging the areas with higher similarity from bottom to top, and simultaneously updating the edge weights in the excessive metric profile; and finally outputting the image segmented by the optimal weight threshold. A smaller weight threshold will produce over-segmentation, with segmentation results retaining only edges of higher significance as the threshold increases.
The super-pixel segmentation algorithm can quickly generate a group of initial closed regions, and the super-pixel regions have more abundant information than single pixels, so the invention provides the segmentation of the unmanned aerial vehicle image by combining linear spectral clustering super-pixels and a super-metric profile map (LSC-UCM).
The invention provides a forest image segmentation method based on super pixels and a super metric profile map, which comprises the following steps:
s1: performing superpixel segmentation on an original image to generate a superpixel image comprising a plurality of superpixel areas, wherein the original image is M multiplied by N and is an RGB image;
referring to fig. 1 and 2, fig. 1 is an original image of an embodiment of the present invention, and fig. 2 is a super-pixel diagram of an embodiment of the present invention.
In this embodiment, a super-pixel map is generated using a linear spectral clustering method, and the number of super-pixel regions in the super-pixel map is initialized to 100. If the number of the initial super pixel areas is set to be too small, the weaker edges are difficult to separate; if the number of the initial super-pixel areas is set to be too large, a plurality of too small super-pixel areas can appear, the complexity of area combination is increased, and the requirement of segmentation definition and the requirement of combination complexity are considered by initializing the number of the super-pixel areas to 100. It should be noted that, the initializing the number of the super pixel regions to 100 means that the number of the super pixel regions is approximately 100, and the number of the super pixel regions in the actually generated super pixel map may be greater than or less than 100, which is a clear technical means for those skilled in the art, and will not be described herein.
S2: converting the super-pixel image from an RGB color space to an HSV color space, and equally dividing a hue component interval, a saturation component interval and a brightness component interval of the converted super-pixel image in the HSV color space into n subintervals;
because the HSV color space is very close to the human visual system and is suitable for processing and analyzing the color perception characteristics, the color histogram under the HSV space is selected as the color characteristic of the super pixel.
In this embodiment, the values of the hue component H, the saturation component S and the brightness component V in the HSV color space are calculated from the values of the R component, the G component and the B component in the RGB color space, specifically, the following processing is performed on each pixel point in the super-pixel map: firstly, the values of R component, G component and B component (in theory, the value range of 3 components is 0-255) of each pixel point in RGB color space are normalized to be 0-1 (namely, the values of R component, G component and B component are divided by 255), and then the values of hue component H, saturation component S and brightness component V of each pixel point in HSV color space are calculated according to the following formula:
V＝255×max
the above three formulas are respectively calculated for each pixel, i.e. the above 3 formulas are executed for each pixel, max is the maximum value of R component, G component and B component of the pixel, and min is the R component of the pixelThe minimum values of the G component and the B component are calculated, and the calculated values of the hue component H, the saturation component S and the brightness component V are all between 0 and 255 (theoretical range). The range of the hue component interval is defined by the minimum value and the maximum value of the calculated hue component H, the range of the saturation component interval is defined by the minimum value and the maximum value of the calculated saturation component S, and the range of the brightness component interval is defined by the minimum value and the maximum value of the calculated brightness component V, and the hue component interval, the saturation component interval and the brightness component interval are all divided into n subintervals on average on the basis of the range. S3: respectively counting and normalizing the number of pixels in each sub-interval of each super-pixel region (converted according to the step S2) in the tone component interval, the saturation component interval and the brightness component interval to obtain a normalized histogramWherein m=3×n, n is an integer greater than 1, i represents the i-th super pixel region,/-th super pixel region>Normalized values of the number of pixels in the 1 st to n th sections among the tone component sections, respectively representing the i-th super-pixel region,/>Normalized values of the number of pixels respectively representing the 1 st to n th sections of the ith super pixel region among the saturation component sections,/th>Pixel number normalization values respectively representing the 1 st to n th sections of the ith super pixel region in the brightness component sections;
in this embodiment, n is 25, so that m=3×n=75, and in other embodiments, the value of n may be adjusted according to practical situations, where the adjustment principle is that the feature dimension can be reduced while extracting finer features of the hue component interval, the saturation component interval, and the brightness component interval, so as to increase the calculation speed.
S4: the dissimilarity D (R) between all adjacent two super-pixel regions is calculated according to i ,R j )：
Wherein R is i 、R j Representing two adjacent super pixel regions;
s5: according to the calculation result of the step S4, a dissimilarity sequence S is obtained according to the sequence from small dissimilarity to large dissimilarity, and each element in the dissimilarity sequence S is the dissimilarity between two adjacent super-pixel areas;
s6: initializing to generate a matrix U, wherein U=M×N and is a zero matrix;
s7: performing one-to-one correspondence between all pixel points located between two adjacent super pixel areas in the super pixel map and elements in the matrix U, and respectively assigning dissimilarities calculated in the step S4 to corresponding elements in the matrix U;
s8: the element U with the minimum value is taken out from the dissimilarity sequence S, two adjacent super-pixel areas corresponding to the element U are combined, the combination rule is that the element in the matrix U corresponding to each pixel point between the two adjacent super-pixel areas corresponding to the element U is updated to the element U, and meanwhile, the histogram of the combined super-pixel area R is calculated to beWhere a () represents the total number of pixels in the super pixel region;
the values in the dissimilar sequence S represent the threshold value when two adjacent super-pixel regions are merged, i.e. the weight value of the edge to be removed, the smaller the weight value is, the more similar the two super-pixel regions are, the earlier the two super-pixel regions are merged, and the larger the weight value is, the later the edge between the two super-pixel regions is disappeared.
It is also found that, when step S8 is performed, if two adjacent super-pixel regions corresponding to the element U are original regions that have not been merged in the super-pixel map, only the histogram of the merged super-pixel region R is calculated in this step without updating the matrix U, and the updating of the adjacent regions is already performed on the matrix U in step S7.
S9: calculating dissimilarity between the merged superpixel region R and all the superpixel regions adjacent to the merged superpixel region R in the step S8, and updating the dissimilarity sequence S according to the dissimilarity;
s10: repeating the steps S8-S9 until the number of elements in the dissimilarity sequence S is zero;
s11: normalizing all elements in the matrix U to be between 0 and 1;
in this embodiment, the normalization function used here is a sigmoid function, and other normalization functions may be selected in other embodiments. Fig. 3 shows an image corresponding to a normalized matrix U according to an embodiment of the present invention, and in fig. 3, the intensity of the edge corresponding to the edge from deep to shallow is from strong to weak, which indicates that the HSV spatial histogram feature can be well used to measure dissimilarity between regions.
S12: selecting a segmentation threshold T, removing adjacent areas smaller than the segmentation threshold T from the super-pixel image obtained in the step S1, and reserving the adjacent areas larger than the segmentation threshold T to obtain a segmentation image containing a plurality of sub-areas;
s13: and (3) calculating the average color of each subarea in the segmentation map obtained in the step S12, and filling each average color in the corresponding subarea to obtain a final segmentation result map.
FIGS. 4 a-4 c are graphs of segmentation results obtained when the segmentation threshold T is 0.5, 0.93 and 0.99, respectively, FIG. 4a being relatively sensitive to weak edges, resulting in over-segmentation; fig. 4b retains the more significant edges; fig. 4c shows that the segmentation effect is optimal when the segmentation threshold T is 0.99.
In step S1 of this embodiment, a linear spectral clustering algorithm is used, so that a group of closed and compact superpixels can be generated while edge information of an image is maintained, and each pixel point in the output marking matrix is marked as a class label to which it belongs. In the algorithm for generating superpixels by linear spectral clustering, the input is an image I, the output is a marking matrix L, and the specific steps are as follows:
step1. Mapping the features of each pixel p in image I to a high-dimensional feature vector using equation (1)
Step2 at horizontal intervals v x And vertical spacing v y The image is uniformly initialized to K classes.
Step3, setting the iteration times T and initializing the class center m k K=1, 2, …, K, which is the characteristic mean of the pixels within the class.
Step4 computing a neighborhood τv of the class center x ×τv y (τ>0.5 Euclidean distance between each pixel and the class center in the pixel p, updating class label B to which the pixel p belongs p Center labels for class closest to p:
step5 update class center m k The weighted mean value is within the class:
pi in k Representing the set of pixels of the k-th class.
Step6 up to m k Convergence or iteration to T times, otherwise go to Step4.
Step7. Merge the too small area into its neighborhood, the class labels of all pixels form a matrix L. The pixels eventually assigned to the same class form a superpixel.
The invention is further illustrated in the following specific segmentation example (second embodiment):
in this example, the experimental data used are forest images acquired by unmanned aerial vehicle aerial photography, and the resolution of the images is 4000×3000, 4608×2592, etc. The experimental images were taken from the original aerial photograph at 512 x 512 pixels for a total of 4. The CPU of the experimental machine is 3.00GHz, the memory is 16GB, the operating system is Linux, and the development tool is Matlab (R2016 b x) and C++ mixed programming.
Fig. 5a to 5d are original images of the second embodiment of the present invention, image segmentation comparison is performed using the algorithm ISODATA, FCM, gPb-OWT-UCM of fig. 5a to 5d and the proposed LSC-UCM of the present invention, fig. 6a to 6d are manual segmentation result diagrams of the second embodiment of the present invention, fig. 7a to 7d are ISODATA segmentation result diagrams of the second embodiment of the present invention, fig. 8a to 8d are FCM segmentation result diagrams of the second embodiment of the present invention, fig. 9a to 9d are gPb-OWT-UCM segmentation result diagrams of the second embodiment of the present invention, and fig. 10a to 10d are LSC-UCM segmentation result diagrams of the second embodiment of the present invention. The number of categories in the segmentation graphs of ISODATA and FCM is set manually, the segmentation results of gPb-OWT-UCM and LSC-UCM are also related to the selection of UCM threshold values, the best segmentation results of each algorithm are selected in the embodiment, and the best threshold values selected by gPb-OWT-UCM and LSC-UCM are different for different images, but can be determined in a certain interval. The selection of the optimal threshold adopts a coarse-to-fine search mode, all segmentation graphs of the UCM threshold in a certain interval are firstly generated, and then the optimal segmentation results are searched. As the threshold increases, the weak edges fade away and the more significant edges remain. Based on a large number of experiments, the optimal threshold range of the LSC-UCM is 0.5-1, and the optimal threshold range of the gPb-OWT-UCM is 0.1-1. The LSC-UCM sets the number of super pixels to 100, and the number of initial areas automatically generated by the gPb-OWT-UCM is about 3000, so that the calculation complexity is higher. The colors in the LSC-UCM segmentation map are represented by the average value of the colors of pixels in the class, and different colors represent different classes, so that the different classes in the manual segmentation map and other algorithm segmentation maps are also uniformly represented by the corresponding colors in the LSC-UCM segmentation map for convenience of comparison.
From the above segmentation results, it can be known that the ISODATA can automatically adjust the number of clusters by virtue of a split merging mechanism, but the number of initial parameters is more, and the excessive segmentation phenomenon can occur due to the larger influence of the initial parameter values; the FCM has better segmentation effect, but pixels with far space distance or large color difference are divided into the same class, and the clustering number is required to be manually set, so that the segmentation result is greatly influenced; gPb-OWT-UCM and LSC-UCM have a small dependence on initial parameters, can generate closed region contours, and have large similar interval differences in regions. The gPb-OWT-UCM has large calculation amount when the edge intensity is obtained, and the targets with uneven colors are easily misjudged into a plurality of categories, so that the segmentation effect is poorer than that of the LSC-UCM algorithm. The LSC-UCM algorithm provided by the invention is simple in calculation, and the contours of different ground object areas after segmentation are clear and accurate, and are the most similar to the artificial segmentation result.
The following further describes the segmentation evaluation index:
when evaluating the performance of the segmentation algorithm, the performance of the segmentation algorithm with similar effect is difficult to distinguish by subjective discrimination, and quantitative segmentation judgment criteria are required to be introduced for scientific, objective and accurate comparison. The four different algorithms are quantitatively analyzed below using three pixel-based evaluation indexes and run time, namely error rate (error), average cross-over (mean intersection over union, mIoU), and gray-level contrast (GC).
The misclassification rate represents the proportion of misclassified pixels to the whole image, and is often used to describe the segmentation accuracy of a target object, and the calculation formula is as follows:
in N i Represents the number of pixels in class i, N i * For the i-th pixel number of the artificial division, N represents the total pixel number of the image, k is the number of the artificial division categories, and when the category number automatically generated by the algorithm is different from the artificial division category number, the part with the difference is the wrong division. The smaller the error division rate, the smaller the error divided part, and the better the algorithm dividing performance.
The intersection ratio is the ratio of the intersection set and the union set of the algorithm segmentation result and the manual segmentation result, and the image is segmented into k classes manually, so that the average intersection ratio is the average value of various intersection ratios, and the calculation formula is as follows:
in n ii For both manual segmentation and algorithms a certain pixel is divided into the number of pixels of class i,dividing the algorithm into the number of pixels of the i-th class,>for the number of pixels manually segmented into the ith class (k' is the number of algorithm segmentation classes), the numerator and the denominator respectively represent the intersection and the union of the pixels divided into the ith class in the manual segmentation and the algorithm segmentation, and the larger the intersection ratio is, the more similar the result of the algorithm segmentation is to the result of the manual segmentation.
The gray contrast judges the quality of the divided image according to the magnitude of the characteristic contrast between the areas, and if the average gray of any two adjacent areas is f i And f j The gray contrast between them can be calculated as follows:
the larger the gray contrast of the segmentation map, the larger the gap between the regions segmented by the algorithm.
The quantitative analysis is performed on fig. 5a to 10d by taking the error rate, the average cross-over ratio, the gray contrast and the running time as evaluation indexes, and the results are shown in fig. 11 to 14, wherein fig. 11 is a schematic diagram of the error rate comparison of each algorithm in the second embodiment of the invention, fig. 12 is a schematic diagram of the average cross-over ratio of each algorithm in the second embodiment of the invention, fig. 13 is a schematic diagram of the gray contrast of each algorithm in the second embodiment of the invention, and fig. 14 is a schematic diagram of the running time comparison of each algorithm in the second embodiment of the invention.
As shown in fig. 11, LSC-UCM has minimum number of pixels divided by mistake and stable performance; secondly, gPb-OWT-UCM and ISODATA, but the segmentation performance is relatively unstable; the FCM has the greatest error rate and the worst segmentation result.
As shown in fig. 12, the LSC-UCM average cross ratio is the largest, indicating closest approach to the artificial segmentation result; the gPb-OWT-UCM segmentation result is also good; the average cross-over ratio of ISODATA and FCM is relatively low, and the difference between the average cross-over ratio and the artificial segmentation result is large.
As shown in fig. 13, gray contrast between adjacent areas divided by ISODATA is highest; the gray contrast of FCM, gPb-OWT-UCM and LSC-UCM is relatively close, and the dissimilarity between the areas of the segmentation result is slightly poor.
As shown in FIG. 14, the FCM and ISODATA have the shortest running time, the LSC-UCM has the fast running speed, and the gPb-OWT-UCM has the running time about 60 times that of the LSC-UCM and the slowest running speed.
From the analysis, the FCM operates at the highest speed, but the error rate is the highest; the ISODATA algorithm is faster, the dissimilarity between the segmentation areas is large, but the segmentation accuracy is slightly poor; gPb-OWT-UCM segmentation accuracy is high, but the operation complexity is high when gPb features are extracted and an edge intensity image is generated, so that the method is not suitable for processing large-size aerial images; the LSC-UCM provided by the invention has high segmentation precision and running speed far higher than gPb-OWT-UCM, so that the algorithm has obvious advantages in segmentation precision and speed.
The method can quickly and accurately divide the ground objects of different categories in the image, the division result not only keeps the obvious edges of the target area, but also combines the areas with higher similarity, the method is superior to other three algorithms in terms of the error division rate and the average intersection and comparison of the two measures reflecting the image division accuracy, and the operation complexity is obviously improved compared with gPb-OWT-UCM.
The forest image segmentation method based on the super-pixel and the super-metric profile map can set the segmentation threshold T autonomously, and can set the segmentation thresholds T with different sizes according to actual needs so as to obtain a better segmentation result. Compared with other existing image segmentation algorithms, the method has the advantages of remarkably reduced operation complexity, higher algorithm speed, good segmentation effect, small dependence on initial parameters and the like, is very suitable for segmenting high-resolution unmanned aerial vehicle forest region aerial images, and has high popularization and application values.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
Those of ordinary skill in the art will appreciate that: the modules in the apparatus of the embodiments may be distributed in the apparatus of the embodiments according to the description of the embodiments, or may be located in one or more apparatuses different from the present embodiments with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further split into a plurality of sub-modules.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. A forest image segmentation method based on super pixels and a super metric profile map is characterized by comprising the following steps:
s1: performing superpixel segmentation on an original image to generate a superpixel image comprising a plurality of superpixel areas, wherein the original image is M multiplied by N and is an RGB image;
s2: converting the super-pixel image from an RGB color space to an HSV color space, and equally dividing a hue component interval, a saturation component interval and a brightness component interval of the converted super-pixel image in the HSV color space into n subintervals;
s3: respectively counting and normalizing the number of pixels in each sub-interval of each super-pixel region in the tone component interval, the saturation component interval and the brightness component interval to obtain a normalized histogramWherein m=3×n, n is an integer greater than 1, i represents the i-th super pixel region,/-th super pixel region>Normalized values of the number of pixels in the 1 st to n th sections among the tone component sections, respectively representing the i-th super-pixel region,/>Normalized values of the number of pixels respectively representing the 1 st to n th sections of the ith super pixel region among the saturation component sections,/th>Pixel number normalization values respectively representing the 1 st to n th sections of the ith super pixel region in the brightness component sections;
s4: the dissimilarity D (R) between all adjacent two super-pixel regions is calculated according to i ,R j )：
Wherein R is i 、R j Representing two adjacent super pixel regions;
s5: according to the calculation result of the step S4, a dissimilarity sequence S is obtained according to the sequence from small dissimilarity to large dissimilarity, and each element in the dissimilarity sequence S is the dissimilarity between two adjacent super-pixel areas;
s6: initializing to generate a matrix U, wherein U=M×N and is a zero matrix;
s7: performing one-to-one correspondence between all pixel points located between two adjacent super pixel areas in the super pixel map and elements in the matrix U, and respectively assigning dissimilarities calculated in the step S4 to corresponding elements in the matrix U;
s8: the element U with the minimum value is taken out from the dissimilarity sequence S, two adjacent super-pixel areas corresponding to the element U are combined, the combination rule is that the element in the matrix U corresponding to each pixel point between the two adjacent super-pixel areas corresponding to the element U is updated to the element U, and meanwhile, the histogram of the combined super-pixel area R is calculated to beWhere a () represents the total number of pixels in the super pixel region;
s9: calculating dissimilarity between the merged superpixel region R and all the superpixel regions adjacent to the merged superpixel region R in the step S8, and updating the dissimilarity sequence S according to the dissimilarity;
s10: repeating the steps S8-S9 until the number of elements in the dissimilarity sequence S is zero;
s11: normalizing all elements in the matrix U to be between 0 and 1;
s12: selecting a segmentation threshold T, removing adjacent areas smaller than the segmentation threshold T from the super-pixel image obtained in the step S1, and reserving the adjacent areas larger than the segmentation threshold T to obtain a segmentation image containing a plurality of sub-areas;
in this step, comparing the segmentation threshold T with each element in the matrix U after the execution of step S11, for each element in the matrix U, analyzing all elements greater than the segmentation threshold T and all elements less than the segmentation threshold T one by one, and corresponding the elements one to one in the super-pixel map obtained in step S1, removing the edges corresponding to the elements less than the segmentation threshold T in the super-pixel map, and retaining the edges corresponding to the elements greater than the segmentation threshold T in the super-pixel map, where "removing" means removing the contour line for segmentation depicted at the pixel point, and "retaining" means retaining the contour line for segmentation depicted at the corresponding pixel point in the super-pixel map;
s13: and (3) calculating the average color of each subarea in the segmentation map obtained in the step S12, and filling each average color in the corresponding subarea to obtain a final segmentation result map.
2. The forest image segmentation method based on the superpixel and the hypermetric profile according to claim 1, wherein in step S1, the number of superpixel areas in the superpixel map is initialized to 100.
3. The method for segmenting a forest image based on the super-pixel and super-metric profile according to claim 1, wherein in step S1, the super-pixel map is generated by using a linear spectral clustering method.
4. The method for segmenting a forest image based on super pixels and super metric profile according to claim 1, wherein the range of values of the hue component interval is between 0 and 255, the range of values of the saturation component interval is between 0 and 255, and the range of values of the brightness component interval is between 0 and 255.
5. The method of forest image segmentation based on super pixels and super metric profiles as set forth in claim 1, wherein n is 25.
6. The forest image segmentation method based on the super-pixel and super-metric profile according to claim 1, wherein the normalization function used in step S11 is a sigmoid function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481471.1A CN112037230B (en) | 2019-06-04 | 2019-06-04 | Forest image segmentation method based on superpixels and hyper-metric profile map |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481471.1A CN112037230B (en) | 2019-06-04 | 2019-06-04 | Forest image segmentation method based on superpixels and hyper-metric profile map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112037230A CN112037230A (en) | 2020-12-04 |
| CN112037230B true CN112037230B (en) | 2023-07-18 |
Family
ID=73575871
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910481471.1A Active CN112037230B (en) | 2019-06-04 | 2019-06-04 | Forest image segmentation method based on superpixels and hyper-metric profile map |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112037230B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115578660B (en) * | 2022-11-09 | 2023-04-07 | 牧马人(山东)勘察测绘集团有限公司 | Land block segmentation method based on remote sensing image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2682936A1 (en) * | 2012-07-02 | 2014-01-08 | TP Vision Holding B.V. | Image processing |
| CN105118049A (en) * | 2015-07-22 | 2015-12-02 | 东南大学 | Image segmentation method based on super pixel clustering |
| CN105930815A (en) * | 2016-05-04 | 2016-09-07 | 中国农业大学 | Underwater organism detection method and system |
| WO2019062092A1 (en) * | 2017-09-30 | 2019-04-04 | 深圳市颐通科技有限公司 | Superpixel- and multivariate color space-based body outline extraction method |
| CN109785329A (en) * | 2018-10-29 | 2019-05-21 | 重庆师范大学 | Based on the purple soil image segmentation extracting method for improving SLIC algorithm |
-
2019
- 2019-06-04 CN CN201910481471.1A patent/CN112037230B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2682936A1 (en) * | 2012-07-02 | 2014-01-08 | TP Vision Holding B.V. | Image processing |
| CN105118049A (en) * | 2015-07-22 | 2015-12-02 | 东南大学 | Image segmentation method based on super pixel clustering |
| CN105930815A (en) * | 2016-05-04 | 2016-09-07 | 中国农业大学 | Underwater organism detection method and system |
| WO2019062092A1 (en) * | 2017-09-30 | 2019-04-04 | 深圳市颐通科技有限公司 | Superpixel- and multivariate color space-based body outline extraction method |
| CN109785329A (en) * | 2018-10-29 | 2019-05-21 | 重庆师范大学 | Based on the purple soil image segmentation extracting method for improving SLIC algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 基于线性谱聚类的林地图像中枯死树监测;宋以宁;刘文萍;骆有庆;宗世祥;;林业科学(第04期);187-195 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112037230A (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106909902B (en) | Remote sensing target detection method based on improved hierarchical significant model | |
| CN107292339B (en) | Unmanned aerial vehicle low-altitude remote sensing image high-resolution landform classification method based on feature fusion | |
| CN105930868B (en) | A kind of low resolution airport target detection method based on stratification enhancing study | |
| CN106127791B (en) | A kind of contour of building line drawing method of aviation remote sensing image | |
| Gheissari et al. | Person reidentification using spatiotemporal appearance | |
| CN104599275B (en) | The RGB-D scene understanding methods of imparametrization based on probability graph model | |
| CN108537239B (en) | Method for detecting image saliency target | |
| CN110111338B (en) | Visual tracking method based on superpixel space-time saliency segmentation | |
| CN107563413B (en) | Unmanned aerial vehicle aerial image farmland block object accurate extraction method | |
| CN109325484B (en) | Flower image classification method based on background prior significance | |
| CN104966085B (en) | A kind of remote sensing images region of interest area detecting method based on the fusion of more notable features | |
| CN106023257B (en) | A kind of method for tracking target based on rotor wing unmanned aerial vehicle platform | |
| CN107392968B (en) | The image significance detection method of Fusion of Color comparison diagram and Color-spatial distribution figure | |
| CN105761238B (en) | A method of passing through gray-scale statistical data depth information extraction well-marked target | |
| CN105740915B (en) | A kind of collaboration dividing method merging perception information | |
| CN104835175B (en) | Object detection method in a kind of nuclear environment of view-based access control model attention mechanism | |
| CN107369158B (en) | Indoor scene layout estimation and target area extraction method based on RGB-D image | |
| Gupta et al. | Object based information extraction from high resolution satellite imagery using eCognition | |
| CN108154157B (en) | Fast spectral clustering method based on integration | |
| WO2016165064A1 (en) | Robust foreground detection method based on multi-view learning | |
| CN106960182A (en) | A kind of pedestrian integrated based on multiple features recognition methods again | |
| CN114359323B (en) | Image target area detection method based on visual attention mechanism | |
| CN107610136B (en) | Salient object detection method based on convex hull structure center query point sorting | |
| CN115690513A (en) | Urban street tree species identification method based on deep learning | |
| CN113095332B (en) | Saliency region detection method based on feature learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |