CN112818775A - Forest road rapid identification method and system based on regional boundary pixel exchange - Google Patents
Forest road rapid identification method and system based on regional boundary pixel exchange Download PDFInfo
- Publication number
- CN112818775A CN112818775A CN202110075332.6A CN202110075332A CN112818775A CN 112818775 A CN112818775 A CN 112818775A CN 202110075332 A CN202110075332 A CN 202110075332A CN 112818775 A CN112818775 A CN 112818775A
- Authority
- CN
- China
- Prior art keywords
- region
- sub
- boundary
- image
- road
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a forest road real-time identification method and a forest road real-time identification system based on regional boundary pixel exchange. Firstly, video images are collected, extracted and initially segmented based on space constraint. Secondly, similarity judgment is carried out on the boundary pixels of the region by constructing an energy function, and the exchange between the boundary pixels and the brother connected domain is determined according to an energy maximization condition. And (4) finishing the attribution judgment and exchange of all sub-region boundary pixels through iteration, realizing rapid region updating and rapid image segmentation, and generating a new region boundary. And then, introducing a sub-region characteristic operator through a support vector machine model, and quickly identifying and classifying each region. And finally, extracting the boundary coordinates of the road area, and generating a smooth road boundary through spline curve fitting to finish the real-time identification of the non-structured road area of the forest area. The method makes up the problem of error identification of the existing algorithm in the forest region environment, and highlights the accuracy and the real-time performance of identifying the non-structured roads in the forest region.
Description
Technical Field
The invention belongs to the technical field of autonomous navigation and unstructured road identification of special vehicles in forest zones, and particularly relates to a forest zone road rapid identification method and system based on regional boundary pixel exchange.
Background
Because forest roads lack effective reference objects and artificial identification, the forest roads have the characteristics of strong nonlinearity and uncertainty, and the forest roads pose serious challenges to both forest work vehicles and vehicle drivers. Therefore, urgent needs are provided for autonomy and intellectualization during vehicle operation in forest areas. The perception of the forest environment and the identification of the road through the visual information are the basis for realizing the autonomous running of the forest vehicle. The existing road recognition algorithm carried by the autonomous driving platform mainly aims at recognizing the structured road in the urban environment. Although image extraction and semantic recognition of surrounding lane lines, sign signs, signal lights, pedestrians, vehicles, and the like have been basically accomplished, urban roads are greatly different from forest-area unstructured roads. Forest road lacks obvious lane identification and general traffic signal indicating identification, and the existing road recognition algorithm can not carry out reasonable semantic segmentation and semantic understanding on forest road lacking reference objects and artificial identification, and the direct transplantation of the forest road recognition algorithm for forest road detection can cause unpredictable risks. And the environmental perception based on the visual image is a key link for realizing the identification of the forest area unstructured road by the autonomous working vehicle in the forest area. Therefore, on one hand, forest region unstructured road identification based on visual information is beneficial to semantic understanding of forest region autonomous navigation vehicles to roads and accessible regions, and on the other hand, rapid processing and real-time resolving of visual images in the vehicle driving process have important practical significance for achieving forest region vehicle autonomous navigation.
The patent application number is CN201110341479.1, and the invention discloses a Chinese patent of an unstructured road detection method based on adaptive edge registration. In the unstructured road identification [0003], the invention mainly aims at non-urban main roads and other road types without obvious lane line marks (such as campuses, residential quarters, rural roads and the like). Although the road of the type has irregular shape compared with the urban structured road, the road surface has uncertain characteristics such as breakage, cracks and the like. However, compared with the forest road, the road area still has obvious characteristics of relatively definite shape and relatively consistent texture of a large area in the image. The roads in the forest area have different shapes, uncertain interferences such as various weeds, humus, stones, rubbles, tree crown covering, shadows and the like, extremely uneven texture characteristics and serious light influence. In this case, it is difficult to find color and texture features similar to those of non-urban main roads, which results in a predicament in the way-finding behavior. In addition, the Canny edge detection algorithm in the algorithm [0040] is realized based on the characteristic that the country road boundary is irregular but still has quite obvious boundary distinguishing characteristics, and the ideal recognition effect is difficult to achieve on the road surface covered by weeds and riprap in the forest region and with a fuzzy boundary.
The patent application number is CN201610812183.6, and the invention name is Chinese patent of 'an unstructured road identification method'. In the invention, a Laplacion second-order differential operator is adopted to carry out sharpening operation in [0018], and on the basis, OTSU segmentation is carried out only by adopting two index components, namely hue and saturation values of a gray level image in [0025 ]. Although the influence of the illumination condition on the image segmentation is objectively eliminated, the algorithm implementation process does not get rid of the limitation of the color threshold value. Under the condition of complex interference of forest environment, the region segmentation and [0059] operation after the image is sharpened can interfere with the judgment of road regions and the stability of the algorithm, and cause hidden danger. In addition, in order to increase the recognition speed, [0049] adopts a strategy of compressing a picture with the size of 1280 pixels 960 pixels into an image with 320 pixels 240 pixels, so that the recognition accuracy and the recognition effect are greatly influenced for the blurred and irregular road boundaries in forest regions, the image quality is improved, the calculation speed is reduced, the real-time requirement cannot be met, and the algorithm needs to sacrifice and balance between the two.
The patent application number is CN201911145132.2, and the invention name is Chinese patent of 'an unstructured road identification method'. According to the method, the ant colony algorithm is adopted to optimize the parameters of the BP neural network in [0009], although the understanding effect of the neural network on image semantics can be objectively enhanced, the ant colony algorithm is high in space complexity, and the optimization process can cause a local optimal solution. In addition, several indexes of gray average, variance, consistency and smoothness are mainly extracted in the process of extracting characteristic values by image blocks [0060 ]. The basic reason that the algorithm can meet the requirements is that the country road has single and obvious characteristic information, is prominent in contrast characteristic relative to the surrounding environment, has high recognition degree, and has poor recognition effect when being directly transplanted to forest road recognition.
The patent application number is CN201811614332.3, and the invention discloses a Chinese patent named as a road recognition model training method, a road recognition method and a road recognition device. In the multi-frame image processing process [0009], a convolutional neural network structure is adopted to extract relevant features between continuous frame images, the convolutional neural network structure is based on traffic identification and category labels under a large number of structured roads and is used as input to obtain a recognition result, but the model is mainly used for target recognition under urban structured road conditions. In contrast, forest road environments are complex, reference objects and artificial identifications are lacked, and accurate identification cannot be achieved under the road condition that no obvious artificial identifications exist in forest regions through the algorithm.
In the above four methods, the first three types of unstructured road identification are based on non-urban arterial roads and rural roads, but the images of the unstructured roads of the type still have very obvious structure, color and texture characteristics, so that the adopted image characteristics can basically realize the identification of the roads only based on gray level images. The fourth method is based on neural network identification of urban structured roads. However, the unstructured roads in the forest region are far from the similar standards, because a large number of uncertain interference factors such as litter, weeds, soil stones and the like are widely distributed on the unstructured roads in the forest region, which greatly affects the identification of the roads and the resolution of passable areas. Although the methods provide different algorithms for the identification of the unstructured road and the structured road, the methods cannot substantially solve the real-time detection and the image semantic identification of the complex unstructured road in the forest region.
Disclosure of Invention
Aiming at the technical defects in the prior art, the embodiment of the invention aims to provide a method and a system for quickly identifying forest road based on regional boundary pixel exchange.
In a first aspect, an embodiment of the present invention provides a method for quickly identifying a forest road based on area boundary pixel exchange, including:
s1, video image acquisition: acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road area;
s2, image initial segmentation: dividing the image to be processed into N rectangular sub-regions with the same size according to the pixel size, numbering each rectangular sub-region, and defining adjacent rectangular sub-regions as brother connected domains;
s3, boundary pixel swap based sub-region reconstruction: constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with brother connected regions or not based on an energy maximization principle so as to optimize the segmentation of the sub-regions, traversing all sub-region boundary points in an iterative mode, and rapidly completing the reconstruction of the sub-regions and realizing the final segmentation of the image to be processed through the region boundary pixel exchange;
s4, extracting characteristics of the subareas: mapping the image to different color spaces by adopting HSV mean values and texture mean values of the sub-regions subjected to the energy maximization detection and the final segmentation in the step S3, extracting the characteristics of each sub-region, and generating corresponding characteristic operators;
s5, fitting and identifying road boundaries: the method comprises the steps of rapidly resolving a feature operator of a subregion based on boundary pixel exchange through a support vector machine model, judging whether the subregion is classified into two categories or not, carrying out binarization processing on an image on the basis of the classification, preliminarily combining passable subregions into a road region according to a binarization result, extracting coordinates of a left boundary and a right boundary of the region, fitting scattered points of the left boundary coordinate and the right boundary coordinate through spline curves to obtain a smooth road boundary, and finishing unstructured road identification of the forest region.
As a specific embodiment of the present application, in step S3, an energy function based on image features is constructed, specifically:
constructing a color distribution item and a boundary pixel item of the sub-region;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
As a specific embodiment of the present application, in step S3, performing energy maximization detection on the region boundary pixels of the image to be processed by using the energy function, specifically:
firstly, counting a color histogram under a specific color space of each subregion, and then constructing an energy function based on probability density distribution of different color channels of the color histogram, wherein the more concentrated the color distribution in the subregion is, the larger the energy function is; extracting region boundary pixels of the sub-region; and realizing the energy maximization detection and updating of the sub-region through edge pixel exchange and iteration.
As a specific embodiment of the present application, step S4 specifically includes:
carrying out image HSV (hue, saturation and value) mean value processing on a single subregion, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subregion;
calculating and extracting the sub-region texture features, wherein the extracted texture feature mean value comprises four indexes of an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor wavelet texture;
combining the seven characteristic values into a region characteristic operator tau, and describing the characteristics of the segmented sub-regions by using the characteristic operator tau;
and sequentially carrying out rapid feature extraction on each sub-region and generating a corresponding feature operator.
In a second aspect, an embodiment of the present invention provides a forest road fast identification system based on area boundary pixel exchange, including:
the video image acquisition module is used for acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road area;
the image initial segmentation module is used for dividing the image to be processed into N rectangular subregions with the same size according to the pixel size, numbering each rectangular subregion, and defining adjacent rectangular subregions as brother connected regions;
the sub-region reconstruction module is used for constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with a brother communication region or not based on an energy maximization principle so as to optimize the segmentation of the sub-regions, traversing all region boundary points in an iterative mode, and rapidly completing the sub-region reconstruction and realizing the final segmentation of the image to be processed through the region boundary pixel exchange;
the subregion feature extraction module is used for mapping the image to different color spaces by adopting HSV mean values and texture mean values of the subregions subjected to energy maximization detection and final segmentation, performing feature extraction on each subregion, and generating corresponding feature operators;
and the road boundary fitting and identifying module is used for rapidly resolving a characteristic operator of the sub-region based on boundary pixel exchange through a support vector machine model, judging whether the sub-region is classified into two categories or not, carrying out binarization processing on the image on the basis of the classification, preliminarily combining passable regions into the road region according to a binarization result, extracting coordinates of the left and right boundaries of the region, carrying out spline curve fitting to obtain a smooth road boundary, and finishing the non-structured road identification of the forest region.
In a preferred embodiment of the present application, the image capturing device is a monocular vision camera.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
the invention discloses a method for rapidly identifying forest road based on regional boundary pixel exchange, which takes energy function construction and energy function maximization as a core and completes rapid detection and identification on an unstructured road lacking reference objects and artificial identification in a forest region. The method has the advantages that the image area division based on the area boundary pixel exchange does not need to judge all pixels of the frame image, but energy maximization judgment is carried out on the attribution of the edge pixels on the basis of the area color distribution density, the calculation amount is small, the calculation efficiency is high, the identification speed is high, the real-time forest area road detection under the medium-low speed moving condition in the safe driving process of special forest area operation vehicles can be met, the defect that only urban structured roads, rural roads and residential roads can be identified in the existing non-structured road identification and detection technology is overcome, the road detection capability of the forest area operation vehicles in the complex and variable forest area environment is greatly improved, the semantic understanding and identification capability is greatly improved, and technical support is provided for improving the autonomous navigation capability of the forest area operation vehicles.
In addition, compared with the prior art, the method for quickly identifying the forest road has the advantages of high speed, high efficiency and the like.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below.
Fig. 1 is a main flowchart of a method for quickly identifying forest road based on area boundary pixel switching according to an embodiment of the present invention.
Fig. 2 is a diagram illustrating the effect of image segmentation generated after fast swap based on region boundaries.
Fig. 3 is a region two classification based on the region feature operator τ.
Fig. 4 is a diagram of forest region unstructured road boundary extraction and left and right boundary segmentation effects.
FIG. 5 is a process of fitting a smooth road boundary in the calculation process of the present invention.
Fig. 6 is a smooth road boundary fitting effect diagram.
Fig. 7 is a block diagram of a forest road fast identification system based on area boundary pixel switching according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a method for quickly identifying a forest road based on regional boundary pixel switching according to an embodiment of the present invention mainly includes the following steps:
s1, video image acquisition: continuous frame images are acquired in real time through image acquisition equipment, and the frame images are extracted to obtain an image to be processed containing a forest region unstructured road area.
Specifically, a monocular vision camera is adopted to acquire real-time continuous frame images, then the frame images are extracted to obtain images containing forest region unstructured road areas, the acquired images are read into a system, and the images are converted into OpenCV Mat multichannel image types.
For example, after parameter presetting and system initialization are performed on the system, five frames of images are read to form an initial basic buffer queue with the length of 5.
S2, image initial segmentation: according to the pixel size, the image to be processed is divided into N rectangular sub-regions with the same size, each rectangular sub-region is numbered, and adjacent rectangular sub-regions are defined as brother connected domains.
Specifically, a read current frame image is preprocessed, the image is divided into regions according to space constraint and division quantity setting, the image is divided into N rectangular subregions with the same size, each subregion is numbered, the frame image is primarily divided into only image sizes and space constraint, adjacent subregions are defined as 'brother connected regions', the image size collected by a camera in the embodiment is 640 pixels by 480 pixels, the image is divided into 25(N is 5 × 5) initial subregions to simultaneously guarantee the calculation speed and the region division effect, and the initial aspect ratio is 4: 3.
S3, boundary pixel swap based sub-region reconstruction: the method comprises the steps of constructing an energy function based on image characteristics, adopting the energy function to carry out energy maximization detection on each sub-region, judging whether region boundary pixels are exchanged with brother connected regions or not based on an energy maximization principle so as to optimize the segmentation of the sub-regions, traversing all region boundary points in an iteration mode, and rapidly completing the reconstruction of the sub-regions and realizing the final segmentation of an image to be processed through region boundary pixel exchange.
Specifically, S3 mainly includes:
step 301, constructing an energy function of image sub-regions, and realizing an optimal segmentation strategy of the regions through energy function maximization. The energy function is composed of two indexes of a color distribution item and a boundary pixel item. Wherein the energy function formula is:
E(s)＝α*D(s)+β*B(s) (1)
in the formula (1), e(s) is a sub-region energy index, which is also a criterion for determining the energy maximization of the region, and when the term reaches the maximum value, all boundary pixels in the region complete assignment. D(s) is a color distribution term, B(s) is a boundary pixel term index, alpha and beta are proportionality coefficients, weights are distributed to the two indexes, and the attribution of the boundary pixels is judged through the maximization of an energy function.
Step 302, constructing a sub-region color distribution item. The color density distribution of the region is used for determining the main component color cluster of the region through a voting mode, and then the quality index of the color distribution of the region is calculated, wherein the color quality of the region is contributed by each pixel in the region. The sub-region color distribution item is mainly used for measuring the color quality of the region. The color distribution term formula is:
in the formula (2), i is a regional pixel point or a super-pixel block in the sub-region,
in order to be a function of the color distribution,
a color distribution histogram for a set of pixels, wherein
The expression of (a) is:
in the formula (3), j is a single color histogram in the color distribution histogram, S is a set of all pixel points in the sub-region a, δ is a proportional function for recording the number of pixel points falling into the distribution region in the color histogram set, i (i) is a pixel color statistical function, if the color is i, 1 is counted, otherwise 0 is counted, and a is the sub-region pixel point set.
In the formula (4), the reaction mixture is,
the construction of the color distribution function is a measure of the color distribution according to the degree of dispersion of the pixel distribution.
Step 303, constructing a boundary pixel item index, wherein the boundary pixel item index is mainly used for controlling the region boundary smoothness. The main action is to limit the boundary pixel exchange condition by adjusting the parameter to limit the regional boundary punishment mechanism so as to achieve the purpose of controlling and evaluating the boundary smoothness, wherein the boundary pixel formula is as follows:
B(s)＝∑i∑k(bi(k))2 (5)
in the formula (5), B(s) is a boundary pixel term index, bi(k) And representing a color distribution histogram of the boundary patch, wherein k is a histogram in the color distribution histogram of the patch, and a boundary penalty function is constructed through the color distribution histogram of a pixel point in the patch. In the formula (6), S is a set of all pixel points in the sub-region a, δ is a proportional function for recording the number of pixel points falling into the distribution region in the color histogram set, i is a pixel point in the patch, and a is a set of sub-region pixel points.
And (3) completing attribute judgment of sub-region boundary pixels through energy function construction, traversing region boundaries to realize definition and rapid division of the region boundaries, and finishing region division when the boundary pixels of the 'brother connected region' are not exchanged any more. The segmentation method and the segmentation effect can be understood in conjunction with fig. 2.
It should be noted that, as can be seen from the foregoing description, step S3 mainly describes an area edge pixel swapping algorithm, and the calculation method thereof is as follows: first, a frame image is initially divided. In order to simultaneously give consideration to the reasonable expression of calculation efficiency and subregion semantics, a frame image is divided into N rectangular initial subregions with the same shape, then a subregion semantic evaluation index function is constructed through an energy function, regional boundary pixels are extracted, subregion energy maximization is realized through the influence of the boundary pixels on the energy function, meanwhile, the attribution of the boundary pixels is judged and exchanged, and finally, all regional boundaries are traversed through iteration, and all subregion optimization and division are completed.
Further, the construction process of the energy function is mainly as follows: and constructing two part indexes of a color distribution item and a boundary pixel item, and carrying out weighted summation on the two indexes. The color distribution item is described based on the color density of the subareas of the statistical histogram, and the color quality of the subareas is obtained by summing the color densities; the boundary pixel item is calculated based on the color quality of the pixel color channel; the weighting coefficient is essentially a penalty coefficient of border pixel attribution, and objectively represents an evaluation index of the sub-region border smoothness.
S4, extracting characteristics of the subareas: and mapping the image to different color spaces by adopting HSV mean values and texture mean values of the sub-regions subjected to the energy maximization detection and the final segmentation in the step S3, performing feature extraction on each sub-region, and generating corresponding feature operators.
Specifically, S4 mainly includes:
in step 401, sub-region feature extraction is to perform image HSV mean processing on a single sub-region, and extract three bases of hue (H), saturation (S), and lightness (V) of the region to perform rapid feature value extraction.
Step 402, mapping the sub-region texture mean value to different color spaces, calculating and extracting the region texture features, wherein the extracted texture mean value comprises four indexes of LBP texture index, gray level co-occurrence matrix, gray level-gradient co-occurrence matrix and Gabor wavelet texture.
In step S403, the seven eigenvalues jointly form a region eigen operator τ, and the feature of the segmented sub-region can be described by the eigen operator τ.
And S404, sequentially performing rapid feature extraction on each sub-region through iteration, and generating a corresponding feature operator.
S5, fitting and identifying road boundaries: the method comprises the steps of rapidly resolving a characteristic operator of a sub-region based on boundary pixel exchange through a support vector machine model, judging whether the sub-region of a frame image is classified into two categories of a road region, carrying out binarization processing on the image on the basis of the categories, preliminarily combining passable regions into the road region according to binarization results, extracting coordinates of left and right boundaries of the region, fitting scattered points of the coordinates of the left and right boundaries through spline curves to obtain a smooth road boundary, and finishing unstructured road identification of the forest region.
Specifically, S5 mainly includes:
step 501, inputting the feature operators describing the sub-regions into a trained support vector machine model, rapidly resolving the sub-region feature operators based on boundary pixel exchange through the support vector machine, judging whether the sub-regions of the frame image are classified into two categories or not, and then preliminarily combining the passable regions into the road regions. The support vector machine classification results based on feature operators can be understood in conjunction with the binary image shown in fig. 3.
Step 502, based on the above road area division, obtaining boundary pixels of the road area boundary, and extracting coordinates corresponding to the boundary pixels, where the extracted road boundary is substantially a scatter set. The initial road boundary can be understood by the line graph formed by the scatter of the left and right boundaries of the road shown in fig. 4.
Step 503, spline curve fitting is performed on the scatter coordinates corresponding to the boundary pixels to obtain a smooth road boundary, and fig. 5 is a spline curve fitting process. The road boundary spline curve obtained by fitting is shown in figure 6 in a mode of equidistant scattered points of the abscissa, and the black scattered points in the middle are the positions of the central lines of the roads, so that the forest area unstructured road identification is completed.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
the invention discloses a method for rapidly identifying forest road based on regional boundary pixel exchange, which takes energy function construction and energy function maximization as a core and completes rapid detection and identification on an unstructured road lacking reference objects and artificial identification in a forest region. The method has the advantages that the image area division based on the area boundary pixel exchange does not need to judge all pixels of the frame image, but energy maximization judgment is carried out on the attribution of the edge pixels on the basis of the area color distribution density, the calculation amount is small, the calculation efficiency is high, the identification speed is high, the real-time forest area road detection under the medium-low speed moving condition in the safe driving process of special forest area operation vehicles can be met, the defect that only urban structured roads, rural roads and residential roads can be identified in the existing non-structured road identification and detection technology is overcome, the road detection capability of the forest area operation vehicles in the complex and variable forest area environment is greatly improved, the semantic understanding and identification capability is greatly improved, and technical support is provided for improving the autonomous navigation capability of the forest area operation vehicles.
Based on the same inventive concept, an embodiment of the present invention further provides a forest road fast identification system based on area boundary pixel exchange, as shown in fig. 7, including:
the video image acquisition module 11 is used for acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road area;
an image initial segmentation module 12, configured to divide the image to be processed into N rectangular sub-regions with the same size according to the pixel size, number each rectangular sub-region, and define an adjacent rectangular sub-region as a sibling connected domain;
the sub-region reconstruction module 13 is configured to construct an energy function based on image features, perform energy calculation and evaluation on the sub-region of the image to be processed by using the energy function, judge whether a region boundary pixel is exchanged with a brother connected domain based on an energy maximization principle to update and optimize the sub-region, traverse all region boundary points to realize region boundary pixel exchange, and quickly complete sub-region reconstruction and realize final segmentation of the image to be processed by iteratively optimizing region segmentation;
a sub-region feature extraction module 14, configured to map the image to different color spaces by using HSV mean values and texture mean values for the sub-regions subjected to energy maximization detection and final segmentation, perform feature extraction on each sub-region, and generate a corresponding feature operator;
the road boundary fitting and identifying module 15 is used for rapidly resolving a characteristic operator of a sub-region based on boundary pixel exchange through a support vector machine model, judging whether the sub-region of a frame image is in two categories or not, carrying out binarization processing on the image on the basis of the categories, preliminarily combining passable regions into road regions according to binarization results, extracting coordinates of left and right boundaries of the regions, fitting scattered points of the coordinates of the left and right boundaries through spline curves to obtain a smooth road boundary, and completing the non-structural road identification of the forest region.
Wherein the sub-region reconstruction module 13 is specifically configured to:
constructing a color distribution item and a boundary pixel item of the sub-region;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
The road boundary fitting identification module 15 is specifically configured to:
carrying out image HSV (hue, saturation and value) mean value processing on a single subregion, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subregion;
calculating and extracting the sub-region texture features, wherein the extracted texture feature mean value comprises four indexes of an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor wavelet texture;
combining the seven characteristic values into a region characteristic operator tau, and describing the characteristics of the segmented sub-regions by using the characteristic operator tau;
and sequentially carrying out rapid feature extraction on each sub-region and generating a corresponding feature operator.
It should be noted that, for a more detailed description of the system embodiment section, please refer to the foregoing method embodiment section, which is not described herein again.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A forest road rapid identification method based on regional boundary pixel exchange is characterized by comprising the following steps:
s1, video image acquisition: acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road area;
s2, image initial segmentation: dividing the image to be processed into N rectangular sub-regions with the same size according to the pixel size, numbering each rectangular sub-region, and defining adjacent rectangular sub-regions as brother connected domains;
s3, boundary pixel swap based sub-region reconstruction: constructing an energy function based on image characteristics, performing energy maximization detection on each sub-region by adopting the energy function, judging whether region boundary pixels are exchanged with brother connected regions or not based on an energy maximization principle so as to optimize the segmentation of the sub-regions, traversing all sub-region boundary points in an iterative mode, and rapidly completing the reconstruction of the sub-regions and realizing the final segmentation of the image to be processed through the region boundary pixel exchange;
s4, extracting characteristics of the subareas: mapping the image to different color spaces by adopting HSV mean values and texture mean values of the sub-regions subjected to the energy maximization detection and the final segmentation in the step S3, performing feature extraction on each sub-region, and generating corresponding feature operators;
s5, fitting and identifying road boundaries: the method comprises the steps of rapidly resolving a feature operator of a subregion based on boundary pixel exchange through a support vector machine model, judging whether the subregion is classified into two categories or not, carrying out binarization processing on an image on the basis of the classification, preliminarily combining passable subregions into a road region according to a binarization result, extracting coordinates of a left boundary and a right boundary of the region, fitting scattered points of the left boundary coordinate and the right boundary coordinate through spline curves to obtain a smooth road boundary, and finishing unstructured road identification of the forest region.
2. The method according to claim 1, wherein in step S3, an energy function based on image features is constructed, specifically:
constructing a color distribution item and a boundary pixel item of the sub-region;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
3. The method of claim 2, wherein constructing the color distribution term is specifically:
and determining the main component color cluster of the subarea according to the area color density distribution in a voting mode, and calculating the quality index of the subarea color distribution.
4. The method according to claim 1, wherein in step S3, the energy function is used to perform energy maximization detection on the region boundary pixels of the image to be processed, specifically:
firstly, counting a color histogram under a specific color space of each subregion, and then constructing an energy function based on probability density distribution of different color channels of the color histogram, wherein the more concentrated the color distribution in the subregion is, the larger the energy function is; extracting region boundary pixels of the sub-region; and realizing the energy maximization detection and updating of the sub-region through edge pixel exchange and iteration.
5. The method according to claim 1, wherein step S4 is specifically:
carrying out image HSV (hue, saturation and value) mean value processing on a single subregion, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subregion;
calculating and extracting the sub-region texture features, wherein the extracted texture feature mean value comprises four indexes of an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor wavelet texture;
combining the seven characteristic values into a region characteristic operator tau, and describing the characteristics of the segmented sub-regions by using the characteristic operator tau;
and sequentially carrying out rapid feature extraction on each sub-region and generating a corresponding feature operator.
6. The utility model provides a forest zone road quick identification system based on regional boundary pixel exchanges which characterized in that includes:
the video image acquisition module is used for acquiring continuous frame images in real time through image acquisition equipment, and extracting the frame images to obtain an image to be processed containing a forest region unstructured road area;
the image initial segmentation module is used for dividing the image to be processed into N rectangular subregions with the same size according to the pixel size, numbering each rectangular subregion, and defining adjacent rectangular subregions as brother connected regions;
the subregion reconstruction module is used for constructing an energy function based on image characteristics, performing energy maximization detection on each subregion by adopting the energy function, judging that region boundary pixels and brother communication domains are exchanged based on an energy maximization principle, traversing all region boundary points, optimizing the segmentation of the subregions in an iteration mode, and quickly finishing the reconstruction of the subregions and realizing the final segmentation of the image to be processed;
the subregion feature extraction module is used for mapping the image to different color spaces by adopting HSV mean values and texture mean values of the subregions subjected to energy maximization detection and final segmentation, extracting features of each subregion and generating corresponding feature operators;
and the road boundary fitting and identifying module is used for rapidly resolving a sub-region characteristic operator based on boundary pixel exchange through a support vector machine model, judging whether the sub-region is classified into two categories or not, carrying out binarization processing on the image on the basis of the classification, preliminarily combining passable regions into road regions according to binarization results, extracting coordinates of left and right boundaries of the regions, fitting scattered points of the coordinates of the left and right boundaries through spline curves to obtain a smooth road boundary, and finishing the non-structural road identification of the forest region.
7. The system of claim 6, wherein the sub-region reconstruction module is specifically configured to:
constructing a color distribution item and a boundary pixel item of the sub-region;
and carrying out weighted summation on the color distribution item and the boundary pixel item to construct an energy function based on image characteristics.
8. The system of claim 6, wherein the road boundary fitting identification module is specifically configured to:
carrying out image HSV (hue, saturation and value) mean value processing on a single subregion, and extracting three characteristic values of hue (H), saturation (S) and brightness (V) of the subregion;
calculating and extracting the sub-region texture features, wherein the extracted texture feature mean value comprises four indexes of an LBP texture index, a gray level co-occurrence matrix, a gray level-gradient co-occurrence matrix and a Gabor wavelet texture;
combining the seven characteristic values into a region characteristic operator tau, and describing the characteristics of the segmented sub-regions by using the characteristic operator tau;
and sequentially carrying out rapid feature extraction on each sub-region and generating a corresponding feature operator.
9. The system of claim 6, wherein the image capture device is a monocular vision camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110075332.6A CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110075332.6A CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112818775A true CN112818775A (en) | 2021-05-18 |
| CN112818775B CN112818775B (en) | 2023-07-25 |
Family
ID=75858416
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110075332.6A Active CN112818775B (en) | 2021-01-20 | 2021-01-20 | Forest road rapid identification method and system based on regional boundary pixel exchange |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818775B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706561A (en) * | 2021-10-29 | 2021-11-26 | 华南理工大学 | Image semantic segmentation method based on region separation |
| CN114332370A (en) * | 2021-12-28 | 2022-04-12 | 埃洛克航空科技（北京）有限公司 | Road image processing method, device, equipment and storage medium |
| CN114419338A (en) * | 2022-03-28 | 2022-04-29 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN116912820A (en) * | 2023-09-13 | 2023-10-20 | 青岛君盛食品股份有限公司 | Visual inspection method for infant food safety |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050232485A1 (en) * | 2000-05-04 | 2005-10-20 | International Business Machines Corporation | Method and apparatus for determining a region in an image based on a user input |
| WO2015010451A1 (en) * | 2013-07-22 | 2015-01-29 | 浙江大学 | Method for road detection from one image |
| CN104392212A (en) * | 2014-11-14 | 2015-03-04 | 北京工业大学 | Method for detecting road information and identifying forward vehicles based on vision |
| CN104700071A (en) * | 2015-01-16 | 2015-06-10 | 北京工业大学 | Method for extracting panorama road profile |
| CN106485715A (en) * | 2016-09-09 | 2017-03-08 | 电子科技大学成都研究院 | A kind of unstructured road recognition methods |
| CN106651872A (en) * | 2016-11-23 | 2017-05-10 | 北京理工大学 | Prewitt operator-based pavement crack recognition method and system |
| CN107292253A (en) * | 2017-06-09 | 2017-10-24 | 西安交通大学 | A kind of visible detection method in road driving region |
| CN108280450A (en) * | 2017-12-29 | 2018-07-13 | 安徽农业大学 | A kind of express highway pavement detection method based on lane line |
| WO2020107716A1 (en) * | 2018-11-30 | 2020-06-04 | 长沙理工大学 | Target image segmentation method and apparatus, and device |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111798459A (en) * | 2020-06-16 | 2020-10-20 | 北京林业大学 | Unmanned aerial vehicle aerial photography tree self-adaptive segmentation method and system based on switching thought |
-
2021
- 2021-01-20 CN CN202110075332.6A patent/CN112818775B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050232485A1 (en) * | 2000-05-04 | 2005-10-20 | International Business Machines Corporation | Method and apparatus for determining a region in an image based on a user input |
| WO2015010451A1 (en) * | 2013-07-22 | 2015-01-29 | 浙江大学 | Method for road detection from one image |
| CN104392212A (en) * | 2014-11-14 | 2015-03-04 | 北京工业大学 | Method for detecting road information and identifying forward vehicles based on vision |
| CN104700071A (en) * | 2015-01-16 | 2015-06-10 | 北京工业大学 | Method for extracting panorama road profile |
| CN106485715A (en) * | 2016-09-09 | 2017-03-08 | 电子科技大学成都研究院 | A kind of unstructured road recognition methods |
| CN106651872A (en) * | 2016-11-23 | 2017-05-10 | 北京理工大学 | Prewitt operator-based pavement crack recognition method and system |
| CN107292253A (en) * | 2017-06-09 | 2017-10-24 | 西安交通大学 | A kind of visible detection method in road driving region |
| CN108280450A (en) * | 2017-12-29 | 2018-07-13 | 安徽农业大学 | A kind of express highway pavement detection method based on lane line |
| WO2020107716A1 (en) * | 2018-11-30 | 2020-06-04 | 长沙理工大学 | Target image segmentation method and apparatus, and device |
| CN111611919A (en) * | 2020-05-20 | 2020-09-01 | 西安交通大学苏州研究院 | Road scene layout analysis method based on structured learning |
| CN111798459A (en) * | 2020-06-16 | 2020-10-20 | 北京林业大学 | Unmanned aerial vehicle aerial photography tree self-adaptive segmentation method and system based on switching thought |
Non-Patent Citations (4)
| Title |
|---|
| 刘步实: "集成学习框架下的道路图像分割算法研究", CNKI * |
| 李向阳: "基于单目视觉的机动车道路检测和跟踪研究", CNKI * |
| 赵婕;张春美;张小勇;姚峰林: "基于区域边界最优映射的图像分割算法", 计算机应用研究, vol. 33, no. 1 * |
| 赵燕东;涂佳炎;: "基于北斗卫星导航***的林区智能巡检测绘***研究", 农业机械学报, no. 07 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113706561A (en) * | 2021-10-29 | 2021-11-26 | 华南理工大学 | Image semantic segmentation method based on region separation |
| CN114332370A (en) * | 2021-12-28 | 2022-04-12 | 埃洛克航空科技（北京）有限公司 | Road image processing method, device, equipment and storage medium |
| CN114419338A (en) * | 2022-03-28 | 2022-04-29 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN116912820A (en) * | 2023-09-13 | 2023-10-20 | 青岛君盛食品股份有限公司 | Visual inspection method for infant food safety |
| CN116912820B (en) * | 2023-09-13 | 2023-12-12 | 青岛君盛食品股份有限公司 | Visual inspection method for infant food safety |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112818775B (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112818775B (en) | Forest road rapid identification method and system based on regional boundary pixel exchange | |
| CN110533684B (en) | Chromosome karyotype image cutting method | |
| CN105260699B (en) | A kind of processing method and processing device of lane line data | |
| CN110263717B (en) | Method for determining land utilization category of street view image | |
| CN106599792B (en) | Method for detecting hand driving violation behavior | |
| CN109657632B (en) | Lane line detection and identification method | |
| CN107273896A (en) | A kind of car plate detection recognition methods based on image recognition | |
| CN108537239B (en) | Method for detecting image saliency target | |
| CN106651872A (en) | Prewitt operator-based pavement crack recognition method and system | |
| CN111738064B (en) | Haze concentration identification method for haze image | |
| CN109284669A (en) | Pedestrian detection method based on Mask RCNN | |
| CN103824081B (en) | Method for detecting rapid robustness traffic signs on outdoor bad illumination condition | |
| CN110738676A (en) | GrabCT automatic segmentation algorithm combined with RGBD data | |
| CN103136537B (en) | Vehicle type identification method based on support vector machine | |
| CN110992381A (en) | Moving target background segmentation method based on improved Vibe + algorithm | |
| CN104715239A (en) | Vehicle color identification method based on defogging processing and weight blocking | |
| CN113409267B (en) | Pavement crack detection and segmentation method based on deep learning | |
| CN108509950B (en) | Railway contact net support number plate detection and identification method based on probability feature weighted fusion | |
| CN110334692A (en) | A kind of blind way recognition methods based on image procossing | |
| CN112561899A (en) | Electric power inspection image identification method | |
| CN111160328A (en) | Automatic traffic marking extraction method based on semantic segmentation technology | |
| CN113705579A (en) | Automatic image annotation method driven by visual saliency | |
| CN106570885A (en) | Background modeling method based on brightness and texture fusion threshold value | |
| CN112906616A (en) | Lane line extraction and generation method | |
| CN115170479A (en) | Automatic extraction method for asphalt pavement repairing diseases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |