CN105987684A - Monocular vision-based agricultural vehicle navigation line detection system and method - Google Patents

Abstract

The invention discloses a monocular vision-based agricultural vehicle navigation line detection system and method and effectively solves the problems of low efficiency and high cost in the prior art. The invention includes establishment of a hardware system of a navigation line detection system, and a navigation lien detection method. The navigation line detection method includes the following steps: extracting crop lines based on farmland greenness characteristics; removing image noise based on median filtering; performing morphological opening operation of images; extracting navigation line characteristic points; fitting a navigation line. Based on a monocular vision sensor in connection with image processing technology, a center line representing a crop line is extracted from an acquired farmland image as a navigation line for autonomous travelling of an agricultural vehicle. The detection method provided herein analyzes farmland image characteristics, employs greenness characteristic-based crop line extraction and centroid-based navigation line detection, eliminates noise interference from the farmland image through median filtering and morphological methods and therefore has the advantages of low computing load and high stability.

Description

A kind of agricultural vehicle leading line detecting system based on monocular vision and method

Technical field

The present invention relates to agricultural machines navigation systems technology field, particularly relate to a kind of agricultural vehicle based on monocular vision Leading line detecting system and method.

Background technology

Automatic navigation technology is the important component part of reading intelligent agriculture, automatic dispenser, apply fertilizer, gather in, middle weeding etc. Aspect has purposes widely.Current most widely used navigation mode is vision guided navigation and GPS navigation.The application of GPS navigation Limited relatively big by environment, when running into when blocking of building, tall and big trees etc., signal is susceptible to lose, and due to its price Indispensable with priori cartographic information so that relatively costly, real-time, the stability of agricultural robot operation are all caused by this The biggest impact.Comparatively speaking, vision guided navigation cost is relatively low, and motility is bigger, particularly can obtain abundant environmental information, real The planning of Shi Jinhang guidance path, is the focus of research at present.

Visual pattern is processed and feature information extraction be vision navigation system application premise and basis, for row Planting crop, the main task of vision guided navigation identifies crop row, exactly as navigational reference line, for determining vehicle from image Position relatively provides foundation.At present, existing much about the researchs of guidance path detection both at home and abroad, as Yuan help cloud application based on Vertical projection method carries out crop row location, determines crop row according to the peak value of grey level histogram, and this method is effectively quick, but When in the ranks there is noise jamming, effect is affected.Such as traditional Hough change matching crop row, stability is high, but calculates simultaneously Measure huge, it is difficult to meet the requirement of real-time of working truck.

Therefore, the present invention is based on monocular vision sensor, for farmland destructuring working environment, proposes a kind of based on agriculture The immediate navigational info detecting system of field green characteristic and anchor point, is efficiently modified traditional method, completes path In real time, extract fast and accurately.

Summary of the invention

The object of the invention is contemplated to make up the defect of prior art, it is provided that a kind of agricultural vehicle based on monocular vision is led Course line detecting system and method.

The present invention is achieved by the following technical solutions:

A kind of agricultural vehicle leading line detecting system based on monocular vision, include industrial camera, USB transmission module, Image processing platform, image storage card and display, described industrial camera is arranged on the top of working truck, pacifies diagonally downward The ambient image of dress collection vehicle dead ahead；Industrial camera is connected by described USB transmission module with image processing platform；Described Image processing platform carry out the process of view data, extract the information needed for navigation；Described image storage card is arranged on figure As the digital picture on processing platform, after preserving industrial camera shooting and processing；Described display passes through USB interface and figure As processing platform is connected, for showing the navigation route information detected in real time.

The method of a kind of agricultural vehicle leading line based on monocular vision detection, comprises the following steps:

Step 1: the farmland image that industrial camera gathers, carries out threshold process according to the excess green of crop, extracts crop OK；

Step 2: for extracting the image of crop row, uses mean value method to carry out gray processing pretreatment, based on gray-scale map Picture, uses medium filtering to remove the noise in image；

Step 3: for the image after removal noise, maximize principle according to inter-class variance and carry out binaryzation, it is thus achieved that black and white Dichromatism binary map；

Step 4: for binary image, utilizes rectangular configuration element to carry out morphology opening operation process, eliminates crop row In tiny hole, and the little agglomerate of independence that plant growth irregularly causes, it is thus achieved that edge-smoothing, the crop row of regular shape Image；

Step 5: for the crop row image after Morphological scale-space, is divided into contour horizontal bar-chart picture, calculates water The centre of form of the white color lump of representation crop row in riglet, obtains the characteristic point that leading line is extracted；

Step 6: for the leading line characteristic point obtained, uses method of least square to be fitted, leading of final acquisition detection Course line.

In step 1, extracting crop row according to excess green, the formula carrying out threshold process is as follows:

Wherein, using RGB color to carry out farmland graphical analysis, (i, j), (i, j), (i j) represents numeral to b to g to r respectively In image, coordinate is that ((i, j) (i, j) (i, j) value is as threshold for-b for-r by the 2*g of pixel for i, pixel red, green, blue channel value j) Value, judges pixel in image, if this value is more than 0, is then crop row, keeps original pixel value constant, otherwise by it As background.

Step 2 uses mean value method to carry out gray processing pretreatment computing formula as follows:

Gray (i, j)=(r (i, j)+g (i, j)+b (i, j))/3

Wherein, (i j) is the grey scale pixel value calculated to Gray.

In step 2, carrying out medium filtering, to remove the formula of noise in image as follows:

G (i, j)=Med{Gray_i-v..., Gray_i..., Gray_i+v, i ∈ N, v=(m-1)/2

Wherein, (i, j) is filtered gray value to G, and Med is to seek median operation, the size of m representative structure element, typically Take odd number, Gray_i-v..., Gray_i..., Gray_i+vIt is Gray_iCentered by the neighborhood territory pixel gray value of pixel.

In step 3, the formula that inter-class variance maximization principle carries out binary conversion treatment is as follows:

$B(i,j)=\left\{\begin{array}{cc}0& G\left(i,j\right)<T\\ 255& G\left(i,j\right)\&GreaterEqual;T\end{array}\right.$

Wherein, (i, is j) pixel value after processing to B, and segmentation threshold T calculates based on method maximization approach between otsu class Go out.

In step 4, described morphology opening operation processes the profile according to crop row, uses 5 pixel * 1 pixels Rectangular configuration element carries out burn into expansive working.

In step 5, the segmentation of horizontal bar uses contour principle, if image size is L × H pixel, in units of height h Image carries out equidistant segmentation, and the size of each horizontal bar is then L × h, and the horizontal rule number of division is H/h, actual application Middle adjustable dividing strip number.

In step 5, calculating of characteristic point uses calculation based on the centre of form, and its formula is as follows:

$f(i,j)=\left\{\begin{array}{cc}1& r(i,j)=g(i,j)=g(i,j)=255\\ 0& r(i,j)=g(i,j)=g(i,j)=0\end{array}\right.$

$i^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}if\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)},j^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}jf\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)}$

Wherein, f (i, j) represents pixel value, (and i ', j ') represent the centre of form coordinate asked for.

The invention have the advantage that the present invention utilize cheap CCD industrial camera as environment sensing sensor, in conjunction with Embedded image processing platform and related detecting method, detect navigational reference line, and the real-time navigation for agricultural vehicle provides Reference frame, the detection method that the present invention provides, by analyzing farmland characteristics of image, uses crop row based on green characteristic to carry Take and leading line based on centre of form detection, and it is dry to eliminate the noise in the image of farmland by medium filtering and morphological method Disturb, have the advantages that amount of calculation is little, stability is high.Present configuration is simple, with low cost simultaneously, independently leads for farmland vehicle Boat technology has huge potential using value.

Accompanying drawing explanation

Fig. 1 is that complement opened up by the hardware of the present invention.

Fig. 2 is guidance path overhaul flow chart in the present invention.

Fig. 3 is the result figure that crop row extracts.

Fig. 4 is the design sketch after picture noise is removed.

Fig. 5 is the result figure of image binaryzation.

Fig. 6 is the design sketch of Morphological scale-space.

Fig. 7 is characterized a result figure extracted.

Fig. 8 is the result figure of leading line matching.

Detailed description of the invention

As it is shown in figure 1, a kind of agricultural vehicle leading line detecting system based on monocular vision, include industrial camera 1, USB transmission module 2, image processing platform 3, image storage card 4 and display 5, described industrial camera 1 is arranged on working truck Top, diagonally downward install collection vehicle dead ahead ambient image 6；Described USB transmission module 2 is by industrial camera 1 He Image processing platform 3 is connected；Described image processing platform 3 carries out the process of view data, extracts the information needed for navigation；Institute The image storage card 4 stated is arranged on image processing platform 3, preserves the digital picture after industrial camera 1 shoots and processes；Institute The display 5 stated is connected with image processing platform 3 by USB interface, for showing the navigation route information detected in real time.

As in figure 2 it is shown, the method for a kind of agricultural vehicle leading line based on monocular vision detection, comprise the following steps:

Step 1: the farmland image that industrial camera 1 gathers, carries out threshold process according to the excess green of crop, extracts crop OK；

Step 2: for extracting the image of crop row, uses mean value method to carry out gray processing pretreatment, based on gray-scale map Picture, uses medium filtering to remove the noise in image, as shown in Figure 4；

Step 3: for the image after removal noise, maximize principle according to inter-class variance and carry out binaryzation, it is thus achieved that black and white Dichromatism binary map；

Step 4: for binary image, utilizes rectangular configuration element to carry out morphology opening operation process, eliminates crop row In tiny hole, and the little agglomerate of independence that plant growth irregularly causes, it is thus achieved that edge-smoothing, the crop row of regular shape Image；

Step 5: for the crop row image after Morphological scale-space, is divided into contour horizontal bar-chart picture, calculates water The centre of form of the white color lump of representation crop row in riglet, obtains the characteristic point that leading line is extracted；

Step 6: for the leading line characteristic point obtained, uses method of least square to be fitted, leading of final acquisition detection Course line.

In step 1, extracting crop row according to excess green, the formula carrying out threshold process is as follows:

Wherein, using RGB color to carry out farmland graphical analysis, (i, j), (i, j), (i j) represents numeral to b to g to r respectively In image, coordinate is that ((i, j) (i, j) (i, j) value is as threshold for-b for-r by the 2*g of pixel for i, pixel red, green, blue channel value j) Value, judges pixel in image, if this value is more than 0, is then crop row, keeps original pixel value constant, otherwise by it As background.

Step 2 uses mean value method to carry out gray processing pretreatment computing formula as follows:

Gray (i, j)=(r (i, j)+g (i, j)+b (i, j))/3

Wherein, (i j) is the grey scale pixel value calculated to Gray.

In step 2, carrying out medium filtering, to remove the formula of noise in image as follows:

G (i, j)=Med{Gray_i-v..., Gray_i..., Gray_i+v, i ∈ N, v=(m-1)/2

Wherein, (i, j) is filtered gray value to G, and Med is to seek median operation, the size of m representative structure element, typically Take odd number, Gray_i-v..., Gray_i..., Gray_i+vIt is Gray_iCentered by the neighborhood territory pixel gray value of pixel.

In step 3, the formula that inter-class variance maximization principle carries out binary conversion treatment is as follows:

$B(i,j)=\left\{\begin{array}{cc}0& G\left(i,j\right)<T\\ 255& G\left(i,j\right)\&GreaterEqual;T\end{array}\right.$

Wherein, (i, is j) pixel value after processing to B, and segmentation threshold T calculates based on method maximization approach between otsu class Go out.

In step 4, described morphology opening operation processes the profile according to crop row, uses 5 pixel * 1 pixels Rectangular configuration element carries out burn into expansive working.

In step 5, the segmentation of horizontal bar uses contour principle, if image size is L × H pixel, in units of height h Image carries out equidistant segmentation, and the size of each horizontal bar is then L × h, and the horizontal rule number of division is H/h, actual application Middle adjustable dividing strip number.

In step 5, calculating of characteristic point uses calculation based on the centre of form, and its formula is as follows:

$f(i,j)=\left\{\begin{array}{cc}1& r(i,j)=g(i,j)=g(i,j)=255\\ 0& r(i,j)=g(i,j)=g(i,j)=0\end{array}\right.$

$i^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}if\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)},j^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}jf\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)}$

Wherein, f (i, j) represents pixel value, (and i ', j ') represent the centre of form coordinate asked for.

The industrial camera used is 5,000,000 pixels, and USB interface, resolution is adjustable, and arranging acquisition resolution is 720X576 Pixel, arranging collection picture format is jpeg format.

Image storage card is arranged on image processing platform, and by USB transmission module by industrial camera and image procossing Platform connects, and during work, farmland image or the video of collected by camera are stored, at image by image pick-up card with digital signaling Platform carries out the process of view data, extracts navigation information.The image processed there is also in image pick-up card.At image Platform uses QS-PTE9 embedded video image processing platform based on Freescale i.MX6Q (Cotrex A9).

Display 5 is connected by USB interface and image processing platform 3, the ring around display working truck in real time Border image, and the guidance path reference information extracted.

As in figure 2 it is shown, guidance path detecting system overall flow, including crop row extraction, picture noise removal, image two Value, Morphological scale-space, feature point extraction, these steps of leading line matching, be implemented as follows.

The first step, the crop row of farmland image extracts.RGB color is used to carry out farmland graphical analysis, by analyzing Understanding, crop and soil and the value being characterized mainly in that green component of other background informations in image, crop row has substantially Green characteristic, i.e. G-value is higher, and Soil Background R value and G-value are higher, and the G-value hence with pixel carries out carrying of crop row Take.Improving based on the tradition super green signature grey scale method of 2G-R-B, directly as judgment threshold, this value is more than 0 Pixel is considered crop, retains original pixel value, on the contrary as background process, as shown in formula (1), wherein r (i, j), g (i, j), (i j) represents that in digital picture, coordinate is (i, pixel red, green, blue channel value j) to b respectively.This method directly carries out crop The segmented extraction of row, simultaneously by background removal, greatly reduces amount of calculation by retain original pixel value, improves in real time Property.The result that crop row extracts is as shown in Figure 3.

Second step, the removal of picture noise.Noise remove comprises gray processing pretreatment, two steps of medium filtering.

1) gray proces: use global statistics Mean Method to calculate the gray value of pixel, concrete as shown in formula (2), (i j) is the grey scale pixel value calculated to Gray.

Gray (i, j)=(r (i, j)+g (i, j)+b (i, j))/3 (2)

2) medium filtering: comprise from road surface in image, crop and the multi-source information of environment, use medium filtering to carry out Smoothing processing, removes salt-pepper noise therein.According to intensity level, pixel in window is arranged, in selected and sorted set of pixels Between be worth as pixel (i, new value j), for the pixel of edge, the directly original gray value of reservation.Such as formula (3) institute Show:

G (i, j)=Med{Gray_i-_v..., Gray_i..., Gray_i+v, i ∈ N, v=(m-1)/2

(3)

Wherein, (i, j) is filtered gray value to G, and Med is to seek median operation, the size of m representative structure element, typically Take odd number, Gray_i-v..., Gray_i..., Gray_i+vIt is Gray_iCentered by the neighborhood territory pixel gray value of pixel.

Taking m value in the present embodiment is 3, uses the template of 3 × 3 pixels to be filtered, namely take Gray (i-1, j-1), Gray(i-1,j),Gray(i-1,j+1),Gray(i,y-1),Gray(i,j),Gray(i,j+1),Gray(i+1,j),Gray (i+1, j+1) } intermediate value as filtering output value.When specifically applying, for different crop map pictures, can take various sizes of Template obtains optimum filtering effect.

3rd step, image binaryzation.Binaryzation it is critical only that choosing of threshold value, based on inter-class variance maximize principle, Travel through in maximum gradation value interval in minimum, select to make target, the gray value T of the direct maximum variance of background two class As segmentation threshold, as shown in formula (4), (i j) is the pixel value after processing to B.Binary conversion treatment result is as shown in Figure 5.

$B\left(i,j\right)=\left\{\begin{array}{cc}0& G\left(i,j\right)<T\\ 255& G\left(i,j\right)\&GreaterEqual;T\end{array}\right.---\left(4\right)$

4th step, Morphological scale-space.For eliminating hole tiny in crop row, and plant growth irregularly cause only Vertical little agglomerate, carries out morphology opening operation process, the convolution operation expanded after namely carrying out first burn into.Consider crop row Profile, uses the rectangular configuration element of 5 pixel * 1 pixels to carry out burn into expansive working, processes and connects adjacent element, simultaneously Remove scattered noise spot.

First etching operation is carried out: by each pixel of structural element scanogram, covered with it by structural element Bianry image does and operates, if pixel value is all 1, then this pixel value is 1, is otherwise 0；Next carries out expansive working, with knot Each pixel of constitutive element scanogram, the bianry image covered with it by structural element does and operates, if being all 0, then This pixel value is 0, is otherwise 1.The effect of Morphological scale-space as shown in Figure 6, effectively eliminates noise tiny in image fast, and The border of smooth crop row.

5th step, the extraction of characteristic point.The method calculated based on horizontal bar segmentation and characteristic point is used to obtain navigation characteristic Point.

1) horizontal bar segmentation: set image size as L × H pixel, in units of height h, image is carried out equidistant segmentation, The size of each horizontal bar is then L × h, and the horizontal rule number of division is H/h, can be adjusted as required.In the present embodiment Selection marks off 8 horizontal bars.

2) characteristic point based on the centre of form calculates: in the image that horizontal bar is partitioned into, black picture element is background, by its pixel value Being expressed as 0, white color lump represents target crop row, and pixel value is expressed as 1, is calculated the centre of form of each white blocks by traversal Position, namely the center of crop row is as the characteristic point of matching leading line.Concrete as shown in computing formula (5), (6), f (i, J) pixel value is represented, (i ', j ') represent the centre of form coordinate asked for.Fig. 7 is the characteristic point image extracted.

$f(i,j)=\left\{\begin{array}{cc}1& r(i,j)=g(i,j)=g(i,j)=255\\ 0& r(i,j)=g(i,j)=g(i,j)=0\end{array}\right.---\left(5\right)$

$i^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}if\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)},j^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}jf\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)}---\left(6\right)$

6th step, the matching of leading line: based on method of least square, all characteristic points are carried out fitting a straight line, obtain masterpiece The reference leading line of thing row.Result is as shown in Figure 8.

Claims (9)

1. an agricultural vehicle leading line detecting system based on monocular vision, it is characterised in that: include industrial camera, USB Transport module, image processing platform, image storage card and display, described industrial camera is arranged on the top of working truck, The ambient image of collection vehicle dead ahead is installed diagonally downward；Industrial camera and image procossing are put down by described USB transmission module Platform is connected；Described image processing platform carries out the process of view data, extracts the information needed for navigation；Described image storage Card is arranged on image processing platform, the digital picture after preserving industrial camera shooting and processing；Described display passes through USB interface is connected with image processing platform, for showing the navigation route information detected in real time.

2. the method for agricultural vehicle leading line based on a monocular vision detection, it is characterised in that: comprise the following steps:

Step 1: the farmland image that industrial camera gathers, carries out threshold process according to the excess green of crop, extracts crop row；

Step 2: for extracting the image of crop row, uses mean value method to carry out gray processing pretreatment, based on gray level image, adopts The noise in image is removed with medium filtering；

Step 3: for the image after removal noise, maximize principle according to inter-class variance and carry out binaryzation, it is thus achieved that black-and-white two color Binary map；

Step 4: for binary image, utilizes rectangular configuration element to carry out morphology opening operation process, eliminates in crop row thin Little hole, and the little agglomerate of independence that plant growth irregularly causes, it is thus achieved that edge-smoothing, the crop row figure of regular shape Picture；

Step 5: for the crop row image after Morphological scale-space, be divided into contour horizontal bar-chart picture, calculated level bar The centre of form of the white color lump of middle representation crop row, obtains the characteristic point that leading line is extracted；

Step 6: for the leading line characteristic point obtained, uses method of least square to be fitted, the final leading line obtaining detection.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 1, extracting crop row according to excess green, the formula carrying out threshold process is as follows:

Wherein, using RGB color to carry out farmland graphical analysis, (i, j), (i, j), (i j) represents digital picture to b to g to r respectively Middle coordinate is that ((i, j) (i, j) (i, j) value is as threshold value, right for-b for-r by the 2*g of pixel for i, pixel red, green, blue channel value j) In image, pixel judges, if this value is more than 0, is then crop row, keeps original pixel value constant, otherwise as the back of the body Scape.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: Step 2 uses mean value method to carry out gray processing pretreatment computing formula as follows:

Gray (i, j)=(r (i, j)+g (i, j)+b (i, j))/3

Wherein, (i j) is the grey scale pixel value calculated to Gray.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 2, carrying out medium filtering, to remove the formula of noise in image as follows:

G (i, j)=Med{Gray_i-v..., Gray_i..., Gray_i+vI ∈ N, v=(m-1)/2

Wherein, (i, j) is filtered gray value to G, and Med is to seek median operation, the size of m representative structure element, typically takes strange Number, Gray_i-v..., Gray_i..., Gray_i+vIt is Gray_iCentered by the neighborhood territory pixel gray value of pixel.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 3, the formula that inter-class variance maximization principle carries out binary conversion treatment is as follows:

$B(i,j)=\left\{\begin{array}{cc}0& G\left(i,j\right)<T\\ 255& G\left(i,j\right)\&GreaterEqual;T\end{array}\right.$

Wherein, (i, is j) pixel value after processing to B, and segmentation threshold T calculates based on method maximization approach between otsu class.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 4, described morphology opening operation processes the profile according to crop row, uses the rectangular configuration of 5 pixel * 1 pixels Element carries out burn into expansive working.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 5, the segmentation of horizontal bar uses contour principle, if image size is L × H pixel, enters image in units of height h The equidistant segmentation of row, the size of each horizontal bar is then L × h, and the horizontal rule number of division is H/h, adjustable in actual application Dividing strip number.

A kind of method of agricultural vehicle leading line based on monocular vision detection, it is characterised in that: In step 5, calculating of characteristic point uses calculation based on the centre of form, and its formula is as follows:

$f(i,j)=\left\{\begin{array}{cc}1& r(i,j)=g(i,j)=g(i,j)=255\\ 0& r(i,j)=g(i,j)=g(i,j)=0\end{array}\right.$

$i^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}if\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)},j^{,}=\frac{{\mathrm{\&Sigma;}}_{L\&times;H}jf\left(i,j\right)}{{\mathrm{\&Sigma;}}_{L\&times;H}f\left(i,j\right)}$

Wherein, f (i, j) represents pixel value, (and i ', j ') represent the centre of form coordinate asked for.