CN113128576A

CN113128576A - Crop row detection method and device based on deep learning image segmentation

Info

Publication number: CN113128576A
Application number: CN202110362037.9A
Authority: CN
Inventors: 马钦; 卫建; 吴才聪
Original assignee: China Agricultural University
Current assignee: China Agricultural University
Priority date: 2021-04-02
Filing date: 2021-04-02
Publication date: 2021-07-16

Abstract

The invention provides a crop row detection method and device based on deep learning image segmentation, wherein the method comprises the following steps: obtaining an image of a farmland crop; inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result; carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result. According to the method, a binary segmentation result and pixel level high-dimensional vector representation are obtained through a convolutional neural network model, crop row characteristics are obtained by fusing information of the two, and then a clustering algorithm and a curve fitting algorithm are used for performing fitting description on crop row lines, so that the crop row identification speed and accuracy are effectively improved.

Description

Crop row detection method and device based on deep learning image segmentation

Technical Field

The invention relates to the technical field of deep learning and image segmentation, in particular to a crop row detection method and device based on deep learning image segmentation.

Background

Under advocating and developing of intelligent agriculture and accurate agricultural, the accurate efficient of crop row detects and provides the basis for mechanized agricultural production activity, provides directness, supplementary navigation for farmland operation machinery, and the accurate chemical fertilizer of using of operation machinery, medicine and weeding provide the guidance, and accurate intelligent operation is implemented and has been reduced the extravagant and excessive input of pesticide of farmland chemical fertilizer, protects ecological environment. Meanwhile, the labor input intensity and the production cost are reduced, the resource utilization rate is improved, and the economic benefit of the farmland is further improved.

Present crop row discernment mainly carries on visual sensor equipment based on two kinds of platforms of on-vehicle and unmanned aerial vehicle, and main flow is: the method comprises the steps of manually extracting image features such as color and texture of crop rows or three-dimensional information features such as elevation and depth, selecting a specific threshold range for target recognition according to a certain environment suitable condition, cutting an interested area, extracting feature points of the crop rows, finally fitting the feature points, and describing the crop rows. Common crop row feature extraction methods include providing various green feature description operators based on different color spaces such as RGB and HSV, determining a crop row threshold range by combining image binarization, threshold segmentation and the like, and segmenting a target region. The crop row line fitting method is mainly based on Hough space transformation, a least square method and an improvement method thereof. The Hough transformation method has the advantages of strong anti-interference capability and small influence by noise, can realize detection under a certain weed density, but has large storage space required by an algorithm and higher time complexity, and cannot meet real-time and accurate farmland operation; the least square method can realize quick measurement, but is greatly interfered by noise.

Therefore, the traditional crop row detection method can achieve better effect only by manually extracting color, texture and three-dimensional characteristics and adjusting the range of the target segmentation threshold according to the specific operation environment. The actual farmland environment is complex and changeable, the crop rows are under different illumination conditions, at different growth stages and at different weed densities, and the crop rows are bent to different degrees due to topographic relief or shaking of operation machinery, so that the traditional method has the defects that the threshold segmentation range is difficult to adapt to a plurality of scenes, and the robustness is poor.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a crop row detection method and device based on deep learning image segmentation.

The invention provides a crop row detection method based on deep learning image segmentation, which comprises the following steps: obtaining an image of a farmland crop; inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result; carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

According to the crop row detection method based on deep learning image segmentation, before the farmland crop image is input into the trained convolution network model, the method further comprises the following steps: obtaining images of farmland crops with different growth stages, different weed densities and different bending degrees, and marking crop rows as training samples; training the constructed improved BiSeNet V2 network model based on a plurality of training samples to obtain the trained convolutional network model; the improved BiSeNet V2 network model is obtained by adding an SCNN network layer between an aggregation layer and a seghead layer of a BiSeNet V2 network.

According to the crop row detection method based on deep learning image segmentation, the training of the constructed convolution network model based on a plurality of training samples comprises the following steps: and training the constructed convolution network model through gradient descent and back propagation.

According to the crop row detection method based on deep learning image segmentation, the clustering method for performing spatial clustering on the high-dimensional feature vectors extracted by the convolutional network model comprises the following steps: mean Shift clustering or DBSCAN clustering.

According to the crop row detection method based on deep learning image segmentation, the farmland crop image is acquired, and the method comprises the following steps: a visual sensor is carried on the agricultural operation mechanical platform to obtain the image of the farmland crop.

According to the crop row detection method based on the deep learning image segmentation, which is disclosed by the embodiment of the invention, crop row fitting is carried out according to the binary image and the different crop row examples to obtain a crop row curve, and the method comprises the following steps: fitting of quadratic or polynomial curves is performed by the least squares method.

According to the crop row detection method based on deep learning image segmentation, after farmland crop images of different growth stages, different weed densities and different bending degrees are obtained, the method further comprises the following steps: cutting an original image through the region of interest to obtain a target and a region; and the data diversity and the anti-interference capability are increased through data enhancement.

The invention also provides a crop row detection device based on deep learning image segmentation, which comprises: the acquisition module is used for acquiring an image of the farmland crop; the processing module is used for inputting the farmland crop image into the trained convolution network model and outputting a binary image of an image segmentation result; the clustering module is used for carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; the fitting module is used for fitting the crop row according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the crop row detection method based on deep learning image segmentation.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for crop row detection based on deep learning image segmentation as described in any of the above.

According to the crop row detection method and device based on deep learning image segmentation, the binary segmentation result and the pixel level high-dimensional vector representation are obtained through the convolutional neural network model, crop row characteristics are obtained through fusion of information of the two, and then the crop row lines are subjected to fitting description through the clustering algorithm and the curve fitting algorithm, so that the crop row identification speed and accuracy are effectively improved.

Drawings

In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a schematic flow chart of a crop row detection method based on deep learning image segmentation according to the present invention;

FIG. 2 is a diagram illustrating a binary segmentation result provided by the present invention;

FIG. 3 is a schematic diagram of the improved BiSeNet V2 network structure provided by the present invention;

FIG. 4 is a schematic diagram of the SCNN network structure provided by the present invention;

FIG. 5 is a second schematic flowchart of the crop row detection method based on deep learning image segmentation according to the present invention;

FIG. 6 is a schematic structural diagram of a crop row detection apparatus based on deep learning image segmentation according to the present invention;

fig. 7 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The crop row detection method and apparatus based on deep learning image segmentation according to the present invention are described below with reference to fig. 1 to 7. Fig. 1 is a schematic flow chart of a crop row detection method based on deep learning image segmentation provided by the present invention, and as shown in fig. 1, the present invention provides a crop row detection method based on deep learning image segmentation, which includes:

101. and obtaining an image of the farmland crop.

The outdoor farmland crop image is collected through the vision sensor, and the image comprises a video, a picture and the like.

102. And inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result.

And carrying out operations such as convolution, pooling and the like on the acquired image by the trained deep learning network model structure, extracting features and carrying out up-sampling to obtain a binary image. The feature extraction backbone network is a multilayer convolutional neural network and can be replaced by other feature extraction backbone networks, such as network structures of VGG16, ResNet series and the like; the decoder is composed of multiple layers of upsampled convolutional layers, such as device convolutional kernels, upsampled convolutional kernels, void convolutional kernels, and pooling layer operations.

Fig. 2 is a schematic diagram of a binary segmentation result provided by the present invention, and as shown in fig. 2, the binary result is an output result of a crop row and a background, and pixels are respectively represented as 1 and 0. The high-dimensional feature vector is a high-dimensional embedded feature vector, and is obtained from a feature extraction network of a convolution network for high-dimensional characterization information of each pixel.

103. And carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples.

Here, different crop row instances are identified by clustering the high-dimensional embedded feature vectors.

104. And fitting the crop row according to the binary image and the different crop row examples to obtain a crop row curve.

And after the example of each crop row is determined, performing curve fitting on the binary image on the basis to obtain a curve of the crop row.

According to the crop row detection method based on deep learning image segmentation, a binary segmentation result and pixel level high dimensional vector representation are obtained through a convolutional neural network model, crop row characteristics are obtained through fusion of information of the two, and then a clustering algorithm and a curve fitting algorithm are used for performing fitting drawing on crop row lines, so that the crop row identification speed and accuracy are effectively improved.

In one embodiment, before inputting the image of the field crop into the trained convolutional network model, the method further comprises: obtaining images of farmland crops with different growth stages, different weed densities and different bending degrees, and marking crop rows as training samples; and training the constructed improved BiSeNet V2 network model based on a plurality of training samples to obtain the trained convolutional network model.

The farmland environment is complex and changeable, and besides a good scene with clear crop rows, a large number of complex scenes exist. Four complex farmland scenes are mainly analyzed, including scenes of crop row breakage, weed interference, different canopy widths and shadow interference. For corn planted in equal row spacing, the rows of the crop are generally independently distributed and not continuous. And a breakage of the seedling band may occur. The recognizable crop row has fewer pixel points, so the division difficulty is higher. Because the color and texture of weeds are similar to those of corn crops, the different weed densities bring great interference to the identification of crop rows. The canopy widths are different at different growth stages of the crop, which provides a challenge for curve fitting. Under different illumination conditions, crop rows have certain differences. The sunlight irradiates objects such as buildings, trees and the like, and shadows are generated. Large areas of shadow occlusion greatly increase the difficulty of segmentation. In summary, different complex scenes have some common characteristics. The image can extract fewer crop row pixels, and parts of the crop row pixels are missing or blocked, and meanwhile, some noise interference exists. However, the convolutional neural network loses image detail information when the image is down-sampled, so that it is difficult to accurately segment crop rows with small areas and fuzzy edges.

The invention carries out comparative analysis on the neural network used for image segmentation, selects a simple and efficient BiSeNet V2 network structure to build a model, and ensures the real-time property. Meanwhile, in order to improve the recognition accuracy in a complex scene, the BiSeNet V2 is optimized, and a network structure of an improved network S-BiSeNet V2 and S-BiSeNet V2 based on the BiSeNet V2 is provided as shown in fig. 3. As can be seen, the SCNN network layer is added between the aggregation layer and the seg Head layer of the BiSeNet V2, and an S-BiSeNet V2 network structure is obtained.

The invention provides an improvement method by analyzing the crop rows in the complex scene. In a farmland scene, different crop rows have a certain position relationship. Meanwhile, the crop rows also have a certain prior shape, and the same crop strip is collinear. The different crop rows are approximately parallel in the real world. Such spatial positional relationships are referred to as spatial context information. The SCNN may be used to extract spatial context information. Through comparison experiments, the method is superior to a probabilistic graphical model (CRF/MRF) which is large in calculation amount and poor in real-time performance. The method has excellent performance on objects which have strong spatial relationship but poor appearance clues, such as lane lines, electric wires, walls and the like. The method is simple and efficient, and can be flexibly applied to each position of the CNN. Therefore, the invention improves the original BiSeNet V2 by combining Spatial CNN, and improves the crop row segmentation precision.

Conventional convolutional neural networks are typically Layer-by-Layer (Layer-by-Layer) convolutions. Each convolutional layer receives the feature map from the previous layer, and after convolution operation and nonlinear activation, passes the feature map to the next convolutional layer. The Spatial CNN generalizes this layer-by-layer convolution to Slice-by-Slice convolution, i.e., the rows and columns of the feature map are regarded as "layers", and these "layers" are convolved in sequence. Assuming that the height of the input three-dimensional feature map is H, the width is W, and the number of channels is C, the structure of Spatial CNN is shown in FIG. 4. SCNN performs slicing first, followed by sequential convolution.

Spatial CNN slices the three-dimensional feature map from top to bottom for H slices. The first patch vector is then convolved with a convolution kernel size C w (w 9). And after the convolution result is subjected to nonlinear activation, updating the second piece of vector, and so on until the end. The whole process is called SCNN _ D. A bottom-up convolution operation, i.e., SCNN _ U, is then performed, similar to the SCNN _ D flow, except for a change in direction. And similarly, carrying out slicing and convolution operation from left to right and from right to left to obtain the final three-dimensional characteristic diagram. The mode of updating piece by piece is similar to a residual error network, and the training difficulty can be reduced. The unique convolution of Spatial CNNs allows pixel information to be transferred between different neurons in the same layer. The capability of extracting image space information is improved. Therefore, the method is applied to the BiSeNet V2 network, and the crop row extraction precision of the network in a complex scene is improved.

Although the Spatial CNN structure can be added at any position of the network structure, the input feature diagram size of the Spatial CNN structure is not too large in order to ensure the real-time performance of the network. In a preferred embodiment, the feature pattern is placed between the encoder and the decoder, and the feature pattern has a width and a height of 32 x 64, respectively, so that the accuracy is guaranteed and the calculation amount is small. A crop row detection data set is constructed, actual operation video or image data is obtained from a data acquisition platform, images are obtained frame by frame, and cutting is carried out according to an interested area, so that data of different illumination conditions, different growth stages, different weed densities and different bending degrees of crop rows are obtained, and the complex operation environment is basically covered. And performing feature extraction learning on input training data, automatically optimizing network parameters, and obtaining a binary segmentation result and high-dimensional features of each pixel through an image segmentation module. By constructing data sets of different illumination conditions, different growth stages, different weed densities and different bending degrees of crop rows and certain data enhancement processing, the robustness of model identification can be improved.

In one embodiment, the training the constructed convolutional network model based on a plurality of training samples includes: and training the constructed convolution network model through gradient descent and back propagation.

Deep learning network training is carried out in an end-to-end mode, a labeled data set is input into a network model, supervised learning is carried out, and model parameters are updated through a gradient descent and back propagation algorithm.

The feature extraction backbone network mainly performs supervised learning and deeply learns the convolutional neural network through gradient descent and back propagation, and has strong advantages in image feature extraction.

And (3) carrying out binary image segmentation, wherein 0 and 1 pixel are respectively used for representing image target categories, namely background and crop row, learned by the network model. The background refers to other non-target objects in the actual working environment, including soil, sky, weeds, shade streets and the like.

And high-dimensional features are obtained, the high-dimensional feature vector of each pixel is learned through a deep learning network, and deep feature information is beneficial to feature clustering work in the later period. Different crop row instances are clustered into the same cluster. End-to-end training learning can be performed through a supervised learning mode, gradient reverse propagation, weight sharing, multi-scale information fusion and the like, and related parameter weights are updated. By weight sharing, multi-scale learning image fusion and efficient image clustering and curve fitting methods, the network scale is effectively reduced, and the algorithm real-time performance is ensured.

In one embodiment, the clustering method for spatially clustering the high-dimensional feature vectors extracted by the convolutional network model comprises: mean Shift clustering or DBSCAN clustering.

The high-dimensional characteristic vector can obtain more deep learning, clustering is convenient to realize by applying a clustering algorithm, and the unsupervised clustering algorithm does not need to determine the number of clusters, so that any number of crop rows can be identified. The clustering algorithm comprises Mean Shift clustering, DBSCAN clustering method and the like. After clustering is finished, the embedded characteristic vectors represented by different crop rows are clustered into a cluster, and corresponding id numbers are respectively given to each cluster for distinguishing.

And combining the binary image crop row coordinate points with different crop row example clustering ids to perform image fusion. Coordinate points of crop row targets are obtained from the binary image, corresponding feature vectors are found in high-dimensional vector representation, and then the vectors are projected to a new mask image, so that fusion of the binary image and the high-dimensional feature image is realized.

In one embodiment, the acquiring the field crop image comprises: a visual sensor is carried on the agricultural operation mechanical platform to obtain the image of the farmland crop.

Agricultural operation machinery platforms include unmanned aerial vehicles, various farm work tractors, combine harvesters and the like. The visual sensor is carried on the operation platform, and other original machines are not required to be modified. The vision sensor is generally mounted on the on-board platform neutral axis and is mounted at a predetermined tilt, pitch, and yaw angle. The vision sensor may be a camera equipped with a camera.

In one embodiment, fitting the crop row according to the binary image and the different crop row examples to obtain a crop row curve, includes: fitting of quadratic or polynomial curves is performed by the least squares method.

Although the output crop row segmentation result of the convolutional neural network is fine, specific parameters of crop rows are needed for the convenience of applications such as autonomous navigation, weeding and pesticide spraying. The invention uses least square method and DBSCAN algorithm to fit the crop row line, and uses quadratic polynomial curve to complete the fitting. And finally, remapping the result back to the original image.

Fitting the obtained crop row characteristic points to obtain curve parameters. Specifically, the obtained crop row characteristic points are further optimized and screened, so that the number of corresponding points of the same type of crop row meets a certain threshold, other background interferences such as weeds and the like are eliminated, then curve fitting is carried out on the same crop row characteristic points, the fitting method can be a least square method, and the fitting result can be a quadratic or polynomial curve.

In one embodiment, after acquiring images of field crops with different growth stages, different weed densities and different degrees of bending, the method further comprises: cutting an original image through the region of interest to obtain a target and a region; and the data diversity and the anti-interference capability are increased through data enhancement.

The original image can be cut through the region of interest to obtain a target and a region; through data enhancement, increase data diversity and interference killing feature, balance data quantity. The data set may be divided into a training set, a validation set, and a test set in a certain proportion.

With reference to the foregoing embodiments, another embodiment of the present invention is obtained, and fig. 5 is a second flowchart of the crop row detection method based on deep learning image segmentation provided by the present invention, and specifically refer to fig. 5, which is not repeated herein.

The S-BiSeNet V2 can successfully identify crop rows, and the segmentation result is fine, so that the better robustness of the model is demonstrated. Due to the similar color and texture characteristics of weeds and the breakage of local seedling belts, pixels are lost, but the model has good spatial information extraction capability, and the influence of complex scene interference on the whole result is small.

FCN, BiSeNet V2 and S-BiSeNet V2 models are compared according to the IoU, fp, fn and acc evaluation indexes, wherein the base network of FCN is VGG 16. Different network models were trained using the same training parameters, see table 1.

Parameter name	Parameter value
		Training batches	32
Validating batches	4
		Training period	500
Weight attenuation	0.0005
		Learning rate decay factor	0.1
Initial learning rate	0.01

Evaluation indexes were respectively calculated on the test sets, and the results are shown in table 2.

TABLE 2 comparison of different models

Model (model)	IoU	fp	fn	acc
					FCN-VGG16	0.7928	0.2408	0.0631	0.9369
BiSeNet V2	0.8648	0.1903	0.0333	0.9667
					S-BiSeNet V2	0.8980	0.1501	0.0189	0.9811

As can be seen from the above table, the segmentation accuracy of the BiSeNet V2 model is far better than that of the FCN model. The S-BiSeNet V2 model IoU was the highest. IoU was 0.0083(3 percentage points or so) higher than original BiSeNet V2.

And (3) comparing the segmentation speeds: on the crop row Data set, the speed of processing a single image by the segmentation model is compared, and the sizes of the training parameter file (Data) and the network structure file (Meta) of different models are compared, and the result is shown in table 3. In consideration of the future practical application cost, the model adopts a low-cost display card to carry out reasoning. The segmentation speed of the BiSeNet V2 model is the fastest, the time for processing each image is 27.3ms, the FCN model is the slowest in segmentation, and the S-BiSeNet V2 is about 35ms slower than that of the original BiSeNet V network. The FCN model parameters and structure are larger due to its larger number of layers. The BiSeNet V2 belongs to a light-weight network, and a Spatial CNN module is added on the basis of the S-BiSeNet V2, so that the number of parameters and the structure of a model are increased. But still within acceptable ranges. The S-BiSeNet V2 takes about 65ms, i.e., about 15FPS, to detect a single image.

TABLE 3 segmentation speed and storage size for different models

Model (model)	Speed (ms)	Reference quantity (MB)	Model structure (MB)
				FCN-VGG16	150.39	408	7.75
BiSeNet V2	27.30	26.6	14.7
				S-BiSeNet V2	65.54	33.3	20.9

The crop row detection device based on deep learning image segmentation provided by the invention is described below, and the crop row detection device based on deep learning image segmentation described below and the crop row detection method based on deep learning image segmentation described above can be referred to correspondingly.

Fig. 6 is a schematic structural diagram of a crop row detection apparatus based on deep learning image segmentation according to the present invention, and as shown in fig. 6, the crop row detection apparatus based on deep learning image segmentation includes: an acquisition module 601, a processing module 602, a clustering module 603, and a fitting module 604. The acquisition module 601 is used for acquiring an image of a farmland crop; the processing module 602 is configured to input the farmland crop image into the trained convolutional network model, and output a binary image of an image segmentation result; the clustering module 603 is configured to perform spatial clustering on the high-dimensional feature vectors extracted by the convolutional network model to obtain different crop row examples; the fitting module 604 is configured to perform crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

The device embodiment provided in the embodiments of the present invention is for implementing the above method embodiments, and for details of the process and the details, reference is made to the above method embodiments, which are not described herein again.

The crop row detection device based on deep learning image segmentation provided by the embodiment of the invention obtains a binarization segmentation result and pixel level high-dimensional vector representation through a convolution neural network model, obtains crop row characteristics through fusing the two information, and then performs fitting description on crop row lines by using a clustering algorithm and a curve fitting algorithm, thereby effectively improving the crop row identification speed and accuracy.

Fig. 7 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 7, the electronic device may include: a processor (processor)701, a communication Interface (Communications Interface)702, a memory (memory)703 and a communication bus 704, wherein the processor 701, the communication Interface 702 and the memory 703 complete communication with each other through the communication bus 704. The processor 701 may invoke logic instructions in the memory 703 to perform a method of crop row detection based on deep learning image segmentation, the method comprising: obtaining an image of a farmland crop; inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result; carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

In addition, the logic instructions in the memory 703 can be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the crop row detection method based on deep learning image segmentation provided by the above methods, the method including: obtaining an image of a farmland crop; inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result; carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

In yet another aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the method for crop row detection based on deep learning image segmentation provided in the foregoing embodiments, the method including: obtaining an image of a farmland crop; inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result; carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples; performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve; and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A crop row detection method based on deep learning image segmentation is characterized by comprising the following steps:

obtaining an image of a farmland crop;

inputting the farmland crop image into the trained convolution network model, and outputting a binary image of an image segmentation result;

carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples;

performing crop row fitting according to the binary image and the different crop row examples to obtain a crop row curve;

and the convolution network model is obtained by training according to the farmland crop image with the known crop row result.

2. The crop row detection method based on deep learning image segmentation as claimed in claim 1, wherein before inputting the farmland crop image into the trained convolutional network model, further comprising:

obtaining images of farmland crops with different growth stages, different weed densities and different bending degrees, and marking crop rows as training samples;

training the constructed improved BiSeNet V2 network model based on a plurality of training samples to obtain the trained convolutional network model;

the improved BiSeNet V2 network model is obtained by adding an SCNN network layer between an aggregation layer and a seghead layer of a BiSeNet V2 network.

3. The crop row detection method based on deep learning image segmentation according to claim 1 or 2, wherein the training of the constructed convolutional network model based on a plurality of training samples comprises:

and training the constructed convolution network model through gradient descent and back propagation.

4. The deep learning image segmentation-based crop row detection method according to claim 1, wherein the clustering method for spatial clustering the high-dimensional feature vectors extracted by the convolutional network model comprises:

mean Shift clustering or DBSCAN clustering.

5. The crop row detection method based on deep learning image segmentation according to claim 1, wherein the obtaining of the farmland crop image comprises:

a visual sensor is carried on the agricultural operation mechanical platform to obtain the image of the farmland crop.

6. The crop row detection method based on deep learning image segmentation according to claim 1, wherein performing crop row fitting according to the binary image and the different crop row instances to obtain a crop row curve comprises:

fitting of quadratic or polynomial curves is performed by the least squares method.

7. The crop row detection method based on deep learning image segmentation according to claim 2, wherein after obtaining the images of the field crops at different growth stages, different weed densities and different degrees of curvature, the method further comprises:

cutting an original image through the region of interest to obtain a target and a region;

and the data diversity and the anti-interference capability are increased through data enhancement.

8. A crop row detection apparatus based on deep learning image segmentation, comprising:

the acquisition module is used for acquiring an image of the farmland crop;

the processing module is used for inputting the farmland crop image into the trained convolution network model and outputting a binary image of an image segmentation result;

the clustering module is used for carrying out spatial clustering on the high-dimensional characteristic vectors extracted by the convolutional network model to obtain different crop row examples;

the fitting module is used for fitting the crop row according to the binary image and the different crop row examples to obtain a crop row curve;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for crop row detection based on deep learning image segmentation according to any one of claims 1 to 7.

10. A non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program, when being executed by a processor, implements the steps of the method for crop row detection based on deep learning image segmentation according to any one of claims 1 to 7.