CN113077452A

CN113077452A - Apple tree pest and disease detection method based on DNN network and spot detection algorithm

Info

Publication number: CN113077452A
Application number: CN202110398406.XA
Authority: CN
Inventors: 李海; 李谊骏; 陈诗果; 杨谋; 兰元帅
Original assignee: Chengdu College of University of Electronic Science and Technology of China
Current assignee: Chengdu College of University of Electronic Science and Technology of China
Priority date: 2021-04-09
Filing date: 2021-04-09
Publication date: 2021-07-06
Anticipated expiration: 2041-04-09
Also published as: CN113077452B

Abstract

The invention discloses an apple tree pest detection method based on a DNN network and a spot detection algorithm, which relates to the technical field of fruit tree pest detection and comprises the following steps: step 1, building a basic DNN neural network, initializing a weight matrix W and a bias parameter b, inputting a data set, and updating the weight matrix W and the bias parameter b through a forward propagation algorithm and a backward propagation algorithm of the neural network; step 2, carrying out image segmentation on the acquired image after the acquired image is zoomed by adopting a Gaussian pyramid algorithm, and separating the foreground and the background of the image; step 3, histogram equalization is carried out on the image which is segmented by the image, the feature points in the image are enhanced, step 4, a LOG algorithm is adopted to extract the feature points in the image, then open operation processing is adopted to remove noise points, and step 5: and (4) inputting the characteristic points processed in the step (4) into the trained DNN neural network for judgment, and identifying whether the leaves have plant diseases and insect pests.

Description

Apple tree pest and disease detection method based on DNN network and spot detection algorithm

Technical Field

The invention relates to the technical field of fruit tree pest detection, in particular to an apple tree pest detection method based on a DNN network and a spot detection algorithm.

Background

China is the biggest apple producing country in the world, and the planting area and the yield of the apples both account for more than 50 percent of the world. However, the quality of the apples in China is still a certain gap compared with the quality of the apples in developed countries, and the laggard pest control level is a main factor for restricting the development of the apples in China. At the present stage, two main methods for preventing and treating apple tree diseases and insect pests in China are available: "prevention and cure calendar" and single pest and disease detection method. The prevention and control of the plant diseases and insect pests through the 'prevention and control calendar' is carried out according to the occurrence condition of the plant diseases and insect pests in the past year, the key period of the plant diseases and insect pests is usually missed, and the effect is not good. The method of R-CNN, Tamura and the like is used for preventing and treating diseases and insect pests, the research is often carried out on the pathological images of apples, and the diseases and insect pests cannot be detected at the initial stage of disease and insect pest invasion. The method for preventing and treating the plant diseases and insect pests by using a single detection method such as a support vector machine and a YOLO (YOLO) method is characterized in that the characteristics of spots on the leaves of the apple trees are compared with the typical characteristics of the plant diseases and insect pests, so that the aim of identification is fulfilled, the problems of local optimal solution, gradient disappearance and the like are easily caused, the accuracy is low, and the requirement for apple tree orchard deployment is difficult to meet.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides an apple tree pest and disease detection method based on a DNN network and a spot detection algorithm.

The purpose of the invention is realized by the following technical scheme:

the apple tree pest and disease damage detection method based on the DNN network and the spot detection algorithm is characterized by comprising the following steps of:

step 1, building a basic DNN neural network, initializing a weight matrix W and a bias parameter b, inputting a data set, updating the weight matrix W and the bias parameter b through a forward propagation algorithm and a backward propagation algorithm of the neural network, and executing step 2;

step 2, carrying out image segmentation on the acquired image after the acquired image is zoomed by adopting a Gaussian pyramid algorithm, separating the foreground and the background of the image, and executing step 3;

step 3, carrying out histogram equalization on the image subjected to image segmentation, enhancing feature points in the image, and executing step 4;

step 4, extracting feature points in the image by using an LOG algorithm, then removing noise by using open operation processing, and executing step 5;

and 5: and (4) inputting the characteristic points processed in the step (4) into the trained DNN neural network for judgment, and identifying whether the leaves have plant diseases and insect pests.

Preferably, the image segmentation extraction process of step 2 includes two steps: firstly, building a color model, and then segmenting by an iterative energy minimization segmentation algorithm.

Preferably, the process of extracting the feature points in the image by using the LOG algorithm in step 4 includes blob detection, where the blob detection includes obtaining a maximum value or a minimum value of a laplacian of gaussian response by performing derivation on a normalized two-dimensional laplacian of gaussian operator, and then obtaining blobs in the image according to the maximum value or the minimum value of the response.

Preferably, the performing of the on operation in step 4 includes performing an erosion operation on the image obtained by the speckle detection, and then performing an expansion operation on the image to remove the black interference block in the image.

The invention has the beneficial effects that:

the invention provides an apple tree pest detection method based on a DNN network and a spot detection algorithm, and realizes an apple tree pest detection system based on machine vision. The result shows that the identification accuracy rate of the detection system which takes the LOG algorithm as the feature extraction and the DNN neural network as the identification model to the apple tree leaf diseases and insect pests can reach 91.17%, the detection effect of the system to the apple tree leaf diseases and insect pests is clear, the disease and insect pest identification accuracy rate is high, the feasibility of the system is proved, and the detection system can be applied to the identification of common diseases and insect pests of apple trees. In conclusion, the disease and pest control capability of the apple trees can be improved by extracting the disease and pest characteristics of the apple tree leaves through the image processing method of machine vision and then constructing the DNN neural network for detecting and identifying the apple tree diseases and pests.

Drawings

FIG. 1 is a schematic view of the detection method of the present invention;

FIG. 2 is a comparison graph before and after histogram equalization;

FIG. 3 is a comparison of a spot test before and after;

FIG. 4 is a comparison of before and after opening operation;

FIG. 5 is a diagram of a neural network architecture;

FIG. 6 is a diagram of a neural network framework;

FIG. 7 is a schematic view of pest detection accuracy;

fig. 8 is a schematic view of loss values in pest detection.

Detailed Description

The technical solutions of the present invention are further described in detail below with reference to the accompanying drawings, but the scope of the present invention is not limited to the following.

As shown in figure 1, the application provides an apple tree pest detection method based on a DNN network and a spot detection algorithm. And carrying out image segmentation on the image acquired by the camera by a GrabCT method, then extracting feature points in the blade by using a Gaussian Laplacian operator and an LOG algorithm, and sending the feature points into a neural network for training to finally obtain a detection result. In order to solve the problem that a neural network is easy to fall into a local optimal solution and gradient disappears, a DNN network is adopted in the system, in order to further improve the accuracy of the algorithm, the DNN network is mainly adopted in the system, a spot detection algorithm is assisted, the apple tree pest and disease damage is efficiently detected, meanwhile, a detection result is fed back to a supervision platform in real time through a cloud platform, data updating is carried out, cloud sharing and data summarization of data are achieved, and fruit growers are helped to know the growth condition of the apple trees in real time.

While neural networks are based on the extension of perceptrons, DNN can be understood as a neural network with many hidden layers, the essence of which is actually a multi-layer linear regression. The DNN internal network layers can be divided into three types, i.e., an input layer, a hidden layer and an output layer, wherein generally, the first layer is the input layer, the last layer is the output layer, and the middle layers are all hidden layers, and the layers are fully connected.

The system adopts an input layer, three hidden layers and an output layer, the structure is shown in figure 5, and the used activation function is a Sigmoid function.

As shown in fig. 6, a basic DNN neural network is established, a weight matrix w and a bias parameter b are initialized, then images in a training set are subjected to image segmentation, histogram equalization, speckle detection and operation to obtain feature points of each image and coordinates of positions of the feature points and the coordinates, the feature points are selected in an original image, the selected feature images are sent to the neural network for learning, then whether the test set images have diseases and insect pests or not is judged through a forward propagation algorithm, and the accuracy is obtained through calculation. And finally, updating the weight matrix w and the bias parameter b through a back propagation algorithm, and then testing until the final identification result reaches the optimum to obtain the optimum model.

The essence of the forward propagation algorithm of the DNN neural network is that a series of linear operations and activation operations are performed by using a plurality of weight coefficient matrixes W, offset vectors b and input vectors x, each layer calculates the output of the next layer according to the output result of the previous layer from the input layer, and the operations are performed backwards layer by layer until the output layer outputs the final result.

The activation function selected by the system isSigmoid function σ (z), assuming that the output values of the hidden layer and the output layer are a. When there are m neurons in layer 1-1, the output to the jth neuron of layer 1

Comprises the following steps:

it can also be represented in a matrix manner:

a^l＝σ(z^l)＝σ(W^la^l-1+b^l)， (28)

it should be noted that the back propagation algorithm of the DNN neural network is actually to perform iterative optimization on the loss function by using a gradient descent method to solve a minimum value, so as to find an optimal linear weight matrix W and an optimal offset vector b, and make the output result of the sample equal to or close to the sample label as much as possible.

The loss of the metric is first derived using a mean square error method, i.e., it is desirable to minimize the following for each sample:

then, according to the loss function, a weight matrix W and an offset vector b of each layer are solved by iteration through a gradient descent method, and W and b satisfy the requirement of L layers of an output layer

a^L＝σ(z^L)＝σ(W^La^L-1+b^L)， (30)

So for the output layer parameters, the loss function becomes:

the gradient of the output layer can be solved as:

the gradient of the L-1 layer and the L-2 layer can be solved by carrying out step-by-step recursion according to the formula (32). From the successively calculated δ of the L layers^lAnd forward propagation algorithm z^l＝W^la^l-1+b^lCan simply solve the W of the L-th layer^LAnd b^lGradient of (2)

The L-th layer delta can be solved according to the above formula^LSolving for delta^lAnd delta^l+1Is to solve for

But according to z^lAnd z^l+1It is possible to find out:

z^l+1＝W^l+1a^l+b^l+1＝W^l+1σ(z^l)+b^l+1， (35)

then substituting equation (36) into δ above^lAnd delta^l+1Can obtain

The weight W can then be updated^lAnd bias b^l

It should be noted that image segmentation is a process of dividing an image into a plurality of specific regions with unique properties, and then extracting a target region of interest from the regions. The system adopts GrabCont algorithm to extract foreground, and extracts apple tree leaves from the whole image. The whole extraction process is divided into two steps: and building a color model and iterating an energy minimization segmentation algorithm.

Wherein, the establishment of the color model comprises the following contents:

in the RGB color space, for each pixel, it is either a gaussian component from some background GMM or a gaussian component from some target GMM [13], so we can model the target and background with a full covariance GMM of k gaussian components, whose Gibbs energy after modeling is:

E(α，k，θ，z)＝U(α，k，θ，z)+V(α，z)， (1)

U(α，k，θ，z)＝∑_nD(α_n，k_n，θ，z_n)， (2)

in formula (2), U is an area term and represents a negative logarithm of the probability that a pixel belongs to the target or the background, i.e., a penalty for classifying the pixel as the target or the background. Because the model of the mixture gaussian density is:

so we get the negative logarithm of the Gaussian mixture model, the original equation becomes:

the formula (5) has three parameters theta of GMM, the first parameter theta is the weight pi of the Gaussian component, the second parameter theta is the mean vector mu of the Gaussian component, and the third parameter theta is the covariance matrix sigma.

θ＝{π(α，k)，μ(α，k)，∑(α，k)，α＝0，1，k＝1...K}， (6)

In other words, the three GMM parameters describing the target and the background need to be determined by learning, and with the determination of these three parameters, the zone energy term of Gibbs energy can be determined.

Generally, the similarity of two pixels in the RGB space is calculated by euclidean distance, and the most critical parameter β in this case is determined by the contrast of the image. If the image has low contrast, i.e. the pixels m and n have differences themselves, the difference i zm-zn i is low, and then a larger parameter β is multiplied to amplify the difference. If the contrast of the image is relatively high, that is, the difference i zm-zn i of m and n belonging to the same target pixel is relatively high, a relatively small parameter β needs to be multiplied to reduce the difference. In order to enable the V term to work normally under the condition of high or low contrast, the difference is reduced by a relatively small parameter β, so when the constant γ is 50, the weight of n-link is:

V(α，z)＝γ∑_(m，n)∈C[α_n≠α_m]exp-β||z_m-z_n||²， (7)

at this point, a color model is successfully built even if the first step of image segmentation is completed.

The iterative energy minimization segmentation algorithm comprises the following contents:

the target pixel and the background pixel are first acquired by framing the target, and then the GMMs of the target and the background can be estimated from these two pixels. There is also a need to cluster pixels belonging to the object and background into k classes, i.e. k gaussian models in GMM, by the k-mean algorithm [14 ]. Wherein, the gaussian component allocated to each pixel is:

the parameter mean and covariance in equation (8) are both estimated from their own RGB values, and the weight of the gaussian component can be obtained from the ratio of the number of pixels belonging to the gaussian component to the total number of pixels:

then, a graph can be established according to the Gibbs energy term, then the weight t-link and the weight n-link are calculated according to the graph,

image segmentation was finally performed using maxflow/mincut algorithm:

wherein the histogram equalization comprises the following:

the system adopts histogram equalization to widen the gray value of apple leaf scabs in the image and merge the gray value without the scabs, so that the contrast is increased, the image is clear, and the purpose of enhancing the characteristic points is achieved.

The nature of the image gray value is actually a one-dimensional discrete function:

h(k)＝n_k k＝0，1，...，L-1， (11)

in formula (11), n_kFor the number of pixels in the image f (x, y) with a gray level of k, the height of each column of the histogram corresponds to oneN is_k. Here, the histogram may be normalized, and the relative frequency of occurrence of gray levels in the normalized histogram may be defined as P_r(k) In that respect Namely, it is

P_r(k)＝n_k/N， (12)

In the formula (12), N is the total number of pixels of the image f (x, y), and N is_kThe number of pixels in the image f (x, y) having a gray level K. In this case, the original image gradation and the histogram-equalized image gradation are represented by r and s, respectively, and when r, s ∈ (0, 1), the gradation value of the pixel changes between black and white. When r is 0, the gray value of the pixel is black. When r is 1, the gray scale value of the pixel is white. That is to say in [0, 1 ]]Any one of r in (b), through the transformation function T (r), can generate a corresponding s, an

s＝T(r)， (13)

In the formula (13), in the range of 0. ltoreq. r.ltoreq.1, T (r) is a monotonically increasing function, so that when 0. ltoreq. r.ltoreq.1, T (r) is 0. ltoreq.1. The probability density of the random variable r is Pr (r) obtained by probability theory, and since the random variable s is a function of r, the distribution function of the random variable s is assumed to be P_s(s), from the definition of the distribution function:

since the derivative of the distribution function is a probability density function, the derivation of s on both sides can be obtained:

knowledge from probability theory can be used to find the interval [ a, b]A function of uniform distribution having a probability density function of

Because r ∈ [0, 1 ] after normalizing it]Therefore, it is

And due to P_s(s)ds＝P_r(r) dr so that ds is P_rThe two-sided integration of (r) dr yields:

for digital images with discrete gray levels, the probability can be replaced by frequency, so the transformation function T (r)_k) May be represented as;

as can be seen from equation (17), the equalized gray level sk of each pixel can be directly calculated from the histogram of the original image. The effect before and after histogram equalization on apple tree leaves is shown in fig. 2.

It should be noted that the blob detection includes the following:

laplacian of gaussian operator: for a two-dimensional gaussian function:

its laplace transform is:

the normalized gaussian lapel transform:

the LOG algorithm: firstly, gaussian low-pass filtering is carried out on the image f (x, y) by adopting a gaussian kernel with variance sigma, and noise points in the image are removed.

L(x，y；σ)＝f(x，y)*G(x，y；σ)

Then carrying out Laplace transform on the image;

namely:

here, laplacian transform is performed on the gaussian kernel, and then the image is convolved.

Multi-scale detection: when the σ scale is fixed, only the blobs of the corresponding radius can be detected. Therefore, multi-scale detection can be performed by deriving the normalized two-dimensional laplacian of gaussian operator. The normalized laplacian of gaussian function is;

solving for

Is equivalent to solving the equation (24)

This gives:

r²-2a²＝0， (26)

when the size is measured

Then, a laplacian of gaussian sound can be obtainedThe corresponding maximum or minimum, and then the speckle in the image can be obtained based on this response, as shown in fig. 3.

The morphological opening operation includes the following steps:

when the open operation is used for processing the image, noise can be eliminated and small interference blocks can be removed on the premise of not influencing the original image. Therefore, after the speckle detection algorithm is used, the opening operation is used, the image obtained by the speckle detection is firstly subjected to the corrosion operation and then subjected to the expansion operation, and then the black interference block in the image can be removed, so that the extracted scab is more vivid. The images before and after the on operation are shown in fig. 4.

One specific embodiment of the present application is as follows:

the main control chip adopts an NLE-AI800 Internet of things platform, the CPU adopts a dual-core A73+ a dual-core A53+ a single-core A53, the AI computing unit adopts a dual-core NNIE @840MHz and is provided with a 4GB (byte-size) memory and a 32G memory, and the system has super-strong operation processing and analysis capability and can meet the requirements of the system on operation speed and storage space. The system also adopts a new continental Internet of things platform, which is an Internet of things system integrating the functions of equipment online acquisition, remote control, wireless transmission, data processing, early warning information release, decision support, integrated control and the like. The distributed data storage and calculation method supports access of various gateways and hardware equipment of the Internet of things, and stores and calculates data in a distributed mode. The data can be transmitted to the cloud end through a TCP protocol, and the data can be shared and visually analyzed conveniently.

In order to verify the accuracy of the apple tree pest and disease detection system more accurately, the test site is specially selected in the Shaanxi region where the apple trees are planted in the largest area and yield in recent years. In a preselected apple tree test field, an experimenter holds the device, acquires image information of apple tree leaves in real time through a camera, analyzes frames one by one, processes each frame of image, sends the processed image into a DNN neural network, and finally displays a recognition result on a screen.

The experiment scientifically selects 100 apple trees as sampling samples, the sampling samples are divided into six groups for testing, and the test results of each time are transmitted to the cloud end through a TCP protocol, so that the experimental data can be analyzed and researched conveniently.

The results of the experiment are shown in table 1 below:

TABLE 1 statistical table of probability of diseases and insect pests of apple trees

Plant diseases and insect pests	Number of samples	Mosaic disease	Rust disease	Grey leaf spot disease	Alternaria leaf spot	Brown spot disease	Rate of accuracy
								Mosaic disease	100	90	4	0	0	6	90.0％
Rust disease	100	4	92	0	2	2	92.0％
								Grey leaf spot disease	100	0	6	89	4	1	89.0％
Alternaria leaf spot	100	3	4	1	90	2	90.0％
								Brown spot disease	100	0	0	2	4	94	94.0％
No disease	100	2	1	1	3	1	92.0％

From the above data, it was found that the pest detection accuracy was 89% or more in each test area, as shown in fig. 7 and 8. The system can meet the requirement of fruit growers on prevention of apple tree diseases and insect pests by identifying the diseases and insect pests with accuracy, and can be widely deployed and applied in apple tree gardens. However, the system collects the image data through the camera, and the collected image is interfered by factors such as visual interference or image distortion and the like probably due to the influence of some factors such as a light source, environment, hardware and the like, so that the system has a certain error in the detection of plant diseases and insect pests, and the error range is 0.2 to 0.6 percent

The foregoing is merely a preferred embodiment of the invention, it being understood that the embodiments described are part of the invention, and not all of it. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. The invention is not intended to be limited to the forms disclosed herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. The apple tree pest and disease damage detection method based on the DNN network and the spot detection algorithm is characterized by comprising the following steps of:

2. The apple tree pest detection method based on the DNN network and the speckle detection algorithm as claimed in claim 1, wherein the image segmentation extraction process of the step 2 comprises two steps: firstly, building a color model, and then segmenting by an iterative energy minimization segmentation algorithm.

3. The apple tree pest and disease detection method based on the DNN network and the speckle detection algorithm according to claim 3, wherein the process of extracting the feature points in the image by using the LOG algorithm in the step 4 comprises speckle detection, wherein the speckle detection comprises obtaining the maximum value or the minimum value of the Gaussian Laplace response by differentiating the normalized two-dimensional Laplace Gaussian operator, and then obtaining the speckles in the image according to the maximum value or the minimum value of the response.

4. The apple tree pest detection method based on the DNN network and the speckle detection algorithm according to claim 3, wherein the step 4 of removing noise by using the open operation comprises the steps of performing corrosion operation on an image obtained by speckle detection, and then performing expansion operation to remove black interference blocks in the image.