CN113128593A - Plant fine-grained identification method based on bilinear convolutional neural network - Google Patents

Plant fine-grained identification method based on bilinear convolutional neural network Download PDF

Info

Publication number
CN113128593A
CN113128593A CN202110425490.XA CN202110425490A CN113128593A CN 113128593 A CN113128593 A CN 113128593A CN 202110425490 A CN202110425490 A CN 202110425490A CN 113128593 A CN113128593 A CN 113128593A
Authority
CN
China
Prior art keywords
bilinear
grained
feature
features
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110425490.XA
Other languages
Chinese (zh)
Inventor
业巧林
范习健
杨紫颖
何文妍
母园
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Forestry University
Original Assignee
Nanjing Forestry University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Forestry University filed Critical Nanjing Forestry University
Priority to CN202110425490.XA priority Critical patent/CN113128593A/en
Publication of CN113128593A publication Critical patent/CN113128593A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a plant fine granularity identification method based on a bilinear convolutional neural network, which comprises the following steps: s1, extracting coarse-grained features by using a Relu activation function to obtain coarse-grained features X1; s2, acquiring integral representation of the bilinear model; s3, in the step S2, bilinear fusion is carried out on the features extracted by the A, B two feature extraction functions in the bilinear model, and a matrix b is obtained; s4, summing and pooling the matrix b to obtain a matrix xi, and carrying out multi-dimensional vector expansion on xi to obtain a feature vector x; s5, carrying out matrix normalization and L2 normalization on the feature vector x to obtain bilinear features, the method is scientific and reasonable in structure and safe and convenient to use, effectively extracts the fine differences among high-flux plant expression species and even among individuals by using a bilinear model, fully exerts the feature extraction advantages of bilinear pooling on fine-grained images, and achieves higher plant fine-grained identification precision.

Description

Plant fine-grained identification method based on bilinear convolutional neural network
Technical Field
The invention relates to the technical field of plant identification algorithm improvement, in particular to a plant fine-grained identification method based on a bilinear convolutional neural network.
Background
In recent years, with the development of deep learning in the field of computer vision, a deep Curl Neural Network (CNN) processing images of two-dimensional natural scenes becomes one of research focuses of common concern at home and abroad, and vast agriculture and forestry research workers also apply a deep learning technology represented by the CNN to plant phenotype research, thereby opening an intelligent research era of plant phenotype, displaying related research, researching a classification network based on a deep learning method on the basis of acquiring a large amount of high-quality plant image data in real time, and providing precision, texture, color and other morphological characteristic phenotypes for agricultural producers at high efficiency and low cost, Reliable plant classification recognition results. The deep learning technology represented by CNN has strong feature extraction and modeling capability on plant phenotype images, the performance of the plant phenotype image is superior to that of a traditional machine learning method to a great extent, the plant phenotype image becomes a common algorithm for phenotype big data analysis, vein texture features are extracted by Grinblat and the like, three leguminous plants of different varieties are identified through the deep learning algorithm, and the identification rate reaches 96.9% +/-0.2%; liu et al, using multi-feature fusion and an improved deep belief network, improved leaf classification accuracy to 93.9% for 220 different plants; nguyen et al utilize CNN and migratory learning ability to construct a universal crop identification system which adapts to uneven distribution of plants in different regions through a flexible data collection mode, and evaluate the classification identification effect of CNN algorithm frameworks constructed by researchers such as AlexNet and VGG under different organ characteristics of crops. The above-mentioned methods all have in common that a deep learning model that processes images of natural scenes is directly used for images of plants, but there are the following problems in practical applications:
1. according to the method, the plant image is not specially analyzed, so that unique characteristics existing in the plant image are ignored, external interference factors such as posture, illumination, shielding and complex background exist in data acquisition, different samples belonging to the same type of image have large difference, the plant has growth variation, the characterization difference of different stages of plant growth is further obvious, and the fine granularity characteristic of 'intra-class difference' is formed;
2. the detailed division of biological subclasses or subclasses exists in plants, and the subclasses are similar to each other in a certain biological form, so that the fine-grained identification problem of 'similarity between classes' is caused;
3. most of domestic and foreign related researches are only suitable for identifying coarse granularity of plants with large intermediate differences, can not effectively solve the problem of identifying fine granularity caused by direct inter-species similarity, and can not meet the requirements of fine agriculture;
therefore, a plant fine-grained identification method based on a bilinear convolutional neural network is urgently needed to solve the problems.
Disclosure of Invention
The invention aims to provide a plant fine-grained identification method based on a bilinear convolutional neural network, so as to solve the problems in the background technology.
In order to solve the technical problems, the invention provides the following technical scheme: a plant fine-grained identification method based on a bilinear convolutional neural network comprises the following steps:
s1, extracting coarse-grained features by using a Relu activation function according to a VGG 16-based bilinear model feature extractor to obtain coarse-grained features X1;
s2, acquiring an integral representation of the bilinear model, pre-training a network of the bilinear model, initializing a basic network, transferring the initialized basic network to a data set for training, and extracting fine-grained features and updating model parameters;
s3, in step S2, bilinear fusion is carried out on the features extracted by the A, B two feature extraction functions in the bilinear model, and an M multiplied by N dimensional matrix b is obtained;
s4, carrying out summation pooling on the matrix b obtained in the step S3 to obtain a matrix xi, and carrying out multi-dimensional vector expansion on xi to obtain a feature vector x;
s5, according to the feature vector x obtained in the step S4, performing moment normalization and L2 normalization on the features to obtain bilinear features, and performing a chain derivation method on the bilinear features to obtain a gradient expression of the loss function to the feature vector;
further, in step S1, globally using a VGG16 model with a convolutional layer of 3 × 3 and a pooling layer of 2 × 2, simplifying the calculation by using a Relu activation function, defining the whole feature extraction function as f (·), and using a pre-network based on a VGG16 bilinear model to obtain a coarse feature X1:
X1＝Hvgg(x，y，{W1，b1，δrelu})
wherein HvggIs the convolution layer of VGG16, and (x, y) is the input characteristic parameter of the image, W1Weight parameters requiring iterative updating for the network model, b1For linear bias, δ is the relu activation function;
further, the input characteristic parameters (x, y) represent the input of the first layer, x is an RGB three-channel value of the image, and y is the numeralization of the classification label;
further, the weight parameter W and the linear bias parameter b are updated according to a random gradient descent method, the gradient is obtained according to a chain derivation method, and the weight parameter W and the new formula are as follows:
Figure BDA0003029371180000031
the linear bias parameter b is updated by the formula:
Figure BDA0003029371180000032
where η represents learning efficiency and δ represents derivation according to a chainThe gradient obtained by the method is used as a rule,
Figure BDA0003029371180000033
an ith neuron output indicating that the l-1 layer has used an activation function;
further, in step S2, the bilinear model is expressed as a whole: b ═ fA，fBP, C), wherein fA，fBAs a feature function for mapping the image I to the location l to obtain fA(l，I)∈Rc×M、fB(l，I)∈Rc×NWherein P is a pooling layer for sampling the feature layer, and C is a classification function;
further, in step S3, the feature function f is setA(l，I)∈Rc×MAnd fB(l，I)∈Rc×NCarrying out bilinear fusion multiplication at the same position to obtain an M multiplied by N dimensional matrix b:
Figure BDA0003029371180000047
further, in step S4, summing and pooling are adopted, the matrix b at all positions is accumulated according to the following formula to obtain a matrix xi, and the matrix xi is subjected to multidimensional vector expansion to obtain a feature vector x:
Figure BDA0003029371180000041
x＝vec(ξ(I))；
further, in step S5, the moment normalization and the L2 normalization are combined to obtain the final feature, and fine-grained classification is performed on the final feature, where the normalized feature description is represented as z, and z is obtained according to the following formula:
Figure BDA0003029371180000042
furthermore, the bilinear model extracts the image features, wherein c is 1, and M and N are featuresThe number of channels represents two characteristics in the bilinear model as vectors respectively
Figure BDA0003029371180000043
(Vector)
Figure BDA0003029371180000044
Obtaining a matrix from the matrix accumulation
Figure BDA0003029371180000045
Denote A as [ a ]1，...，aL]∈RM×LAnd B is represented as B ═ B1，...，bL]∈RN×LAnd generating xi by using a matrix relation, wherein xi is expressed as:
ξ(I)＝ABT∈RM×N；
further, a characteristic function fAAnd a characteristic function fBThe output tensor is used for obtaining M multiplied by N dimensions, the position L is M multiplied by N position points, each position point obtains P multiplied by P dimensions after bilinear transformation, the feature vector of PP multiplied by 1 is obtained after accumulation and pooling, back propagation is carried out through a chain derivation rule for finishing end-to-end training, and the bilinear feature after accumulation and pooling is expressed as x-ATB, the gradient of the loss function to the feature vector is expressed as
Figure BDA0003029371180000046
Where P is the pooling function.
Compared with the prior art, the invention has the following beneficial effects:
1. the invention uses the bilinear model to effectively extract the subtle differences among the expression species of the high-flux plants and even among individuals, and the plant fine-grained identification model based on the transfer learning, the model constructs an identification model which can effectively represent the tiny local differences of different categories and capture the shared information of the same category to mine the knowledge structure of the plant image, designs a bilinear local feature descriptor, constructs a plant phenotype fine-grained identification model based on the bilinear CNN, fully exerts the feature extraction advantages of bilinear pooling on the fine-grained image, achieves higher plant fine-grained identification precision, enhances the robustness of a network model, and is compared with the previous plant coarse-grained identification method only suitable for the larger interspecies differences, the fine-grained identification method can effectively solve the problem of fine-grained identification caused by similarity among plant species, and realizes effective identification of plant images with fine-grained characteristics.
2. The bilinear model adopted by the invention is minimized in the aspect of training loss, has fast convergence, solves the problem that the traditional coarse-grained deep learning network is difficult to adapt to a plant fine-grained data set, can effectively improve the identification accuracy and generalization robustness of the model, and can effectively finish the identification of the plant phenotype fine-grained data set.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of a method for identifying plant fine granularity based on a bilinear convolutional neural network according to the invention;
FIG. 2 is a schematic process diagram of a plant fine-grained identification method based on a bilinear convolutional neural network according to the present invention;
FIG. 3 is a data sample diagram in an embodiment of a plant fine-grained identification method based on a bilinear convolutional neural network according to the present invention;
FIG. 4 is a diagram of a comparison model recognition result in an embodiment of a plant fine-grained recognition method based on a bilinear convolutional neural network according to the present invention;
FIG. 5 is a comparative model loss curve diagram in an embodiment of a plant fine-grained identification method based on a bilinear convolutional neural network.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example (b): please refer to fig. 1-2: the invention provides the technical scheme that: a plant fine-grained identification method based on a bilinear convolutional neural network comprises the following steps: s1, extracting coarse-grained features by using a Relu activation function according to a VGG 16-based bilinear model feature extractor to obtain coarse-grained features X1;
s2, acquiring an integral representation of the bilinear model, pre-training a network of the bilinear model, initializing a basic network, transferring the initialized basic network to a data set for training, and extracting fine-grained features and updating model parameters;
s3, in step S2, bilinear fusion is carried out on the features extracted by the A, B two feature extraction functions in the bilinear model, and an M multiplied by N dimensional matrix b is obtained;
s4, carrying out summation pooling on the matrix b obtained in the step S3 to obtain a matrix xi, and carrying out multi-dimensional vector expansion on xi to obtain a feature vector x;
and S5, performing moment normalization and L2 normalization on the features according to the feature vector x obtained in the step S4 to obtain bilinear features, and performing a chain derivation method on the bilinear features to obtain a gradient expression of the loss function to the feature vector.
In step S1, globally using a VGG16 model with convolutional layer of 3 × 3 and pooling layer of 2 × 2, simplifying the calculation with the Relu activation function, defining the whole feature extraction function as f (·), and using a pre-network based on VGG16 bilinear model to obtain coarse feature X1:
X1＝Hvgg(x，y，{W1，b1，δrelu})
wherein HvggIs the convolution layer of VGG16, and (x, y) is the input characteristic parameter of the image, W1Weight parameters requiring iterative updating for the network model, b1For linear biasing, δ is the relu activation function.
The input characteristic parameters (x, y) represent the input of the first layer, x is the RGB three-channel value of the image, and y is the numeralization of the classification label.
The weight parameter W and the linear bias parameter b are updated according to a random gradient descent method, the gradient is obtained according to a chain type derivation method, and the weight parameter W and the new formula are as follows:
Figure BDA0003029371180000071
the linear bias parameter b is updated by the formula:
Figure BDA0003029371180000072
where η represents the learning efficiency, δ represents the gradient obtained according to the chain-derivative rule,
Figure BDA0003029371180000073
indicating that layer l-1 has used the ith neuron output of the activation function.
The adopted bilinear model feature extractor is composed of two weak supervision networks, the preposed networks of two fine-grained models can perform coarse-grained feature extraction based on VGG16, the networks are pre-trained on ImageNet to initialize two basic networks, then the networks are migrated to a small Chinese cabbage dataset in the embodiment to be trained so as to perform fine-grained feature extraction and model parameter updating, VGG16 is composed of 5 layers of convolution layers, 3 layers of full connection layers and softmax output layers, the layers are separated by max-poolling maximum pooling, all activation units of the hidden layers adopt Relu functions, the VGG model is a preferred algorithm for extracting CNN features from images, the VGG16 network model provides that the depth increase of a convolutional neural network and the use of a small convolutional kernel have great effect on the final classification recognition effect of the networks, the VGG16 is finely tuned in the embodiment, the final pooling layer and the two full connection layers and the softmax layers are removed, the convolution layers used in the VGG16 are all 3 x 3, the pooling layers are all 2 x 2, gradient calculation and back propagation can be more effectively carried out while calculation is simplified by adopting a relu activation function, gradient extinction and gradient explosion are avoided, and the expression of the relu activation function is
Figure BDA0003029371180000074
The activation function being artificialThe function run on the neuron of the neural network is responsible for mapping the input of the neuron to the output end, the activation function is used for adding nonlinear factors, the expression ability of the neural network to the model is improved, and the problem which cannot be solved by a linear model can be solvedTThe result of the nonlinear output after x + b, i.e. for the input vector x of the neuron from the neural network of the previous layer, the neuron using the linear rectification activation function will output max (0. w)Tx + b) to the output of the next layer of neurons, the definition of rele activation function is f (x) max (0, x), the whole feature extraction function is defined as f (·), fAAnd fBThe corresponding alignment is f (·).
In step S2, the entirety of the bilinear model is represented as: b ═ fA，fBP, C), wherein fA，fBIs a feature function for mapping the image I and the position l into dimensional features to obtain fA(l，I)∈Rc×M、fB(l，I)∈Rc×NWherein P is a pooling layer for sampling the feature layer, and C is a classification function; common classifiers are based on logistic regression, support vector machines and naive bayes, etc.
In step S3, the feature function f is setA(l，I)∈Rc×MAnd fB(l，I)∈Rc×NCarrying out bilinear fusion multiplication at the same position to obtain an M multiplied by N dimensional matrix b:
Figure BDA0003029371180000081
fA、fBtwo feature functions f, which are feature functions for mapping the image I and the position l into dimensional featuresA、fBThe extracted feature dimensions are C M and C N, respectively, the output b of the pooling function P will be an M N matrix, position I is the position in the image, x isijThe characteristic value of the ith row and the jth column in the image is represented.
In step S4, summing pooling is adopted, the matrix b at all positions is accumulated according to the following formula to obtain a matrix xi, and the matrix xi is subjected to multidimensional vector expansion to obtain a feature vector x:
Figure BDA0003029371180000082
x＝vec(ξ(I))。
in step S5, the moment normalization and the L2 normalization are combined to obtain the final feature, and fine-grained classification is performed on the final feature, where the normalized feature description is represented as z, and z is obtained according to the following formula:
Figure BDA0003029371180000083
in the bilinear model extraction image features, c is 1, M and N are the channel number of the features, and two features in the bilinear model are respectively expressed as vectors
Figure BDA0003029371180000084
(Vector)
Figure BDA0003029371180000085
Obtaining a matrix from the matrix accumulation
Figure BDA0003029371180000091
Denote A as [ a ]1，...，aL]∈RM×LAnd B is represented as B ═ B1，...，bL]∈RN×LAnd generating xi by using a matrix relation, wherein xi is expressed as: xi (I) ═ ABT∈RM×N。
Characteristic function fAAnd a characteristic function fBThe output tensor is used for obtaining M multiplied by N dimensions, the position L is M multiplied by N position points, each position point obtains P multiplied by P dimensions after bilinear transformation, the feature vector of PP multiplied by 1 is obtained after accumulation and pooling, back propagation is carried out through a chain derivation rule for finishing end-to-end training, and the bilinear feature after accumulation and pooling is expressed as x-ATB, the gradient of the loss function to the feature vector is expressed as
Figure BDA0003029371180000092
Wherein P is a pooling function;
at present, the implementation modes of bilinear CNNs are divided into two types, one is multi-Mode Bilinear Pooling (MBP), that is, two features extracted from a unified sample come from two different feature extraction functions, and the other is Homologous Bilinear Pooling (HBP) or Second-order Pooling (Second-order Pooling), that is, two features are extracted through the same feature extractor.
Chain-type derivation law: the input is x and the target value is y, wfIs a variable in the function f, f ═ f (x, w)f) The variable w included in each map is used to change the data so that the target becomes smallerfConducting derivation and calculation
Figure BDA0003029371180000093
Can pass through
Figure BDA0003029371180000094
In this example, arabidopsis thaliana was selected as the target of the experiment because it has a good genomic sequence and is an important plant type for plant phenotype research, and this example uses a published depth phenotype data set as the data set of this experiment, which is composed of 4 consecutive top view images of arabidopsis thaliana of different species, SF-2, Cvi, Landsberg (Ler-1) and Columbia (Col-0), respectively, and data samples are shown in fig. 3, different species of arabidopsis thaliana were planted in a substrate that strictly controls environmental conditions such as soil and light, a fixed camera was installed above the plant to take the top view images at a fixed rate, and a data sequence of each plant was constructed with potted images recorded at 12:00pm per day, and each plant acquisition sequence was designed to involve 22 consecutive top view images;
it is an example that 8/11 (1552 total) of each class of arabidopsis data set were randomly used as training sets, and the remaining 582 were used as test sets. On the basis of the partition, a cloud server platform is constructed by taking ubuntu 16.04LTS as a system, the platform carries a dual-Core Intel Core [email protected] x8 processor, a memory 256G and a 4 x 4T solid state hard disk, a video card is an NVIDIA Tesla p40 GPU, a computing cache is 96G, and a deep learning framework pytorch3.0 is adopted;
in order to verify the performance advantages of the bilinear CNN model in plant fine-grained identification, the model trains a plurality of comparison models, including a plurality of coarse-grained deep networks such as VGG16, ResNet18 and DenseNet161, parameters related to the training process include batch _ size, learning rate, activation function, optimization function and iteration times, the training parameters of the bilinear CNN model are set as shown in Table 1, the training parameters of the coarse-grained deep networks are set as shown in Table 2, the four network models are trained for multiple times, and the comparison test results obtained by calculation are shown in Table 3:
Batch-size learning rate Activating a function Optimization function Iterative function
Training parameter values 32 0.001 relu SGD 55
TABLE 1
Batch-size Learning rate Activating a function Optimization function Iterative function
Training parameter values 64 1 relu SGD 55
TABLE 2
Figure BDA0003029371180000101
Figure BDA0003029371180000111
TABLE 3
In table 3, VGG16 represents Visual Geometry Group Networks Visual Geometry Networks Group Networks-16 layers, ResNet18 represents Residual Networks-18 layers, densnet 161 represents Dense Networks density Networks-161 layers, and bilinar-CNN represents Bilinear CNN Networks, and as can be seen from table 3, the recognition accuracy on VGG16 and ResNet18 test sets is low, because coarse-grained Networks easily focus on only significant differences on feature maps and ignore fine differences among classes, the recognition effect is poor, and the average accuracy on densnet 161 test sets is 94.85%, and the network has a deeper network structure, can extract deep-level plant phenotypic features, but does not consider fine differences among different classes. The model adopts the bilinear convolution neural network to carry out fine-grained identification, the accuracy rate on a test set can reach 97.25%, and the model has higher identification accuracy rate compared with a common coarse-grained network.
In order to further compare the classification performance of each network model, the average accuracy and the loss function curve of each model identification are plotted, as shown in fig. 4 and 5 respectively. After 25 updating iterations, the comparison result shows that the loss function trends of all models are reduced and tend to be stable in convergence. The loss functions of VGG16, ResNet18 and DenseNet161 fluctuate greatly in the descending process, which shows that the traditional coarse-grained deep learning network is difficult to adapt to fine-grained plant data sets, the recognition robustness needs to be improved, the model obtains a good result in the aspect of training loss, the whole training loss is minimum, the convergence is fast, the recognition accuracy and the generalization robustness of the model are improved more effectively, and the plant phenotype fine-grained recognition work can be effectively performed.
The working principle of the invention is as follows: the invention discloses a plant fine-grained identification model based on transfer learning, which is used for constructing an identification model capable of effectively representing different types of tiny local differences, capturing the same type of shared information to mine the knowledge structure of a plant image, and providing the plant fine-grained identification model based on transfer learning, wherein coarse-grained feature extraction is carried out based on a VGG16 model, then a network of a bilinear model is pre-trained, a basic network is initialized, and the initialized basic network is transferred to a data set to be trained so as to extract fine-grained features and update model parameters.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A plant fine-grained identification method based on a bilinear convolutional neural network is characterized by comprising the following steps: the identification method comprises the following steps:
s1, extracting coarse-grained features by using a Relu activation function according to a VGG 16-based bilinear model feature extractor to obtain coarse-grained features X1;
s2, acquiring an integral representation of the bilinear model, pre-training a network of the bilinear model, initializing a basic network, transferring the initialized basic network to a data set for training, and extracting fine-grained features and updating model parameters;
s3, in step S2, bilinear fusion is carried out on the features extracted by the A, B two feature extraction functions in the bilinear model, and an M multiplied by N dimensional matrix b is obtained;
s4, carrying out summation pooling on the matrix b obtained in the step S3 to obtain a matrix xi, and carrying out multi-dimensional vector expansion on xi to obtain a feature vector x;
and S5, performing moment normalization and L2 normalization on the features according to the feature vector x obtained in the step S4 to obtain bilinear features, and performing a chain derivation method on the bilinear features to obtain a gradient expression of the loss function to the feature vector.
2. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 1, characterized in that: in step S1, globally using a VGG16 model with convolutional layer of 3 × 3 and pooling layer of 2 × 2, simplifying the calculation with the Relu activation function, defining the whole feature extraction function as f (·), and using a pre-network based on VGG16 bilinear model to obtain coarse feature X1:
X1＝Hvgg(x，y，{W1，b1，δrelu})
wherein HvggIs the convolution layer of VGG16, and (x, y) is the input characteristic parameter of the image, W1Weight parameters requiring iterative updating for the network model, b1For linear biasing, δ is the relu activation function.
3. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 2, characterized in that: the input characteristic parameters (x, y) represent the input of the first layer, x is the RGB three-channel value of the image, and y is the numeralization of the classification label.
4. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 2, characterized in that: the weight parameter W and the linear bias parameter b are updated according to a random gradient descent method, the gradient is obtained according to a chain derivation method, and the weightThe heavy parameter W and the new formula are as follows:
Figure FDA0003029371170000021
the linear bias parameter b is updated by the formula:
Figure FDA0003029371170000022
where η represents the learning efficiency, δ represents the gradient obtained according to the chain-derivative rule,
Figure FDA0003029371170000023
indicating that layer l-1 has used the ith neuron output of the activation function.
5. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 1, characterized in that: in step S2, the entirety of the bilinear model is represented as: b ═ fA，fBP, C), wherein fA，fBIs a feature function for mapping the image I and the position l into dimensional features to obtain fA(l，I)∈Rc×M、fB(l，I)∈Rc×NWherein P is a pooling layer for sampling the feature layer, and C is a classification function.
6. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 1, characterized in that: in step S3, the feature function f is setA(l，I)∈Rc×MAnd fB(l，I)∈Rc×NCarrying out bilinear fusion multiplication at the same position to obtain an M multiplied by N dimensional matrix b:
Figure FDA0003029371170000024
7. the plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 1, characterized in that: in step S4, summing pooling is adopted, the matrix b at all positions is accumulated according to the following formula to obtain a matrix xi, and the matrix xi is subjected to multidimensional vector expansion to obtain a feature vector x:
Figure FDA0003029371170000025
x＝vec(ξ(I))。
8. the plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 1, characterized in that: in step S5, the moment normalization and the L2 normalization are combined to obtain the final feature, and fine-grained classification is performed on the final feature, where the normalized feature description is represented as z, and z is obtained according to the following formula:
Figure FDA0003029371170000031
9. the plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 6, characterized in that: in the bilinear model extraction image features, c is 1, M and N are the channel number of the features, and two features in the bilinear model are respectively expressed as vectors
Figure FDA0003029371170000032
(Vector)
Figure FDA0003029371170000033
Obtaining a matrix from the matrix accumulation
Figure FDA0003029371170000034
Denote A as [ a ]1，...，aL]∈RM×LAnd B is represented as B ═ B1，...，bL]∈RN×LAnd generating xi by using a matrix relation, wherein xi is expressed as: xi (I) ═ ABT∈RM×N。
10. The plant fine-grained identification method based on the bilinear convolutional neural network as claimed in claim 9, wherein: characteristic function fAAnd a characteristic function fBThe output tensor is used for obtaining M multiplied by N dimensions, the position L is M multiplied by N position points, each position point obtains P multiplied by P dimensions after bilinear transformation, the feature vector of PP multiplied by 1 is obtained after accumulation and pooling, back propagation is carried out through a chain derivation rule for finishing end-to-end training, and the bilinear feature after accumulation and pooling is expressed as x-ATB, the gradient of the loss function to the feature vector is expressed as
Figure FDA0003029371170000035
Where P is the pooling function.
CN202110425490.XA 2021-04-20 2021-04-20 Plant fine-grained identification method based on bilinear convolutional neural network Pending CN113128593A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110425490.XA CN113128593A (en) 2021-04-20 2021-04-20 Plant fine-grained identification method based on bilinear convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110425490.XA CN113128593A (en) 2021-04-20 2021-04-20 Plant fine-grained identification method based on bilinear convolutional neural network

Publications (1)

Publication Number Publication Date
CN113128593A true CN113128593A (en) 2021-07-16

Family

ID=76778159

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110425490.XA Pending CN113128593A (en) 2021-04-20 2021-04-20 Plant fine-grained identification method based on bilinear convolutional neural network

Country Status (1)

Country Link
CN (1) CN113128593A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113850136A (en) * 2021-08-24 2021-12-28 中国船舶重工集团公司第七0九研究所 Yolov5 and BCNN-based vehicle orientation identification method and system
CN113920313A (en) * 2021-09-29 2022-01-11 北京百度网讯科技有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN114445660A (en) * 2022-01-21 2022-05-06 华东交通大学 Fine-grained image recognition method
CN114722928A (en) * 2022-03-29 2022-07-08 河海大学 Blue-green algae image identification method based on deep learning
CN115035389A (en) * 2022-08-10 2022-09-09 华东交通大学 Fine-grained image identification method and device based on reliability evaluation and iterative learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108875827A (en) * 2018-06-15 2018-11-23 广州深域信息科技有限公司 A kind of method and system of fine granularity image classification
CN109086792A (en) * 2018-06-26 2018-12-25 上海理工大学 Based on the fine granularity image classification method for detecting and identifying the network architecture
CN109685115A (en) * 2018-11-30 2019-04-26 西北大学 A kind of the fine granularity conceptual model and learning method of bilinearity Fusion Features
CN110263863A (en) * 2019-06-24 2019-09-20 南京农业大学 Fine granularity mushroom phenotype recognition methods based on transfer learning Yu bilinearity InceptionResNetV2

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108875827A (en) * 2018-06-15 2018-11-23 广州深域信息科技有限公司 A kind of method and system of fine granularity image classification
CN109086792A (en) * 2018-06-26 2018-12-25 上海理工大学 Based on the fine granularity image classification method for detecting and identifying the network architecture
CN109685115A (en) * 2018-11-30 2019-04-26 西北大学 A kind of the fine granularity conceptual model and learning method of bilinearity Fusion Features
CN110263863A (en) * 2019-06-24 2019-09-20 南京农业大学 Fine granularity mushroom phenotype recognition methods based on transfer learning Yu bilinearity InceptionResNetV2

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113850136A (en) * 2021-08-24 2021-12-28 中国船舶重工集团公司第七0九研究所 Yolov5 and BCNN-based vehicle orientation identification method and system
CN113920313A (en) * 2021-09-29 2022-01-11 北京百度网讯科技有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN114445660A (en) * 2022-01-21 2022-05-06 华东交通大学 Fine-grained image recognition method
CN114445660B (en) * 2022-01-21 2023-04-07 华东交通大学 Fine-grained image recognition method
CN114722928A (en) * 2022-03-29 2022-07-08 河海大学 Blue-green algae image identification method based on deep learning
CN114722928B (en) * 2022-03-29 2024-04-16 河海大学 Blue algae image recognition method based on deep learning
CN115035389A (en) * 2022-08-10 2022-09-09 华东交通大学 Fine-grained image identification method and device based on reliability evaluation and iterative learning
CN115035389B (en) * 2022-08-10 2022-10-25 华东交通大学 Fine-grained image identification method and device based on reliability evaluation and iterative learning

Similar Documents

Publication Publication Date Title
CN113128593A (en) Plant fine-grained identification method based on bilinear convolutional neural network
Taghavi Namin et al. Deep phenotyping: deep learning for temporal phenotype/genotype classification
Liu et al. Plant diseases and pests detection based on deep learning: a review
Kong et al. Multi-stream hybrid architecture based on cross-level fusion strategy for fine-grained crop species recognition in precision agriculture
Karthik et al. Attention embedded residual CNN for disease detection in tomato leaves
Yang et al. A survey of few-shot learning in smart agriculture: developments, applications, and challenges
CN114220124A (en) Near-infrared-visible light cross-modal double-flow pedestrian re-identification method and system
CN110728694B (en) Long-time visual target tracking method based on continuous learning
Reddy et al. Resnet-based modified red deer optimization with DLCNN classifier for plant disease identification and classification
CN108256486A (en) A kind of image-recognizing method and device based on non-negative low-rank and semi-supervised learning
CN114121163B (en) Culture medium prediction system, training and culture medium prediction method based on ensemble learning
CN109284741A (en) A kind of extensive Remote Sensing Image Retrieval method and system based on depth Hash network
CN112184734A (en) Long-time animal posture recognition system based on infrared images and wearable optical fibers
Yu et al. A recognition method of soybean leaf diseases based on an improved deep learning model
Gu et al. Image-based hot pepper disease and pest diagnosis using transfer learning and fine-tuning
CN115393631A (en) Hyperspectral image classification method based on Bayesian layer graph convolution neural network
Yang et al. Identification of navel orange lesions by nonlinear Deep Learning algorithm
CN116883364A (en) Apple leaf disease identification method based on CNN and Transformer
Ma et al. Cross-source image retrieval based on ensemble learning and knowledge distillation for remote sensing images
Rahman et al. A CNN Model-based ensemble approach for Fruit identification using seed
Li et al. Early drought plant stress detection with bi-directional long-term memory networks
CN114627496A (en) Robust pedestrian re-identification method based on depolarization batch normalization of Gaussian process
Roy et al. Disease Detection in Paddy Crop using Machine Learning Techniques
Costales et al. Development of a Prototype Application for Rice Disease Detection Using Convolutional Neural Networks
CN114821632A (en) Method for re-identifying blocked pedestrians

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination