CN115050014A - Small sample tomato disease identification system and method based on image text learning - Google Patents
Small sample tomato disease identification system and method based on image text learning Download PDFInfo
- Publication number
- CN115050014A CN115050014A CN202210673683.1A CN202210673683A CN115050014A CN 115050014 A CN115050014 A CN 115050014A CN 202210673683 A CN202210673683 A CN 202210673683A CN 115050014 A CN115050014 A CN 115050014A
- Authority
- CN
- China
- Prior art keywords
- tomato
- text
- disease
- information
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 235000007688 Lycopersicon esculentum Nutrition 0.000 title claims abstract description 130
- 201000010099 disease Diseases 0.000 title claims abstract description 128
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 128
- 238000000034 method Methods 0.000 title claims abstract description 30
- 240000003768 Solanum lycopersicum Species 0.000 title claims description 124
- 238000000605 extraction Methods 0.000 claims description 90
- 230000003902 lesion Effects 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000002457 bidirectional effect Effects 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 125000004122 cyclic group Chemical group 0.000 claims description 7
- 241000894007 species Species 0.000 claims description 5
- 241000227653 Lycopersicon Species 0.000 abstract 6
- 238000012549 training Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 5
- 235000013311 vegetables Nutrition 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000003745 diagnosis Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000000306 recurrent effect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 241000894006 Bacteria Species 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 241000233866 Fungi Species 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006806 disease prevention Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 244000052769 pathogen Species 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a small sample tomato disease identification system and method based on image text learning, which comprises the following steps: the system comprises an image classification module, a text classification module and a combined classification module; the image classification module is used for obtaining a first prediction probability of tomato disease types based on a tomato image; the text classification module is used for obtaining a second prediction probability of the tomato disease type based on the tomato text information; and the joint classification module is used for carrying out joint output on the first prediction probability and the second prediction probability to obtain the disease category. Through the technical scheme, the tomato diseases can be accurately identified without a large number of disease images.
Description
Technical Field
The invention belongs to the technical field of image model recognition, and particularly relates to a small sample tomato disease recognition system and method based on image text learning.
Background
During the production process of tomatoes, tomato diseases frequently occur due to the existence of pathogens such as bacteria, fungi, viruses and the like. These diseases seriously affect the quality and yield of vegetables, resulting in very serious economic losses each year. The traditional tomato disease diagnosis is mainly identified and evaluated by an agricultural expert or a technician according to experience, has the problems of time consumption, labor waste and low efficiency, and is difficult to adapt to the real-time and accuracy requirements of rapid disease prevention and control. The development of deep convolutional networks has prompted the rapid development of universal visual recognition on large-scale reference datasets, such as ImageNet, over the past few years. These CNN-based models and algorithms have proven useful in solving disease identification problems. Therefore, more and more researches are focused on vegetable disease detection and classification, and have been successful to some extent.
The tomato leaf disease identification model based on deep learning generally needs large-scale disease leaf images as training data sets, the data sets in different disease stages are time-consuming and labor-consuming to construct, and in a complex environment, the tomato disease images often contain various backgrounds such as other plants, soil, mulching films, water pipes and the like. Therefore, in the existing tomato disease data set, on one hand, the number of disease images is not enough to support the recognition model with large training parameter quantity, and on the other hand, the background of the disease images is single, and the data set of the type is directly used for training, so that a large error exists in the test under a real environment.
Disclosure of Invention
The invention aims to provide a small sample tomato disease identification system based on image text learning, and aims to solve the problems in the prior art.
In order to achieve the above object, the present invention provides a small sample tomato disease recognition system based on image text learning, comprising: the system comprises an image classification module, a text classification module and a combined classification module;
the image classification module is used for obtaining a first prediction probability of tomato disease types based on a tomato image;
the text classification module is used for obtaining a second prediction probability of the tomato disease types based on the tomato text information;
and the joint classification module is used for performing joint output on the first prediction probability and the second prediction probability to obtain the disease category.
Preferably, the image classification module comprises: a first feature extraction network and a first probability calculation unit;
the first feature extraction network is used for extracting features of the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels;
the first probability calculation unit calculates a first prediction probability based on the first extraction result.
Preferably, the text classification module comprises: a second feature extraction network and a second probability calculation unit;
the second feature extraction network is used for extracting features of the tomato text information to obtain a text information extraction result; wherein the tomato text information comprises: the position information of the growth of the disease spots, the front and back information of the leaves and the self information of the disease characteristics, wherein the text information extraction result comprises the following steps: tomato characteristic text and tomato characteristic text labels;
the second probability calculation unit is configured to calculate a second prediction probability based on the text information extraction result.
Preferably, the second feature extraction network comprises a context network and a current text network;
the context network performs feature extraction on context information through a bidirectional cyclic neural network to obtain a second extraction result; wherein the context information comprises position information of the lesion growth and front and back information of the leaf;
the current text network performs feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics.
Preferably, the joint classification module comprises: a combined output unit and a disease identification unit;
the combined output unit is used for performing combined output on the tomato image and the tomato text to obtain a third prediction probability;
and the disease identification unit identifies the tomato diseases based on the third prediction probability to obtain disease types.
On the other hand, in order to achieve the technical purpose, the invention provides a small sample tomato disease identification method based on image text learning, which comprises the following steps:
obtaining a first prediction probability of tomato disease types based on a tomato image;
obtaining a second prediction probability of the tomato disease type based on the tomato text information;
and jointly outputting the first prediction probability and the second prediction probability to obtain the disease category.
Preferably, the process of obtaining a first predicted probability of a tomato disease species comprises:
performing feature extraction on the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels; a first prediction probability is calculated based on the first extraction result.
Preferably, the process of obtaining the second predicted probability of the tomato disease species comprises:
carrying out feature extraction on the tomato text information to obtain a text information extraction result; calculating a second prediction probability based on the text information extraction result; wherein the tomato text information comprises: the position information of the growth of the lesion, the front and back information of the leaf and the information of the disease characteristics, wherein the text information extraction result comprises the following steps: tomato feature text and tomato feature text labels.
Preferably, the process of extracting the features of the tomato text information comprises the following steps:
extracting the characteristics of the context information through a bidirectional cyclic neural network to obtain a second extraction result; wherein the context information comprises position information of the lesion growth and front and back information of the leaf;
performing feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics;
wherein the text information extraction result comprises: the position information of the growth of the lesion, the front and back information of the blade and the self information of the disease characteristics.
Preferably, the process of obtaining disease categories comprises:
jointly outputting the tomato image and the tomato text to obtain a third prediction probability; and identifying the tomato diseases based on the third prediction probability to obtain the disease types.
The invention has the technical effects that: the method comprises the steps of obtaining a first prediction probability of tomato disease types based on a tomato image, obtaining a second prediction probability of the tomato disease types based on a tomato text, and classifying the disease types in a parallel mode of an image channel and a text channel; the disease description information is added in a text form on the basis of the image data set, so that a disease image-text pair is formed as a data set of a disease identification model. According to the invention, by describing disease symptoms in the text channel, the characteristic expression of the disease is further enhanced, and the influence of the disease image background on the identification process is weakened. According to the invention, through the combined classification module, the image-text pair can be input into the model, and the model can output the tomato disease category finally.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 is a schematic diagram of a small sample tomato disease identification network structure in an embodiment of the invention;
fig. 2 is a flow chart of a small sample tomato disease identification method in the embodiment of the invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Example one
As shown in fig. 1, the present embodiment provides a small sample tomato disease recognition system based on image text learning, including: the system comprises an image classification module, a text classification module and a combined classification module;
the image classification module is used for obtaining a first prediction probability of the tomato disease type based on the tomato image;
the text classification module is used for obtaining a second prediction probability of the tomato disease type based on the tomato text information;
and the joint classification module is used for carrying out joint output on the first prediction probability and the second prediction probability to obtain the disease category.
In some embodiments, the image classification module comprises: a first feature extraction network and a first probability calculation unit; the first feature extraction network is used for extracting features of the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels; a first probability calculation unit that calculates a first prediction probability based on the first extraction result.
In this embodiment, the image classification module specifically includes: the image feature extraction networks with different structures show different advantages of different recognition tasks, and the networks are mostly composed of convolution layers, pooling layers and full-connection layers. In the network structure, the network performance and the model size are comprehensively considered, and ResNet18 is used as a feature extraction network Img-Net of the image branch.
Giving an image I and an image label L as the input of Img-Net, and obtaining the output [ P ] of an image branch after feature extraction I1 ,P I2 ,P I3 ,P I4 ,P I5 ] img As shown in formula (1).
[P I1 ,P I2 ,P I3 ,P I4 ,P I5 ] img ＝S(W(I),L) (1)
Wherein W (-) represents the extraction result of the feature extraction network Img-Net, S (-) is a Softmax function, and P represents the prediction probability of each disease type.
In some embodiments, the text classification module comprises: a second feature extraction network and a second probability calculation unit; the second feature extraction network is used for extracting features of the tomato text information to obtain a text information extraction result; wherein the tomato text information comprises: the position information of the growth of the disease spots, the front and back information of the leaves and the information of the disease characteristics, and the text information extraction result comprises the following steps: tomato characteristic text and tomato characteristic text labels; and a second probability calculation unit for calculating a second prediction probability based on the text information extraction result.
In some embodiments, the second feature extraction network comprises a contextual network and a current text network; the context network is used for extracting the characteristics of the context information through the bidirectional cyclic neural network to obtain a second extraction result; the context information comprises position information of lesion growth and front and back information of the leaves; the current text network performs feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics.
In this embodiment, the text classification module specifically includes: different from the image feature extraction network, in order to better extract context information between texts, the Text feature extraction network is mostly composed of a recurrent neural network layer, but in a vegetable disease description Text, the feature extraction network not only needs to extract context information between texts (such as a lesion growth position, front and back information of leaves, and the like), but also is very important for information of disease features, and compared with the recurrent neural network layer, a convolutional layer has the advantage of extracting specific features, so that in the network structure, the feature extraction network Text-Net using TextRCNN as a Text branch is used.
Given preprocessed text C (T) i ) And the Text label L is used as the input of Text-Net, and the Text C (T) is firstly input i ) Obtaining context characteristic C of text after passing through bidirectional recurrent neural network (LSTM) l (T i ) And C r (T i ) The calculation formulas are shown in formulas (2) and (3).
C l (T i )＝f(W (l) C l (T i-1 )+W (sl) e(T i-1 )) (2)
C r (T i )＝f(W (r) C r (T i+1 )+W (sr) e(T i+1 )) (3)
Where f (-) is the Tanh activation function, T i For the current vectorized text, T i-1 And T i+1 Upper and lower, W, respectively, of vectorized text (l) And W (r) Respectively, the left and right cyclic neural network hidden layers and the conversion matrix of the next hidden layer, W (sl) And W (sr) A combination matrix of the left and right semantics of the current and next vectorized text, C l And C r Left and right quantized text, e (T), respectively, for the current vectorized text i-1 ) And e (T) i+1 ) A left word embedding vector and a right word embedding vector, respectively, representing the current vectorized text.
Splicing the obtained context features and the current text features to be used as the input of a specific feature extractor, wherein the specific feature extractor extracts the text features with the most obvious features in the vectorized text by using a maximum pooling layer (MaxPool), and finally obtains the output [ P ] of text branches T1 ,P T2 ,P T3 ,P T4 ,P T5 ] text As shown in formula (4).
[P T1 ,P T2 ,P T3 ,P T4 ,P T5 ] text ＝S(fc(M(C l (T i )+C(T i )+C r (T i ))),L) (4)
Wherein S (-) is a softmax function, fc (-) is a full link layer, M (-) is a maximum pooling layer (maxpool), and P represents the prediction probability of each disease type.
In some embodiments, the joint classification module comprises: a joint output unit and a disease identification unit; the combined output unit is used for performing combined output on the tomato image and the tomato text to obtain a third prediction probability; and the disease identification unit is used for identifying the tomato diseases based on the third prediction probability to obtain the disease types.
In this embodiment, the joint classification module specifically includes: the Img-Net and the Text-Net respectively extract the characteristics of different modes in the image-Text pair from different angles, and the characteristics of the two modes can be combined after the characteristics are fused, so that the difference between the large probability classification value and the small probability classification value is further increased, and the classification confidence of the combined classifier is increased. The joint output formula is shown in formula (5).
[P J1 ,P J2 ,P J3 ,P J4 ,P J5 ] joint ＝[P I1+T1 ,P I2+T2 ,P I3+T3 ,P I4+T4 ,P I5+T5 ] (5)
The loss function of the characteristic combination is shown as the formula (6).
Where T is the number of sample classes,
and
is a label value, and the two values are the same,
and
the predicted result probability values of Text-Net and Img-Net are respectively.
In this embodiment, the learning rates of the image classification module and the text classification module are both 0.0001, Adam is adopted as the optimizer, and the batch processing size is 16. The model training process comprises the following steps:
step one, data preprocessing. The original data set needs to be preprocessed in order to increase the diversity of the original data set, and because the original data set is shot in a real field and the situation in real application is considered, data enhancement processing is not carried out on disease images, the original images are only uniformly resized into 224 x 224 pixels, and the disease description text data are vectorized by adopting a word bag model, wherein the vectorization length is 20.
And step two, performing 50 rounds of training on the whole network.
And step three, network testing, namely testing the network on the test set.
The beneficial effects of this embodiment:
in the embodiment, disease types are classified in a mode that an image channel and a text channel are parallel; in the embodiment, disease description text information is added on the basis of disease image data to assist the diagnosis process, so that the disease identification model can obtain a good effect on the basis of a small number of image-text pairs. The appearance of different diseases on the blade is different, and according to the condition that the disease characteristics exist on the front and the back of the blade, the back image of the diseased blade is added to assist the disease diagnosis process.
Compared with the existing disease recognition model, the vegetable disease recognition model (ITC-Net) for image and text collaborative representation learning is constructed, the correlation and complementarity between the disease image characteristics and the text description are utilized, the effects of the image mode independent training and the text mode independent training are obtained in the complex environment vegetable disease small sample data set, and the accuracy, the precision, the sensitivity and the specificity of the test set are respectively 99.48%, 98.90%, 98.78% and 99.66%. The work of the method provides a feasible scheme for small sample disease identification based on image text collaborative representation learning in an actual agricultural scene.
Example two
As shown in fig. 2, the embodiment provides a small sample tomato disease identification method based on image text learning, which includes the following steps:
obtaining a first prediction probability of tomato disease types based on a tomato image;
obtaining a second prediction probability of the tomato disease type based on the tomato text information;
and carrying out joint output on the first prediction probability and the second prediction probability to obtain the disease category.
In some embodiments, the process of obtaining a first predicted probability of a tomato disease species comprises: performing feature extraction on the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels; a first prediction probability is calculated based on the first extraction result.
In some embodiments, the process of obtaining a second predicted probability of a tomato disease species comprises: carrying out feature extraction on the tomato text information to obtain a text information extraction result; calculating a second prediction probability based on the text information extraction result; wherein the tomato text information comprises: the position information of the growth of the lesion, the front and back information of the leaf and the information of the disease characteristics, wherein the text information extraction result comprises the following steps: tomato feature text and tomato feature text labels.
In some embodiments, the process of feature extraction of the tomato text information comprises: extracting the characteristics of the context information through a bidirectional cyclic neural network to obtain a second extraction result; wherein the context information comprises position information of the lesion growth and front and back information of the leaf; performing feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics; wherein the text information extraction result comprises: position information of lesion growth, front and back information of leaves and self information of disease characteristics.
In some embodiments, the process of obtaining disease categories comprises: the tomato image and the tomato text are jointly output to obtain a third prediction probability; and identifying the tomato diseases based on the third prediction probability to obtain the disease types.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A small sample tomato disease identification system based on image text learning is characterized by comprising: the system comprises an image classification module, a text classification module and a combined classification module;
the image classification module is used for obtaining a first prediction probability of tomato disease types based on a tomato image;
the text classification module is used for obtaining a second prediction probability of the tomato disease types based on the tomato text information;
and the joint classification module is used for performing joint output on the first prediction probability and the second prediction probability to obtain the disease category.
2. The small-sample tomato disease recognition system based on image text learning of claim 1,
the image classification module comprises: a first feature extraction network and a first probability calculation unit;
the first feature extraction network is used for extracting features of the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels;
the first probability calculation unit calculates a first prediction probability based on the first extraction result.
3. The small-sample tomato disease recognition system based on image text learning of claim 1,
the text classification module comprises: a second feature extraction network and a second probability calculation unit;
the second feature extraction network is used for extracting features of the tomato text information to obtain a text information extraction result; wherein the tomato text information comprises: the position information of the growth of the lesion, the front and back information of the leaf and the information of the disease characteristics, wherein the text information extraction result comprises the following steps: tomato characteristic text and tomato characteristic text labels;
the second probability calculation unit is used for calculating a second prediction probability based on the text information extraction result.
4. The small-sample tomato disease recognition system based on image text learning of claim 3,
the second feature extraction network comprises a context network and a current text network;
the context network performs feature extraction on context information through a bidirectional cyclic neural network to obtain a second extraction result; wherein the context information comprises position information of the lesion growth and front and back information of the leaf;
the current text network performs feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics.
5. The small-sample tomato disease recognition system based on image text learning of claim 1,
the joint classification module comprises: a combined output unit and a disease identification unit;
the combined output unit is used for performing combined output on the tomato image and the tomato text to obtain a third prediction probability;
and the disease identification unit identifies the tomato diseases based on the third prediction probability to obtain disease categories.
6. A small sample tomato disease identification method based on image text learning is characterized by comprising the following steps:
obtaining a first prediction probability of tomato disease types based on a tomato image;
obtaining a second prediction probability of the tomato disease type based on the tomato text information;
and jointly outputting the first prediction probability and the second prediction probability to obtain the disease category.
7. The small-sample tomato disease recognition method based on image text learning according to claim 6,
the process of obtaining a first predicted probability of a tomato disease species comprises:
performing feature extraction on the tomato image to obtain a first extraction result; wherein the first extraction result comprises: tomato characteristic images and tomato characteristic image labels; a first prediction probability is calculated based on the first extraction result.
8. The small-sample tomato disease recognition method based on image text learning according to claim 6,
the process of obtaining the second prediction probability of the tomato disease category comprises the following steps:
carrying out feature extraction on the tomato text information to obtain a text information extraction result; calculating a second prediction probability based on the text information extraction result; wherein the tomato text information comprises: the position information of the growth of the lesion, the front and back information of the leaf and the information of the disease characteristics, wherein the text information extraction result comprises the following steps: tomato feature text and tomato feature text labels.
9. The small-sample tomato disease recognition method based on image text learning according to claim 8,
the process of extracting the characteristics of the tomato text information comprises the following steps:
extracting the characteristics of the context information through a bidirectional cyclic neural network to obtain a second extraction result; wherein the context information comprises position information of the lesion growth and front and back information of the leaf;
performing feature extraction on the current text information through a convolutional neural network to obtain a third extraction result; wherein the current text information is the self information of the disease characteristics;
wherein the text information extraction result comprises: the position information of the growth of the lesion, the front and back information of the blade and the self information of the disease characteristics.
10. The small-sample tomato disease recognition method based on image text learning according to claim 6,
the process of obtaining disease categories comprises the following steps:
the tomato image and the tomato text are jointly output to obtain a third prediction probability; and identifying the tomato diseases based on the third prediction probability to obtain the disease types.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210673683.1A CN115050014A (en) | 2022-06-15 | 2022-06-15 | Small sample tomato disease identification system and method based on image text learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210673683.1A CN115050014A (en) | 2022-06-15 | 2022-06-15 | Small sample tomato disease identification system and method based on image text learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115050014A true CN115050014A (en) | 2022-09-13 |
Family
ID=83162289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210673683.1A Pending CN115050014A (en) | 2022-06-15 | 2022-06-15 | Small sample tomato disease identification system and method based on image text learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115050014A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117292174A (en) * | 2023-09-06 | 2023-12-26 | 中化现代农业有限公司 | Apple disease identification method, apple disease identification device, electronic equipment and storage medium |
| CN117407833A (en) * | 2023-10-30 | 2024-01-16 | 山东农业大学 | Automatic monitoring system and method for identifying pathogenic spores of crops based on neural network |
| CN117611924A (en) * | 2024-01-17 | 2024-02-27 | 贵州大学 | Plant leaf phenotype disease classification method based on graphic subspace joint learning |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682704A (en) * | 2017-01-20 | 2017-05-17 | 中国科学院合肥物质科学研究院 | Method of disease image identification based on hybrid convolutional neural network fused with context information |
| CN107563439A (en) * | 2017-08-31 | 2018-01-09 | 湖南麓川信息科技有限公司 | A kind of model for identifying cleaning food materials picture and identification food materials class method for distinguishing |
| CN107808375A (en) * | 2017-09-28 | 2018-03-16 | 中国科学院合肥物质科学研究院 | Merge the rice disease image detecting method of a variety of context deep learning models |
| CN110033000A (en) * | 2019-03-21 | 2019-07-19 | 华中科技大学 | A kind of text detection and recognition methods of bill images |
| CN110148120A (en) * | 2019-05-09 | 2019-08-20 | 四川省农业科学院农业信息与农村经济研究所 | A kind of disease intelligent identification Method and system based on CNN and transfer learning |
| CN110705460A (en) * | 2019-09-29 | 2020-01-17 | 北京百度网讯科技有限公司 | Image category identification method and device |
| CN112633422A (en) * | 2021-03-10 | 2021-04-09 | 北京易真学思教育科技有限公司 | Training method of text recognition model, text recognition method, device and equipment |
| CN112749675A (en) * | 2021-01-20 | 2021-05-04 | 云南农业大学 | Potato disease identification method based on convolutional neural network |
| CN113159095A (en) * | 2021-01-30 | 2021-07-23 | 华为技术有限公司 | Model training method, image retrieval method and device |
| CN113312987A (en) * | 2021-05-11 | 2021-08-27 | 东南大学 | Unmanned aerial vehicle aerial photographing based pavement crack image identification method |
| CN113344009A (en) * | 2021-06-03 | 2021-09-03 | 大连大学 | Light and small network self-adaptive tomato disease feature extraction method |
| CN113989536A (en) * | 2020-07-09 | 2022-01-28 | 天津科技大学 | Tomato disease identification method based on cuckoo search algorithm |
| CN114328944A (en) * | 2021-11-09 | 2022-04-12 | 中国科学院合肥物质科学研究院 | Construction method of agricultural information visualization interactive system based on knowledge graph |
| CN114550164A (en) * | 2021-12-28 | 2022-05-27 | 吉林农业大学 | Tomato leaf disease identification research method based on deep learning |
| CN114550334A (en) * | 2021-12-28 | 2022-05-27 | 江苏燕宁工程科技集团有限公司 | Bridge robot inspection teaching training system, method and storage medium |
| CN114693990A (en) * | 2022-03-11 | 2022-07-01 | 中国科学院合肥物质科学研究院 | Small sample crop disease identification method based on feature extraction and storage medium |
-
2022
- 2022-06-15 CN CN202210673683.1A patent/CN115050014A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106682704A (en) * | 2017-01-20 | 2017-05-17 | 中国科学院合肥物质科学研究院 | Method of disease image identification based on hybrid convolutional neural network fused with context information |
| CN107563439A (en) * | 2017-08-31 | 2018-01-09 | 湖南麓川信息科技有限公司 | A kind of model for identifying cleaning food materials picture and identification food materials class method for distinguishing |
| CN107808375A (en) * | 2017-09-28 | 2018-03-16 | 中国科学院合肥物质科学研究院 | Merge the rice disease image detecting method of a variety of context deep learning models |
| CN110033000A (en) * | 2019-03-21 | 2019-07-19 | 华中科技大学 | A kind of text detection and recognition methods of bill images |
| CN110148120A (en) * | 2019-05-09 | 2019-08-20 | 四川省农业科学院农业信息与农村经济研究所 | A kind of disease intelligent identification Method and system based on CNN and transfer learning |
| CN110705460A (en) * | 2019-09-29 | 2020-01-17 | 北京百度网讯科技有限公司 | Image category identification method and device |
| CN113989536A (en) * | 2020-07-09 | 2022-01-28 | 天津科技大学 | Tomato disease identification method based on cuckoo search algorithm |
| CN112749675A (en) * | 2021-01-20 | 2021-05-04 | 云南农业大学 | Potato disease identification method based on convolutional neural network |
| CN113159095A (en) * | 2021-01-30 | 2021-07-23 | 华为技术有限公司 | Model training method, image retrieval method and device |
| CN112633422A (en) * | 2021-03-10 | 2021-04-09 | 北京易真学思教育科技有限公司 | Training method of text recognition model, text recognition method, device and equipment |
| CN113312987A (en) * | 2021-05-11 | 2021-08-27 | 东南大学 | Unmanned aerial vehicle aerial photographing based pavement crack image identification method |
| CN113344009A (en) * | 2021-06-03 | 2021-09-03 | 大连大学 | Light and small network self-adaptive tomato disease feature extraction method |
| CN114328944A (en) * | 2021-11-09 | 2022-04-12 | 中国科学院合肥物质科学研究院 | Construction method of agricultural information visualization interactive system based on knowledge graph |
| CN114550164A (en) * | 2021-12-28 | 2022-05-27 | 吉林农业大学 | Tomato leaf disease identification research method based on deep learning |
| CN114550334A (en) * | 2021-12-28 | 2022-05-27 | 江苏燕宁工程科技集团有限公司 | Bridge robot inspection teaching training system, method and storage medium |
| CN114693990A (en) * | 2022-03-11 | 2022-07-01 | 中国科学院合肥物质科学研究院 | Small sample crop disease identification method based on feature extraction and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| SIWEI LAI 等: "Recurrent Convolutional Neural Networks for Text Classification", 《AAAI\'15: PROCEEDINGS OF THE TWENTY-NINTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE》 * |
| 王春山 等: "采用双模态联合表征学习方法识别作物病害" * |
| 王春山: "改进Multi-scale ResNet的蔬菜叶部病害识别" * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117292174A (en) * | 2023-09-06 | 2023-12-26 | 中化现代农业有限公司 | Apple disease identification method, apple disease identification device, electronic equipment and storage medium |
| CN117292174B (en) * | 2023-09-06 | 2024-04-19 | 中化现代农业有限公司 | Apple disease identification method, apple disease identification device, electronic equipment and storage medium |
| CN117407833A (en) * | 2023-10-30 | 2024-01-16 | 山东农业大学 | Automatic monitoring system and method for identifying pathogenic spores of crops based on neural network |
| CN117611924A (en) * | 2024-01-17 | 2024-02-27 | 贵州大学 | Plant leaf phenotype disease classification method based on graphic subspace joint learning |
| CN117611924B (en) * | 2024-01-17 | 2024-04-09 | 贵州大学 | Plant leaf phenotype disease classification method based on graphic subspace joint learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Krishnamoorthy et al. | Rice leaf diseases prediction using deep neural networks with transfer learning | |
| CN110532900B (en) | Facial expression recognition method based on U-Net and LS-CNN | |
| CN115050014A (en) | Small sample tomato disease identification system and method based on image text learning | |
| CN111680706B (en) | Dual-channel output contour detection method based on coding and decoding structure | |
| Ninomiya | High-throughput field crop phenotyping: current status and challenges | |
| CN113657294B (en) | Crop disease and insect pest detection method and system based on computer vision | |
| CN111105393B (en) | Grape disease and pest identification method and device based on deep learning | |
| CN112732921B (en) | False user comment detection method and system | |
| CN114816497B (en) | Link generation method based on BERT pre-training model | |
| CN115048537A (en) | Disease recognition system based on image-text multi-mode collaborative representation | |
| Wu | Identification of maize leaf diseases based on convolutional neural network | |
| CN114758132B (en) | Fruit tree disease and pest identification method and system based on convolutional neural network | |
| CN114693616A (en) | Rice disease detection method, equipment and medium based on improved target detection model and convolutional neural network | |
| CN114841961B (en) | Wheat scab detection method based on image enhancement and improved YOLOv5 | |
| CN114913425A (en) | Corn disease degree detection method based on belief learning and fine-grained feature extraction | |
| CN114399661A (en) | Instance awareness backbone network training method | |
| Wang et al. | Rapid detection of Yunnan Xiaomila based on lightweight YOLOv7 algorithm | |
| Jia et al. | Polar-Net: Green fruit instance segmentation in complex orchard environment | |
| Cho et al. | Fruit ripeness prediction based on DNN feature induction from sparse dataset | |
| CN117496325A (en) | Rice pest detection method and device based on improved YOLOv5s model | |
| Bhange et al. | A review of image processing for pomegranate disease detection | |
| Mohona et al. | Efficient computation of leaf disease classification techniques using deep learning | |
| Kalpana et al. | Diagnosis of major foliar diseases in black gram (vigna mungo l.) using convolution neural network (cnn) | |
| CN116778391A (en) | Multi-mode crop disease phenotype collaborative analysis model and device | |
| Kumar et al. | Deep Learning-Based Web Application for Real-Time Apple Leaf Disease Detection and Classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220913 |