CN116246176B - Crop disease detection method and device, electronic equipment and storage medium - Google Patents
Crop disease detection method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116246176B CN116246176B CN202310532458.0A CN202310532458A CN116246176B CN 116246176 B CN116246176 B CN 116246176B CN 202310532458 A CN202310532458 A CN 202310532458A CN 116246176 B CN116246176 B CN 116246176B
- Authority
- CN
- China
- Prior art keywords
- feature vector
- disease
- historical
- determining
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 201000010099 disease Diseases 0.000 title claims abstract description 226
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 title claims abstract description 226
- 238000001514 detection method Methods 0.000 title claims abstract description 39
- 239000013598 vector Substances 0.000 claims abstract description 284
- 230000004927 fusion Effects 0.000 claims abstract description 65
- 238000000605 extraction Methods 0.000 claims abstract description 20
- 238000000034 method Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 239000003086 colorant Substances 0.000 claims description 4
- 244000038559 crop plants Species 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 8
- 239000000284 extract Substances 0.000 description 3
- 210000000056 organ Anatomy 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011369 optimal treatment Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/188—Vegetation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Resources & Organizations (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Agronomy & Crop Science (AREA)
- Animal Husbandry (AREA)
- Marine Sciences & Fisheries (AREA)
- Mining & Mineral Resources (AREA)
- Computing Systems (AREA)
- Economics (AREA)
- Evolutionary Computation (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a disease detection method and device for crops, electronic equipment and a storage medium, and relates to the technical field of intelligent agriculture. Determining weight image data according to the color range and the shot image data by acquiring shot image data of crops; acquiring word description data of crops, and determining a keyword sequence according to part-of-speech categories and the word description data; feature extraction is carried out on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and a fusion feature vector is determined according to the image feature vector and the text feature vector; and matching the fusion feature vector with a disease database to obtain disease category and disease degree. According to the invention, the image shooting data and the text description data are fused, the disease characteristics of crops are accurately obtained, and the disease category and the disease degree are rapidly searched through the disease database. Solves the problem that in the prior art, the disease condition of crops is diagnosed by farm technical personnel, and misdiagnosis is easy to occur.
Description
Technical Field
The invention relates to the technical field of intelligent agriculture, in particular to a disease detection method and device for crops, electronic equipment and a storage medium.
Background
The development of traditional agriculture mainly depends on the long-term work of agricultural personnel in the field, the diagnosis of crops is also totally solved by the observation of farmers in the field, the treatment mode is simple and easy to implement, and the diagnosis is totally dependent on the field experience of the farmers. However, when the insect damage or disaster suffered by the crops is beyond the experience range of farmers, misdiagnosis can be caused, the optimal treatment period of the crops is delayed, and crop harvest is affected.
Accordingly, there is a need for improvement and development in the art.
Disclosure of Invention
The invention provides a disease detection method, a device, electronic equipment and a storage medium for crops, which are used for solving the problem that misdiagnosis is easy to occur by agricultural personnel in the prior art when disease conditions of crops are diagnosed, and realizing automatic disease detection.
The invention provides a disease detection method for crops, which comprises the following steps:
acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
and carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
According to the disease detection method for crops provided by the invention, the determination method of each characteristic vector label comprises the following steps:
acquiring historical disease data sets corresponding to the disease categories respectively, wherein each historical disease data set comprises a plurality of historical shooting image data and historical text description data corresponding to the historical shooting image data respectively;
determining the characteristic vector labels respectively corresponding to the father nodes according to the historical disease data sets;
acquiring preset image similarity and text similarity, and determining a plurality of sub-data sets corresponding to each historical disease data set according to each historical disease data set, the image similarity and the text similarity, wherein the similarity between the historical shot image data in each sub-data set is larger than the image similarity, and the similarity between the historical text description data corresponding to each historical shot image data is larger than the text similarity;
and determining the feature vector labels respectively corresponding to the child nodes according to the child data sets.
According to the disease detection method for crops provided by the invention, any one of the historical disease data set or the sub data set is used as a target data set, and the method for determining the characteristic vector label of the target node corresponding to the target data set comprises the following steps:
acquiring a historical image feature vector corresponding to each piece of historical shooting image data in the target data set and a historical text feature vector corresponding to each piece of historical text description data;
fusing each historical image feature vector and each historical text feature vector to obtain a historical fusion feature vector;
acquiring an image public feature vector corresponding to each historical image feature vector and a text public feature vector corresponding to each historical text feature vector, and fusing the image public feature vector and the text public feature vector to obtain a public feature vector;
and determining the feature vector label of the target node according to the historical fusion feature vector and the public feature vector, wherein when the fusion feature vector is matched with the feature vector label of the target node, the fusion feature vector is respectively matched with the public feature vector and the historical fusion feature vector to obtain two matching results, and the matching result of the fusion feature vector and the feature vector label of the target node is determined according to the two matching results.
According to the disease detection method for crops provided by the invention, the weight image data is determined according to the preset color range and the shooting image data, and the disease detection method comprises the following steps:
converting pixel points outside the color range in the shot image data into preset background colors to obtain simplified image data, wherein the color range is determined based on the color category of the crops;
determining a standard color of the crop in a disease-free state from the color range;
and determining weight values corresponding to all pixel points in the simplified image data according to the background color and the standard color to obtain the weight image data, wherein the weight value of the pixel point corresponding to the background color is the lowest, and the weight value of the pixel point closer to the standard color is lower except for the pixel point corresponding to the background color.
According to the disease detection method for crops provided by the invention, the part-of-speech class is noun and adjective, and the keyword sequence is determined according to the preset part-of-speech class and the text description data, and the method comprises the following steps:
traversing the text description data, and picking nouns and adjectives in the text description data as keywords through traversing;
and generating the keyword sequence according to the keywords and the extraction sequence of the keywords.
According to the disease detection method for crops provided by the invention, the fusion feature vector is determined according to the image feature vector and the text feature vector, and the method comprises the following steps:
inputting the character feature vector into a preset vector conversion model to obtain a predicted image feature vector corresponding to the character feature vector;
and fusing the image feature vector and the predicted image feature vector to obtain the fused feature vector.
The invention provides a disease detection method for crops, which further comprises the following steps:
and determining a disease solution corresponding to the crop according to the target disease category and the target disease degree.
The invention also provides a disease detection device for crops, which comprises:
the image processing module is used for acquiring shooting image data corresponding to crops and determining weight image data according to a preset color range and the shooting image data;
the word processing module is used for acquiring word description data corresponding to the crops and determining a keyword sequence according to a preset part-of-speech class and the word description data;
the feature extraction module is used for carrying out feature extraction on the weight image data and the keyword sequence respectively to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
the disease determining module is used for carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the crop disease detection method according to any one of the above.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method of disease detection of a crop as described in any of the above.
The invention has the beneficial effects that: according to the crop disease detection method, device, electronic equipment and storage medium provided by the invention, the disease characteristics of crops can be accurately obtained by fusing the image shooting data and the text description data. And the disease type and the disease degree of crops can be quickly found out based on the disease characteristics through the disease database in the form of father-son nodes, so that the artificial participation degree in the crop disease detection process is reduced. Solves the problem that in the prior art, the disease condition of crops is diagnosed by farm technical personnel, and misdiagnosis is easy to occur.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for detecting diseases of crops;
fig. 2 is a schematic structural view of a disease detection device for crops provided by the invention;
fig. 3 is a schematic structural diagram of an electronic device provided by the present invention.
Reference numerals:
210: an image processing module; 220: a word processing module; 230: a feature extraction module; 240: a disease determination module; 310: a processor; 320: a communication interface; 330: a memory; 340: a communication bus.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes a disease detection method, device, electronic equipment and storage medium for crops in accordance with fig. 1 to 3.
As shown in fig. 1, the method comprises the steps of:
s100, acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
s200, acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
s300, respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
s400, carrying out vector matching on the fusion feature vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, feature vector labels of the father-son nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the feature vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
Specifically, agricultural personnel can acquire the shot image data of crops with diseases through shooting modes such as mobile phones, unmanned aerial vehicles and the like, and the shot image data can reflect visual characteristics such as colors and shapes of the crops. However, in the captured image data, not only the crop but also other redundant backgrounds are provided, in order to reduce the interference of the redundant backgrounds, in this embodiment, a color range is preset for the true color of the crop, the pixel points with the color values outside the color range in the captured image data represent the area where the non-crop is located, the pixel points with the color values within the color range represent the area where the crop is located, and then the weight distribution is performed on each pixel point in the captured image data through the color range, so as to obtain the weight image data, thereby enabling the model to pay more attention to the area where the crop is located, and reducing the interference of the redundant backgrounds. In addition, other non-visual features that cannot be represented by images, such as rough touch, bad smell, and the like, may exist in crops suffering from diseases, so in order to improve accuracy of disease detection, it is also necessary to acquire text description data input by agricultural personnel for crops, and in order to reduce interference of invalid words in the text description data, such as nonsensical words, connective words, and the like, part-of-speech categories of keywords are preset in this embodiment, and the keyword is extracted from the text description data through the part-of-speech categories to obtain a keyword sequence. And then respectively carrying out feature extraction on the weight image data and the keyword sequence, and fusing to obtain a fused feature vector capable of accurately reflecting the disease features of crops. And then matching the fusion feature vector with a disease database constructed in advance. The disease database is a storage structure of parent-child nodes, and has a plurality of parent nodes, each parent node is associated with a plurality of child nodes, and in this embodiment, each parent node corresponds to a disease class, each child node corresponds to a disease degree of a disease class, and each parent node/child node includes a specific feature vector label. When the fusion feature vector is matched with the disease database, the fusion feature vector is matched with the feature vector labels of all the father nodes, and the disease category of the crops is determined according to the father node with the highest matching degree. And then matching with each child node of the father node with the highest matching degree, and determining the disease degree of the crops according to the child node with the highest matching degree. According to the embodiment, the disease characteristics of crops can be accurately obtained by fusing the image shooting data and the text description data. And the disease type and the disease degree of crops can be quickly found out based on the disease characteristics through the disease database in the form of father-son nodes, so that the artificial participation degree in the crop disease detection process is reduced. Solves the problems that in the prior art, the disease condition of crops is diagnosed by farm technical personnel and misdiagnosis is easy to occur.
In one implementation, the determining the weighted image data according to the preset color range and the captured image data includes:
converting pixel points outside the color range in the shot image data into preset background colors to obtain simplified image data, wherein the color range is determined based on the color category of the crops;
determining a standard color of the crop in a disease-free state from the color range;
and determining weight values corresponding to all pixel points in the simplified image data according to the background color and the standard color to obtain the weight image data, wherein the weight value of the pixel point corresponding to the background color is the lowest, and the weight value of the pixel point closer to the standard color is lower except for the pixel point corresponding to the background color.
In short, the pixels in the captured image data whose color values lie outside the color range represent areas where no crop is located, and in order to reduce the interference of such pixels, such pixels are converted into a uniform background color and given the lowest weight value, so that the model does not distribute excessive attention to such pixels. Pixels in the captured image data whose color values lie within the color range represent the region in which the crop is located, and therefore such pixels are given a higher weight value than pixels of the background color. However, the area where the crop is located is not every area where the crop is located, so that the standard color of the crop in the disease-free state is predetermined in this embodiment, the probability that the pixel point with the color value closer to the standard color is the disease area is lower, the probability that the pixel point with the color value farther away from the standard color is the disease area is higher, and the weight value of the pixel point with the color value lower than the similarity value of the standard color is higher, so that the model places more attention on the disease area, and the accuracy of disease detection is improved.
In one implementation, the part-of-speech category is a noun or an adjective, and the determining the keyword sequence according to the preset part-of-speech category and the text description data includes:
traversing the text description data, and picking nouns and adjectives in the text description data as keywords through traversing;
and generating the keyword sequence according to the keywords and the extraction sequence of the keywords.
Specifically, since nouns may reflect plant organs involved in the textual description data and adjectives may reflect the expressions of each plant organ involved in the textual description data, the present embodiment extracts nouns and adjectives as keywords from the textual description data, and generates a keyword sequence in the form of a time series according to the extraction order of each keyword. And simplifying the information of the text description data through the keyword sequence.
In one implementation, the determining a fusion feature vector according to the image feature vector and the text feature vector includes:
inputting the character feature vector into a preset vector conversion model to obtain a predicted image feature vector corresponding to the character feature vector;
and fusing the image feature vector and the predicted image feature vector to obtain the fused feature vector.
Specifically, a vector conversion model is pre-built, a text feature vector is input into the vector conversion model, the vector conversion model determines image data with similar meaning according to the input text feature vector, and a feature vector of the image data is extracted to obtain a predicted image feature vector corresponding to the text feature vector. And finally, fusing the image feature vector and the predicted image feature vector to obtain a fused feature vector, and comprehensively and accurately reflecting disease features of crops through the fused feature vector.
In one implementation, the working principle of the vector conversion model is: determining a plurality of image elements and filter parameters corresponding to the image elements respectively according to the character feature vectors, wherein the image elements correspond to different plant organs respectively, and the filter parameters correspond to different adjectives respectively; generating image data corresponding to the character feature vector according to each image element and the filter parameters of each image element; and extracting the characteristics of the image data to obtain the predicted image characteristic vector.
In one implementation, the filter parameter may be one or more of contrast, brightness, roughness.
In another implementation, the determining a fusion feature vector from the image feature vector and the text feature vector includes: and inputting the image feature vector and the text feature vector into a preset multi-mode fusion model to obtain the fusion feature vector.
In one implementation manner, the method for determining each feature vector label includes:
acquiring historical disease data sets corresponding to the disease categories respectively, wherein each historical disease data set comprises a plurality of historical shooting image data and historical text description data corresponding to the historical shooting image data respectively;
determining the characteristic vector labels respectively corresponding to the father nodes according to the historical disease data sets;
acquiring preset image similarity and text similarity, and determining a plurality of sub-data sets corresponding to each historical disease data set according to each historical disease data set, the image similarity and the text similarity, wherein the similarity between the historical shot image data in each sub-data set is larger than the image similarity, and the similarity between the historical text description data corresponding to each historical shot image data is larger than the text similarity;
and determining the feature vector labels respectively corresponding to the child nodes according to the child data sets.
In short, in order to construct a disease database, the embodiment acquires a large number of historical photographed image data and historical text description data of crops under different disease categories in advance, so as to obtain a historical disease data set of different disease categories. For each disease category, a parent node corresponding to the disease category is first constructed. And then, carrying out feature extraction on the historical disease data set of the disease category to obtain a feature vector label corresponding to the father node. Then dividing the historical shot image data in the historical disease data set of the disease category into a plurality of sets through the preset image similarity; judging whether each set needs to be segmented again through preset character similarity, wherein for each set, if the similarity between the historical character description data of each historical shooting image data in the set is larger than the character similarity, the segmentation is not needed again, otherwise, the segmentation is needed again until the similarity between the obtained historical shooting image data in the set is larger than the image similarity, and the similarity between the character description data of each historical shooting image data is larger than the character similarity, the segmentation is stopped, and a plurality of sub-data sets corresponding to the disease category are obtained. And respectively constructing a child node associated with the father node of the disease category according to each child data set. And aiming at each child node, extracting the characteristics of the child data set corresponding to the child node to obtain the characteristic vector label corresponding to the child node. And finally, determining the disease degree corresponding to each child node respectively in a manual labeling mode, and determining whether the disease category corresponding to each father node respectively is correct in a manual confirmation mode. According to the embodiment, the disease database is constructed in the form of the father-son nodes, so that the structural storage of historical disease data is realized, and the disease type and the disease degree of crops can be conveniently and quickly searched and found.
In one implementation manner, any one of the historical disease data set or the sub data set is taken as a target data set, and the method for determining the feature vector label of the target node corresponding to the target data set includes:
acquiring a historical image feature vector corresponding to each piece of historical shooting image data in the target data set and a historical text feature vector corresponding to each piece of historical text description data;
fusing each historical image feature vector and each historical text feature vector to obtain a historical fusion feature vector;
acquiring an image public feature vector corresponding to each historical image feature vector and a text public feature vector corresponding to each historical text feature vector, and fusing the image public feature vector and the text public feature vector to obtain a public feature vector;
and determining the feature vector label of the target node according to the historical fusion feature vector and the public feature vector, wherein when the fusion feature vector is matched with the feature vector label of the target node, the fusion feature vector is respectively matched with the public feature vector and the historical fusion feature vector to obtain two matching results, and the matching result of the fusion feature vector and the feature vector label of the target node is determined according to the two matching results.
In short, the feature vector label of the target node actually comprises two feature vectors, one is to respectively extract features of all the historical shot image data and all the historical text description data in the target data set, and then to perform multi-mode fusion on all the obtained historical image feature vectors and all the historical text feature vectors to obtain a historical fusion feature vector; and the other is to extract feature vectors shared by all the historical image feature vectors to obtain an image public feature vector, extract feature vectors shared by all the historical text feature vectors to obtain a text public feature vector, and finally perform multi-mode fusion on the image public feature vector and the text public feature vector to obtain a public feature vector. When the fusion feature vector of the crops is matched with the disease database, the fusion feature vector of the crops is mainly matched with the feature vector labels of the father and son nodes. Therefore, when the fusion feature vector is matched with the feature vector label of the target node, the fusion feature vector is required to be matched with the public feature vector and the historical fusion feature vector respectively to obtain two matching results, and the final matching result is determined based on the two matching results. It can be understood that the common feature vector reflects the unique and exclusive features of the disease type/disease degree corresponding to the target node from a local part, and the history fusion feature vector reflects the broad features of the disease type/disease degree corresponding to the target node from a global part, so that the feature vector label formed by combining the two feature vectors can further improve the matching precision of the disease database and obtain a more accurate disease detection result.
In one implementation, the method further comprises: acquiring weight values respectively corresponding to the historical fusion feature vector and the public feature vector, wherein the weight value of the historical fusion feature vector is lower than that of the public feature vector; weighting calculation is carried out according to the matching result and the weight value respectively corresponding to the historical fusion feature vector and the public feature vector, so as to obtain a weight matching result; and determining a matching result of the fusion feature vector and the target node according to the weight matching result.
In one implementation, the method further comprises:
and determining a disease solution corresponding to the crop according to the target disease category and the target disease degree.
Specifically, the embodiment can intelligently recommend a proper disease solution for farm technicians according to the detected target disease category and target disease degree of crops. For example, the name of the crop, the target disease category, and the target disease degree may be sent to a cloud server, through which the corresponding disease solution is searched and fed back; the disease solution databases containing different disease categories and disease degrees can also be constructed in advance, and corresponding disease solutions can be found out from the disease solution databases according to the target disease categories and the target disease degrees; the disease problems of crops can be generated according to the target disease types and the target disease degrees, and the disease problems are sent to a plurality of expert personnel on line, so that disease solutions fed back by the expert personnel are obtained.
The disease detection device for crops provided by the invention is described below, and the disease detection device for crops described below and the disease detection method for crops described above can be referred to correspondingly.
As shown in fig. 2, the apparatus includes:
the image processing module 210 is configured to obtain captured image data corresponding to a crop, and determine weighted image data according to a preset color range and the captured image data;
the word processing module 220 is configured to obtain word description data corresponding to the crop, and determine a keyword sequence according to a preset part-of-speech category and the word description data;
the feature extraction module 230 is configured to perform feature extraction on the weighted image data and the keyword sequence, respectively, to obtain an image feature vector and a text feature vector, and determine a fusion feature vector according to the image feature vector and the text feature vector;
the disease determining module 240 is configured to perform vector matching on the fused feature vector and a preset disease database to obtain a matching result, and determine a target disease category and a target disease degree corresponding to the crop according to the matching result, where the disease database is a storage structure of parent-child nodes, feature vector labels of the parent nodes respectively correspond to different disease categories, each parent node is associated with a plurality of child nodes, and feature vector labels of the child nodes respectively correspond to different disease degrees of the disease categories.
Fig. 3 illustrates a physical schematic diagram of an electronic device, as shown in fig. 3, where the electronic device may include: processor 310, communication interface (Communications Interface) 320, memory 330 and communication bus 340, wherein processor 310, communication interface 320, memory 330 accomplish communication with each other through communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a disease detection method for a crop, the method comprising:
acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
and carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
Further, the logic instructions in the memory 330 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the method of disease detection of a crop provided by the methods described above, the method comprising:
acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
and carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
In yet another aspect, the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program which when executed by a processor is implemented to perform the method of disease detection of crop plants provided by the methods described above, the method comprising:
acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
and carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (8)
1. A method for disease detection of crops, the method comprising:
acquiring shooting image data corresponding to crops, and determining weight image data according to a preset color range and the shooting image data;
acquiring text description data corresponding to the crops, and determining a keyword sequence according to a preset part-of-speech class and the text description data;
respectively carrying out feature extraction on the weight image data and the keyword sequence to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
vector matching is carried out on the fusion characteristic vector and a preset disease database to obtain a matching result, and a target disease category and a target disease degree corresponding to the crops are determined according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories;
the method for determining the feature vector labels comprises the following steps:
acquiring historical disease data sets corresponding to the disease categories respectively, wherein each historical disease data set comprises a plurality of historical shooting image data and historical text description data corresponding to the historical shooting image data respectively;
determining the characteristic vector labels respectively corresponding to the father nodes according to the historical disease data sets;
acquiring preset image similarity and text similarity, and determining a plurality of sub-data sets corresponding to each historical disease data set according to each historical disease data set, the image similarity and the text similarity, wherein the similarity between the historical shot image data in each sub-data set is larger than the image similarity, and the similarity between the historical text description data corresponding to each historical shot image data is larger than the text similarity;
determining the feature vector labels respectively corresponding to the child nodes according to the child data sets;
the method for determining the feature vector label of the target node corresponding to the target data set comprises the following steps of:
acquiring a historical image feature vector corresponding to each piece of historical shooting image data in the target data set and a historical text feature vector corresponding to each piece of historical text description data;
fusing each historical image feature vector and each historical text feature vector to obtain a historical fusion feature vector;
acquiring an image public feature vector corresponding to each historical image feature vector and a text public feature vector corresponding to each historical text feature vector, and fusing the image public feature vector and the text public feature vector to obtain a public feature vector;
and determining the feature vector label of the target node according to the historical fusion feature vector and the public feature vector, wherein when the fusion feature vector is matched with the feature vector label of the target node, the fusion feature vector is respectively matched with the public feature vector and the historical fusion feature vector to obtain two matching results, and the matching result of the fusion feature vector and the feature vector label of the target node is determined according to the two matching results.
2. The disease detection method of crops according to claim 1, wherein the determining weight image data from a preset color range and the photographed image data includes:
converting pixel points outside the color range in the shot image data into preset background colors to obtain simplified image data, wherein the color range is determined based on the color category of the crops;
determining a standard color of the crop in a disease-free state from the color range;
and determining weight values corresponding to all pixel points in the simplified image data according to the background color and the standard color to obtain the weight image data, wherein the weight value of the pixel point corresponding to the background color is the lowest, and the weight value of the pixel point closer to the standard color is lower except for the pixel point corresponding to the background color.
3. The disease detection method of crops according to claim 1, wherein the part-of-speech categories are nouns and adjectives, the determining a keyword sequence according to a preset part-of-speech category and the word description data comprises:
traversing the text description data, and picking nouns and adjectives in the text description data as keywords through traversing;
and generating the keyword sequence according to the keywords and the extraction sequence of the keywords.
4. The disease detection method of crop plants according to claim 1, wherein the determining a fusion feature vector from the image feature vector and the text feature vector comprises:
inputting the character feature vector into a preset vector conversion model to obtain a predicted image feature vector corresponding to the character feature vector;
and fusing the image feature vector and the predicted image feature vector to obtain the fused feature vector.
5. The disease detection method of crops according to claim 1, wherein after determining the target disease category and the target disease degree corresponding to the crops according to the matching result, the method further comprises:
and determining a disease solution corresponding to the crop according to the target disease category and the target disease degree.
6. A disease detection device for crops, the device comprising:
the image processing module is used for acquiring shooting image data corresponding to crops and determining weight image data according to a preset color range and the shooting image data;
the word processing module is used for acquiring word description data corresponding to the crops and determining a keyword sequence according to a preset part-of-speech class and the word description data;
the feature extraction module is used for carrying out feature extraction on the weight image data and the keyword sequence respectively to obtain an image feature vector and a text feature vector, and determining a fusion feature vector according to the image feature vector and the text feature vector;
the disease determining module is used for carrying out vector matching on the fusion characteristic vector and a preset disease database to obtain a matching result, and determining a target disease category and a target disease degree corresponding to the crops according to the matching result, wherein the disease database is a storage structure of father-son nodes, characteristic vector labels of the father nodes respectively correspond to different disease categories, each father node is associated with a plurality of son nodes, and the characteristic vector labels of the son nodes respectively correspond to different disease degrees of the disease categories;
the method for determining the feature vector labels comprises the following steps:
acquiring historical disease data sets corresponding to the disease categories respectively, wherein each historical disease data set comprises a plurality of historical shooting image data and historical text description data corresponding to the historical shooting image data respectively;
determining the characteristic vector labels respectively corresponding to the father nodes according to the historical disease data sets;
acquiring preset image similarity and text similarity, and determining a plurality of sub-data sets corresponding to each historical disease data set according to each historical disease data set, the image similarity and the text similarity, wherein the similarity between the historical shot image data in each sub-data set is larger than the image similarity, and the similarity between the historical text description data corresponding to each historical shot image data is larger than the text similarity;
determining the feature vector labels respectively corresponding to the child nodes according to the child data sets;
the method for determining the feature vector label of the target node corresponding to the target data set comprises the following steps of:
acquiring a historical image feature vector corresponding to each piece of historical shooting image data in the target data set and a historical text feature vector corresponding to each piece of historical text description data;
fusing each historical image feature vector and each historical text feature vector to obtain a historical fusion feature vector;
acquiring an image public feature vector corresponding to each historical image feature vector and a text public feature vector corresponding to each historical text feature vector, and fusing the image public feature vector and the text public feature vector to obtain a public feature vector;
and determining the feature vector label of the target node according to the historical fusion feature vector and the public feature vector, wherein when the fusion feature vector is matched with the feature vector label of the target node, the fusion feature vector is respectively matched with the public feature vector and the historical fusion feature vector to obtain two matching results, and the matching result of the fusion feature vector and the feature vector label of the target node is determined according to the two matching results.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, implements the method for disease detection of crops according to any one of claims 1 to 5.
8. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the disease detection method of a crop plant as claimed in any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310532458.0A CN116246176B (en) | 2023-05-12 | 2023-05-12 | Crop disease detection method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310532458.0A CN116246176B (en) | 2023-05-12 | 2023-05-12 | Crop disease detection method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116246176A CN116246176A (en) | 2023-06-09 |
| CN116246176B true CN116246176B (en) | 2023-09-19 |
Family
ID=86631705
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310532458.0A Active CN116246176B (en) | 2023-05-12 | 2023-05-12 | Crop disease detection method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116246176B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473530A (en) * | 2013-08-30 | 2013-12-25 | 天津理工大学 | Adaptive action recognition method based on multi-view and multi-mode characteristics |
| CN108256549A (en) * | 2017-12-13 | 2018-07-06 | 北京达佳互联信息技术有限公司 | Image classification method, device and terminal |
| CN111444960A (en) * | 2020-03-26 | 2020-07-24 | 上海交通大学 | Skin disease image classification system based on multi-mode data input |
| AU2020102885A4 (en) * | 2020-10-20 | 2020-12-17 | Xijing University | Disease recognition method of winter jujube based on deep convolutional neural network and disease image |
| AU2020103613A4 (en) * | 2020-11-23 | 2021-02-04 | Agricultural Information and Rural Economic Research Institute of Sichuan Academy of Agricultural Sciences | Cnn and transfer learning based disease intelligent identification method and system |
| CN113094464A (en) * | 2021-03-31 | 2021-07-09 | 哈尔滨理工大学 | Establishment and auxiliary identification method of extensible crop disease analysis library |
| CN114842240A (en) * | 2022-04-06 | 2022-08-02 | 盐城工学院 | Method for classifying images of leaves of MobileNet V2 crops by fusing ghost module and attention mechanism |
| WO2022178961A1 (en) * | 2021-02-23 | 2022-09-01 | 北京亿药科技有限公司 | Compound function prediction method based on neural network and connectivity map algorithm |
| CN115048537A (en) * | 2022-07-11 | 2022-09-13 | 河北农业大学 | Disease recognition system based on image-text multi-mode collaborative representation |
| CN115660291A (en) * | 2022-12-12 | 2023-01-31 | 广东省农业科学院植物保护研究所 | Plant disease occurrence and potential occurrence identification and evaluation method and system |
| CN115761529A (en) * | 2023-01-09 | 2023-03-07 | 阿里巴巴（中国）有限公司 | Image processing method and electronic device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220036006A1 (en) * | 2020-07-30 | 2022-02-03 | International Business Machines Corporation | Feature vector generation for probabalistic matching |
| US20220076828A1 (en) * | 2020-09-10 | 2022-03-10 | Babylon Partners Limited | Context Aware Machine Learning Models for Prediction |
-
2023
- 2023-05-12 CN CN202310532458.0A patent/CN116246176B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103473530A (en) * | 2013-08-30 | 2013-12-25 | 天津理工大学 | Adaptive action recognition method based on multi-view and multi-mode characteristics |
| CN108256549A (en) * | 2017-12-13 | 2018-07-06 | 北京达佳互联信息技术有限公司 | Image classification method, device and terminal |
| CN111444960A (en) * | 2020-03-26 | 2020-07-24 | 上海交通大学 | Skin disease image classification system based on multi-mode data input |
| AU2020102885A4 (en) * | 2020-10-20 | 2020-12-17 | Xijing University | Disease recognition method of winter jujube based on deep convolutional neural network and disease image |
| AU2020103613A4 (en) * | 2020-11-23 | 2021-02-04 | Agricultural Information and Rural Economic Research Institute of Sichuan Academy of Agricultural Sciences | Cnn and transfer learning based disease intelligent identification method and system |
| WO2022178961A1 (en) * | 2021-02-23 | 2022-09-01 | 北京亿药科技有限公司 | Compound function prediction method based on neural network and connectivity map algorithm |
| CN113094464A (en) * | 2021-03-31 | 2021-07-09 | 哈尔滨理工大学 | Establishment and auxiliary identification method of extensible crop disease analysis library |
| CN114842240A (en) * | 2022-04-06 | 2022-08-02 | 盐城工学院 | Method for classifying images of leaves of MobileNet V2 crops by fusing ghost module and attention mechanism |
| CN115048537A (en) * | 2022-07-11 | 2022-09-13 | 河北农业大学 | Disease recognition system based on image-text multi-mode collaborative representation |
| CN115660291A (en) * | 2022-12-12 | 2023-01-31 | 广东省农业科学院植物保护研究所 | Plant disease occurrence and potential occurrence identification and evaluation method and system |
| CN115761529A (en) * | 2023-01-09 | 2023-03-07 | 阿里巴巴（中国）有限公司 | Image processing method and electronic device |
Non-Patent Citations (3)
| Title |
|---|
| Crop disease identification and interpretation method based on multimodal deep learning;Ji Zhou 等;《Computers and Electronics in Agriculture》;全文 * |
| 基于特征向量的中文文献相似度评价;杜薇薇;;情报理论与实践(第04期);全文 * |
| 基于特征融合与局部判别映射的苹果叶部病害识别方法;李超;彭进业;张善文;;广东农业科学(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116246176A (en) | 2023-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11487995B2 (en) | Method and apparatus for determining image quality | |
| CN110866140B (en) | Image feature extraction model training method, image searching method and computer equipment | |
| CN109117857B (en) | Biological attribute identification method, device and equipment | |
| US20220237917A1 (en) | Video comparison method and apparatus, computer device, and storage medium | |
| CN112070069A (en) | Method and device for identifying remote sensing image | |
| CN113987236B (en) | Unsupervised training method and unsupervised training device for visual retrieval model based on graph convolution network | |
| CN112966758B (en) | Crop disease, insect and weed identification method, device and system and storage medium | |
| WO2023087063A1 (en) | Method and system for analysing medical images to generate a medical report | |
| CN111444313B (en) | Knowledge graph-based question and answer method, knowledge graph-based question and answer device, computer equipment and storage medium | |
| CN112015966A (en) | Image searching method and device, electronic equipment and storage medium | |
| CN112784054A (en) | Concept graph processing apparatus, concept graph processing method, and computer-readable medium | |
| CN111401193A (en) | Method and device for obtaining expression recognition model and expression recognition method and device | |
| KR102363961B1 (en) | Analysis method based on product image, apparatus and program | |
| CN109583358A (en) | A kind of Medical Surveillance fast accurate enforcement approach | |
| CN116246176B (en) | Crop disease detection method and device, electronic equipment and storage medium | |
| CN110839242A (en) | Abnormal number identification method and device | |
| CN111598000A (en) | Face recognition method, device, server and readable storage medium based on multiple tasks | |
| CN111191065A (en) | Homologous image determining method and device | |
| CN113743251B (en) | Target searching method and device based on weak supervision scene | |
| CN116052286A (en) | Living body detection model training method, living body detection method and related equipment | |
| CN111723872B (en) | Pedestrian attribute identification method and device, storage medium and electronic device | |
| Okawa et al. | Detection of abnormal fish by image recognition using fine-tuning | |
| CN112270334A (en) | Few-sample image classification method and system based on abnormal point exposure | |
| CN111291594A (en) | Image identification method and system | |
| Suciningtyas et al. | Activation Function Comparison On Potato Leaf Disease Classification Performance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |