CN112818855A - Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image - Google Patents
Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image Download PDFInfo
- Publication number
- CN112818855A CN112818855A CN202110138934.1A CN202110138934A CN112818855A CN 112818855 A CN112818855 A CN 112818855A CN 202110138934 A CN202110138934 A CN 202110138934A CN 112818855 A CN112818855 A CN 112818855A
- Authority
- CN
- China
- Prior art keywords
- crown
- peach
- image
- images
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 235000006040 Prunus persica var persica Nutrition 0.000 title claims abstract description 134
- 238000000034 method Methods 0.000 title claims abstract description 64
- 240000006413 Prunus persica var. persica Species 0.000 title 1
- 244000144730 Amygdalus persica Species 0.000 claims abstract description 133
- 239000002420 orchard Substances 0.000 claims abstract description 33
- 230000011218 segmentation Effects 0.000 claims abstract description 33
- 238000000605 extraction Methods 0.000 claims abstract description 23
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 238000012549 training Methods 0.000 claims description 26
- 241000196324 Embryophyta Species 0.000 claims description 14
- 238000012795 verification Methods 0.000 claims description 9
- 238000013136 deep learning model Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000002310 reflectometry Methods 0.000 claims description 5
- 230000003321 amplification Effects 0.000 claims description 3
- 238000010191 image analysis Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 235000013399 edible fruits Nutrition 0.000 abstract description 8
- 238000013135 deep learning Methods 0.000 abstract description 5
- 238000003384 imaging method Methods 0.000 abstract description 3
- 238000011161 development Methods 0.000 abstract description 2
- 230000018109 developmental process Effects 0.000 abstract description 2
- 238000003860 storage Methods 0.000 description 10
- 238000013461 design Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005259 measurement Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 5
- 230000003416 augmentation Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000011835 investigation Methods 0.000 description 4
- 230000003042 antagnostic effect Effects 0.000 description 3
- 238000009395 breeding Methods 0.000 description 3
- 230000001488 breeding effect Effects 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013210 evaluation model Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 230000029553 photosynthesis Effects 0.000 description 2
- 238000010672 photosynthesis Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000029058 respiratory gaseous exchange Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000003337 fertilizer Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000575 pesticide Substances 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Multimedia (AREA)
- Biophysics (AREA)
- Remote Sensing (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Astronomy & Astrophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, a device and a system for extracting peach crown phenotype information based on unmanned aerial vehicle images, wherein the method comprises the following steps: respectively acquiring images of the peach orchard under different growth conditions; then preprocessing the image in splicing software to obtain sub-images; inputting the subimages into a target network model to obtain a crown segmentation result; and obtaining a crown volume result according to the crown segmentation result. The method not only establishes an efficient and accurate peach crown extraction process based on the unmanned aerial vehicle visible light imaging and deep learning framework; the peach tree crown volume calculation method based on the peach tree crown extraction result, the peach tree elevation information and the relevant geometric model is low in cost and high in precision. By utilizing the method, the high-throughput, rapid and lossless extraction of information such as growth and development of different peach tree objects in natural environment can be realized, and further data support is provided for orchard accurate control and fruit yield-quality prediction.
Description
Technical Field
The invention relates to the technical field of computer image processing, in particular to a method, a device and a system for extracting canopy phenotype information of a peach tree based on unmanned aerial vehicle images.
Background
The peach tree originates from China, and is used as an important organ in the processes of photosynthesis, respiration and the like of the peach tree, so that the fruit bearing amount and the fruit quality of the peach tree are directly influenced. Therefore, in fruit tree breeding, yield measurement and orchard fine management, indexes such as crown appearance, volume and the like need to be accurately measured so as to evaluate a tree structure and improve yield estimation precision.
The traditional peach tree canopy information extraction method mainly comprises a ground manual investigation method, a remote sensing image visual interpretation method and the like. The ground manual investigation method comprises the steps of manually obtaining parameter factors such as the width of a peach crown, the diameter of the crown, the plant height and the like by using tools such as a tape measure and a height gauge manually, and calculating data such as the volume of the crown layer according to a relevant geometric formula. The manual measurement consumes a large amount of labor cost, wastes time and labor, has poor measurement accuracy stability, and is easily influenced by human subjective factors to generate large fluctuation. In addition, the manual measurement method can only be carried out on a single crown, and continuous and synchronous measurement on a large number of peach crowns cannot be realized.
The remote sensing image visual interpretation method obtains data such as area, shape and the like of a canopy part by manually describing a satellite image shot by satellite transit, and obtains information such as leaf area, plant height and the like by coupling related vegetation indexes and a surveying and mapping technology. Satellite remote sensing mainly depends on a transit satellite to shoot and image the ground, and continuous earth observation in a key growth period cannot be realized; the resolution of satellite images is generally meter-level, and the size of the crown of the peach tree is 3-5 meters, so that the accurate parameter extraction of a single plant is difficult to realize; the satellite image quality is easily interfered by the factors of bad weather such as fog, snow, cloud, rain and the like, so that the observation is difficult. Therefore, the traditional peach tree canopy information extraction method has many defects, such as time and labor waste, high labor cost, expensive instruments, complex use and the like.
Disclosure of Invention
The invention provides a method, a device and a system for extracting phenotype information of a peach crown based on an unmanned aerial vehicle image, which can not only be based on an unmanned aerial vehicle visible light imaging and deep learning framework, but also be used for extracting the peach crown; the peach tree crown volume calculation method based on the peach tree crown extraction result, the peach tree elevation information and the relevant geometric model is low in cost and high in precision, so that the peach tree crown information extraction precision is improved, the instrument use cost is reduced, and the working efficiency is improved.
In a first aspect, an embodiment of the present invention provides a method for extracting phenotype information of a canopy of a peach tree based on an unmanned aerial vehicle image, including:
respectively acquiring images of the peach orchard under different growth conditions;
inputting the peach garden image into image splicing software for preprocessing to obtain sub-images;
inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model;
and obtaining a crown volume result according to the crown segmentation result.
In one possible design, the respectively acquiring images of the peach orchard under different growth conditions comprises:
adopting an unmanned aerial vehicle to respectively obtain peach garden images under different growth parameter conditions, wherein the growth parameters comprise at least one of the following: height, growth period, light intensity, and ground reflectivity.
In one possible design, the image of the peach garden is input into image stitching software for preprocessing, and the method comprises the following steps:
and aligning the peach garden images, generating dense point clouds, generating a DEM (digital elevation model) and obtaining the peach garden ortho-image.
In one possible design, before obtaining the sub-image, the method further includes:
and marking the peach orchard orthographic images to respectively obtain peach crown area images under different growth parameters.
In one possible design, deriving the sub-image includes:
and after the peach orchard orthographic image is cut, obtaining subimages and a training set corresponding to the subimages through rotation and mirror image analysis and amplification processing.
In one possible design, before inputting the sub-image into the target network model, the method further includes:
constructing an initial network model, wherein the initial network model comprises: the tree crown classification method comprises a generator, a discriminator and introduction condition information related to the generator and the discriminator, wherein the introduction condition information is used for optimizing the generator and the discriminator, and the generator is used for classifying sub-images to obtain tree crown classification types; the discriminator is used for obtaining the crown segmentation result according to the peach tree crown region image;
and training the initial network model by adopting the training set corresponding to the subimages to obtain the target network model.
In one possible design, obtaining a crown volume result according to the crown segmentation result includes:
obtaining the plant height of the peach tree according to the crown area and the edge point in the corresponding output image of the crown segmentation result and the parameter information corresponding to the edge point and the center of the crown area;
and obtaining a crown volume result based on an ellipsoid volume calculation method according to the plant height of the peach tree.
In one possible design, after obtaining the crown volume result, the method further includes:
and respectively carrying out precision verification on the output image corresponding to the crown segmentation result and the crown volume result.
In a second aspect, an apparatus for extracting information of a crown of a peach tree provided in an embodiment of the present invention includes:
the acquisition module is used for respectively acquiring images of the peach orchard under different growth conditions;
the obtaining module is used for inputting the peach garden image into image splicing software for preprocessing to obtain sub-images;
the input module is used for inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model;
and the obtaining module is used for obtaining a crown volume result according to the crown segmentation result.
In a third aspect, the system for extracting information from a crown of a peach tree provided by the embodiment of the present invention includes a memory and a processor, where the memory stores executable instructions of the processor; wherein the processor is configured to perform the method for peach tree crown information extraction of any one of the first aspect via execution of the executable instructions.
The invention provides a method, a device and a system for extracting peach crown phenotype information based on unmanned aerial vehicle images, wherein the method comprises the following steps: respectively acquiring images of the peach orchard under different growth conditions; inputting the peach garden image into image splicing software for preprocessing to obtain sub-images; inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model; and obtaining a crown volume result according to the crown segmentation result. The invention provides an efficient and accurate peach crown extraction process based on an unmanned aerial vehicle visible light imaging and deep learning framework; the method comprises the steps of establishing a condition-generation countermeasure network-based peach crown extraction method, and acquiring peach crown layer orthoimages under the conditions of different heights, different growth periods, different illumination intensities and different ground reflectivities through an unmanned aerial vehicle; and then, training by using a condition-generation antagonistic network pair to obtain an accurate tree crown segmentation result. The peach tree crown volume calculation method is low in cost and high in precision, the peach tree crown volume is estimated by the ellipsoid volume method based on the peach tree crown extraction result, the estimation result is higher in correlation compared with the ground laser radar estimation result, the use cost of an instrument is reduced, and the working efficiency is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for extracting information of a crown of a peach tree according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a network model structure for extracting information of a canopy of a peach tree according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a verification result of the volume estimation precision of the crown of the peach tree according to the embodiment of the present invention;
fig. 4 is a schematic diagram of a system for extracting information of a crown of a peach tree according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The peach trees originate from China and are fruit tree varieties widely planted worldwide, and the planting area of the peach trees in 2018 in the whole world reaches 160 ten thousand hectares. The crown of the peach tree is used as an important organ in the processes of photosynthesis, respiration and the like of the peach tree, and directly influences the fruit bearing amount and the fruit quality of the peach tree. Therefore, in fruit tree breeding, yield measurement and orchard fine management, indexes such as crown appearance, volume and the like need to be accurately measured so as to evaluate a tree structure and improve yield estimation precision. The traditional peach tree canopy information extraction method mainly comprises a ground manual investigation method, a remote sensing image visual interpretation method and the like. The ground manual investigation method comprises the steps of manually obtaining parameter factors such as the width of a peach crown, the diameter of the crown, the plant height and the like by using tools such as a tape measure and a height gauge manually, and calculating data such as the volume of the crown layer according to a relevant geometric formula. The remote sensing image visual interpretation method obtains data such as area, shape and the like of a canopy part by manually describing a satellite image shot by satellite transit, and obtains information such as leaf area, plant height and the like by coupling related vegetation indexes and a surveying and mapping technology. However, the traditional method for extracting the information of the canopy of the peach tree has many defects, such as time and labor waste, high labor cost, expensive instrument, complex use and the like. In recent years, with the continuous development of unmanned aerial vehicle remote sensing, image processing and deep learning technologies, how to quickly and accurately acquire peach crown information in an orchard by using high-resolution low-altitude remote sensing visible light data becomes a key point in the current orchard phenotype and peach breeding research.
The invention provides a fast and efficient peach tree canopy information extraction method by combining unmanned aerial vehicle remote sensing and deep learning technologies. The following describes the technical solutions of the present invention and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
The invention adopts the unmanned aerial vehicle to collect peach orchard images, the unmanned aerial vehicle platform in the embodiment selects DJI-P4 four-rotor unmanned aerial vehicle, the bottom of the platform is provided with 1 integrated five-channel high-definition digital camera, the top of the platform is integrated with an illuminometer which can monitor and correct the illumination in real time, and the detailed parameter settings of the unmanned aerial vehicle and the camera are shown in table 1.
TABLE 1
The unmanned aerial vehicle flies at the position 15-20 meters above the orchard at the speed of 2.5 m/s to ensure that the longitudinal and lateral overlapping rates of the shot images are more than 75 percent. According to the flight scheme, the data acquisition for peach gardens with the floor area of about 7 hectares is about 10 minutes. And simultaneously acquiring the volume data of the crown of the single peach tree as a verification set by using an LMS291-S05 vehicle-mounted laser radar on the ground synchronously while acquiring the data of the unmanned aerial vehicle.
Fig. 1 is a flowchart of a method for extracting information of a crown of a peach tree according to an embodiment of the present invention, and as shown in fig. 1, the method in this embodiment may include:
and S101, respectively acquiring images of the peach orchard under different growth conditions.
Specifically, adopt unmanned aerial vehicle to obtain the peach garden image under the different growth parameter condition respectively, wherein the growth parameter includes following at least one: height, growth period, light intensity, and ground reflectivity.
In the embodiment, the peach orchard flies at the speed of 2.5 m/s at the position 15-20 m above the ground, and peach orchard images (namely peach orchard low-altitude remote sensing visible light images) at different heights, different growth periods, different illumination intensities and different ground reflectivity can be obtained.
And S102, inputting the peach garden image into image splicing software for preprocessing to obtain sub-images.
Specifically, in this embodiment, image stitching software is used to perform geometric correction, radiation correction and other processing on the obtained images, and then stitching is performed to form a high-definition ortho image covering the whole peach garden area; subsequently, a series of image blocks are cut out from the ortho-image, and the image blocks are subjected to augmentation processing. And aligning the peach garden images, generating dense point clouds and generating a DEM (digital elevation model) to obtain the peach garden ortho-image. And then marking the peach orchard orthographic image to respectively obtain peach crown area images under different growth parameters. And then cutting the peach orchard orthographic image, and obtaining a subimage and a training set corresponding to the subimage by rotation and mirror image analysis and amplification processing.
2.1 data splicing
And importing all peach garden images obtained by the unmanned aerial vehicle into Agisoft Photoscan Professional software, aligning all images by the software according to the correlation characteristics (such as image shooting according to preset shooting time or presentation of all structures of the shot peach tree in the images) among the images and coordinate information, generating dense point clouds, generating a DEM (digital elevation model) and the like, and finally obtaining high-definition peach garden orthographic images covering all orchards.
2.2 Artificial marking
And (3) marking the generated peach garden orthographic image by using Labelme software, and manually marking individual peach tree crown area images of different varieties, different growth periods and different environmental conditions on the image by adopting a manual visual method.
2.3 data cutting and augmentation
In order to reduce the complexity of the training time of the initial network model and improve the working efficiency of the display card, the peach orchard orthographic image is cut into a series of sub-image blocks with the size of 256 multiplied by 256 pixels. Because the deep learning model has higher demand on the data volume, the cut sub-image blocks are subjected to augmentation operations such as rotation, mirroring, principal component analysis and the like to improve the training data volume and enrich the training set information, so that the sub-images are obtained. Finally, the sub-images are divided into several sets, named training set, validation set and test set, respectively, see table 2.
TABLE 2 data set details
| Training set | Verification set | Test set | |
| Number of images | 5868 | 3912 | 1956 |
S103, inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model.
Specifically, before the sub-image is input into the target network model, the method further includes:
constructing an initial network model, wherein the initial network model comprises: the tree crown classification method comprises a generator, a discriminator and introduction condition information related to the generator and the discriminator, wherein the introduction condition information is used for optimizing the generator and the discriminator, and the generator is used for classifying sub-images to obtain tree crown classification types; the discriminator is used for obtaining the crown segmentation result according to the peach tree crown region image; and training the initial network model by adopting the training set corresponding to the subimages to obtain the target network model.
The present invention classifies pixels in an image using a semi-supervised type of condition-generating antagonistic network (e.g., an initial network model). The training strategy for generating the countermeasure network (GAN) is realized by mutual game between the generator and the discriminator. Since the traditional generation countermeasure network cannot control the mode of generating data, the model training process is guided by introducing additional introduction condition information, namely, the condition-generation countermeasure network is used for obtaining the target network model. The objective function in a conditionally-generative antagonistic network is defined as follows:
where x represents the samples in the training data set pdata (x), D (discriminator) and G (generator) represent two neural networks contained in the model, z represents the random noise subject to the prior distribution pz (z), and y represents the conditional image. Referring to fig. 2 representing a model structure of a conditional-generative countermeasure network, fig. 2 is a schematic diagram of a network model structure for extracting information of a canopy of a peach tree according to an embodiment of the present invention. As shown in fig. 2, an input image (e.g., a sub-image) is sequentially passed through a generator and a discriminator to obtain a crown segmentation result, wherein the generator includes a plurality of convolutional layers (e.g., Conv-1, Conv-2 … … Conv-8, DeConv-1, DeConv-2 … … DeConv-8) and a Fully Connected layer FC to obtain a crown classification type, and the discriminator includes a connection active layer (Connected ReLU) and a Fully Connected activated layer (Fully Connected signed) to obtain the crown segmentation result. Here, we train the model using Adam optimization algorithm (Adam optimization algorithm). Condition-Generation confrontational network model training for 30 rounds, using batch size (batch size) of 15, momentum parameters set to 0.5 and 0.999, respectively, learning rate of 0.00007, and stride of 2 × 2.
And S104, obtaining a crown volume result according to the crown segmentation result.
Specifically, the plant height of the peach tree is obtained according to a crown area and an edge point in an output image corresponding to the crown segmentation result, and parameter information corresponding to the edge point and the center of the crown area; and obtaining a crown volume result based on an ellipsoid volume calculation method according to the plant height of the peach tree.
The accurate measurement of the volume of the tree crown is crucial to the growth prediction and the determination of the application amount of pesticides and fertilizers. The method adopts an Ellipsoid Volume Method (EVM) to estimate the volume of the crown of the peach tree. The parameters Ea and Eb are obtained by the distance between the farthest marginal point of the crown region extracted in the last step in the due north and the due east and the center of the crown region. The parameter Ec (i.e. nDSM) is expressed by the plant height of the corresponding peach tree, and the calculation formula of the plant height of the peach tree is as follows:
nDSM＝DSM-DEM (2)
the nDSM represents the plant height, the DSM represents a digital surface model, the DEM is a digital elevation model, and the DSM and the DEM can obtain corresponding information in unmanned aerial vehicle image splicing. The volume calculation formula of the peach tree crown based on the ellipsoid volume method is as follows:
in an alternative embodiment, after obtaining the crown volume result, the method further comprises: and respectively carrying out precision verification on the output image corresponding to the crown segmentation result and the crown volume result.
4.1 peach crown region extraction precision verification
Using QsegThe Precision of the crown extraction model is evaluated by five index evaluation models of Sr, Precision, Recall and F-measure, and the calculation formula is as follows:
c (-) represents the pixel contained in the image of the peach tree crown region extracted by the computer; b (-) represents pixels contained in the manually marked peach tree crown area image; a and b represent the row and column coordinates, respectively, of the pixels contained in the pictures with row and column sizes i and j, respectively; TP, FP and FN represent true positive, false positive and false negative, respectively. The extraction accuracy of the condition-generating countermeasure network of the present invention was compared with FCN, U-Net, SegNet, ExG, and CIVE, and the results are shown in Table 3.
TABLE 3 comparison of crown extraction accuracy for different models
| Model (model) | Qseg | Sr | Precision | Recall | IoU |
| CGAN | 0.913 | 0.898 | 0.903 | 0.907 | 0.911 |
| FCN | 0.887 | 0.862 | 0.882 | 0.876 | 0.899 |
| U-Net | 0.784 | 0.752 | 0.765 | 0.754 | 0.769 |
| SegNet | 0.712 | 0.708 | 0.722 | 0.704 | 0.716 |
| ExG | 0.717 | 0.703 | 0.657 | 0.632 | 0.618 |
| CIVE | 0.554 | 0.534 | 0.537 | 0.575 | 0.549 |
As can be seen from Table 3, the precision of the conditional-generated countermeasure network as the peach crown extraction model is significantly improved compared with that of the comparative method, and the improvement rate is about 10% -30%, which indicates that the method has a great application value in orchard observation.
4.2 peach crown volume estimation precision verification
By calculating the correlation coefficient (R)2) And Root Mean Square Error (RMSE) which is calculated according to a formula (9) and is used for carrying out precision comparison on the tree-shaped volume calculated by the EVM and the execution result of the ground laser radar scanner.
VdVolume reference value V of peach tree crown obtained by scanning laser radarEThe volume of the crown of the peach tree estimated by the ellipsoid method, and N is the number of the peach trees.
Fig. 3 is a schematic diagram of a result of verifying the volume estimation accuracy of the crown of the peach tree according to the embodiment of the present invention. Wherein VLRepresenting the lidar estimation result, VpThe volume value of the peach tree crown calculated based on the EVM method is shown. As shown in fig. 3, the method for estimating the volume of the crown according to the present invention achieves better results.
The embodiment of the present invention further provides a device (not shown) for extracting information of a crown of a peach tree, which specifically includes:
the acquisition module is used for respectively acquiring images of the peach orchard under different growth conditions;
the obtaining module is used for inputting the peach garden image into image splicing software for preprocessing to obtain sub-images;
the input module is used for inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model;
and the obtaining module is used for obtaining a crown volume result according to the crown segmentation result.
Fig. 4 is a schematic diagram of a system for extracting information of a peach tree crown according to an embodiment of the present invention, as shown in fig. 4, acquiring peach orchard images under different production conditions based on an unmanned aerial vehicle, performing orthorectification, inputting the peach orchard images into image stitching software for preprocessing to generate a DSM (digital surface model) and a DEM (digital elevation model) to obtain peach orchard orthoimages, further cutting the peach orchard orthoimages to obtain sub-image blocks, performing augmentation operations such as rotation, mirroring, principal component analysis and the like on the cut sub-image blocks to obtain sub-images, inputting the sub-images into a target network model (for example, a trained condition-generation countermeasure network) to obtain a crown segmentation result, and further obtaining a crown volume result according to the crown segmentation result. And respectively carrying out precision verification on the output image corresponding to the crown segmentation result and the crown volume result. Specific aspect uses QsegAnd the Precision of the crown extraction model is evaluated by five index evaluation models of Sr, Precision, Recall and F-measure, and the condition-generation countermeasure network is compared with the extraction Precision of FCN, U-Net, SegNet, ExG and CIVE, so that the robustness is better. On the other hand, by calculating the correlation coefficient (R)2) And Root Mean Square Error (RMSE), and the tree volume calculated by the EVM is compared with the accuracy of the execution result of the ground laser radar scanner, so that the result obtained by the tree crown volume estimation method provided by the invention is better.
In addition, embodiments of the present application further provide a computer-readable storage medium, in which computer-executable instructions are stored, and when at least one processor of the user equipment executes the computer-executable instructions, the user equipment performs the above-mentioned various possible methods.
Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in user equipment. Of course, the processor and the storage medium may reside as discrete components in a communication device.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for extracting phenotype information of a peach tree crown based on unmanned aerial vehicle images is characterized by comprising the following steps:
respectively acquiring images of the peach orchard under different growth conditions;
inputting the peach garden image into image splicing software for preprocessing to obtain sub-images;
inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model;
and obtaining a crown volume result according to the crown segmentation result.
2. The method as claimed in claim 1, wherein the separately acquiring images of the peach orchard under different growing conditions comprises:
adopting an unmanned aerial vehicle to respectively obtain peach garden images under different growth parameter conditions, wherein the growth parameters comprise at least one of the following: height, growth period, light intensity, and ground reflectivity.
3. The method as claimed in claim 2, wherein the inputting of the image of the peach garden into image stitching software for preprocessing comprises:
and aligning the peach garden images, generating dense point clouds, generating a DEM (digital elevation model) and obtaining the peach garden ortho-image.
4. The method of claim 3, further comprising, prior to obtaining the sub-image:
and marking the peach orchard orthographic images to respectively obtain peach crown area images under different growth parameters.
5. The method of claim 4, wherein obtaining the sub-image comprises:
and after the peach orchard orthographic image is cut, obtaining subimages and a training set corresponding to the subimages through rotation and mirror image analysis and amplification processing.
6. The method of claim 5, further comprising, prior to inputting the sub-image into a target network model:
constructing an initial network model, wherein the initial network model comprises: the tree crown classification method comprises a generator, a discriminator and introduction condition information related to the generator and the discriminator, wherein the introduction condition information is used for optimizing the generator and the discriminator, and the generator is used for classifying sub-images to obtain tree crown classification types; the discriminator is used for obtaining the crown segmentation result according to the peach tree crown region image;
and training the initial network model by adopting the training set corresponding to the subimages to obtain the target network model.
7. The method of claim 6, wherein obtaining a crown volume result from the crown segmentation result comprises:
obtaining the plant height of the peach tree according to the crown area and the edge point in the corresponding output image of the crown segmentation result and the parameter information corresponding to the edge point and the center of the crown area;
and obtaining a crown volume result based on an ellipsoid volume calculation method according to the plant height of the peach tree.
8. The method of claim 7, after obtaining the crown volume result, further comprising:
and respectively carrying out precision verification on the output image corresponding to the crown segmentation result and the crown volume result.
9. The utility model provides a device that peach crown phenotype information extracted based on unmanned aerial vehicle image, its characterized in that includes:
the acquisition module is used for respectively acquiring images of the peach orchard under different growth conditions;
the obtaining module is used for inputting the peach garden image into image splicing software for preprocessing to obtain sub-images;
the input module is used for inputting the subimages into a target network model to obtain a crown segmentation result, wherein the target network model is obtained by inputting the subimages into an initial network model for training, and the initial network model is a semi-supervised deep learning model;
and the obtaining module is used for obtaining a crown volume result according to the crown segmentation result.
10. A peach tree crown phenotype information extraction system based on unmanned aerial vehicle images is characterized by comprising a memory and a processor, wherein executable instructions of the processor are stored in the memory; wherein the processor is configured to perform the method of peach tree crown information extraction of any one of claims 1-8 via execution of the executable instructions.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110138934.1A CN112818855A (en) | 2021-02-01 | 2021-02-01 | Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110138934.1A CN112818855A (en) | 2021-02-01 | 2021-02-01 | Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112818855A true CN112818855A (en) | 2021-05-18 |
Family
ID=75861218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110138934.1A Pending CN112818855A (en) | 2021-02-01 | 2021-02-01 | Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818855A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110956187A (en) * | 2019-11-28 | 2020-04-03 | 中国农业科学院农业信息研究所 | Unmanned aerial vehicle image plant canopy information extraction method based on ensemble learning |
| CN111738165A (en) * | 2020-06-24 | 2020-10-02 | 中国农业科学院农业信息研究所 | Method for extracting individual plant canopy from high-resolution unmanned aerial vehicle visible light remote sensing image |
-
2021
- 2021-02-01 CN CN202110138934.1A patent/CN112818855A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110956187A (en) * | 2019-11-28 | 2020-04-03 | 中国农业科学院农业信息研究所 | Unmanned aerial vehicle image plant canopy information extraction method based on ensemble learning |
| CN111738165A (en) * | 2020-06-24 | 2020-10-02 | 中国农业科学院农业信息研究所 | Method for extracting individual plant canopy from high-resolution unmanned aerial vehicle visible light remote sensing image |
Non-Patent Citations (2)
| Title |
|---|
| 丁为民 等: "《基于机器视觉的果树树冠体积测量方法研究》", 《农业机械学报》, vol. 47, no. 6, 30 June 2016 (2016-06-30), pages 2 - 3 * |
| 张向荣 等: "大数据挖掘技术及其在医药领域的应用", 西安电子科技大学业出版社, pages: 183 - 114 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Iglhaut et al. | Structure from motion photogrammetry in forestry: A review | |
| Kalisperakis et al. | Leaf area index estimation in vineyards from UAV hyperspectral data, 2D image mosaics and 3D canopy surface models | |
| Wu et al. | Inter-comparison of remote sensing platforms for height estimation of mango and avocado tree crowns | |
| WO2022067598A1 (en) | Method of individual tree crown segmentation from airborne lidar data using novel gaussian filter and energy function minimization | |
| CN109117811A (en) | A kind of system and method based on low-altitude remote sensing measuring technique estimation urban vegetation coverage rate | |
| CN111767865A (en) | Method for inverting mangrove forest biomass by using aerial image and laser data | |
| CN109784300A (en) | A kind of crops science survey production method and system | |
| CN110969654A (en) | Corn high-throughput phenotype measurement method and device based on harvester and harvester | |
| Brocks et al. | Toward an automated low-cost three-dimensional crop surface monitoring system using oblique stereo imagery from consumer-grade smart cameras | |
| CN109816542A (en) | A kind of crop production reduction Claims Resolution method and system | |
| Zhou et al. | An automated, high-performance approach for detecting and characterizing broccoli based on UAV remote-sensing and transformers: A case study from Haining, China | |
| Liu et al. | An efficient approach based on UAV orthographic imagery to map paddy with support of field-level canopy height from point cloud data | |
| CN115453555A (en) | Unmanned aerial vehicle rapid monitoring method and system for grassland productivity | |
| KR102479284B1 (en) | Vegetation index acquisition unit and apparatus for monitoring plant comprising the same | |
| CN117114147A (en) | Method and device for estimating forest vegetation carbon reserves based on radar and satellite remote sensing | |
| CN116561509A (en) | Urban vegetation overground biomass accurate inversion method and system considering vegetation types | |
| Sangjan et al. | Phenotyping architecture traits of tree species using remote sensing techniques | |
| Yin et al. | Individual tree parameters estimation for chinese fir (cunninghamia lanceolate (lamb.) hook) plantations of south china using UAV Oblique Photography: Possibilities and Challenges | |
| Pätzig et al. | Allometric relationships for selected macrophytes of kettle holes in northeast Germany as a basis for efficient biomass estimation using unmanned aerial systems (UAS) | |
| CN115619286B (en) | Method and system for evaluating quality of sample plot of breeding field district | |
| Rinnamang et al. | Estimation of aboveground biomass using aerial photogrammetry from unmanned aerial vehicle in teak (Tectona grandis) plantation in Thailand | |
| CN108399650B (en) | Chlorophyll content prediction method | |
| CN115223062B (en) | Eucalyptus artificial forest area stand accumulation amount time difference correction method based on UAV data | |
| CN113514402B (en) | System and method for predicting chlorophyll content of winter wheat | |
| CN112818855A (en) | Method, device and system for extracting peach tree crown phenotype information based on unmanned aerial vehicle image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210518 |