CN116228782B - Wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition - Google Patents
Wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition Download PDFInfo
- Publication number
- CN116228782B CN116228782B CN202211654903.2A CN202211654903A CN116228782B CN 116228782 B CN116228782 B CN 116228782B CN 202211654903 A CN202211654903 A CN 202211654903A CN 116228782 B CN116228782 B CN 116228782B
- Authority
- CN
- China
- Prior art keywords
- wheat
- unmanned aerial
- aerial vehicle
- density map
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000209140 Triticum Species 0.000 title claims abstract description 175
- 235000021307 Triticum Nutrition 0.000 title claims abstract description 175
- 238000000034 method Methods 0.000 title claims abstract description 132
- 238000005457 optimization Methods 0.000 claims abstract description 66
- 241000495841 Oenanthe oenanthe Species 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 37
- 230000008569 process Effects 0.000 claims abstract description 24
- 230000003044 adaptive effect Effects 0.000 claims abstract description 21
- 238000005070 sampling Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 27
- 241001024327 Oenanthe <Aves> Species 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 10
- 238000005520 cutting process Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 210000005069 ears Anatomy 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000011835 investigation Methods 0.000 abstract description 38
- 238000010586 diagram Methods 0.000 description 23
- 238000005259 measurement Methods 0.000 description 9
- 238000001514 detection method Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000003709 image segmentation Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 235000013339 cereals Nutrition 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 235000013305 food Nutrition 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 239000002028 Biomass Substances 0.000 description 1
- 241000196324 Embryophyta Species 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009418 agronomic effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000003306 harvesting Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30242—Counting objects in image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition, comprising the following steps: according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity; acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity; dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images; determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model; splicing the optimized mass density map of each sub-image, and determining a wheat head density map; and determining the wheat head quantity of the wheat field according to the wheat head density chart. According to the invention, a density map regression optimization model is constructed to adapt to unmanned aerial vehicle images, a training process of the adaptive Gaussian kernel participation model is adopted, and large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, so that the field investigation efficiency of the wheat spike number is improved.
Description
Technical Field
The invention relates to the technical field of agricultural image processing, in particular to a wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Wheat is a world food crop with strong adaptability and wide distribution, and is also important trade food and assistance food. Among them, the wheat yield estimation method is the most important one. The number of ears, the number of grains per ear and the grain weight are the most important yield components of wheat. Several studies have shown that there is a dynamic compensation mechanism between agronomic yield components, other factors fluctuate drastically with yield-related differences, but the correlation of ear number and yield remains always at the stable strongest level. Thus, the number of wheat ears becomes important information for studying the yield of wheat, and accurate estimation of the number of wheat ears is critical for the grower to predict the harvest and growth trend of wheat.
The traditional wheat ear counting can only rely on manual investigation, is tedious, laborious and limited in sampling area, is easy to make mistakes and takes too long time, so that the accuracy of yield prediction is severely limited, and the estimation error is overlarge.
Therefore, how to provide a new solution to the above technical problem is a technical problem to be solved in the art.
Disclosure of Invention
The embodiment of the invention provides a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, a density map regression optimization model is constructed to adapt to unmanned aerial vehicle images, a training process of a self-adaptive Gaussian kernel participation model is adopted, large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, field investigation efficiency of wheat spike numbers is improved, and the method has important significance for estimating wheat yield of unmanned aerial vehicles, and comprises the following steps:
according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity;
acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity;
dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images;
determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
splicing the optimized mass density map of each sub-image, and determining a wheat head density map;
and determining the wheat head quantity of the wheat field according to the wheat head density chart.
The embodiment of the invention also provides a wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition, which comprises the following components:
The unmanned aerial vehicle acquisition parameter and image acquisition quantity determining module is used for determining unmanned aerial vehicle acquisition parameters and image acquisition quantity according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters;
the unmanned aerial vehicle wheat ear image acquisition module is used for acquiring unmanned aerial vehicle wheat ear images according to unmanned aerial vehicle acquisition parameters and image acquisition quantity;
the sub-image determining module is used for dividing the wheat head image of the unmanned aerial vehicle and determining a plurality of sub-images;
the optimized mass density map determining module is used for determining an optimized mass density map of each sub-image by using the trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
the wheat head density map determining module is used for splicing the optimized mass density map of each sub-image to determine a wheat head density map;
and the wheat Tian Maisui quantity determining module is used for determining the wheat head quantity of the wheat field according to the wheat head density map.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition when being executed by a processor.
The embodiment of the invention also provides a computer program product, which comprises a computer program, wherein the computer program realizes the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition when being executed by a processor.
The embodiment of the invention provides a wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition, comprising the following steps: according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity; acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity; dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images; determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model; splicing the optimized mass density map of each sub-image, and determining a wheat head density map; and determining the wheat head quantity of the wheat field according to the wheat head density chart. The invention constructs a density map regression optimization model to adapt to the unmanned aerial vehicle image, and adopts a training process of a self-adaptive Gaussian kernel participation model to adapt to the unmanned aerial vehicle image. The large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, so that the field investigation efficiency of the wheat spike number is improved, and the method has important significance for estimating the wheat yield by using the unmanned aerial vehicle. Aiming at the problems that manual investigation is time-consuming and labor-consuming and the near-image detection area is limited, the method is popularized to a large-area measurement scale by means of the unmanned aerial vehicle platform, and combines a five-point sampling investigation method and unmanned aerial vehicle acquisition parameters, so that planning of sampling investigation acquisition paths of the unmanned aerial vehicle is realized, and investigation efficiency is improved. Because the conventional truth value density map generation method is of a fixed Gaussian kernel size, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:
fig. 1 is a schematic diagram of a wheat Tian Sui counting method based on unmanned aerial vehicle acquisition in an embodiment of the invention.
Fig. 2 is a schematic diagram of unmanned aerial vehicle sampling and collecting based on a wheat Tian Sui counting method collected by the unmanned aerial vehicle in an embodiment of the invention.
Fig. 3 is a schematic diagram of an image segmentation marker based on a wheat Tian Sui counting method acquired by an unmanned aerial vehicle according to an embodiment of the invention.
Fig. 4 is a schematic diagram of a density chart regression optimization model of a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition in an embodiment of the invention.
Fig. 5 is a schematic diagram of wheat ear characteristic difference under different camera angles based on a wheat Tian Sui number counting method collected by an unmanned aerial vehicle according to an embodiment of the invention.
Fig. 6 is a schematic diagram of the size of a self-adaptive gaussian kernel based on a wheat Tian Sui number counting method acquired by an unmanned aerial vehicle according to an embodiment of the invention.
Fig. 7 is a density chart of different gaussian kernel generation based on a wheat Tian Sui number counting method acquired by an unmanned aerial vehicle according to an embodiment of the invention.
Fig. 8 is a density chart of different models of a wheat Tian Sui counting method based on unmanned aerial vehicle acquisition in an embodiment of the invention.
Fig. 9 is a chart of wheat ear density obtained by splicing the wheat Tian Sui count method based on unmanned aerial vehicle acquisition in the embodiment of the invention.
Fig. 10 is a result of a sampling investigation method based on a wheat Tian Sui count method collected by an unmanned aerial vehicle according to an embodiment of the present invention.
Fig. 11 is a schematic diagram of a computer device running a method for counting the number of wheat Tian Sui based on unmanned aerial vehicle acquisition.
Fig. 12 is a schematic diagram of a wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition in an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.
Fig. 1 is a schematic diagram of a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, and as shown in fig. 1, the embodiment of the invention provides a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, which adopts a self-adaptive gaussian kernel to generate a ground real density map and constructs a density map regression algorithm to adapt to unmanned aerial vehicle images. The large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, the field investigation efficiency of the wheat spike number is improved, and the method has important significance for estimating the wheat yield by using the unmanned aerial vehicle and comprises the following steps:
Step 101: according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity;
step 102: acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity;
step 103: dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images;
step 104: determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
step 105: splicing the optimized mass density map of each sub-image, and determining a wheat head density map;
step 106: and determining the wheat head quantity of the wheat field according to the wheat head density chart.
The embodiment of the invention provides a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, which comprises the following steps: according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity; acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity; dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images; determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model; splicing the optimized mass density map of each sub-image, and determining a wheat head density map; and determining the wheat head quantity of the wheat field according to the wheat head density chart. The invention constructs a density map regression optimization model to adapt to the unmanned aerial vehicle image, and adopts a training process of a self-adaptive Gaussian kernel participation model to adapt to the unmanned aerial vehicle image. The large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, so that the field investigation efficiency of the wheat spike number is improved, and the method has important significance for estimating the wheat yield by using the unmanned aerial vehicle. Aiming at the problems that manual investigation is time-consuming and labor-consuming and the near-image detection area is limited, the method is popularized to a large-area measurement scale by means of the unmanned aerial vehicle platform, and combines a five-point sampling investigation method and unmanned aerial vehicle acquisition parameters, so that planning of sampling investigation acquisition paths of the unmanned aerial vehicle is realized, and investigation efficiency is improved. Because the conventional truth value density map generation method is of a fixed Gaussian kernel size, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved.
The nondestructive wheat ear counting method can be roughly classified into 3 types. Firstly, a traditional image processing algorithm aims at realizing wheat head counting by manually screening characteristic parameters (color characteristics and texture characteristics). The traditional image algorithm has low cost and high detection speed, but needs to manually screen features, the features need to be manually designed with thresholds, and the robustness is poor when the wheat spike counting is carried out in a complex environment. And secondly, a target detection and segmentation algorithm based on deep learning is used for adaptively generating characteristics by using a convolutional neural network and fitting a candidate frame or segmenting wheat ears so as to realize the counting purpose of the wheat ears. The method can adaptively generate the characteristics, is not influenced by complex environments, and can position the wheat ears. It is not suitable for very small scale and too dense scenes. Thirdly, a density map regression algorithm based on deep learning aims at counting the wheat head quantity by utilizing a convolutional neural network to generate a high-definition density map. The method is advanced compared to the former method in that it uses the position information of the wheat ears in exchange for a good adaptability to small scale and dense scenes.
Unmanned aerial vehicles are one of the main ways to obtain remote sensing data at present. Compared with a satellite remote sensing platform, unmanned aerial vehicle remote sensing has the advantages of simplicity in operation, flexibility, rapidness in response, low use cost and the like, is widely applied to the field of agriculture in recent years, and has been used for multiple aspects such as crop yield evaluation, crop height monitoring, crop weed mapping, biomass monitoring and the like. The wheat ear images acquired by the unmanned aerial vehicle are generally characterized by resolution diversity, illumination inconsistency, high density and the like, and great difficulty is brought to a wheat ear counting task. As previously described, deep learning algorithms can overcome this problem. However, there are many deep learning algorithms. It is worth considering to choose a suitable algorithm to match the unmanned aerial vehicle image in order to achieve an efficient measurement of the wheat head number.
Crowd counting studies provide a solution for the algorithm selection of the present invention. The density map regression algorithm mainly focuses on small-scale and dense crowd scenes and has been developed. The algorithm converts the labeled target information (e.g., geometrically adaptive gaussian kernels) into a probability density map (ground truth), then uses end-to-end training to generate an optimized mass density map (predicted value), and finally determines the quantity by integration. The wheat head counting scene of the unmanned aerial vehicle image is very similar to the crowd counting scene. In addition, the location information of each wheat ear is not considered during the investigation, so the weaknesses of the algorithm can be ignored.
Although it has been determined initially that the density map regression algorithm type is suitable for wheat head counting on unmanned aerial vehicle images. In the actual measurement process, unmanned aerial vehicle image acquisition and algorithm matching have problems. Therefore, we designed a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, and adopted a self-adaptive Gaussian kernel to generate a ground real density map, and constructed a density map regression algorithm to adapt to unmanned aerial vehicle images. According to the method, large-area wheat spike counting can be achieved by using a small number of unmanned aerial vehicle images, so that the field investigation efficiency of the wheat spike number is improved, and the method has important significance in estimating the wheat yield by using the unmanned aerial vehicle. The efficient and automatic large-area wheat spike counting method developed by the application has great significance for wheat yield prediction, and can provide theoretical basis and technical support for grain safety early warning.
When the method for counting the number of the wheat Tian Sui based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, the method comprises the following steps:
according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity;
acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity;
dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images;
determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
splicing the optimized mass density map of each sub-image, and determining a wheat head density map;
and determining the wheat head quantity of the wheat field according to the wheat head density chart.
Fig. 2 is a schematic diagram of unmanned aerial vehicle sampling and collecting based on a wheat Tian Sui number counting method collected by an unmanned aerial vehicle according to an embodiment of the present invention, when the method for counting the wheat Tian Sui number collected by an unmanned aerial vehicle according to an embodiment of the present invention is implemented, in one embodiment, the unmanned aerial vehicle parameters include: image sensor parameters and fly height parameters;
according to wheat Tian Shuxing and unmanned aerial vehicle parameter, confirm unmanned aerial vehicle acquisition parameter and image acquisition quantity, include:
According to the image sensor parameters and the flight height parameters, determining the occupation area of a single image of the unmanned aerial vehicle;
determining the number of images required by the whole domain according to the area of a wheat field acquisition region and the occupied area of a single image of the unmanned aerial vehicle;
determining the image acquisition quantity according to the ratio Q of the five-point sampling method and the quantity of images required by the whole domain;
according to the field type in the wheat Tian Shuxing, determining an acquisition route and a photographing rule as unmanned aerial vehicle acquisition parameters; the field type includes: wide field type, medium type and narrow field type.
Aiming at the problems that manual investigation is time-consuming and labor-consuming and the near-image detection area is limited, the method is popularized to a large-area measurement scale by means of the unmanned aerial vehicle platform, and the unmanned aerial vehicle acquisition parameters are obtained by combining a five-point sampling investigation method, so that the planning of the unmanned aerial vehicle sampling investigation acquisition path is realized, and the investigation efficiency is improved.
The image acquisition scheme is shown in fig. 2, and the unmanned aerial vehicle is used for replacing the manual work by referring to a five-point sampling investigation method. The sampling points are flexibly determined by the type and size of the field, the parameters of the image sensor and the flight height parameters. Firstly, determining the occupied area S of an image acquired by an unmanned aerial vehicle through two parameters, namely an image sensor parameter and a flight height parameter sample Then according to the area S of the wheat field collecting area region Divided by S sample Equal to the number N of images required for the universe region According to the ratio Q of the five-point sampling method, the number N of samples to be acquired by the unmanned aerial vehicle is converted sample And finally, determining a collection route and a photographing rule for sample acquisition according to the field type. In one example summary, a DJI Mavic 2pro drone equipped with a 1 inch CMOS camera was used, with image sensor parameters, principally: effective pixels 2000 ten thousand, equivalent focal length 28mm, image size 5472×3648. The flying height is set to be 5 meters, and the automatic exposure and the automatic focusing are performed. Determination of S by means of an image sensor and a flying height sample 15m of 2 . Investigation region area S region 1hm of 2 . Number of images required for universe N region 666 images, Q is 3/33, therefore, the unmanned aerial vehicle acquires the number N of sample images to be acquired sample 60 sheets. Because the test field is a strip field with different treatments, the test field belongs to a narrow field type, the route is set to be equidistant straight flight, and the photographing rule is a hovering photographing mode.
When the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, an unmanned aerial vehicle wheat ear image is segmented, a plurality of sub-images are determined, and the method comprises the following steps:
And dividing the wheat ear image of the unmanned aerial vehicle into a plurality of sub-images in the length and width dimensions according to the preset number.
Fig. 3 is an image segmentation marking schematic diagram of a wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition according to an embodiment of the present invention, as shown in fig. 3, in the embodiment, due to the high similarity of unmanned aerial vehicle wheat ear images in a large field environment, model training can be performed by using a small number of images, so as to reduce manual marking work and ensure data quality. In one example, 22 images are used for model training and verification, and 38 images are used for comparison testing with manual measurement data. Since the original image contains too much target information, the length and width of the original image are divided into six equal parts in order to facilitate the marking work. Thus, one original image is divided into 36 sub-images of 912×608 pixels in size. Thus, there are 22×36=792 sub-images in total. Finally, 396 sub-images are used for model training, 396 sub-images are used for verification. Wheat ears were marked using Labelme 4.5.10, the mark location was the center point of the ear, the tag name "ear", and the file was saved in json format.
When the method for counting the number of the wheat Tian Sui based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, the method further comprises the following steps:
Constructing a density map regression optimization model;
determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel;
training the density map regression optimization model according to the true value density map of the preset number of sub-images and the corresponding preset number of sub-images, and determining the trained density map regression optimization model.
In an embodiment, the density map regression optimization model is built in advance, then a true value density map of a preset number of sub-images is determined by utilizing the self-adaptive Gaussian kernel, and finally training is carried out on the density map regression optimization model according to the true value density map of the preset number of sub-images and the corresponding preset number of sub-images, so that a trained density map regression optimization model is determined.
Because the conventional truth value density map generation method is used for fixing the size of the Gaussian kernel, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved. Furthermore, when the density map regression optimization model is trained, due to the high similarity of the wheat ear images of the unmanned aerial vehicle in a large field environment, partial true value density maps and corresponding partial sub-images are adopted, full quantum images are not adopted, and accurate training of the density map regression optimization model can be realized, so that the trained density map regression optimization model can accurately obtain the optimized quality density map of each sub-image.
When the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, a density map regression optimization model is constructed, and the method comprises the following steps:
constructing a density map regression model according to a density map regression algorithm;
and taking the VGG16 network as a front-end network of the density map regression model, adding a multi-scale feature fusion structure as a rear-end network at the rear end corresponding to the front-end network, and determining a density map regression optimization model.
In an embodiment, the VGG16 network is used as a front-end network in the study, and multi-scale feature fusion is added on a back-end network thereof so as to improve the quality of density map generation. The density map regression optimization model is shown in FIG. 4. VGG16 networks, VGGs, also called VGG-16, as the name implies, are 16 layers, including 13 convolutional layers and 3 fully connected layers, and convolutional neural network models were proposed by Simonyan and Zisselman of group Visual Geometry Group, which mainly work to demonstrate that increasing the depth of the network can affect the final performance of the network to some extent.
When the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, the true value density diagram of each sub-image is determined by utilizing the adaptive Gaussian kernel, and the method comprises the following steps:
Randomly acquiring a preset number of sub-images from all the sub-images;
determining a function between the diagonal and the distance according to the diagonal length of the wheat ears in the preset number of sub-images and combining the relationship between the wheat ears and the center points of the images;
establishing an adaptive Gaussian kernel according to a function between the diagonal and the distance;
and determining a true value density chart of the preset number of sub-images by using the adaptive Gaussian kernel.
When the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition provided by the embodiment of the invention is implemented, in one embodiment, a true value density chart of a sub-image is determined according to the following mode:
wherein D is GT A true value density map for the sub-image;is an adaptive Gaussian kernel for the transformation of delta (x-x i ) Carrying out convolution; x is the position of a pixel in the image; x is x i Annotating the target wheat ears for each of the truth values delta; n is the number of ears of wheat annotations; d (D) i As a function between diagonal and distance, D i ＝0.01×d i +25, β is the conversion coefficient between the diagonal and the kernel size, d i Is the distance between the center point of the wheat ear and the center point of the image.
The foregoing expressions for determining the true value density map of the sub-image are exemplary, and those skilled in the art will appreciate that the above-described formulas may be modified and added with other parameters or data as needed in practice, or that other specific formulas may be provided, and such modifications are within the scope of the present invention.
Because the conventional truth value density map generation method is used for fixing the size of the Gaussian kernel, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved.
Fig. 5 is a schematic diagram of wheat ear characteristic difference under different camera view angles based on a wheat Tian Sui number counting method collected by an unmanned aerial vehicle according to an embodiment of the invention, as shown in fig. 5, the size and characteristics of wheat ears in an edge view are greatly different from those in a center view. To reduce the effects of this discrepancy, a truth density map is generated by a geometry adaptation kernel. I.e. based on camera visionAn adaptive gaussian kernel of the distance between the target point and the image center point in the figure. D (D) GT The definition is shown in the above formula (1). Annotating target ear x for each of the truth values delta i We use gaussian kernelFor delta (x-x) i ) Convolving, where x is the position of the pixel in the image, N is the number of ear notes,/>Is an adaptive Gaussian kernel, D i Is a function between diagonal and distance, as shown in fig. 6, the left is a schematic diagram of different attempts of the unmanned aerial vehicle wheat ear image collected by the unmanned aerial vehicle, the right is the relationship between the diagonal length of the wheat ear and the distance from the center of the image, and D is determined by researching the relationship between the diagonal length of the wheat ear and the distance from the center of the image i ＝0.01×d i +25, β is the conversion coefficient between diagonal and kernel size, with β=0.1 being the best result tested. The density map results generated by the different gaussian kernels are shown in fig. 7.
In a specific implementation of the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition, in one embodiment, training a density map regression optimization model according to a true value density map of a preset number of sub-images and a preset number of sub-images corresponding to the true value density map, and determining a trained density map regression optimization model, wherein the training comprises the following steps:
randomly cutting a true value density map of a preset number of sub-images and a corresponding preset number of sub-images into pixel images with preset sizes, and inputting a density map regression optimization model to be trained;
generating a basic feature map by using the front-end network as a backbone network, and transmitting the basic feature map to the back-end network;
the quality of the basic feature map is improved by using a 1 multiplied by 1 convolution kernel, a 3 multiplied by 3 convolution kernel and up-sampling in a back-end network, an optimized quality density map of the sub-image is determined, and a trained density map regression optimization model is output; in the process of improving the quality of the basic feature map, the feature layers with the same size in the front-end network are concerned, and the feature layers with the same size in the front-end network are combined with the up-sampled feature layers.
In an embodiment, the partial sub-images and corresponding truth density maps are randomly cut into 400 x 400 pixels and input into the model. The front-end network uses VGG16 as the backbone to generate the base profile. In back-end networks, the use of 1 x 1 convolution kernels, 3 x 3 convolution kernels, and upsampling continuously improves the density map quality. By focusing on the same size feature layers in the front-end network and combining them with the upsampled feature layer. And finally, generating an optimized quality density map with the same size as the cut image, and outputting a trained density map regression optimization model. Wherein the quality of the optimized mass density map is higher than the base feature map.
Furthermore, in order to evaluate the wheat head counting method, the proposed density map regression optimization model network is compared with other algorithms. The accuracy of all models is shown in table 1. The density map regression optimization model optimized by the invention has the highest precision, and the Mean Absolute Error (MAE) and the Mean Square Error (MSE) are respectively 9.01 and 11.85. Compared with CSRNet, the improvement is 16.42%. An estimated density map for the different models is shown in fig. 8. Wherein SFANet and the method of the present invention are closest to the true value density plot. The method of the present invention is better because of the influence of the density map generation method.
TABLE 1 accuracy of regression algorithm for different density maps
In an embodiment, the optimized mass density map of each sub-image is determined by the output trained density map regression optimization model.
When the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition is implemented, in one embodiment, an overlapping area is arranged between adjacent sub-images obtained by dividing the wheat ear image of the unmanned aerial vehicle; the width of the overlapping area is set according to the length of the wheat ears;
splicing the optimized mass density map of each sub-image to determine a wheat head density map, comprising:
identifying overlapping regions of the optimized mass density map for each sub-image;
determining the adjacent relation of the optimized mass density map of the sub-images according to the adjacent relation between the sub-images;
and according to the adjacent relation of the optimized mass density maps of the sub-images, arranging the optimized mass density maps of the sub-images, cutting and deleting half of the overlapping area of the optimized mass density maps of the adjacent sub-images, and splicing to obtain the wheat head density map.
Aiming at the problem that the large-scale image of the unmanned plane needs to be segmented and reprocessed and the repeated counting of the segmentation edges of the sub-images, the method well solves the repeated counting problem by setting the overlapping area among the sub-images in the image segmentation and merging process.
In an embodiment, determining the number of wheat Tian Maisui from the wheat head density map comprises: and determining the wheat head quantity of the wheat field in the investigation region according to the wheat head density map and the occupied area of the single image.
In an embodiment, the final goal is to count regional wheat ears in the form of a sampling survey, which is one way to increase the efficiency of the work. The density map regression algorithm proposed above is to divide the sample image into sub-images for further estimation. Therefore, it is necessary to stitch the estimation results of the sub-images into a complete sample estimation result. In the reasoning phase, the image segmentation method is different from the training phase. As shown in fig. 9, an overlapping region is provided between adjacent sub-images, and the width (or height) of the overlapping region is determined by the ear size. Generally, the length of the wheat head is not less than the maximum value, and it is recommended to set the length to 2 times the maximum value, so that both precision and efficiency can be achieved. And (3) obtaining an estimated density map after model reasoning of the segmented sub-images, cutting and deleting half of the overlapping area of the adjacent sub-images, and then seamlessly splicing the sub-images into an estimated density map of a complete sample. The method can solve the problem of repeated counting caused by splitting the spliced part of the image, and has certain reference value for other repeated counting researches.
FIG. 10 is a diagram of an embodiment of the invention based on unmanned aerial vehicleAs shown in fig. 10, the left side is the relation between the estimated value of the sample image and the true value thereof, and the right side is the comparison of the area sampling investigation method and the manual investigation method; the invention integrates the density map estimation results of each acquired unmanned aerial vehicle wheat ear image sample, and compares the density map estimation results with a true value (left side of fig. 10), R 2 0.9919, p<0.01, reaching very significant correlation levels. Illustrating that the present invention is effective from true value density map generation, network construction, and density map stitching. All test data are estimated by sampling area and converted into standard units (10 4 /hm 2 ) Comparison with the manual measurement (right side of fig. 10). The results showed that the Root Mean Square Error (RMSE) and the Mean Absolute Percentage Error (MAPE) of Ningmai 13 were 18.95 10, respectively 4 /hm 2 And 3.37%. Yangmian No. 4 RMSE and MAPE were 13.65.10 respectively 4 /hm 2 And 2.94%. Thus, the method proposed by the present invention is reliable and efficient.
According to the invention, by means of the unmanned aerial vehicle platform and combining a five-point sampling investigation method, the sampling investigation collection path of the unmanned aerial vehicle is planned, and the investigation efficiency is improved. And dynamically adjusting the size of the Gaussian kernel according to the distance between the wheat ears and the center point of the image, wherein the generated truth value density map is more in line with the actual scene. In the image segmentation process, the problem of repeated counting is well solved by setting the overlapping area between the sub-images.
As shown in fig. 11, an embodiment of the present invention further provides a computer device 1100, including a memory 1110, a processor 1120, and a computer program 1130 stored in the memory and capable of running on the processor, where the processor implements the above-mentioned method for counting the number of wheat Tian Sui based on unmanned aerial vehicle collection when executing the computer program.
The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition when being executed by a processor.
The embodiment of the invention also provides a computer program product, which comprises a computer program, wherein the computer program realizes the wheat Tian Sui number counting method based on unmanned aerial vehicle acquisition when being executed by a processor.
Fig. 11 is a schematic diagram of a computer device for running a method for counting the number of wheat Tian Sui based on unmanned aerial vehicle acquisition, and in an embodiment of the present invention, a device for counting the number of wheat Tian Suishu based on unmanned aerial vehicle acquisition is further provided, as described in the following embodiment. Because the principle of the device for solving the problem is similar to that of the wheat Tian Sui counting method based on unmanned aerial vehicle acquisition, the implementation of the device can be referred to the implementation of the wheat Tian Sui counting method based on unmanned aerial vehicle acquisition, and the repetition is omitted.
Fig. 12 is a schematic diagram of a wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition, and as shown in fig. 12, the embodiment of the invention also provides a wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle collection is implemented, in one embodiment, the device comprises:
the unmanned aerial vehicle acquisition parameter and image acquisition quantity determining module 1201 is used for determining unmanned aerial vehicle acquisition parameters and image acquisition quantity according to the wheat Tian Shuxing and unmanned aerial vehicle parameters;
the unmanned aerial vehicle wheat ear image acquisition module 1202 is used for acquiring unmanned aerial vehicle wheat ear images according to unmanned aerial vehicle acquisition parameters and image acquisition quantity;
the sub-image determining module 1203 is configured to segment the wheat head image of the unmanned aerial vehicle, and determine a plurality of sub-images;
an optimized mass density map determining module 1204, configured to determine an optimized mass density map for each sub-image using the trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
the ear density map determining module 1205 is configured to splice the optimized mass density maps of each sub-image to determine an ear density map;
The wheat Tian Maisui quantity determining module 1206 is used for determining the wheat ear quantity of the wheat field according to the wheat ear density map.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle collection is specifically implemented, in one embodiment, the unmanned aerial vehicle parameters include: image sensor parameters and fly height parameters;
the unmanned aerial vehicle acquisition parameter and image acquisition quantity determining module is specifically used for:
according to the image sensor parameters and the flight height parameters, determining the occupation area of a single image of the unmanned aerial vehicle;
determining the number of images required by the whole domain according to the area of a wheat field acquisition region and the occupied area of a single image of the unmanned aerial vehicle;
determining the image acquisition quantity according to the ratio Q of the five-point sampling method and the quantity of images required by the whole domain;
according to the field type in the wheat Tian Shuxing, determining an acquisition route and a photographing rule as unmanned aerial vehicle acquisition parameters; the field type includes: wide field type, medium type and narrow field type.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the sub-image determining module is specifically configured to:
and dividing the wheat ear image of the unmanned aerial vehicle into a plurality of sub-images in the length and width dimensions according to the preset number.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the device further comprises a model building training module for:
constructing a density map regression optimization model;
determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel;
training the density map regression optimization model according to the true value density map of the preset number of sub-images and the corresponding preset number of sub-images, and determining the trained density map regression optimization model.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the model building training module is further used for:
constructing a density map regression model according to a density map regression algorithm;
and taking the VGG16 network as a front-end network of the density map regression model, adding a multi-scale feature fusion structure as a rear-end network at the rear end corresponding to the front-end network, and determining a density map regression optimization model.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the model building training module is further used for:
Randomly acquiring a preset number of sub-images from all the sub-images;
determining a function between the diagonal and the distance according to the diagonal length of the wheat ears in the preset number of sub-images and combining the relationship between the wheat ears and the center points of the images;
establishing an adaptive Gaussian kernel according to a function between the diagonal and the distance;
and determining a true value density chart of the preset number of sub-images by using the adaptive Gaussian kernel.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the model building training module is further used for determining a true value density chart of the sub-image according to the mode of the formula (1).
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is implemented, in one embodiment, the model building training module is further used for:
randomly cutting a true value density map of a preset number of sub-images and a corresponding preset number of sub-images into pixel images with preset sizes, and inputting a density map regression optimization model to be trained;
generating a basic feature map by using the front-end network as a backbone network, and transmitting the basic feature map to the back-end network;
the quality of the basic feature map is improved by using a 1 multiplied by 1 convolution kernel, a 3 multiplied by 3 convolution kernel and up-sampling in a back-end network, an optimized quality density map of the sub-image is determined, and a trained density map regression optimization model is output; in the process of improving the quality of the basic feature map, the feature layers with the same size in the front-end network are concerned, and the feature layers with the same size in the front-end network are combined with the up-sampled feature layers.
When the wheat Tian Suishu counting device based on unmanned aerial vehicle acquisition is specifically implemented, in one embodiment, an overlapping area is arranged between adjacent sub-images obtained by dividing the wheat ear image of the unmanned aerial vehicle; the width of the overlapping area is set according to the length of the wheat ears;
the wheat ear density map determining module is specifically configured to:
identifying overlapping regions of the optimized mass density map for each sub-image;
determining the adjacent relation of the optimized mass density map of the sub-images according to the adjacent relation between the sub-images;
and according to the adjacent relation of the optimized mass density maps of the sub-images, arranging the optimized mass density maps of the sub-images, cutting and deleting half of the overlapping area of the optimized mass density maps of the adjacent sub-images, and splicing to obtain the wheat head density map.
In summary, the method and the device for counting the number of the wheat Tian Sui based on unmanned aerial vehicle acquisition provided by the embodiment of the invention comprise the following steps: according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity; acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity; dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images; determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model; splicing the optimized mass density map of each sub-image, and determining a wheat head density map; and determining the wheat head quantity of the wheat field according to the wheat head density chart. The invention constructs a density map regression optimization model to adapt to the unmanned aerial vehicle image, and adopts a training process of a self-adaptive Gaussian kernel participation model to adapt to the unmanned aerial vehicle image. The large-area wheat spike counting can be realized by using a small number of unmanned aerial vehicle images, so that the field investigation efficiency of the wheat spike number is improved, and the method has important significance for estimating the wheat yield by using the unmanned aerial vehicle. Aiming at the problems that manual investigation is time-consuming and labor-consuming and the near-image detection area is limited, the method is popularized to a large-area measurement scale by means of the unmanned aerial vehicle platform, and combines a five-point sampling investigation method and unmanned aerial vehicle acquisition parameters, so that planning of sampling investigation acquisition paths of the unmanned aerial vehicle is realized, and investigation efficiency is improved. Because the conventional truth value density map generation method is of a fixed Gaussian kernel size, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved.
Aiming at the problems that manual investigation is time-consuming and labor-consuming and the near-image detection area is limited, the method is popularized to a large-area measurement scale by means of the unmanned aerial vehicle platform, and the unmanned aerial vehicle acquisition parameters are obtained by combining a five-point sampling investigation method, so that the planning of the unmanned aerial vehicle sampling investigation acquisition path is realized, and the investigation efficiency is improved. Because the conventional truth value density map generation method is used for fixing the size of the Gaussian kernel, the problem that accuracy is damaged due to the fact that wheat ear scenes with different scales cannot be reflected is solved. Aiming at the problem that the large-scale image of the unmanned plane needs to be segmented and reprocessed and the repeated counting of the segmentation edges of the sub-images, the method well solves the repeated counting problem by setting the overlapping area among the sub-images in the image segmentation and merging process.
According to the technical scheme, the data acquisition, storage, use, processing and the like all meet the relevant regulations of national laws and regulations, and various types of data such as personal identity data, operation data, behavior data and the like related to individuals, clients, crowds and the like acquired by the method are authorized.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The utility model provides a wheat Tian Sui number counting method based on unmanned aerial vehicle gathers which characterized in that includes:
according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters, determining unmanned aerial vehicle acquisition parameters and image acquisition quantity;
acquiring wheat ear images of the unmanned aerial vehicle according to the unmanned aerial vehicle acquisition parameters and the image acquisition quantity;
dividing the wheat ear image of the unmanned aerial vehicle, and determining a plurality of sub-images;
determining an optimized mass density map of each sub-image by using a trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
splicing the optimized mass density map of each sub-image, and determining a wheat head density map;
determining the number of wheat Tian Maisui according to the wheat spike density map;
according to wheat Tian Shuxing and unmanned aerial vehicle parameter, confirm unmanned aerial vehicle acquisition parameter and image acquisition quantity, include:
according to the image sensor parameters and the flight height parameters, determining the occupation area of a single image of the unmanned aerial vehicle;
determining the number of images required by the whole domain according to the area of a wheat field acquisition region and the occupied area of a single image of the unmanned aerial vehicle;
determining the image acquisition quantity according to the ratio Q of the five-point sampling method and the quantity of images required by the whole domain;
According to the field type in the wheat Tian Shuxing, determining an acquisition route and a photographing rule as unmanned aerial vehicle acquisition parameters; the field type includes: wide-field type, medium type and narrow-field type;
further comprises:
constructing a density map regression optimization model;
determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel;
training a density map regression optimization model according to the true value density map of the preset number of sub-images and the corresponding preset number of sub-images, and determining a trained density map regression optimization model;
determining a true value density map for each sub-image using an adaptive gaussian kernel, comprising:
randomly acquiring a preset number of sub-images from all the sub-images;
determining a function between the diagonal and the distance according to the diagonal length of the wheat ears in the preset number of sub-images and combining the relationship between the wheat ears and the center points of the images;
establishing an adaptive Gaussian kernel according to a function between the diagonal and the distance;
determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel;
the true value density map of the sub-image is determined as follows:
withθ i ＝βD i
wherein D is GT A true value density map for the sub-image;is an adaptive Gaussian kernel for the transformation of delta (x-x i ) Carrying out convolution; x is the position of a pixel in the image; x is x i Annotating the target wheat ears for each of the truth values delta; n is the number of ears of wheat annotations; d (D) i As a function between diagonal and distance, D i ＝0.01×d i +25, β is the conversion coefficient between the diagonal and the kernel size, d i Is the distance between the center point of the wheat ear and the center point of the image.
2. The method of claim 1, wherein the unmanned aerial vehicle parameters comprise: image sensor parameters and fly-height parameters.
3. The method of claim 1, wherein segmenting the image of the unmanned aerial vehicle ear to determine a plurality of sub-images comprises:
and dividing the wheat ear image of the unmanned aerial vehicle into a plurality of sub-images in the length and width dimensions according to the preset number.
4. The method of claim 1, wherein constructing a density map regression optimization model comprises:
constructing a density map regression model according to a density map regression algorithm;
and taking the VGG16 network as a front-end network of the density map regression model, adding a multi-scale feature fusion structure as a rear-end network at the rear end corresponding to the front-end network, and determining a density map regression optimization model.
5. The method of claim 1, wherein training the density map regression optimization model based on the true value density map of the predetermined number of sub-images and the corresponding predetermined number of sub-images to determine a trained density map regression optimization model comprises:
Randomly cutting a true value density map of a preset number of sub-images and a corresponding preset number of sub-images into pixel images with preset sizes, and inputting a density map regression optimization model to be trained;
generating a basic feature map by using the front-end network as a backbone network, and transmitting the basic feature map to the back-end network;
the quality of the basic feature map is improved by using a 1 multiplied by 1 convolution kernel, a 3 multiplied by 3 convolution kernel and up-sampling in a back-end network, an optimized quality density map of the sub-image is determined, and a trained density map regression optimization model is output; in the process of improving the quality of the basic feature map, the feature layers with the same size in the front-end network are concerned, and the feature layers with the same size in the front-end network are combined with the up-sampled feature layers.
6. The method of claim 1, wherein an overlapping region is provided between adjacent sub-images obtained by segmenting the image of the unmanned aerial vehicle ear; the width of the overlapping area is set according to the length of the wheat ears;
splicing the optimized mass density map of each sub-image to determine a wheat head density map, comprising:
identifying overlapping regions of the optimized mass density map for each sub-image;
determining the adjacent relation of the optimized mass density map of the sub-images according to the adjacent relation between the sub-images;
And according to the adjacent relation of the optimized mass density maps of the sub-images, arranging the optimized mass density maps of the sub-images, cutting and deleting half of the overlapping area of the optimized mass density maps of the adjacent sub-images, and splicing to obtain the wheat head density map.
7. Wheat Tian Suishu counting assembly based on unmanned aerial vehicle gathers, its characterized in that includes:
the unmanned aerial vehicle acquisition parameter and image acquisition quantity determining module is used for determining unmanned aerial vehicle acquisition parameters and image acquisition quantity according to the wheat Tian Shuxing and the unmanned aerial vehicle parameters;
the unmanned aerial vehicle wheat ear image acquisition module is used for acquiring unmanned aerial vehicle wheat ear images according to unmanned aerial vehicle acquisition parameters and image acquisition quantity;
the sub-image determining module is used for dividing the wheat head image of the unmanned aerial vehicle and determining a plurality of sub-images;
the optimized mass density map determining module is used for determining an optimized mass density map of each sub-image by using the trained density map regression optimization model; the self-adaptive Gaussian kernel is used for participating in the training process of the density map regression optimization model;
the wheat head density map determining module is used for splicing the optimized mass density map of each sub-image to determine a wheat head density map;
the wheat Tian Maisui quantity determining module is used for determining the quantity of wheat Tian Maisui according to the wheat spike density map;
The unmanned aerial vehicle acquisition parameter and image acquisition quantity determining module is specifically used for: according to the image sensor parameters and the flight height parameters, determining the occupation area of a single image of the unmanned aerial vehicle; determining the number of images required by the whole domain according to the area of a wheat field acquisition region and the occupied area of a single image of the unmanned aerial vehicle; determining the image acquisition quantity according to the ratio Q of the five-point sampling method and the quantity of images required by the whole domain; according to the field type in the wheat Tian Shuxing, determining an acquisition route and a photographing rule as unmanned aerial vehicle acquisition parameters; the field type includes: wide-field type, medium type and narrow-field type;
the model building training module is used for: constructing a density map regression optimization model; determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel; training a density map regression optimization model according to the true value density map of the preset number of sub-images and the corresponding preset number of sub-images, and determining a trained density map regression optimization model;
the model building training module is further used for: randomly acquiring a preset number of sub-images from all the sub-images; determining a function between the diagonal and the distance according to the diagonal length of the wheat ears in the preset number of sub-images and combining the relationship between the wheat ears and the center points of the images; establishing an adaptive Gaussian kernel according to a function between the diagonal and the distance; determining a true value density chart of a preset number of quantum images by using an adaptive Gaussian kernel; the true value density map of the sub-image is determined as follows:
withθ i ＝βD i
Wherein D is GT A true value density map for the sub-image;is an adaptive Gaussian kernel for the transformation of delta (x-x i ) Carrying out convolution; x is the position of a pixel in the image; x is x i Annotating the target wheat ears for each of the truth values delta; n is the number of ears of wheat annotations; d (D) i As a function between diagonal and distance, D i ＝0.01×d i +25, β is the conversion coefficient between the diagonal and the kernel size, d i Is the distance between the center point of the wheat ear and the center point of the image.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 6 when executing the computer program.
9. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any of claims 1 to 6.
10. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211654903.2A CN116228782B (en) | 2022-12-22 | 2022-12-22 | Wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211654903.2A CN116228782B (en) | 2022-12-22 | 2022-12-22 | Wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116228782A CN116228782A (en) | 2023-06-06 |
| CN116228782B true CN116228782B (en) | 2024-01-12 |
Family
ID=86583389
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211654903.2A Active CN116228782B (en) | 2022-12-22 | 2022-12-22 | Wheat Tian Sui number counting method and device based on unmanned aerial vehicle acquisition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116228782B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107465906A (en) * | 2017-08-09 | 2017-12-12 | 广东欧珀移动通信有限公司 | Panorama shooting method, device and the terminal device of scene |
| CN109740721A (en) * | 2018-12-19 | 2019-05-10 | 中国农业大学 | Wheat head method of counting and device |
| CN111242036A (en) * | 2020-01-14 | 2020-06-05 | 西安建筑科技大学 | Crowd counting method based on encoding-decoding structure multi-scale convolutional neural network |
| CN111444872A (en) * | 2020-03-31 | 2020-07-24 | 广西善图科技有限公司 | Danxia landform parameter measuring method |
| CN112115988A (en) * | 2020-09-03 | 2020-12-22 | 中国农业大学 | Wheat ear counting method and device and self-walking trolley |
| CN113435282A (en) * | 2021-06-18 | 2021-09-24 | 南京农业大学 | Unmanned aerial vehicle image ear recognition method based on deep learning |
| CN114266752A (en) * | 2021-12-23 | 2022-04-01 | 中国农业科学院作物科学研究所 | Wheat spike number identification method, system and medium based on fast R-CNN |
| CN114418861A (en) * | 2022-03-31 | 2022-04-29 | 南京云创大数据科技股份有限公司 | Camera image splicing processing method and system |
| CN114758293A (en) * | 2022-03-28 | 2022-07-15 | 暨南大学 | Deep learning population counting method based on auxiliary branch optimization and local density block enhancement |
| CN114882385A (en) * | 2022-05-13 | 2022-08-09 | 中国农业科学院农业信息研究所 | Method for counting wheat ears in field based on unmanned aerial vehicle platform |
-
2022
- 2022-12-22 CN CN202211654903.2A patent/CN116228782B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107465906A (en) * | 2017-08-09 | 2017-12-12 | 广东欧珀移动通信有限公司 | Panorama shooting method, device and the terminal device of scene |
| CN109740721A (en) * | 2018-12-19 | 2019-05-10 | 中国农业大学 | Wheat head method of counting and device |
| CN111242036A (en) * | 2020-01-14 | 2020-06-05 | 西安建筑科技大学 | Crowd counting method based on encoding-decoding structure multi-scale convolutional neural network |
| CN111444872A (en) * | 2020-03-31 | 2020-07-24 | 广西善图科技有限公司 | Danxia landform parameter measuring method |
| CN112115988A (en) * | 2020-09-03 | 2020-12-22 | 中国农业大学 | Wheat ear counting method and device and self-walking trolley |
| CN113435282A (en) * | 2021-06-18 | 2021-09-24 | 南京农业大学 | Unmanned aerial vehicle image ear recognition method based on deep learning |
| CN114266752A (en) * | 2021-12-23 | 2022-04-01 | 中国农业科学院作物科学研究所 | Wheat spike number identification method, system and medium based on fast R-CNN |
| CN114758293A (en) * | 2022-03-28 | 2022-07-15 | 暨南大学 | Deep learning population counting method based on auxiliary branch optimization and local density block enhancement |
| CN114418861A (en) * | 2022-03-31 | 2022-04-29 | 南京云创大数据科技股份有限公司 | Camera image splicing processing method and system |
| CN114882385A (en) * | 2022-05-13 | 2022-08-09 | 中国农业科学院农业信息研究所 | Method for counting wheat ears in field based on unmanned aerial vehicle platform |
Non-Patent Citations (2)
| Title |
|---|
| 基于无人机图像的多尺度感知麦穗计数方法;孙俊等;《农业工程学报》;第136-144页 * |
| 徐文兵等.《数字地形图测绘原理与方法》.2020,第229页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116228782A (en) | 2023-06-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800736B (en) | Road extraction method based on remote sensing image and deep learning | |
| US20230316555A1 (en) | System and Method for Image-Based Remote Sensing of Crop Plants | |
| CN111738165B (en) | Method for extracting individual plant canopy from high-resolution unmanned aerial vehicle visible light remote sensing image | |
| US10509987B1 (en) | Learning method and learning device for object detector based on reconfigurable network for optimizing customers' requirements such as key performance index using target object estimating network and target object merging network, and testing method and testing device using the same | |
| CN110096994B (en) | Small sample PolSAR image classification method based on fuzzy label semantic prior | |
| US10402692B1 (en) | Learning method and learning device for fluctuation-robust object detector based on CNN using target object estimating network adaptable to customers' requirements such as key performance index, and testing device using the same | |
| Bonnet et al. | Comparison of UAS photogrammetric products for tree detection and characterization of coniferous stands | |
| Pham et al. | Road damage detection and classification with YOLOv7 | |
| CN108416353A (en) | Crop field spike of rice fast partition method based on the full convolutional neural networks of depth | |
| KR20200095336A (en) | Method for optimizing hyperparameters of auto-labeling device which auto-labels training images for use in deep learning network to analyze images with high precision, and optimizing device using the same | |
| CN110942454A (en) | Agricultural image semantic segmentation method | |
| CN113160062B (en) | Infrared image target detection method, device, equipment and storage medium | |
| CN111160396B (en) | Hyperspectral image classification method of graph convolution network based on multi-graph structure | |
| CN103996047A (en) | Hyperspectral image classification method based on compression spectrum clustering integration | |
| CN112749654A (en) | Deep neural network model construction method, system and device for video fog monitoring | |
| CN112528058B (en) | Fine-grained image classification method based on image attribute active learning | |
| Agarwal et al. | Efficient NetB3 for Automated Pest Detection in Agriculture | |
| CN113052108A (en) | Multi-scale cascade aerial photography target detection method and system based on deep neural network | |
| CN113887472A (en) | Remote sensing image cloud detection method based on cascade color and texture feature attention | |
| CN113012111A (en) | Rice lodging degree grading monitoring method based on edge calculation | |
| CN117409339A (en) | Unmanned aerial vehicle crop state visual identification method for air-ground coordination | |
| CN116310852A (en) | Double-time-phase remote sensing image unsupervised classification and change detection method and system | |
| CN116205879A (en) | Unmanned aerial vehicle image and deep learning-based wheat lodging area estimation method | |
| CN117197668A (en) | Crop lodging level prediction method and system based on deep learning | |
| CN111242134A (en) | Remote sensing image ground object segmentation method based on feature adaptive learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |