CN114967495A - Orchard virtual simulation inspection system and method based on Internet of things cloud control platform - Google Patents
Orchard virtual simulation inspection system and method based on Internet of things cloud control platform Download PDFInfo
- Publication number
- CN114967495A CN114967495A CN202210438992.0A CN202210438992A CN114967495A CN 114967495 A CN114967495 A CN 114967495A CN 202210438992 A CN202210438992 A CN 202210438992A CN 114967495 A CN114967495 A CN 114967495A
- Authority
- CN
- China
- Prior art keywords
- inspection
- orchard
- module
- inspection robot
- virtual simulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B17/00—Systems involving the use of models or simulators of said systems
- G05B17/02—Systems involving the use of models or simulators of said systems electric
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
- Y02A40/25—Greenhouse technology, e.g. cooling systems therefor
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an orchard virtual simulation inspection system and method based on an Internet of things cloud control platform, wherein the system is based on the Internet of things and comprises an inspection robot, environmental information acquisition equipment, a 5G communication system and an Internet of things big data cloud control platform, the Internet of things big data cloud control platform comprises a user terminal and a cloud server, the user terminal is connected with the cloud server, the 5G communication system comprises a first 5G communication module and a second 5G communication module, the inspection robot is connected with the cloud server through the first 5G communication module, and the environmental information acquisition equipment is connected with the cloud server through the second 5G communication module; the cloud server is used for realizing autonomous cooperative operation of the plurality of inspection robots in the target orchard; the cloud server is used for realizing the matching of the virtual simulation model and the scheduling system and completing the linkage between the virtual simulation model and the scheduling system. The invention adopts the virtual simulation technology to carry out path optimization simulation on the inspection robot, and uses the simulation result in the maneuver control of the inspection robot to realize intelligent management.
Description
Technical Field
The invention relates to an orchard virtual simulation inspection system and method based on an Internet of things cloud control platform, and belongs to the technical field of intelligent agriculture.
Background
With the gradual upgrade of the fruit planting industry, the fruit planting mode gradually changes from the original small-scale and dispersed household contract land to the large-scale and intensive planting mode, and the standardized planting requirement is difficult to meet by depending on the traditional orchard management mode. For example, with the increase of the planting scale, the traditional method is relied on to complete various plant protection links such as pest and disease investigation, fruit tree maturity inspection and the like with manpower, so that the efficiency is low and the labor cost is increased; in addition, the orchard is generally far away from the house of the grower, and the grower cannot guarantee to go to the orchard every day to check the growth environment of the fruit trees, such as soil pH value, air temperature and humidity, soil water content and the like. Therefore, the monitoring mode of the orchard is upgraded more importantly.
Disclosure of Invention
In view of the above, the invention provides an orchard virtual simulation inspection system, an orchard inspection method, an orchard virtual simulation inspection device, a cloud server and a storage medium based on an Internet of things cloud control platform, which are used for acquiring fruit growth information and orchard environment information in real time by utilizing technologies such as a sensor technology, an autonomous navigation technology, a deep learning technology, an Internet of things and the like, and carrying out intelligent decision making through a cloud; in addition, the virtual simulation technology is adopted to carry out path optimization simulation on the inspection robot, and the simulation result is used for the maneuvering control of the inspection robot, so that intelligent management of an orchard is realized, and fruit trees can grow better.
The invention aims to provide an orchard inspection method.
The second purpose of the invention is to provide an orchard inspection device.
The third purpose of the invention is to provide an orchard virtual simulation inspection system based on an Internet of things cloud control platform.
A fourth object of the present invention is to provide a cloud server.
A fifth object of the present invention is to provide a storage medium.
The first purpose of the invention can be achieved by adopting the following technical scheme:
an orchard inspection method is applied to a cloud server and comprises the following steps:
determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
initializing each inspection robot;
after the initialization processing is completed, based on the data obtained by the GPS positioning module and the laser radar of each inspection robot and the data obtained by each inspection robot scanning the RFID electronic tag, the accurate position of each inspection robot is obtained according to the Kalman filtering algorithm and the data fusion algorithm;
planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
controlling each inspection robot to move along a corresponding inspection path, and detecting all fruit trees on the inspection path so as to obtain a plurality of detection results;
providing a suggestion in the aspect of orchard agriculture management for a planting user according to each detection result;
after the target orchard is patrolled and examined and accomplished, let every robot of patrolling and examining return the district that charges to patrol and examine next time.
Further, the initializing each inspection robot specifically includes:
selecting one of the inspection robots;
based on a mapping algorithm, continuously moving and calculating the position of the patrol robot according to data obtained by a GPS positioning module and a laser radar of the patrol robot, and meanwhile, constructing a two-dimensional map of a target orchard by using the laser radar;
and transmitting the constructed two-dimensional map of the target orchard to the rest inspection robots, so as to finish the initialization processing of each inspection robot.
The second purpose of the invention can be achieved by adopting the following technical scheme:
the utility model provides an orchard inspection device, is applied to cloud ware, the device includes:
the system comprises a determining module, a judging module and a judging module, wherein the determining module is used for determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
the initialization module is used for initializing each inspection robot;
the positioning module is used for acquiring the accurate position of each inspection robot according to a Kalman filtering algorithm and a data fusion algorithm based on the data acquired by the GPS positioning module and the laser radar of each inspection robot and the data acquired by the RFID electronic tag scanned by each inspection robot after the initialization processing is finished;
the planning module is used for planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
the inspection module is used for controlling each inspection robot to move along a corresponding inspection path and detecting all fruit trees on the inspection path so as to obtain a plurality of detection results;
the analysis module is used for providing suggestions in the aspect of orchard agriculture management for planting users according to each detection result;
and the charging module is used for returning each inspection robot to the charging area for charging after the target orchard is inspected and finished so as to inspect next time.
The third purpose of the invention can be achieved by adopting the following technical scheme:
an orchard virtual simulation inspection system based on an Internet of things cloud control platform comprises an inspection robot, environment information acquisition equipment, a 5G communication system and an Internet of things big data cloud control platform, wherein the Internet of things big data cloud control platform comprises a user terminal and a cloud server, the user terminal is connected with the cloud server, the 5G communication system comprises a first 5G communication module and a second 5G communication module, the inspection robot is connected with the cloud server through the first 5G communication module, and the environment information acquisition equipment is connected with the cloud server through the second 5G communication module;
the cloud server is used for executing the orchard patrol method.
Further, the cloud server comprises a scheduling system, a virtual simulation model and a big data analysis module;
the scheduling system comprises a path planning module, a database module and a task allocation module, wherein the path planning module, the database module and the task allocation module are used for realizing autonomous cooperative operation of the plurality of inspection robots in a target orchard;
the establishing process of the virtual simulation model comprises the following steps: modeling a real object in a target orchard through modeling software, and importing the built model into a cloud server for rendering;
the virtual simulation model is matched with the scheduling system to realize linkage between the virtual simulation model and the scheduling system;
and the big data analysis module is used for analyzing the data stored by the database module and providing an agronomic suggestion to the user terminal according to the analysis result.
Further, the virtual simulation model is matched with the scheduling system to realize linkage between the two, and the method specifically comprises the following steps:
the virtual simulation model acquires the working state information of the inspection robot by calling a program interface of a database module in the dispatching system, and drives the inspection robot model in the virtual simulation model according to the working state information, or the user terminal controls the inspection robot model in the virtual simulation model by calling the program interface of the database module in the dispatching system, so that the real inspection robot is driven, and the real-to-virtual or virtual-to-real mapping is realized.
Furthermore, the inspection robot comprises a walking device, a computer, a laser radar, an industrial camera, a GPS positioning module and a radio frequency module, wherein the walking device is loaded with the computer, the laser radar, the industrial camera, the GPS positioning module and the radio frequency module, and the first 5G communication module, the laser radar, the industrial camera, the walking device, the GPS positioning module and the radio frequency module are respectively connected with the computer;
the industrial camera is used for acquiring a fruit tree image of a target orchard;
the computer is used for identifying the fruit tree image of the target orchard by using the trained fruit maturity classification model and the trained disease and insect pest classification model, so that the fruit maturity of the fruit tree and whether the disease and insect pest exist in the fruit tree are detected.
Furthermore, the environmental information collection equipment comprises an embedded board card, an air temperature and humidity sensor, a soil pH value sensor, a soil moisture sensor and a solar power supply, the second 5G communication module, the air temperature and humidity sensor, the soil pH value sensor and the soil moisture sensor are respectively connected with the embedded board card, and the solar power supply supplies power to the embedded board card, the air temperature and humidity sensor, the soil pH value sensor, the soil moisture sensor and the second 5G communication module.
Further, the computer is Jetson TX2, and the walking device is a mountain crawler; the user terminal comprises a webpage end and a mobile end, the webpage end and the mobile end both comprise interaction components, and the cloud server is further used for guiding the inspection robot to return to a charging area for charging.
The fourth purpose of the invention can be achieved by adopting the following technical scheme:
the cloud server comprises a processor and a memory for storing an executable program of the processor, and when the processor executes the program stored in the memory, the orchard inspection method is realized.
The fifth purpose of the invention can be achieved by adopting the following technical scheme:
a storage medium storing a program, wherein the program, when executed by a processor, implements the orchard inspection method described above.
Compared with the prior art, the invention has the following beneficial effects:
1. according to the fruit tree disease and pest monitoring system, technologies such as internet of things, autonomous navigation, deep learning and cloud computing are adopted, a grower can remotely monitor an orchard, the inspection robot can identify whether a fruit tree has diseases and pests and the types of the diseases and pests according to a shot fruit tree picture, the label of the disease tree can be fed back to a user terminal, the grower can be helped to quickly find the disease tree and administer medicine according to symptoms, and in addition, the inspection robot can also identify that fruits are in different growth periods and remind the grower to harvest timely; the sensor monitors the pH value, the water content and the air temperature and humidity of the soil in real time, and once the soil property or the air temperature and humidity change, the user terminal immediately gives a prompt to a planting user.
2. According to the orchard maintenance system, an agricultural vehicle autonomous navigation technology, a multi-vehicle cooperative operation technology and a multi-sensor associated positioning technology are adopted, data of a GPS, a laser radar and an RFID electronic tag are fused for relative positioning, a plurality of inspection robots can move autonomously and orderly in a standardized planted orchard, and compared with a scheme of laying fixed cameras on a large scale, the orchard maintenance system can realize that a complete orchard can be inspected by only a few inspection robots, only an inspection area needs to be planned again even if the orchard is expanded in future, no additional hardware is needed, and the orchard maintenance cost is reduced while the orchard maintenance difficulty of orchard equipment is reduced.
3. The virtual reality technology is adopted, the orchard environment is built into a virtual simulation model, the dispatching of the inspection robot can be dynamically displayed in the model, a grower can visually see the operation condition of the inspection robot, the inspection robot can be dispatched and planned through the virtual simulation model, the inspection robot is ensured to operate autonomously without a person, meanwhile, the control right of the grower to a dispatching system is also taken into consideration, and the flexibility of the whole dispatching system is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.
Fig. 1 is a block diagram of a structure of an orchard virtual simulation inspection system based on an internet of things cloud control platform in embodiment 1 of the present invention.
Fig. 2 is a block diagram of the inspection robot according to embodiment 1 of the present invention.
Fig. 3 is a block diagram of an environment information acquisition apparatus according to embodiment 1 of the present invention.
Fig. 4 is a schematic diagram of a relationship between a cloud server and a user terminal in embodiment 1 of the present invention.
Fig. 5 is a flowchart of an orchard inspection method according to embodiment 1 of the present invention.
Fig. 6 is a flowchart of a method for accurately positioning an inspection robot according to embodiment 1 of the present invention.
Fig. 7 is a schematic diagram of a two-dimensional map of a target orchard according to embodiment 1 of the invention.
Fig. 8 is a flowchart of establishing a pest classification model in embodiment 1 of the present invention.
Fig. 9 is a flowchart of establishing a fruit maturity classification model according to embodiment 1 of the present invention.
Fig. 10 is a block diagram of a cloud server according to embodiment 2 of the present invention.
Fig. 11 is a block diagram of the orchard inspection device according to embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the protection scope of the present invention.
Example 1:
as shown in fig. 1-3, the embodiment provides an orchard virtual simulation inspection system based on an internet of things cloud control platform, which is based on the internet of things and comprises an inspection robot 101, an environmental information acquisition device 102, a 5G communication system 103 and an internet of things big data cloud control platform; wherein: the internet of things big data cloud control platform comprises a user terminal 105 and a cloud server 104, the user terminal 105 is connected with the cloud server 104, the 5G communication system 103 comprises a first 5G communication module 1031 and a second 5G communication module 1032, the inspection robot 101 is connected with the cloud server 104 through the first 5G communication module 1031, and the environmental information collection equipment 102 is connected with the cloud server 104 through the second 5G communication module 1032.
Further, as shown in fig. 2, the inspection robot 101 uses a mountain crawler 201 as a carrier, and the mountain crawler 201 is equipped with a computer 202, a laser radar 203, an industrial camera 204, a first 5G communication module 1031, a radio frequency module and a GPS positioning module; wherein, mountain crawler 201, laser radar 203, industry camera 204, first 5G communication module 1031, radio frequency module and GPS orientation module are connected with computer 202 respectively.
Specifically, the radio frequency module in this embodiment includes an RFID reader 205 and an RFID tag 206; the RFID reader 205 is used for scanning the RFID tag 206 on the fruit tree.
The GPS positioning module in the embodiment is used for absolute positioning of the inspection robot and realizing high-precision relative positioning by matching with the laser radar and the RFID electronic tag; in addition to the positioning function, the laser radar is also used for scanning and generating a two-dimensional map of the orchard, and simultaneously detecting front obstacles (detecting dynamic obstacles around the inspection robot, such as people or moving animals mistakenly entering the orchard).
In order to save the installation space and ensure the high performance of the computer, the computer 202 adopts a high-performance and low-power consumption computer module Jetson TX 2; the computer module Jetson TX2 is used for processing information from the laser radar and generating a control signal to the tracked vehicle so as to realize autonomous navigation and obstacle avoidance of the tracked vehicle in the orchard.
Further, as shown in fig. 3, the environmental information collecting device 102 includes a solar power supply 301, an embedded board 302, an air temperature and humidity sensor 303, a soil PH sensor 304, and a soil moisture sensor 305; the air temperature and humidity sensor 303, the soil PH sensor 304, the soil moisture sensor 305 and the second 5G communication module 1032 are respectively connected with the embedded board card 302 through an I/O interface, and the solar power supply 301 supplies power to the embedded board card 302, the air temperature and humidity sensor 303, the soil PH sensor 304, the soil moisture sensor 305 and the second 5G communication module 1032.
This embodiment can be according to the size in orchard, a plurality of environmental information collection equipment of installation in the orchard.
Specifically, the air temperature and humidity sensor 303 is used for collecting the temperature and humidity of air, the soil PH sensor 304 is used for collecting the PH value of soil, and the soil moisture sensor 305 is used for collecting the moisture value of soil; data collected by the three types of sensors are collectively referred to as air soil data, the three types of sensors transmit the air soil data to the embedded board card 302 for processing, and after data processing is completed, the processed data are fed back to the internet of things big data cloud control platform through the second 5G communication module 3032.
In the embodiment, secondary development is performed on the existing cloud computing platform in the market, so that the Internet of things big data cloud control platform is obtained.
As shown in fig. 4, the internet of things big data cloud control platform includes a cloud server 104 and a user terminal 105; wherein: the cloud server 104 includes a scheduling system 401, a virtual simulation model 402 and a big data analysis module 403, and the user terminal 105 includes a PC-side web page and a mobile-side APP, both of which include interactive components.
The cloud server 104 and the user terminal 105 in this embodiment have the following functions:
(1) the cloud server functions as follows: the system is responsible for storing data and pictures returned by the inspection robot and the environmental information acquisition equipment; calling a big data analysis module to provide a recommendation in the aspect of orchard accurate agricultural management for a planting user; operating the dispatching system and outputting a decision result to the inspection robot; and rendering a virtual simulation model of the orchard environment.
It should be noted that a planting user can deploy a certain number of inspection robots according to the size of the orchard area; the cloud server can store data and pictures returned by the plurality of inspection robots and the plurality of environmental information acquisition devices; the cloud server can also operate the scheduling system and output decision results to the plurality of inspection robots.
(2) Functions of the user terminal: the system is responsible for visualizing the data of the cloud server, the decision result and the rendered virtual simulation model of the orchard environment; providing a good user interaction experience.
More specifically, scheduling system 401 includes a path planning module, a database module, and a task assignment module.
The path planning module, the database module and the task allocation module in the embodiment are used for realizing autonomous cooperative operation of a plurality of inspection robots in an orchard, and specifically comprise the following steps:
1. the task allocation module is used for dividing the orchard into a plurality of inspection areas and allocating corresponding inspection areas to each inspection robot by integrating factors such as the electric quantity and the position of each inspection robot.
2. The path planning module is used for planning a corresponding routing inspection route according to the position of each routing inspection robot and the routing inspection area which needs to be responsible for the position, and guiding each routing inspection robot to run according to the planned corresponding routing inspection route.
3. The database module is used for storing the routing inspection route of each inspection robot, the two-dimensional map of the orchard, the electric quantity, the position, the speed and other information.
In this embodiment, the database module has a program interface, which facilitates an external system or device to call the database module information.
The big data analysis module in this embodiment is used for analyzing the data stored by the database module, obtaining the suggestion in the aspect of orchard accurate agricultural management according to the analysis result, and transmitting the suggestion to the user terminal, thereby providing the suggestion in the aspect of orchard accurate agricultural management for the planting user.
The process of establishing the virtual simulation model in the embodiment is as follows: modeling is carried out on a real object in the orchard through modeling software, and the built model is guided into a cloud server to be rendered, wherein the modeling specifically comprises the following steps: establishing an orchard environment and a charging area into a three-dimensional scene by using modeling software, establishing real objects such as fruit trees, inspection robots and the like into a three-dimensional model, and introducing the model into the scene so as to construct and obtain a virtual simulation model of the orchard; by compiling an interface program, the orchard virtual simulation model can be linked with a dispatching system and can be controlled by a planting user through a client terminal, and then dispatching of the inspection robot in reality is controlled.
The virtual simulation model in this embodiment is matched with the scheduling system to realize linkage between the two, which specifically includes: the virtual simulation model obtains the states of the inspection robot, such as the information of electric quantity, inspection routes, speed, position and the like by calling a program interface of a database module in the scheduling system, and drives the inspection robot model in the virtual simulation model according to the information to realize the mapping from reality to virtual. Therefore, a planting user can visually see the dynamic scheduling of the inspection robot through the virtual simulation model; in addition, thanks to the program interface of the database module, the planting user can arrange the routing inspection operation plan of each routing inspection robot through the virtual simulation model according to the self idea, and perform routing inspection path simulation in the virtual environment to obtain the optimized path of the routing inspection robot, thereby realizing the mapping from virtual to real.
As shown in fig. 5, this embodiment further provides an orchard inspection method, where the method is implemented by the cloud server, and includes the following steps:
s501, determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas.
After a target orchard is determined, attaching a RFID electronic tag subjected to waterproof treatment to each fruit tree trunk of the target orchard to finish the labeling of each fruit tree; wherein the RFID electronic tag and the RFID reader carried by the inspection robot have the same height from the ground.
And S502, initializing each inspection robot.
Step S502 specifically includes: the method comprises the steps that one inspection robot in a plurality of inspection robots is selected to conduct two-dimensional mapping by using a SLAM mapping algorithm, wherein the inspection robot can continuously move and calculate the position of the inspection robot according to data obtained by a GPS positioning module and a laser radar, and simultaneously continuously construct a two-dimensional map of a target orchard by using information scanned by the laser radar, so that the inspection robot can obtain map information of the target orchard (the two-dimensional map of the target orchard), and after the two-dimensional map of the target orchard is generated, the map information is uploaded to a dispatching system through a first 5G communication module and then distributed to the rest inspection robots by the dispatching system, and all the inspection robots are enabled to complete initialization processing.
S503, after the initialization processing is completed, based on the data obtained by the GPS positioning module and the laser radar of each inspection robot and the data obtained by the inspection robot scanning the RFID electronic tag, the accurate position of each inspection robot is obtained according to a Kalman filtering algorithm and a data fusion algorithm.
Step S502 is as shown in fig. 6, data obtained by the GPS positioning module and the laser radar of each inspection robot are respectively position and speed estimation based on the GPS and position and speed estimation based on the laser radar, and data obtained by each inspection robot scanning the RFID tag is position estimation based on the RFID tag.
Specifically, the GPS positioning module utilizes the received satellite data packet to calculate the absolute position of the inspection robot, the laser radar scans the environmental characteristic points and the RFID reader scans the RFID electronic tags on the tree trunks to obtain the relative position of the inspection robot on an orchard map, then the Kalman filtering algorithm is used for filtering noise, finally the data fusion algorithm is used for fusing the data of the laser radar scanning environmental characteristic points and the RFID reader to perform relative positioning, the accurate position of the inspection robot is obtained, and finally the accurate position of each inspection robot is obtained.
And S504, planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area.
Step S504, specifically: after each inspection robot obtains the accurate position of the inspection robot, each inspection robot moves to an inspection area which is distributed by a dispatching system in advance, an inspection path can be planned in the corresponding inspection area according to a path planning algorithm of the inspection robot, and at the moment, the purpose of the operation of the laser radar is not to scan a map, but to monitor dynamic obstacles around the inspection robot in real time, such as people or beasts entering an orchard by mistake.
And S505, controlling each inspection robot to move along a corresponding inspection path, and detecting all fruit trees on the inspection path to obtain a plurality of detection results.
When each inspection robot passes through a fruit tree, the RFID reader scans an RFID electronic tag on a trunk of the fruit tree and feeds the RFID electronic tag back to the computer module Jetson TX2, the computer module Jetson TX2 controls the mountain crawler to stop immediately, then the industrial camera starts working, a shot fruit tree picture is transmitted into the computer module Jetson TX2, the computer module Jetson TX2 calls a pre-trained pest classification model and a pre-trained fruit maturity classification model to recognize, and whether pests and fruit maturity in a current picture exist in the fruit tree are detected.
If the detection result shows that the current fruit tree has diseases and insect pests, the computer module Jetson TX2 feeds back the types of the diseases and insect pests and the labels of the disease and insect trees to the Internet of things big data cloud control platform through the first 5G communication module to remind a grower of taking prevention and control measures as soon as possible, and after the analysis is finished, the computer module Jetson TX2 sends an instruction to control the inspection robot to continuously inspect forwards.
The construction steps of the pest classification model and the fruit maturity classification model in the embodiment are as follows:
A. acquiring an image of a fruit tree suffering from diseases and insect pests, and classifying the image according to the types of the diseases and insect pests; constructing a classification network by using a deep learning library Pythrch, and sending classified plant disease and insect pest images into the network for training to obtain a plant disease and insect pest identification model of the fruit tree; reference may be made to the following training and validation process of pest classification models.
B. Acquiring images of fruits in different growth periods, and classifying the images according to the different growth periods; constructing a classification network by using a deep learning library Pythrch, and sending the classified fruit growth image into the network for training to obtain a fruit maturity classification model; the following process of training and verifying the fruit maturity classification model may be referred to.
S506, providing suggestions in orchard agriculture management for planting users according to each detection result.
S507, after the target orchard is patrolled and examined, enabling each patrolling and examining robot to return to a charging area for charging so as to carry out next patrolling and examining.
As shown in fig. 7, after the target orchard is inspected, the scheduling system in the cloud server schedules each inspection robot to return to the charging area for charging, so as to wait for the next inspection schedule.
As shown in fig. 8, the training and verification process of the pest classification model specifically includes:
s801, collecting image sets of fruit trees suffering from different diseases and insect pests.
S802, selecting image parts suffering from diseases and insect pests in the image set by using a marking tool frame, and constructing a disease and insect pest data set;
and S803, training the neural network by using the training set in the pest and disease data set to obtain a neural network model capable of identifying pests and diseases.
And S804, adjusting the model parameters by using a verification set in the pest and disease damage data set.
As shown in fig. 9, the training and verifying process of the fruit maturity classification model specifically includes:
and S901, collecting image sets of the fruits in different growth periods.
S902, selecting a fruit part in the image set by using a marking tool box, and constructing a fruit maturity data set;
and S903, training the neural network by using the training set in the fruit maturity data set to obtain a neural network model capable of identifying the fruit maturity.
And S904, adjusting the model parameters by utilizing the verification set in the fruit maturity data set.
It should be noted that although the method operations described above are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Rather, the depicted steps may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
Example 2:
as shown in fig. 10, the cloud server 104 includes a processor 1042, a first memory, and a network interface 1043 connected through a system bus 1041; the processor 1042 is configured to provide calculation and control capabilities, the memory includes a nonvolatile storage medium 1044 and an internal memory 1045, the nonvolatile storage medium 1044 stores an operating system, a computer program, and a database, the internal memory 1045 provides an environment for the operating system and the computer program in the nonvolatile storage medium to run, and when the computer program is executed by the processor 1042, the orchard patrol method according to the embodiment 1 is implemented as follows:
determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
initializing each inspection robot;
after the initialization processing is completed, based on the data obtained by the GPS positioning module and the laser radar of each inspection robot and the data obtained by each inspection robot scanning the RFID electronic tag, the accurate position of each inspection robot is obtained according to the Kalman filtering algorithm and the data fusion algorithm;
planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
controlling each inspection robot to move along a corresponding inspection path, and detecting all fruit trees on the inspection path so as to obtain a plurality of detection results;
providing a suggestion in the aspect of orchard agriculture management for a planting user according to each detection result;
after the target orchard is patrolled and examined and accomplished, let every robot of patrolling and examining return to the district that charges to patrol and examine next time.
Example 3:
as shown in fig. 11, the present embodiment provides an orchard inspection device, which is applied to a cloud server, and includes an obtaining module 1101, an initializing module 1102, a positioning module 1103, a planning module 1104, a detecting module 1105, an analyzing module 1106, and a charging module 1107, where specific functions of the modules are as follows:
the determining module 1101 is used for determining a target orchard, throwing a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
an initialization module 1102, configured to initialize each inspection robot;
the positioning module 1103 is configured to obtain an accurate position of each inspection robot according to a kalman filtering algorithm and a data fusion algorithm based on data obtained by the GPS positioning module and the laser radar of each inspection robot and data obtained by each inspection robot scanning the RFID tag after the initialization process is completed;
the planning module 1104 is used for planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
the inspection module 1105 is configured to control each inspection robot to move along a corresponding inspection path, and detect all fruit trees on the inspection path, so as to obtain a plurality of detection results;
an analysis module 1106, configured to provide a recommendation in terms of orchard agronomic management for a planting user according to each detection result;
and the charging module 1107 is used for returning each inspection robot to the charging area for charging after the target orchard is inspected completely, so that the next inspection can be performed.
For specific implementation of each module in this embodiment, reference may be made to embodiment 1, which is not described herein again. It should be noted that, the apparatus provided in this embodiment is only illustrated by dividing the functional modules, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above.
It is to be understood that the terms "first", "second", and the like, as used in the apparatus of the present embodiment, may be used to describe various units, but the units are not limited by these terms. These terms are only used to distinguish one module from another. For example, a first hinting module can be referred to as a second hinting module, and similarly, a second hinting module can be referred to as a first hinting module, both the first hinting module and the second hinting module being hinting modules, but not the same hinting module, without departing from the scope of the invention.
Example 4:
the present embodiment provides a storage medium, which is a computer-readable storage medium, and stores a computer program, and when the program is executed by a processor and the processor executes the computer program stored in the memory, the orchard inspection method of embodiment 1 above is implemented, as follows:
determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
initializing each inspection robot;
after the initialization processing is completed, based on the data obtained by the GPS positioning module and the laser radar of each inspection robot and the data obtained by each inspection robot scanning the RFID electronic tag, the accurate position of each inspection robot is obtained according to the Kalman filtering algorithm and the data fusion algorithm;
planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
controlling each inspection robot to move along a corresponding inspection path, and detecting all fruit trees on the inspection path so as to obtain a plurality of detection results;
providing a proposal in the aspect of orchard agriculture management for planting users according to each detection result;
after the target orchard is patrolled and examined and accomplished, let every robot of patrolling and examining return the district that charges to patrol and examine next time.
It should be noted that the computer readable storage medium of the present embodiment may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this embodiment, however, a computer readable signal medium may include a propagated data signal with a computer readable program embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable storage medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program embodied on the computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer-readable storage medium may be written with a computer program for performing the present embodiments in one or more programming languages, including an object oriented programming language such as Java, Python, C + +, and conventional procedural programming languages, such as C, or similar programming languages, or combinations thereof. The program may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In conclusion, the technology of internet of things, autonomous navigation, deep learning, cloud computing and the like is adopted, the planters can remotely monitor the orchard, the inspection robot can identify whether the fruit trees suffer from diseases and insect pests and the types of the diseases and insect pests according to the shot fruit tree pictures, and can feed back the labels of the disease trees to the user terminal to help the planters to quickly find the disease trees and prescribe medicines according to the symptoms; in addition, the virtual reality technology is adopted, the orchard environment is built into a virtual simulation model, the dispatching of the inspection robot can be dynamically displayed in the model, a grower can visually see the operation condition of the inspection robot, and the inspection robot can be dispatched and planned through the virtual simulation model, so that the inspection robot is ensured to operate autonomously without a person, the control right of the grower on a dispatching system is also taken into consideration, and the flexibility of the whole dispatching system is greatly improved.
The above description is only for the preferred embodiments of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the technical solution and the inventive concept of the present invention within the scope of the present invention.
Claims (10)
1. An orchard inspection method is applied to a cloud server and is characterized by comprising the following steps:
determining a target orchard, putting a plurality of inspection robots in the target orchard, and dividing the target orchard to obtain a plurality of inspection areas;
initializing each inspection robot;
after the initialization processing is finished, based on the data obtained by the GPS positioning module and the laser radar of each inspection robot and the data obtained by the scanning of the RFID electronic tag by each inspection robot, the accurate position of each inspection robot is obtained according to a Kalman filtering algorithm and a data fusion algorithm;
planning a corresponding inspection path according to the accurate position of each inspection robot based on each inspection area;
controlling each inspection robot to move along a corresponding inspection path, and detecting all fruit trees on the inspection path so as to obtain a plurality of detection results;
providing a suggestion in the aspect of orchard agriculture management for a planting user according to each detection result;
after the target orchard is patrolled and examined and accomplished, let every robot of patrolling and examining return the district that charges to patrol and examine next time.
2. The orchard inspection method according to claim 1, wherein the initializing process of each inspection robot specifically includes:
selecting one of the inspection robots;
based on a mapping algorithm, continuously moving and calculating the position of the inspection robot according to data obtained by a GPS positioning module and a laser radar of the inspection robot, and simultaneously constructing a two-dimensional map of a target orchard by using the laser radar;
and transmitting the constructed two-dimensional map of the target orchard to the rest inspection robots, so as to finish the initialization processing of each inspection robot.
3. The orchard virtual simulation inspection system based on the Internet of things cloud control platform is characterized by comprising an inspection robot, environmental information acquisition equipment, a 5G communication system and an Internet of things big data cloud control platform, wherein the Internet of things big data cloud control platform comprises a user terminal and a cloud server, the user terminal is connected with the cloud server, the 5G communication system comprises a first 5G communication module and a second 5G communication module, the inspection robot is connected with the cloud server through the first 5G communication module, and the environmental information acquisition equipment is connected with the cloud server through the second 5G communication module;
the cloud server is used for executing the orchard inspection method according to any one of claims 1-2.
4. The orchard virtual simulation inspection system according to claim 3, wherein the cloud server comprises a scheduling system, a virtual simulation model and a big data analysis module;
the scheduling system comprises a path planning module, a database module and a task allocation module, wherein the path planning module, the database module and the task allocation module are used for realizing autonomous cooperative operation of the plurality of inspection robots in a target orchard;
the establishing process of the virtual simulation model comprises the following steps: modeling a real object in a target orchard through modeling software, and importing the built model into a cloud server for rendering;
the virtual simulation model is matched with the scheduling system to realize linkage between the virtual simulation model and the scheduling system;
and the big data analysis module is used for analyzing the data stored by the database module and providing an agronomic suggestion to the user terminal according to an analysis result.
5. The orchard virtual simulation inspection system according to claim 4, wherein the virtual simulation model is matched with the scheduling system to realize linkage between the virtual simulation model and the scheduling system, and specifically comprises:
the virtual simulation model acquires the working state information of the inspection robot by calling a program interface of a database module in the dispatching system, and drives the inspection robot model in the virtual simulation model according to the working state information, or the user terminal controls the inspection robot model in the virtual simulation model by calling the program interface of the database module in the dispatching system, so that the real inspection robot is driven, and the real-to-virtual or virtual-to-real mapping is realized.
6. The orchard virtual simulation inspection system according to claim 3, wherein the inspection robot comprises a walking device, a computer, a laser radar, an industrial camera, a GPS positioning module and a radio frequency module, the walking device is provided with the computer, the laser radar, the industrial camera, the GPS positioning module and the radio frequency module, and the first 5G communication module, the laser radar, the industrial camera, the walking device, the GPS positioning module and the radio frequency module are respectively connected with the computer;
the industrial camera is used for acquiring a fruit tree image of a target orchard;
the computer is used for identifying the fruit tree image of the target orchard by using the trained fruit maturity classification model and the trained disease and insect pest classification model, so that the fruit maturity of the fruit tree and whether the disease and insect pest exist in the fruit tree are detected.
7. The orchard virtual simulation inspection system according to claim 3, wherein the environment information acquisition equipment comprises an embedded board card, an air temperature and humidity sensor, a soil pH value sensor, a soil moisture sensor and a solar power supply, the second 5G communication module, the air temperature and humidity sensor, the soil pH value sensor and the soil moisture sensor are respectively connected with the embedded board card, and the solar power supply supplies power to the embedded board card, the air temperature and humidity sensor, the soil pH value sensor, the soil moisture sensor and the second 5G communication module.
8. The orchard virtual simulation inspection system according to any one of claims 3 to 7, wherein the computer is Jetson TX2, and the walking device is a mountain crawler; the user terminal comprises a webpage end and a mobile end, the webpage end and the mobile end both comprise interaction components, and the cloud server is further used for guiding the inspection robot to return to a charging area for charging.
9. A cloud server comprising a processor and a memory for storing a program executable by the processor, wherein the processor implements the orchard inspection method according to any one of claims 1-2 when executing the program stored in the memory.
10. A storage medium storing a program, wherein the program, when executed by a processor, implements the orchard inspection method according to any one of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210438992.0A CN114967495A (en) | 2022-04-25 | 2022-04-25 | Orchard virtual simulation inspection system and method based on Internet of things cloud control platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210438992.0A CN114967495A (en) | 2022-04-25 | 2022-04-25 | Orchard virtual simulation inspection system and method based on Internet of things cloud control platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114967495A true CN114967495A (en) | 2022-08-30 |
Family
ID=82980502
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210438992.0A Pending CN114967495A (en) | 2022-04-25 | 2022-04-25 | Orchard virtual simulation inspection system and method based on Internet of things cloud control platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114967495A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117499439A (en) * | 2023-11-14 | 2024-02-02 | 北京理工大学前沿技术研究院 | Inspection data processing system and method based on industrial Internet of things |
-
2022
- 2022-04-25 CN CN202210438992.0A patent/CN114967495A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117499439A (en) * | 2023-11-14 | 2024-02-02 | 北京理工大学前沿技术研究院 | Inspection data processing system and method based on industrial Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | An AIoT based smart agricultural system for pests detection | |
| Bacco et al. | The digitisation of agriculture: a survey of research activities on smart farming | |
| Namani et al. | Smart agriculture based on IoT and cloud computing | |
| Bargoti et al. | A pipeline for trunk detection in trellis structured apple orchards | |
| KR20150058163A (en) | Forestry management system | |
| Meivel et al. | Remote sensing analysis of agricultural drone | |
| Orengo et al. | New developments in drone‐based automated surface survey: towards a functional and effective survey system | |
| CN111352404A (en) | Method, system, medium and device for acquiring agricultural condition by cooperation of unmanned aerial vehicle and ground mobile robot | |
| Molin et al. | Precision agriculture and the digital contributions for site-specific management of the fields | |
| CN114967495A (en) | Orchard virtual simulation inspection system and method based on Internet of things cloud control platform | |
| Raptis et al. | CoFly: An automated, AI-based open-source platform for UAV precision agriculture applications | |
| Xiong et al. | Real-time localization and 3D semantic map reconstruction for unstructured citrus orchards | |
| McGlynn et al. | Agricultural robots: Future trends for autonomous farming | |
| Al-Mashhadani et al. | Survey of agricultural robot applications and implementation | |
| Kaivosoja | Future possibilities and challenges for UAV-based imaging development in smart farming | |
| McClune | Joining the dots: reconstructing 3D environments and movement paths using animal-borne devices | |
| Fichtel et al. | Tree localization and monitoring on autonomous drones employing deep learning | |
| Venkatesh et al. | eagri: Smart agriculture monitoring scheme using machine learning strategies | |
| Marreiros et al. | Progress in Artificial Intelligence: 20th EPIA Conference on Artificial Intelligence, EPIA 2021, Virtual Event, September 7–9, 2021, Proceedings | |
| Bala et al. | An IoT-based autonomous robot system for maize precision agriculture operations in sub-Saharan Africa | |
| Juan et al. | Rapid density estimation of tiny pests from sticky traps using Qpest RCNN in conjunction with UWB-UAV-based IoT framework | |
| Chen et al. | Novel intelligent grazing strategy based on remote sensing, herd perception and UAVs monitoring | |
| Apeinans et al. | Smart fruit growing through digital twin paradigm: systematic review and technology gap analysis | |
| Benalaya et al. | UAV search path planning for livestock monitoring | |
| Mitrofanova et al. | Development of a robot for agricultural field scouting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |