CN113848208B - Plant phenotype platform and control system thereof - Google Patents
Plant phenotype platform and control system thereof Download PDFInfo
- Publication number
- CN113848208B CN113848208B CN202111169603.0A CN202111169603A CN113848208B CN 113848208 B CN113848208 B CN 113848208B CN 202111169603 A CN202111169603 A CN 202111169603A CN 113848208 B CN113848208 B CN 113848208B
- Authority
- CN
- China
- Prior art keywords
- mechanical arm
- control
- information
- trolley
- controlling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001228 spectrum Methods 0.000 claims abstract description 25
- 238000010146 3D printing Methods 0.000 claims description 6
- 230000005855 radiation Effects 0.000 claims description 3
- 238000001816 cooling Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 12
- 238000010276 construction Methods 0.000 description 7
- 238000000034 method Methods 0.000 description 5
- 230000036544 posture Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012800 visualization Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000000877 morphologic effect Effects 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 2
- 230000008676 import Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 229920000742 Cotton Polymers 0.000 description 1
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 229910052744 lithium Inorganic materials 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 239000002420 orchard Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J3/00—Spectrometry; Spectrophotometry; Monochromators; Measuring colours
- G01J3/28—Investigating the spectrum
- G01J3/2823—Imaging spectrometer
- G01J2003/2826—Multispectral imaging, e.g. filter imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N2021/8466—Investigation of vegetal material, e.g. leaves, plants, fruits
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Engineering & Computer Science (AREA)
- Immunology (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Health & Medical Sciences (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Pathology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biochemistry (AREA)
- Electromagnetism (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Health & Medical Sciences (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a plant phenotype platform and a control system thereof. A control device is arranged in the movable crawler trolley in the platform; a laser radar is arranged at the front part of the movable track trolley; a mechanical arm is arranged on the movable crawler trolley; an RGB camera, a multispectral camera and a laser scanner are arranged on the mechanical arm; the control device is used for controlling the laser radar to acquire three-dimensional point cloud data of a scene and carrying out path planning according to the three-dimensional point cloud data of the scene so as to control the mobile crawler trolley to move; controlling the movement of the mechanical arm according to the movement instruction of the mechanical arm; controlling an RGB camera to acquire image information, controlling a multispectral camera to acquire spectrum information, and controlling a laser scanner to acquire three-dimensional laser information; and sending out the three-dimensional point cloud data, the image information, the spectrum information and the three-dimensional laser information of the scene. The plant phenotype platform can flexibly move, can collect comprehensive plant information, and realizes automatic plant information collection in a greenhouse or a plant factory.
Description
Technical Field
The invention relates to the field of plant phenotype information acquisition, in particular to a plant phenotype platform and a control system thereof.
Background
Plant phenotyping is an important content of intelligent agriculture and modern crop breeding, and a high-throughput plant phenotype information acquisition platform is a key for automatic acquisition of plant phenotype information. At present, high-throughput plant phenotype platforms facing different agricultural scenes are classified into plant phenotype information acquisition platforms facing non-controllable environments such as fields, orchards and the like and controllable environments such as greenhouses or laboratories and the like, and the plant phenotype platforms can be classified into facility plant phenotype platforms and self-propelled plant phenotype platforms according to the deployment flexibility degree and mechanical structure of the plant phenotype platforms. Common self-propelled plant phenotype platforms in the field, such as a field unmanned aerial vehicle low-altitude remote sensing platform, a field unmanned vehicle phenotype platform and the like, and common facility plant phenotype platforms, such as a large Tian Guidao portal frame type plant phenotype platform, a field distributed internet of things phenotype acquisition platform and the like. The indoor environment is controlled, so that the facility utilization rate is higher, and the facility construction is more beneficial to the field environment, so that the indoor environment type plant phenotype platform is mainly used, for example, an indoor fixed plant phenotype platform is generally composed of a conveyor belt and a collection chamber, and plants enter the collection chamber through the conveyor belt to collect phenotype information. If the indoor space is large, indoor portal frame loading equipment is adopted to collect phenotype information. Such facility plant phenotype platforms typically require site construction and device construction in advance, and some conveyor mechanisms require track laying in advance for machine movement, which would require significant fixed cost investment, and have no portability across the greenhouse, i.e., facility phenotype platforms in one greenhouse are difficult to move to other indoor environments for use due to the fixed installation. Domestic greenhouses are usually provided with cultivation bedframes for cultivating potted crops, but the position setting of the cultivation bedframes often does not have unified standards, and the indoor space utilization conditions are different. This also adds significant difficulty to the construction of indoor facility plant phenotype platforms and weakens the cross-greenhouse mobility of the platforms.
The sensor devices carried on the plant phenotype platform are generally determined by the type of information to be acquired, and are generally classified into sensors (laser radar, depth camera, three-dimensional scanner, etc.) for acquiring three-dimensional morphology of plants, RGB cameras for acquiring two-dimensional texture features of plants, and high (multi) spectrum cameras for acquiring two-dimensional spatial spectrum distribution of plants. However, conventional phenotypic platforms do not fully meet the use requirements of these sensors: three-dimensional sensors may require multi-view acquisition of three-dimensional spatial information, RGB cameras or spectral cameras require custom shooting poses, fusion of three-dimensional information and spectral information requires accurate spatial pose information, etc. These requirements are difficult to meet simultaneously on conventional facility-type platforms.
Therefore, the existing plant phenotype platform has the problems of poor mobility across a greenhouse and single plant information acquisition.
Disclosure of Invention
Based on the above, the embodiment of the invention provides a movable plant phenotype platform with diversified plant information collection and a control system thereof, so as to realize automatic plant information collection in a greenhouse or a plant factory.
In order to achieve the above object, the present invention provides the following solutions:
a plant phenotype platform comprising: the device comprises a movable crawler trolley, a control device, a mechanical arm, a laser radar, an RGB camera, a multispectral camera and a laser scanner;
the control device is arranged in the movable crawler trolley; the laser radar is arranged at the front part of the movable crawler trolley; the mechanical arm is arranged on the movable crawler trolley; the RGB camera, the multispectral camera and the laser scanner are arranged on the mechanical arm; the movable crawler trolley, the mechanical arm, the laser radar, the RGB camera, the multispectral camera and the laser scanner are all electrically connected with the control device;
the control device is used for:
controlling the laser radar to acquire three-dimensional point cloud data of a scene, and planning a path according to the three-dimensional point cloud data of the scene so as to control the movement of the movable crawler trolley;
controlling the mechanical arm to move according to the mechanical arm movement instruction;
controlling the RGB camera to collect image information, controlling the multispectral camera to collect spectrum information, and controlling the laser scanner to collect three-dimensional laser information;
and sending out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information.
Optionally, the mobile crawler trolley comprises a box body, a chassis and crawler wheels;
the crawler wheels are connected with the box body through the chassis.
Optionally, the control device comprises an internal industrial personal computer, an external industrial personal computer and a mechanical arm control cabinet;
the built-in industrial personal computer is arranged in the chassis; the external industrial personal computer and the mechanical arm control cabinet are arranged in the box body; the internal industrial personal computer and the mechanical arm control cabinet are electrically connected with the external industrial personal computer;
the built-in industrial personal computer is used for controlling the laser radar to collect three-dimensional point cloud data of the scene and planning a path according to the three-dimensional point cloud data of the scene so as to control the mobile crawler trolley to move;
the mechanical arm control cabinet is used for receiving a mechanical arm movement instruction of the external industrial personal computer so as to control the mechanical arm to move;
the external industrial personal computer is used for acquiring three-dimensional point cloud data of a scene, controlling the RGB camera to acquire the image information, controlling the multispectral camera to acquire the spectrum information, controlling the laser scanner to acquire the three-dimensional laser information, and sending out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information.
Optionally, the plant phenotype platform further comprises: a bracket; the laser radar is arranged at the front part of the movable crawler trolley through the bracket.
Optionally, the plant phenotype platform further comprises: 3D printing connection; the RGB camera, the multispectral camera and the laser scanner are connected through the 3D printing connecting piece.
Optionally, the plant phenotype platform further comprises: a heat radiation fan;
the cooling fan is arranged on the movable crawler trolley.
Optionally, the plant phenotype platform further comprises: a transmitting/receiving antenna;
the receiving and transmitting antenna is arranged on the movable crawler trolley and is electrically connected with the control device.
The invention also provides a plant phenotype platform control system, which is used for the plant phenotype platform; the control system includes:
the trolley control module is used for:
acquiring a trolley control instruction input by a user, and sending the trolley control instruction to a control device of the plant phenotype platform so as to control a mobile crawler trolley to start moving according to the trolley control instruction;
acquiring scene three-dimensional point cloud data sent by the control device, constructing a scene map according to the scene three-dimensional point cloud data, planning a path, and generating a map after path planning;
adopting an obstacle avoidance algorithm to avoid obstacles in the map after path planning, generating a collision-free path, and sending the collision-free path to the control device so as to control the mobile crawler trolley to move according to the collision-free path;
displaying the collision-free path and the trolley state information in real time;
the mechanical arm control module is used for:
acquiring a mechanical arm control instruction input by a user, generating a mechanical arm movement path, and sending the mechanical arm movement path to the control device so as to control the mechanical arm to move according to the mechanical arm movement path;
displaying the movement path of the mechanical arm and the state information of the mechanical arm in real time;
a sensor control module for:
acquiring a sensor control instruction input by a user, and sending the sensor control instruction to the control device so as to control the RGB camera to acquire image information, the multispectral camera to acquire spectrum information and the laser scanner to acquire three-dimensional laser information;
and displaying the image information, the spectrum information, the three-dimensional laser information and the sensor state information in real time.
Optionally, the plant phenotype platform control system further comprises:
the manual control module is used for:
and manually controlling the trolley control module to be executed at each working point, and manually controlling the mechanical arm control module and the sensor control module to be executed at each road point of each working point.
Optionally, the plant phenotype platform control system further comprises:
the automatic control module is used for:
and executing the trolley control module according to the pre-stored sequence of the working points, and executing the mechanical arm control module and the sensor control module according to the pre-stored sequence of the road points at each working point.
Compared with the prior art, the invention has the beneficial effects that:
the embodiment of the invention provides a plant phenotype platform and a control system thereof, and the plant phenotype platform can flexibly move by arranging a movable crawler trolley (AutomatedGuidedVehicle, AGV); the mechanical arm is arranged on the mobile crawler trolley, the laser radar is arranged at the front part of the mobile crawler trolley, the RGB camera, the multispectral camera and the laser scanner are arranged on the mechanical arm, so that the setting of various sensors is realized, the sensors can sample plants at different positions and different postures conveniently, comprehensive plant information can be acquired, and compared with a phenotype platform for acquiring single plant information, the sensor integration of the phenotype platform is higher; the control device is arranged, so that automatic plant information acquisition in a greenhouse or a plant factory is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a block diagram of a plant phenotype platform provided by an embodiment of the present invention;
FIG. 2 is a front view of a plant phenotype platform provided by an embodiment of the present invention;
FIG. 3 is a side view of a plant phenotype platform provided by an embodiment of the present invention;
FIG. 4 is a top view of a plant phenotype platform provided by an embodiment of the present invention;
FIG. 5 is an internal block diagram of a mobile track cart provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram of a software interface according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of an AGV control page according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a control page of a mechanical arm according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a sensor control page according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of an auto mode page according to an embodiment of the present invention;
FIG. 11 is a schematic workflow diagram of a plant phenotype platform provided by an embodiment of the present invention;
fig. 12 is a schematic diagram of a working point A, B, C, D in a large circulation provided by an embodiment of the present invention;
fig. 13 is a schematic diagram of a middle-cycle waypoint a, b, c, d according to an embodiment of the present invention;
fig. 14 is a schematic diagram of an external industrial personal computer according to an embodiment of the present invention;
fig. 15 is a schematic diagram of a control center with a built-in industrial personal computer according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Referring to fig. 1-4, the plant phenotype platform provided in this embodiment includes: a mobile track car (AGV), a control device, a robotic arm 1, a lidar 2, an RGB camera 3, a multispectral camera 4, and a laser scanner 5.
The control device is arranged in the movable crawler trolley; the laser radar 2 is arranged at the front part of the movable crawler; the mechanical arm 1 is arranged on the movable crawler trolley; the RGB camera 3, the multispectral camera 4 and the laser scanner 5 are arranged at the tail end of the tool end of the mechanical arm 1; the movable crawler trolley, the mechanical arm 1, the laser radar 2, the RGB camera 3, the multispectral camera 4 and the laser scanner 5 are all electrically connected with the control device. The RGB camera 3 used in the present embodiment is an RGB-D camera.
The control device is used for:
controlling the laser radar 2 to collect three-dimensional point cloud data of a scene, and planning a path according to the three-dimensional point cloud data of the scene so as to control the movement of the movable crawler;
controlling the mechanical arm 1 to move according to the mechanical arm movement instruction;
controlling the RGB camera 3 to collect image information, controlling the multispectral camera 4 to collect spectrum information, and controlling the laser scanner 5 to collect three-dimensional laser information;
and sending out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information.
In one example, the mobile track trolley comprises a box 6, a chassis 7 and track wheels 8; the crawler wheel 8 is connected with the box 6 through the chassis 7. The box body 6 is provided with a box body side door 9.
The control device comprises an internal industrial personal computer 15, an external industrial personal computer and a mechanical arm control cabinet 16.
Referring to fig. 5, the built-in industrial personal computer 15 is disposed inside the chassis 7; the external industrial personal computer and the mechanical arm control cabinet 16 are arranged inside the box body 6; the internal industrial personal computer 15 and the mechanical arm control cabinet 16 are electrically connected with the external industrial personal computer. The external industrial personal computer and the mechanical arm control cabinet 16 are fixed by the bracket in the box and are used for placing damping cotton.
The built-in industrial personal computer 15 is used for controlling the laser radar 2 to collect the three-dimensional point cloud data of the scene, and performing path planning according to the three-dimensional point cloud data of the scene so as to control the movement of the movable crawler trolley.
The mechanical arm control cabinet 16 is used for receiving data of the external industrial personal computer and mechanical arm movement instructions, driving the mechanical arm 1 to move and feeding the data back to the external industrial personal computer.
The external industrial personal computer is global master control and is responsible for receiving and transmitting data of sensors (the laser scanner 5, the RGB camera 3 and the multispectral camera 4) and devices (the built-in industrial personal computer 15 of the AGV and the mechanical arm control cabinet 16) of each part of the platform and providing corresponding instructions. Specifically, the external industrial personal computer is configured to obtain the three-dimensional point cloud data of the scene, control the RGB camera 3 to collect the image information, control the multispectral camera 4 to collect the spectrum information, control the laser scanner 5 to collect the three-dimensional laser information, and send out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information. The external industrial personal computer comprises a first industrial personal computer 17 and a second industrial personal computer 18, wherein the first industrial personal computer 17 is used for sending a mechanical arm movement instruction to the mechanical arm control cabinet 16; the second industrial personal computer 18 is used for controlling the multispectral camera 4 to collect the spectrum information and controlling the laser scanner 5 to collect the three-dimensional laser information; the first industrial personal computer 17 is electrically connected with the second industrial personal computer 18.
In one example, the plant phenotype platform further comprises: a bracket 10; the laser radar 2 is installed at the front end of the box 6 near the head of the vehicle through the bracket 10, and is fixed at a high point by adopting the bracket 10 so as to provide scene three-dimensional point cloud scanning map-building sensing of the front area visual field, and the scene three-dimensional point cloud data is transmitted to the built-in industrial personal computer 15 of the AGV, so that synchronous positioning and map building (SLAM) and obstacle avoidance are realized.
In one example, the plant phenotype platform further comprises: a 3D printing connection 11; the RGB camera 3, the multispectral camera 4 and the laser scanner 5 are integrally connected through the 3D printing connector 11. In the aspect of carrying sensors, the three-dimensional morphological structure of plants and the acquisition of two-dimensional multiband spectrum texture information are mainly oriented, and sensors of different types can be assembled on a platform for use after suitable connectors are designed and integrated as long as the sensors can acquire two-dimensional and three-dimensional information of the plants.
In one example, the power for the AGV and other components is provided by an internal lithium battery, and the inverter 21 is used to transform the power for devices requiring 220V or 380V voltages. Wherein, a first storage battery 19 is arranged in the chassis 7 to supply power for the built-in industrial personal computer 15; a second storage battery 20 is arranged in the box body 6 and is used for supplying power to the external industrial personal computer and the mechanical arm control cabinet 16; the inverter 21 is provided inside the case 6.
In one example, the plant phenotype platform further comprises: a heat radiation fan 12; the radiator fan 12 is provided on the moving crawler.
In one example, the plant phenotype platform further comprises: a transmitting/receiving antenna 13; the transceiver antenna 13 is disposed on the mobile crawler and electrically connected to the control device.
In one example, the plant phenotype platform further comprises: a system switch 14 electrically connected to the control device.
The invention also provides a plant phenotype platform control system, which is used for the plant phenotype platform in the embodiment.
The control system of the embodiment adopts a remote desktop based on a Windows10 system to control the plant phenotype platform, the main body of the control end is an external industrial personal computer in a box body of an AGV trolley, an upper computer graphical interface is compiled, configuration and control of all hardware (AGVs, mechanical arms and sensors) integrated in the plant phenotype platform are realized by utilizing Software Development Kits (SDKs) of all system components, configuration information and control strategies can be modified and saved, and acquisition and saving of information acquired by the sensors and running states can be realized. The specific information to be interacted is: pose information, RGB image, depth image and internal and external parameters of RGB-D camera; three-dimensional point cloud and pose information of the laser scanner; pose information of the multispectral camera, internal and external parameters and multispectral images; pose information and running state information of the mechanical arm; pose information and running state information of the AGV and the like.
The control system includes: the device comprises a trolley control module, a mechanical arm control module, a sensor control module, a manual control module and an automatic control module.
The trolley control module is used for:
acquiring a trolley control instruction (an operation instruction for basic control such as forward movement, backward movement, clockwise and anticlockwise steering, speed adjustment and the like) input by a user, and sending the trolley control instruction to a control device of the plant phenotype platform so as to control a mobile crawler trolley to start moving according to the trolley control instruction;
acquiring scene three-dimensional point cloud data sent by the control device, constructing a scene map according to the scene three-dimensional point cloud data, planning a path, and generating a map after path planning;
adopting an obstacle avoidance algorithm (for example, a rapid-expansion random tree (RRT) algorithm) to avoid obstacles in the map after path planning, generating a collision-free path, and sending the collision-free path to the control device so as to control the mobile crawler trolley to move according to the collision-free path;
the collision-free path and the car status information (version information of laser radar, odometer, GPS, etc., navigation status information, and system status information) are displayed in real time.
The mechanical arm control module is used for:
acquiring a mechanical arm control instruction input by a user, generating a mechanical arm movement path, and sending the mechanical arm movement path to the control device so as to control the mechanical arm to move according to the mechanical arm movement path;
and displaying the movement path of the mechanical arm and the state information of the mechanical arm in real time.
A sensor control module for:
acquiring a sensor control instruction input by a user, and sending the sensor control instruction to the control device so as to control the RGB camera to acquire image information, the multispectral camera to acquire spectrum information and the laser scanner to acquire three-dimensional laser information;
and displaying the image information, the spectrum information, the three-dimensional laser information and the sensor state information in real time.
The manual control module is used for:
and manually controlling the trolley control module to be executed at each working point, and manually controlling the mechanical arm control module and the sensor control module to be executed at each road point of each working point.
The automatic control module is used for:
and executing the trolley control module according to the pre-stored sequence of the working points, and executing the mechanical arm control module and the sensor control module according to the pre-stored sequence of the road points at each working point.
Each module in the control system (upper computer) forms a corresponding software interface, 6 tag pages are respectively an AGV control page, a mechanical arm control page, an RGB-D camera control page, a laser scanner control page, a multispectral camera control page and an automatic mode page, the first 5 pages are respectively used for manually controlling each part of devices of the plant phenotype platform system, the last automatic mode page is used for automating a preset acquisition process, and each interface is provided with a functional module for controlling the corresponding device for a user to call, as shown in fig. 6.
The AGV control interface provides manual azimuth control buttons for controlling basic operations such as forward movement, backward movement, clockwise and anticlockwise steering, speed adjustment and the like of the AGV crawler, and further provides manual map construction functions and path planning (map construction start/end, map editing, map reading/storage, selection/marking of points, path navigation, map visualization, state information visualization and the like). The map construction method adopts manual control AGVs to move in a scene, and laser radars acquire three-dimensional point cloud data of the scene in real time and display the three-dimensional point cloud data visually, and the AGVs are controlled until the map information is completely acquired. The user may make modifications to the existing map, such as adding or subtracting obstacles. The AGV collision-free path can be planned by selecting path points on the map, and the adopted algorithm is a rapid extended random tree (RRT) algorithm. The window displays the collision free path and the trolley status information in real time and can be derived as shown in fig. 7.
The mechanical arm control page provides manual control of six joints with independent degrees of freedom to move control keys in joint space, movement control of a mechanical arm tool end (TCP) in three-dimensional space coordinates, running speed and other parameter settings (saved configuration can be loaded), and in addition, point positions can be selected/marked, paths can be generated and tracked, and mechanical arm states can be obtained for visual display and export (mechanical arm pose parameters, running states and the like), as shown in fig. 8.
The sensor control page (RGB-D camera control page, laser scanner control page, multispectral camera control page are similar) is designed with parameter configuration function, manual click execution data acquisition function, sensor state data image storage function, and can visually display acquired data or images and working state information, as shown in fig. 9.
The automatic mode page provides the information acquisition working function of automatically completing the whole greenhouse according to the working flow. After the user completes the configuration of the page, the user can start the automatic mode, as shown in fig. 10.
The workflow of the plant phenotype platform provided in this example is described below.
The plant phenotype platform has two working modes, one is a manual mode, and in the manual mode, the platform is manually controlled by a user to move by utilizing control pages of all components in the upper computer software, and the mechanical arm drives and sensors collect data. The other is an automatic execution mode, in the automatic mode, after the front setting is completed, the platform can automatically reach the set point position in the greenhouse, the mechanical arm reaches a series of road point positions along the planned path, and the sensor sequentially collects data. The automatic execution of the pre-work is a preset work cycle of the work mode and parameter configuration of each sensor. In indoor environments such as greenhouses or plant factories, a front-end arrangement for three nested loops is required. As shown in fig. 11, specifically:
(1) Large circulation
(1) AGVs manually construct or import maps (related functions: clicking a direction control key (or mobile phone control) in the host computer), manually drive, construct/import/save maps, visualize maps, edit maps, etc.
(2) The working point positions A, B, C, D of the AGVs in the greenhouse are preset (related functions include clicking selected points on a map, marking the point positions and postures of the current AGVs, sorting the point positions and importing/storing/visualizing the point positions).
(3) The AGV plans the path according to the selected working point and sequence (related functions: collision-free path planning, path planning parameter setting (cycle times, speed, etc.)).
(4) The AGVs sequentially pass through the working points A, B, C, D and the like (related functions of manual starting/suspending/re-executing/stopping in the driving process, automatic suspending when reaching the working points, visualization of the AGV position in a map, and acquisition/display/storage of AGV running state information and pose information).
(5) The AGV reaches the end point to complete the large circulation in the front working. As shown in fig. 12.
(2) Middle circulation
The middle loops are nested in the large loop (4), and one middle loop is nested in each working point A/B/C/D and the like, as follows:
(1) the mechanical arm manually sets the road points a, b, c, d, e and the like (related functions are that clicking each joint rotation control (joint space) in the upper computer, clicking a front, back, left, right, upper and lower control tool end (three-dimensional space) in the upper computer, manually dragging the mechanical arm to move, and sequencing, saving, importing and displaying all-position pose information).
(2) Robot arm path planning (related functions: collision-free path planning (avoiding self-collision and environmental collision), path planning parameter settings (number of cycles, speed, etc.)).
(3) The mechanical arm sequentially passes through the access points a, b, c, d, e and the like (related functions of manual starting/suspending/re-executing/suspending in the operation process of the mechanical arm, automatic suspending when reaching each access point, mechanical arm operation state information, pose information acquisition/display/storage and scram prompt).
(4) The mechanical arm reaches the end point to finish one middle cycle in the large cycle. The initial pose of the mechanical arm sampling waypoint and the state of each waypoint a, b, c, d, e are shown in fig. 13.
(3) Small circulation
The small loops are nested in the middle loop (3), and one small loop is nested at each waypoint a/b/c/d and the like, as follows:
(1) sensor parameter settings (related functions: manually settable/importable configuration/save configuration, applicable to all acquisition processes/partial acquisition, effect real-time visualization).
(2) Sensor pre-acquisition (related functions: manual click to acquire information and data, save and display of data, buffer data clear).
The sensor setting is completed.
In addition, the external industrial personal computer is adopted as a control center for controlling and information interaction with other parts in the platform, as shown in fig. 14, and the external industrial personal computer can be replaced by an internal industrial personal computer of an AGV as the center for respectively controlling the mechanical arm control cabinet and the external industrial personal computer and realizing information interaction, as shown in fig. 15. The two control architectures are suitable for the configuration of different dispatching systems, the use of a plurality of industrial personal computers can realize the control (LinuxROS, windows) across the operation systems, the control architecture taking the built-in industrial personal computers of the AGVs as the center is a single-center architecture, and the information receiving and transmitting interaction is based on the system carried by the AGVs rather than an upper computer system in the external industrial personal computers; the control architecture taking the external industrial personal computer as the center is a multi-center architecture, and different industrial personal computers can adopt different operating systems, so that different kinds of resources in Linux or Windows can be more conveniently scheduled and utilized.
The invention has the following advantages:
1. compared with the existing facility type platform for acquiring plant phenotype information indoors, the mobile station type plant phenotype platform based on the crawler type chassis AGV and the mechanical arm has the capability of sensing an unknown indoor planting and cultivating environment such as a greenhouse or a plant factory, and can establish a map in the unknown scene and realize synchronous positioning and map establishment (SLAM) and obstacle avoidance, so that the universality of the plant phenotype information acquisition platform in different indoor environments is greatly improved. The advantage is derived from a laser radar and a data processing system assembled on the plant phenotype platform, three-dimensional information in a scene can be perceived, point cloud data can be processed in real time, a RRT algorithm is utilized for path planning, and the platform is convenient for automatic sampling in an indoor planting and cultivating environment.
2. Compared with a wheel type mobile platform, the crawler-type AGV chassis has better obstacle trafficability, can steer by taking the centroid of the AGV chassis as a rotation center, has the smallest turning radius, and can steer in a narrow space.
3. The platform integrates sensors including a laser scanner, an RGB-D camera and a multispectral camera, can collect comprehensive plant optical imaging information such as three-dimensional morphological information, RGB texture information, spectral space reflection information and the like of plants, and has higher sensor integration compared with a phenotype platform for collecting single plant information.
4. The sensor is integrated to the tail end of the tool end of the mechanical arm, so that the sensor can be combined with the path planning of the mechanical arm to perform multi-view information acquisition, which is very important for unstructured three-dimensional morphological information acquisition and two-dimensional texture information acquisition of plants. The traditional facility type plant phenotype platform can only collect various information of plants from a single position and posture due to the fact that the path and the installation posture are fixed, and the diversity of the information and the data is weakened. The mechanical arm can provide accurate pose information, and the sensor can sample plants at different positions and different poses conveniently. The indoor plant planting environment is not constant, and the flexibility of the gesture change also improves the universality of the cross-scene.
5. The accurate pose information provided by the mechanical arm is also beneficial to realizing multi-source data fusion of the sensor carried by the platform. The three-dimensional point cloud provided by the laser scanner or the RGB-D camera can be fused with spectrum data when space pose information is provided, so that a more accurate and robust prediction result is provided for plant physiological and biochemical indexes.
6. By combining the indoor global pose information provided by the AGV, the GPS and the gyroscope and the local information pose information provided by the mechanical arm, an accurate indoor plant digital map can be constructed, and management convenience is provided for an indoor agricultural information system.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (10)
1. A plant phenotype platform comprising: the device comprises a movable crawler trolley, a control device, a mechanical arm, a laser radar, an RGB camera, a multispectral camera and a laser scanner;
the control device is arranged in the movable crawler trolley; the laser radar is arranged at the front part of the movable crawler trolley; the mechanical arm is arranged on the movable crawler trolley; the RGB camera, the multispectral camera and the laser scanner are arranged on the mechanical arm; the movable crawler trolley, the mechanical arm, the laser radar, the RGB camera, the multispectral camera and the laser scanner are all electrically connected with the control device;
the control device is used for:
controlling the laser radar to acquire three-dimensional point cloud data of an indoor scene, and planning a path according to the three-dimensional point cloud data of the indoor scene so as to control the mobile crawler trolley to move according to a collision-free path; the collision-free path is generated by adopting an obstacle avoidance algorithm in the generated map after path planning after constructing a scene map according to the three-dimensional point cloud data of the indoor scene and performing path planning;
controlling the mechanical arm to move according to the mechanical arm movement instruction;
controlling the RGB camera to collect image information, controlling the multispectral camera to collect spectrum information, and controlling the laser scanner to collect three-dimensional laser information;
and sending out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information.
2. A plant phenotype platform according to claim 1 wherein the mobile track trolley comprises a box, a chassis and track wheels;
the crawler wheels are connected with the box body through the chassis.
3. The plant phenotype platform according to claim 2, wherein the control device comprises an internal industrial personal computer, an external industrial personal computer and a mechanical arm control cabinet;
the built-in industrial personal computer is arranged in the chassis; the external industrial personal computer and the mechanical arm control cabinet are arranged in the box body; the internal industrial personal computer and the mechanical arm control cabinet are electrically connected with the external industrial personal computer;
the built-in industrial personal computer is used for controlling the laser radar to collect three-dimensional point cloud data of the scene and planning a path according to the three-dimensional point cloud data of the scene so as to control the mobile crawler trolley to move;
the mechanical arm control cabinet is used for receiving a mechanical arm movement instruction of the external industrial personal computer so as to control the mechanical arm to move;
the external industrial personal computer is used for acquiring three-dimensional point cloud data of a scene, controlling the RGB camera to acquire the image information, controlling the multispectral camera to acquire the spectrum information, controlling the laser scanner to acquire the three-dimensional laser information, and sending out the three-dimensional point cloud data of the scene, the image information, the spectrum information and the three-dimensional laser information.
4. A plant phenotype platform according to claim 1 further comprising: a bracket; the laser radar is arranged at the front part of the movable crawler trolley through the bracket.
5. A plant phenotype platform according to claim 1 further comprising: 3D printing connection; the RGB camera, the multispectral camera and the laser scanner are connected through the 3D printing connecting piece.
6. A plant phenotype platform according to claim 1 further comprising: a heat radiation fan;
the cooling fan is arranged on the movable crawler trolley.
7. A plant phenotype platform according to claim 1 further comprising: a transmitting/receiving antenna;
the receiving and transmitting antenna is arranged on the movable crawler trolley and is electrically connected with the control device.
8. A plant phenotype platform control system for controlling a plant phenotype platform according to any one of claims 1 to 7; the control system includes:
the trolley control module is used for:
acquiring a trolley control instruction input by a user, and sending the trolley control instruction to a control device of the plant phenotype platform so as to control a mobile crawler trolley to start moving according to the trolley control instruction;
acquiring three-dimensional point cloud data of an indoor scene sent by the control device, constructing a scene map according to the three-dimensional point cloud data of the indoor scene, planning a path, and generating a map after path planning;
adopting an obstacle avoidance algorithm to avoid obstacles in the map after path planning, generating a collision-free path, and sending the collision-free path to the control device so as to control the mobile crawler trolley to move according to the collision-free path;
displaying the collision-free path and the trolley state information in real time;
the mechanical arm control module is used for:
acquiring a mechanical arm control instruction input by a user, generating a mechanical arm movement path, and sending the mechanical arm movement path to the control device so as to control the mechanical arm to move according to the mechanical arm movement path;
displaying the movement path of the mechanical arm and the state information of the mechanical arm in real time;
a sensor control module for:
acquiring a sensor control instruction input by a user, and sending the sensor control instruction to the control device so as to control the RGB camera to acquire image information, the multispectral camera to acquire spectrum information and the laser scanner to acquire three-dimensional laser information;
and displaying the image information, the spectrum information, the three-dimensional laser information and the sensor state information in real time.
9. The plant phenotype platform control system of claim 8 further comprising:
the manual control module is used for:
and manually controlling the trolley control module to be executed at each working point, and manually controlling the mechanical arm control module and the sensor control module to be executed at each road point of each working point.
10. The plant phenotype platform control system of claim 8 further comprising:
the automatic control module is used for:
and executing the trolley control module according to the pre-stored sequence of the working points, and executing the mechanical arm control module and the sensor control module according to the pre-stored sequence of the road points at each working point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169603.0A CN113848208B (en) | 2021-10-08 | 2021-10-08 | Plant phenotype platform and control system thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111169603.0A CN113848208B (en) | 2021-10-08 | 2021-10-08 | Plant phenotype platform and control system thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113848208A CN113848208A (en) | 2021-12-28 |
| CN113848208B true CN113848208B (en) | 2023-12-19 |
Family
ID=78977747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111169603.0A Active CN113848208B (en) | 2021-10-08 | 2021-10-08 | Plant phenotype platform and control system thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113848208B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| IL296946B1 (en) * | 2022-09-29 | 2024-05-01 | C Crop Ltd | Plant phenotyping |
| CN116147490B (en) * | 2023-04-17 | 2023-06-30 | 慧诺瑞德（北京）科技有限公司 | High-flux plant phenotype information acquisition and measurement system and method |
| CN116539613B (en) * | 2023-07-07 | 2023-10-13 | 佳木斯大学 | Mobile agricultural informatization online detection equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108981569A (en) * | 2018-07-09 | 2018-12-11 | 南京农业大学 | A kind of high-throughput hothouse plants phenotype measuring system based on the fusion of multispectral cloud |
| CN209215940U (en) * | 2018-11-08 | 2019-08-06 | 西北农林科技大学 | A kind of self-travel type field crop phenotypic information acquisition device |
| WO2020230126A1 (en) * | 2019-05-13 | 2020-11-19 | Carmel Haifa University Economic Corporation Ltd. | Systems and methods for phenotyping |
| CN212965386U (en) * | 2020-05-21 | 2021-04-13 | 南京农业大学 | Crawler-type vehicle-mounted lifting laser radar platform for crop phenotype measurement |
| CN112702565A (en) * | 2020-12-03 | 2021-04-23 | 浙江大学 | System and method for acquiring field plant phenotype information |
| CN113674402A (en) * | 2021-08-23 | 2021-11-19 | 浙江大学 | Plant three-dimensional hyperspectral point cloud model generation method, correction method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10520482B2 (en) * | 2012-06-01 | 2019-12-31 | Agerpoint, Inc. | Systems and methods for monitoring agricultural products |
-
2021
- 2021-10-08 CN CN202111169603.0A patent/CN113848208B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108981569A (en) * | 2018-07-09 | 2018-12-11 | 南京农业大学 | A kind of high-throughput hothouse plants phenotype measuring system based on the fusion of multispectral cloud |
| CN209215940U (en) * | 2018-11-08 | 2019-08-06 | 西北农林科技大学 | A kind of self-travel type field crop phenotypic information acquisition device |
| WO2020230126A1 (en) * | 2019-05-13 | 2020-11-19 | Carmel Haifa University Economic Corporation Ltd. | Systems and methods for phenotyping |
| CN212965386U (en) * | 2020-05-21 | 2021-04-13 | 南京农业大学 | Crawler-type vehicle-mounted lifting laser radar platform for crop phenotype measurement |
| CN112702565A (en) * | 2020-12-03 | 2021-04-23 | 浙江大学 | System and method for acquiring field plant phenotype information |
| CN113674402A (en) * | 2021-08-23 | 2021-11-19 | 浙江大学 | Plant three-dimensional hyperspectral point cloud model generation method, correction method and device |
Non-Patent Citations (2)
| Title |
|---|
| 赵杰等.<基于三维激光雷达的环境建模>.《智能机器人技术：安保、巡逻、处置类警用机器人研究实践》.机械工业出版社,2021,9-28. * |
| 辛斌.《运动体路径规划算法》.《面向复杂优化问题求解的智能优化方法》.北京理工大学出版社,2017,296-301. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113848208A (en) | 2021-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113848208B (en) | Plant phenotype platform and control system thereof | |
| US11044842B2 (en) | Path planning for area coverage | |
| Ball et al. | Vision‐based obstacle detection and navigation for an agricultural robot | |
| CN109240284B (en) | Autonomous path planning method and device for unmanned agricultural machine | |
| US10278333B2 (en) | Pruning robot system | |
| CN105607635B (en) | Automatic guided vehicle panoramic optical vision navigation control system and omnidirectional's automatic guided vehicle | |
| EP3686704B1 (en) | Method for generating a representation and system for teaching an autonomous device operating based on such representation | |
| US20190023396A1 (en) | System for measuring three-dimensional environment data, particularly for plant care, as well as sensor module | |
| CN111348161B (en) | Resource environment monitoring system applied to ocean pasture and operation method thereof | |
| Xu et al. | A modular agricultural robotic system (MARS) for precision farming: Concept and implementation | |
| CN105014675A (en) | Intelligent mobile robot visual navigation system and method in narrow space | |
| Holz et al. | Continuous 3D sensing for navigation and SLAM in cluttered and dynamic environments | |
| CN112684785A (en) | Self-walking mowing system and outdoor walking equipment | |
| Bechtsis et al. | Scheduling and Control of Unmanned Ground Vehicles for Precision Farming: A Real-time Navigation Tool. | |
| Cunha et al. | Vision-based control for rigid body stabilization | |
| Wang et al. | Intelligent robotic lawn mower design | |
| CN106873602A (en) | A kind of mobile robot of autonomous positioning in real time | |
| Tiozzo Fasiolo et al. | Recent trends in mobile robotics for 3D mapping in agriculture | |
| Barbosa et al. | Vision-based autonomous crop row navigation for wheeled mobile robots using super-twisting sliding mode control | |
| CN215707042U (en) | Resource environment monitoring system applied to marine ranching | |
| Jarvis | An autonomous heavy duty outdoor robotic tracked vehicle | |
| Liu et al. | Processed RGB-D slam using open-source software | |
| Yue et al. | Intelligent control system of agricultural unmanned tractor tillage trajectory | |
| Hameed et al. | Task and motion planning for selective weed conrol using a team of autonomous vehicles | |
| Klaser et al. | Vision-based autonomous navigation with a probabilistic occupancy map on unstructured scenarios |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |