CN111508059B - Crop picking demonstration method, device, system and medium based on 3D model - Google Patents
Crop picking demonstration method, device, system and medium based on 3D model Download PDFInfo
- Publication number
- CN111508059B CN111508059B CN202010212188.1A CN202010212188A CN111508059B CN 111508059 B CN111508059 B CN 111508059B CN 202010212188 A CN202010212188 A CN 202010212188A CN 111508059 B CN111508059 B CN 111508059B
- Authority
- CN
- China
- Prior art keywords
- picking
- crop
- user
- sand table
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D46/00—Picking of fruits, vegetables, hops, or the like; Devices for shaking trees or shrubs
- A01D46/30—Robotic devices for individually picking crops
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Robotics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention belongs to the technical field of crop picking, and relates to a crop picking demonstration method, device and system based on a 3D model. The method comprises the following steps: constructing a 3D virtual sand table system; receiving growth parameters, and generating and displaying a 3D standard tree model according to the growth parameters; receiving picking stages and picking parameters; judging whether the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage; and if yes, displaying the picked snapshot of the 3D standard tree model after picking according to the picking parameters. The invention can improve the intuitiveness and interactivity of the crop picking process.
Description
Technical Field
The invention relates to the technical field of crop picking, in particular to a crop picking demonstration method, device and system based on a 3D model and a medium.
Background
At present, technical guidance for picking crops is transmitted and learned in the form of electronic books or information pictures and the like. The electronic book or the information picture is not strong in intuitiveness and interactivity. The electronic book or the information picture is only suitable for single crops, a large amount of related electronic materials are required to be stored when a plurality of crop plants are planted, so that the operation is complicated and inconvenient, and the action key in the picking process cannot be dynamically and vividly displayed in the mode; the picking technical difference of different crop varieties and trees cannot be dynamically and vividly simulated; the picking process cannot be intuitively simulated, and standard technical prompts and guidance, notice reminding and other labeling words are absent in the picking process.
Disclosure of Invention
The invention aims to provide a crop picking demonstration method, device, system and medium based on a 3D model, which are used for solving the technical problems of poor guiding intuitiveness and interactivity of crop picking in the prior art.
The first aspect of the invention provides a crop picking demonstration method based on a 3D model, which comprises the following steps:
constructing a 3D virtual sand table system for crop picking;
receiving growth parameters of a 3D virtual sand table system set by a user, and generating and displaying a 3D standard tree model according to the growth parameters;
receiving picking stages and picking parameters of a user for picking the 3D standard tree model;
judging whether the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage;
if the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage; and displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters.
Preferably, the generating and displaying a virtual 3D standard tree model according to the picking parameters includes:
acquiring and updating the 3D material model files corresponding to the picking parameters in a server; 3D material model files of different crops are stored in the server;
the 3D material model file is analyzed and rendered into a 3D standard tree model;
displaying the 3D standard tree model in the sand table system working area.
Preferably, before the picking stage and the picking parameters of the picking of the 3D standard tree model by the receiving user, the method further comprises:
the picking stages of the 3D standard tree model, the picking parameters of each picking stage, and the notes of each picking stage are displayed.
Preferably, the method further comprises:
after each picking, adjusting a 3D standard tree model according to the picking parameters of the picking and displaying the actual effect after the picking;
and when the user finishes the operation, storing the 3D standard tree model so as to call the 3D standard tree model for 3D printing.
Preferably, the method further comprises:
when an undo instruction of a user is received, the picking operation of the 3D standard tree model is undone last time.
Preferably, the method further comprises:
recording picking stages and picking parameters of a user, and inputting the picking stages and the picking parameters into a deep learning model for optimization;
when the user logs in again, the optimized picking phase and picking parameters are acquired and displayed.
Preferably, the 3D virtual sand table system for constructing crop picking further comprises:
s11, collecting topographic data of crops, and constructing a 3D electronic sand table of an area of the crops by combining with GIS;
s12, acquiring a 3D image of the crop, identifying the type of the crop, and constructing a 3D virtual sand table system containing the crop by combining the 3D electronic sand table of the environment where the crop is positioned;
s13, inputting picking information required by a user before picking and during picking and after picking;
s14, respectively performing image display processing on characters and graphics in the 3D virtual sand table system, adopting 2D display on the characters on the same display interface in the 3D virtual sand table system, and adopting 3D stereoscopic display on crops, fruits and surrounding environments of the crops.
The second aspect of the present invention also provides a crop picking demonstration device based on a 3D model, the device comprising:
the construction module is used for constructing a 3D virtual sand table system for picking crops;
the receiving display module is used for receiving the growth parameters of the 3D virtual sand table system set by a user, and generating and displaying a 3D standard tree model according to the growth parameters;
the receiving module is used for receiving picking stages and picking parameters of a user for picking the 3D standard tree model;
the judging module is used for judging whether the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage;
the display module is used for judging whether the picking parameters meet the pre-stored picking parameter standards of the 3D standard tree model in the picking stage; and displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters.
The third aspect of the invention provides a crop picking demonstration system based on a 3D model, which comprises a server and a crop picking demonstration device based on the 3D model.
A fourth aspect of the invention provides a storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as claimed in any preceding claim.
In summary, the crop picking demonstration method, device, system and medium based on the 3D model provided by the invention adopt the 3D virtual sand table system to replace the traditional operation teaching mode of picking, so that the intuitiveness and interactivity of the crop picking process are improved, and the teaching demonstration cost is saved; and automatically calling a built-in material library by the 3D virtual sand table system to generate a 3D standard tree by using picking parameters set by a user. Virtual picking operations in different picking periods can be carried out on the 3D standard tree, and picking snapshots of the 3D standard tree model after picking are displayed by judging whether the picking parameters of each picking stage meet the picking parameter standard of the current crop in the growing period. Compared with the prior art, the invention has the following beneficial effects:
1. the intuitiveness and the interactivity of the crop picking process are improved, and the teaching demonstration cost is saved;
2. crops and varieties which need to be operated can be selected, and the limitation of single variety in the traditional picking teaching mode is broken through;
3. virtual picking operations can be performed through the 3D sand table system and can prompt and guide picking stages, picking sites and notes.
4. The 3D standard tree is moved, picked and withdrawn by the button of "move, pick and withdraw".
Drawings
Fig. 1 shows a flow diagram of a crop picking demonstration method based on a 3D model of the present invention.
FIG. 2 shows a schematic diagram of a crop picking demonstration based on a 3D model according to the present invention
FIG. 3 shows another schematic view of a crop picking demonstration based on a 3D model of the present invention;
fig. 4 shows a schematic structural diagram of a crop picking demonstration device based on a 3D model according to the present invention;
fig. 5 shows a schematic structural diagram of a crop picking demonstration device based on a 3D model according to the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely configured to illustrate the invention and are not configured to limit the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the invention by showing examples of the invention.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
Referring to fig. 1, the present invention provides a crop picking demonstration method based on a 3D model, the method comprising:
s1, constructing a 3D virtual sand table system for picking crops;
the virtual sand table system refers to a virtual technology in the field of computers, and the basic principle is as follows: according to the real running environment of the computer program, a virtual running environment is created, when a certain program tries to function, the program can be run in a sand table, and all changes caused by the program in the sand table cannot influence the real running environment. The sand table system is applied to demonstration of crop fertilization and pesticide application, and can play a role in constructing a demonstration environment. The virtual sand table system provided by the invention comprehensively considers the characteristics of different crops, the picking environment, the size of the crops fruits, the illumination shadows facing in the picking actual environment, the influence of wind on the movement of the crops and the fruits, the positioning of the fruits and the like on the basis of researching the military sand table and the building sand table, can meet the fruit picking and storage technology demonstration of various crop plants through the 3D virtual sand table system, can dynamically and vividly show the action key ways in the picking process, comprises but is not limited to the fruit picking method, the storage method, the attention matters and the like, and can dynamically and vividly simulate different geographic positions, different climates and different technical differences of fertilization and application in the environment under temperature difference. The main method of the embodiment of the invention is as follows:
s11, collecting topographic data of crops, and constructing a 3D electronic sand table of an area of the crops by combining with GIS;
mainly comprises the following steps: an unmanned aerial vehicle is used for carrying a binocular camera or a variable-focus lens to collect images of terrains where crops are located, and the collected engineering images of the terrains of the crops, such as landforms, three-dimensional coordinates, heights, slopes, rivers, roads and surrounding manual settings are obtained through selecting preset wave band combinations and preset time phases in combination with colors of satellite remote sensing images, wherein the preset wave band combinations and the preset time phases are determined by users according to actual needs;
identifying different objects from the image, and extracting texture feature points of each object;
and constructing the 3D electronic sand table without crops.
S12, acquiring a 3D image of the crop, identifying the type of the crop, and constructing a 3D virtual sand table system containing the crop by combining the 3D electronic sand table of the environment where the crop is positioned;
the unmanned aerial vehicle is used for carrying a binocular camera or a variable-focus lens to shoot a 3D image of the crop, the type of the crop is identified, a background image and a shadow image of the crop in the picture are removed according to illumination intensity and shooting angles, and the 3D outline shape of the crop is displayed; of course, the construction of the 3D image of the fruit of the crop is considered, and the fruit is randomly distributed on the 3D contour shape of the crop, so that the fruit needs to be distinguished from the leaves, the branches and the like of the crop (the image segmentation technology can be adopted) during construction, and the position of the fruit on the 3D contour shape of the crop is positioned; judging the maturity of fruits, such as oranges, wherein the orange color is lighter when the fruits are not ripe before picking, and judging the maturity of the fruits according to the color. The positioning of the fruit on the 3D contour of the crop comprises the steps of: after the fruit image is acquired, correcting the fruit image, performing image processing on the fruit image, detecting by using a Hough circle, selecting 8 characteristic points on the fruit, and obtaining the barycenter coordinates of the fruit by using a least square method. The Hough circle detection and least square method adopted herein belongs to a common algorithm in the image field, and is mainly characterized in that the combination of the Hough circle detection and the least square method is adopted, so that the shape of a fruit can be obtained accurately, and if the fruit is possibly elliptical, and if the fruit is possibly circular. But the position of the fruit on the crop can be more accurately located by finding the centroid.
The method for identifying the crop type and constructing the 3D outline shape of the crop can be added with a deep learning algorithm, and the accuracy of the 3D outline shape can be improved through continuous training. The method mainly comprises the following steps: extracting image features of crops and image features of environments where the crops are located; extracting image features of different crops and image features of the environment where the crops are located for multiple times to form a training set; extracting a training set by adopting a CNN (convolutional neural network) algorithm; then, learning and classifying by using an image classifier; and marking the classified crop characteristics and the environmental characteristics of the crops.
After the position of the fruit on the crops is positioned, collecting the growth parameters of the crops and the fruits thereof in different growth stages;
constructing a 3D initial standard tree model with fruits according to growth parameters of the fruits and the crops at each stage and corresponding 3D outline shapes;
s13, inputting picking information required by a user before picking, during picking and after picking, wherein the method comprises the following steps of: variety name, climate parameters, preparation, picking operation flow, notes, etc.
S14, respectively performing image display processing on characters and graphics in the 3D virtual sand table system, adopting 2D display on the characters on the same display interface in the 3D virtual sand table system, and adopting 3D stereoscopic display on crops, fruits and surrounding environments of the crops. The text displayed on the display interface typically includes: picking information to be displayed at each stage before picking, during picking and after picking, or interaction information of operation of some 3D virtual sand table systems. The display of the 3D virtual sand table system adopts a 2D/3D blending display screen. According to the embodiment of the invention, 3D display is adopted for the image part, 2D display is adopted for the text part, 2D/3D co-fusion display is realized, the dizziness and discomfort of people for reading 3D text are overcome, meanwhile, the fidelity of the displayed stereoscopic image is improved, and the user experience is improved.
The above is a detailed description of the construction of a 3D virtual sand table system for crop harvesting.
S2, receiving growth parameters of the 3D virtual sand table system set by a user, and generating and displaying a 3D standard tree model according to the growth parameters; here, based on the 3D initial standard tree model, the growth parameters and the image rendering technology are combined (the image rendering can be used for rendering colors according to the preference of the user)
Wherein the growth parameters include, but are not limited to, parameters of crop variety, tree shape, etc., wherein the tree shape refers to a specific shape of the generated 3D standard tree, which may specifically include: circular, oval, etc. The user may enter growth parameters into the 3D virtual sand table system through a user interface. In this step, the generating and displaying the 3D standard tree model according to the growth parameters includes:
s21, acquiring and updating the 3D material model file corresponding to the growth parameter in a server.
The server stores 3D material model files of different crops; the 3D material modeling can be completed by using an Autodesk 3D max tool to standard tree type, root system, illumination and the like of crop plants; uploading the modeling-completed ".3ds" material model source file to a 3D virtual sand table system management background, and setting data such as crops, variety names, climate parameters and the like corresponding to each model file. After the client of the sand table system is started, the material data of the server corresponding to crops, varieties and attributes are automatically updated and acquired, and are cached to a local hard disk.
S22, analyzing and rendering the 3D material model file into a 3D standard tree model;
the 3ds file analysis engine is arranged in the 3D virtual sand table system, and 3D model files output by the 3D MAX can be read and analyzed and rendered into a 3D standard tree model. And generating a virtual 3D standard tree model (such as a granulated sugar orange tree generated by a granulated sugar orange) which accords with the crop variety and the tree type of the growth parameters by analyzing and rendering the corresponding 3D model materials, and adjusting the characteristics of the 3D standard tree model generated in different picking stages according to the picking period. Wherein the different picking stages comprise: three picking stages before picking, during picking and after picking.
S23, displaying the 3D standard tree model in the sand table system working area.
As shown in fig. 2, the 3D standard tree model of the citrus reticulate before picking is displayed in the 3D virtual sand table system work area.
S3, receiving picking stages and picking parameters of a user for picking the 3D standard tree model;
wherein the picking parameters include, but are not limited to, picking site, picking action location. The picking part can be the position of the geometric center of the fruit, the picking action position is the position acted by the scissors pointer when picking the fruit, and the picking action position can be the position of the root of the fruit which is in a preset range from the fruit. The invention can generate scissor-shaped pointers at the picking positions of the 3D standard tree model, and act on the picking action positions of the 3D standard tree model to perform pruning and picking operations. The picking stage comprises three stages of pre-picking, mid-picking and post-picking.
S4, judging whether the picking parameters meet the pre-stored picking parameter standards of the 3D standard tree model in the picking stage;
in the invention, picking parameter standards of different picking stages of each 3D standard tree model can be stored in advance, and whether the picking parameters accord with the pre-stored picking parameter standards of the 3D standard tree model in the picking stages or not is judged by comparing the picking parameters with the pre-stored picking parameter standards of the picking stages of the 3D standard tree model. For example, for a sugar orange tree, the picking parameter standard before picking may be set to be within a first preset range in which the picking action position is the picking part, the picking parameter standard in picking may be set to be within a second preset range in which the picking action position is the picking part, and so on. Specifically, the second preset range is set to be 1CM, if the picking action position in picking exceeds the range of 1CM of the picking part, the picking is automatically cancelled, and the picking is prompted to be not in accordance with the picking specification.
And S5, if the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage, displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters.
As in fig. 3, after one picking operation, the working area in the upper right corner of the sand table system will display a picking snapshot of the 3D standard tree model after picking.
In addition, in order to increase interestingness, an animation area which gives life to crops is arranged on a display interface of the 3D virtual sand table system, for example, a mature fruit system can send out voice when picking, namely, the mature fruit system is mature to pick me, and for immature fruits, the system can send out voice when picking, namely, the system is not mature, and the like. Therefore, the display interface of the 3D virtual sand table system divides a preset number of display areas according to the needs of users, and interaction interestingness is enhanced through combination of sound and animation.
In one embodiment of the present invention, in order to enhance the learning effect of the demonstration, the display of the 3D virtual sand table system adopts a touch display screen, on which the characters are 2D displayed, and the crops, picking related tools, etc. are 3D displayed, and the embodiment is characterized in that: the real picking environment is simulated, teaching is performed in the study, and the specific interaction control method is as follows:
after receiving a user touch instruction, starting a 3D virtual sand table system, entering a starting interface of a touch screen, and enabling a crop type dialog box to be arranged on the starting interface;
receiving a touch instruction of selecting crop types by a user, and displaying corresponding crop 3D standard tree forms;
receiving a growth stage selection instruction of a user for the selected crops;
a dialog box for popping up whether the fruits in the growing stage can be picked up;
if the picking tool can pick, the holding mode of the picking tool is displayed;
then controlling the picking tool to move to the fruits on the screen, prompting the position relation of the picking tool on the fruits, and controlling how the picking tool picks the fruits;
inputting a fruit picking instruction by touch, picking fruits, and moving the fruits to a fruit basket of a screen;
prompting how to put the fruits into the fruit basket; the problem here is mainly that fruits are easily damaged when the fruits are put into practice, and the quality of the fruits is deteriorated.
According to the prompt, the fruit is moved into the fruit basket on the screen.
In addition, the invention can realize remote wireless control learning, the 3D virtual sand table system is installed in the server, the user is connected to the server at a far end even in a crop picking site through the handheld electronic equipment terminal through a wireless network, remotely logs in the 3D virtual sand table system to watch picking demonstration, and meanwhile, the handheld electronic equipment terminal and the 3D virtual sand table system are used for real-time interaction.
With a great deal of application of the augmented reality technology, the embodiment of the invention can also combine the 3D virtual sand table system with the augmented reality technology, and the crop picking demonstration method based on the 3D model further comprises the following steps:
displaying the collected crops and fruits on the crops and environmental images thereof in real time;
identifying crop species;
the 3D sand table system is pre-stored with various crops and picking modes thereof, and a 3D standard tree model with approximate current growth conditions and a 3D sand table with similar environments; it should be noted that, the picking mode of apples is different from the picking mode of grapes, litchi and the like, and the picking mode of some hard shell fruits such as walnut and the like is also different from the previous picking mode.
The 3D virtual sand table system demonstrates picking videos of the crops;
prompting a user wearing the AR equipment to watch a picking video through a display screen of the AR equipment, guiding the user to conduct picking operation on crops according to the video, collecting pictures picked on site by the user while demonstrating, prompting the user to improve the operation if the picked pictures are greatly different from picked pictures prestored in the 3D virtual sand table system until the picking action of the user meets the requirements, thus avoiding the problem that the picking is not standardized and the damage to the fruits is avoided, and improving the picking technical level and the fruit quality of the user.
According to the crop picking demonstration method based on the 3D model, a 3D virtual sand table system is adopted to replace a traditional operation teaching mode of picking, so that intuitiveness and interactivity of a crop picking process are improved, and teaching demonstration cost is saved; and automatically calling a built-in material library by the 3D virtual sand table system to generate a 3D standard tree by using picking parameters set by a user. Virtual picking operations in different picking periods can be carried out on the 3D standard tree, and picking snapshots of the 3D standard tree model after picking are displayed by judging whether the picking parameters of each picking stage meet the picking parameter standard of the current crop in the growing period. Compared with the prior art, the invention has the following beneficial effects:
1. the intuitiveness and the interactivity of the crop picking process are improved, and the teaching demonstration cost is saved;
2. crops and varieties which need to be operated can be selected, and the limitation of single variety in the traditional picking teaching mode is broken through;
3. virtual picking operations can be performed through the 3D sand table system and can prompt and guide picking stages, picking sites and notes.
4. The 3D standard tree is moved, picked and withdrawn by the button of "move, pick and withdraw".
The invention also provides another crop picking demonstration method based on a 3D model, which is different from the method in FIG. 1 in that:
between the steps S2 and S3, the method further comprises:
s21, displaying picking stages of the 3D standard tree model, picking parameters of each picking stage and notes of each picking stage.
Wherein the picking stage comprises pre-picking, mid-picking and post-picking. As shown in fig. 2, before picking, picking parameters and notes before picking are prompted at the bottom of the 3D sand table system workspace.
After the step S5, the method further includes:
s51, after each picking, adjusting a 3D standard tree model according to the picking parameters of the picking to display the actual effect of the picking.
Referring specifically to fig. 3, the remaining fruit condition of the 3D standard tree model after this picking is displayed in the 3D virtual sand table system, and at the same time, the picking snapshot of the 3D standard tree model after picking is displayed in the working area in the upper right corner.
And S52, when receiving an undo instruction of a user, undo the picking operation of the 3D standard tree model last time.
As shown in fig. 3, the 3d virtual sand table system workspace shows: moving, picking and cancelling, wherein a user clicks a cancelling button, so that the picking operation of the 3D standard tree model can be cancelled last time; the user clicks the move button to move the current 3D standard tree, and after the user inputs the picking parameters, the user clicks the pick button to pick the 3D standard tree.
S53, recording picking stages and picking parameters of a user, and inputting the picking stages and the picking parameters into a deep learning model for optimization; when the user logs in again, the optimized picking phase and picking parameters are acquired and displayed.
In the invention, a Google TensorFlow AI framework can be adopted for deep learning and algorithm optimization.
And S54, when the user finishes the operation, storing the 3D standard tree model so as to finish printing of the 3D standard tree model.
Fig. 4 is a 3D model-based crop picking demonstration device provided by the present invention, the device includes:
a construction module 41 for constructing a 3D virtual sand table system for crop picking;
the receiving display module 42 is configured to receive a growth parameter of the 3D virtual sand table system set by a user, and generate and display a 3D standard tree model according to the growth parameter;
a receiving module 43, configured to receive picking stages and picking parameters of a user picking the 3D standard tree model;
a judging module 44, configured to judge whether the picking parameter meets a pre-stored picking parameter standard of the 3D standard tree model in the picking stage;
the display module 45 is configured to, if the picking parameters meet pre-stored picking parameter standards of the 3D standard tree model in the picking stage; and displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters.
In one embodiment, the receiving display module 42 includes:
the acquisition module is used for acquiring and updating the 3D material model files corresponding to the picking parameters in the server; 3D material model files of different crops are stored in the server;
the analysis rendering module is used for resolving and rendering the 3D material model file into a 3D standard tree model;
and the sub-display module is used for displaying the 3D standard tree model in the working area of the sand table system.
Further, the device further comprises:
and the first display module is used for displaying the picking stages of the 3D standard tree model, the picking parameters of each picking stage and the notes of each picking stage.
And the second display module is used for adjusting the 3D standard tree model to display the actual effect after the picking according to the picking parameters of the picking after each picking.
And the undo module is used for undoing the picking operation of the 3D standard tree model last time when receiving the undo instruction of the user.
The optimizing module is used for recording picking stages and picking parameters of a user and inputting the picking stages and the picking parameters into the deep learning model for optimization;
and the third display module is used for acquiring and displaying the optimized picking stage and picking parameters when the user logs in again.
And the storage module is used for storing the 3D standard tree model when the user finishes the operation so as to finish printing of the 3D standard tree model.
The invention also provides a crop picking demonstration system based on the 3D model, which comprises a server and any crop picking demonstration device based on the 3D model.
In addition, the crop picking demonstration method based on the 3D model according to the embodiment of the present invention described in connection with fig. 1 may be implemented by a crop picking demonstration apparatus based on the 3D model. Fig. 5 shows a schematic hardware structure of a crop picking demonstration device based on a 3D model according to an embodiment of the present invention.
The crop picking presentation device based on the 3D model may comprise at least one processor 401, at least one memory 402 and computer program instructions stored in the memory 402, which when executed by the processor 401, implement the method described in the present embodiment.
In particular, the processor 401 described above may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured as one or more integrated circuits implementing embodiments of the present invention.
Memory 402 may include mass storage for data or instructions. By way of example, and not limitation, memory 402 may comprise a Hard Disk Drive (HDD), floppy Disk Drive, flash memory, optical Disk, magneto-optical Disk, magnetic tape, or universal serial bus (Universal Serial Bus, USB) Drive, or a combination of two or more of the foregoing. Memory 402 may include removable or non-removable (or fixed) media, where appropriate. Memory 402 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 402 is a non-volatile solid state memory. In a particular embodiment, the memory 402 includes Read Only Memory (ROM). The ROM may be mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these, where appropriate.
The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement the crop picking presentation method based on the 3D model in the above embodiment.
In one example, the 3D model-based crop harvesting presentation apparatus may further include a communication interface 403 and a bus 410. As shown in fig. 5, the processor 401, the memory 402, and the communication interface 403 are connected by a bus 410 and perform communication with each other.
The communication interface 403 is mainly used to implement communication between each module, device, unit and/or apparatus in the embodiment of the present invention.
Bus 410 includes hardware, software, or both, coupling components of the 3D model-based crop harvesting presentation apparatus to each other. By way of example, and not limitation, the buses may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. Bus 410 may include one or more buses, where appropriate. Although embodiments of the invention have been described and illustrated with respect to a particular bus, the invention contemplates any suitable bus or interconnect.
In addition, in combination with the crop picking demonstration method based on the 3D model in the above embodiment, the embodiment of the invention can be implemented by providing a computer readable storage medium. The computer readable storage medium has stored thereon computer program instructions; the computer program instructions, when executed by the processor, implement any of the 3D model-based crop harvesting demonstration methods of the above embodiments.
In summary, the crop picking demonstration method, device and system based on the 3D model provided by the invention adopts the 3D virtual sand table system to replace the traditional operation teaching mode of picking, improves the intuitiveness and interactivity of the crop picking process, and saves the teaching demonstration cost; and automatically calling a built-in material library by the 3D virtual sand table system to generate a 3D standard tree by using picking parameters set by a user. Virtual picking operations in different picking periods can be carried out on the 3D standard tree, and picking snapshots of the 3D standard tree model after picking are displayed by judging whether the picking parameters of each picking stage meet the picking parameter standard of the current crop in the growing period. Compared with the prior art, the invention has the following beneficial effects:
1. the intuitiveness and the interactivity of the crop picking process are improved, and the teaching demonstration cost is saved;
2. crops and varieties which need to be operated can be selected, and the limitation of single variety in the traditional picking teaching mode is broken through;
3. virtual picking operations can be performed through the 3D sand table system and can prompt and guide picking stages, picking sites and notes.
4. The 3D standard tree is moved, picked and withdrawn by the button of "move, pick and withdraw".
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this disclosure describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, or may be performed in a different order from the order in the embodiments, or several steps may be performed simultaneously.
In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present invention is not limited thereto, and any equivalent modifications or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and they should be included in the scope of the present invention.
Claims (10)
1. Crop picking demonstration method based on 3D model, which is characterized by comprising the following steps:
constructing a 3D virtual sand table system for crop picking;
receiving growth parameters of a 3D virtual sand table system set by a user, and generating and displaying a 3D standard tree model according to the growth parameters;
receiving picking stages and picking parameters of a user for picking the 3D standard tree model;
judging whether the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage;
if the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage, displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters;
the 3D virtual sand table system is provided with a display of a touch display screen: receiving a user touch instruction, starting a 3D virtual sand table system, and entering a starting interface of a touch screen; receiving a touch instruction of selecting crop types by a user, and displaying corresponding crop 3D standard tree forms; receiving a growth stage selection instruction of a user for the selected crops; a dialog box for popping up whether the fruits in the growing stage can be picked up; if the picking tool can pick, the holding mode of the picking tool is displayed; controlling the picking tool to move to the fruits on the screen, prompting the position relation of the picking tool on the fruits, and controlling how the picking tool picks the fruits; inputting a fruit picking instruction by touch, picking fruits, and moving the fruits to a fruit basket of a screen; prompting how to put the fruits into the fruit basket; according to the prompt, moving fruits on the screen into the fruit basket;
the method comprises the steps of displaying collected crops, fruits on the crops and environmental images of the fruits on the crops in real time; identifying crop species; the 3D virtual sand table system is pre-stored with various crops and picking modes thereof, and the 3D standard tree model with approximate current growth conditions and the 3D sand table with similar environments; the 3D virtual sand table system demonstrates picking videos of the crops; prompting a user wearing the AR equipment to watch a picking video through a display screen of the AR equipment, guiding the user to conduct picking operation on crop fruits according to the video, collecting pictures picked on site by the user while demonstrating, and prompting the user to improve the operation until the picking action of the user meets the requirement if the picked pictures are greatly different from the picked pictures prestored in the 3D virtual sand table system.
2. The 3D model-based crop harvesting demonstration method of claim 1, wherein the generating and displaying a 3D standard tree model according to the growth parameters comprises:
acquiring and updating a 3D material model file corresponding to the growth parameters in a server; 3D material model files of different crops are stored in the server;
analyzing and rendering the 3D material model file into a 3D standard tree model;
displaying the 3D standard tree model in the sand table system working area.
3. The 3D model-based crop picking demonstration method of claim 2 wherein prior to the picking stage and picking parameters of the 3D standard tree model by the receiving user, the method further comprises:
the picking stages of the 3D standard tree model, the picking parameters of each picking stage, and the notes of each picking stage are displayed.
4. A crop picking presentation method based on a 3D model as claimed in claim 3, the method further comprising:
after each picking, adjusting a 3D standard tree model according to the picking parameters of the picking and displaying the actual effect after the picking;
and when the user finishes the operation, storing the 3D standard tree model so as to call the 3D standard tree model for 3D printing.
5. A crop picking presentation method based on a 3D model as claimed in claim 3, the method further comprising:
when an undo instruction of a user is received, the picking operation of the 3D standard tree model is undone last time.
6. A crop picking presentation method based on a 3D model as claimed in claim 3, the method further comprising:
recording picking stages and picking parameters of a user, and inputting the picking stages and the picking parameters into a deep learning model for optimization;
when the user logs in again, the optimized picking phase and picking parameters are acquired and displayed.
7. The 3D model-based crop picking presentation method as claimed in any one of claims 1 to 6, wherein the constructing a 3D virtual sand table system for crop picking further comprises:
s11, collecting topographic data of crops, and constructing a 3D electronic sand table of an area of the crops by combining with GIS;
s12, acquiring a 3D image of the crop, identifying the type of the crop, and constructing a 3D virtual sand table system containing the crop by combining the 3D electronic sand table of the environment where the crop is positioned;
s13, inputting picking information required by a user before picking and during picking and after picking;
s14, respectively performing image display processing on characters and graphics in the 3D virtual sand table system, adopting 2D display on the characters on the same display interface in the 3D virtual sand table system, and adopting 3D stereoscopic display on crops, fruits and surrounding environments of the crops.
8. Crop picking demonstration device based on 3D model, characterized in that the device comprises:
the construction module is used for constructing a 3D virtual sand table system for picking crops;
the receiving display module is used for receiving the growth parameters of the 3D virtual sand table system set by a user, and generating and displaying a 3D standard tree model according to the growth parameters;
the receiving module is used for receiving picking stages and picking parameters of a user for picking the 3D standard tree model;
the judging module is used for judging whether the picking parameters accord with pre-stored picking parameter standards of the 3D standard tree model in the picking stage;
the display module is used for judging whether the picking parameters meet the pre-stored picking parameter standards of the 3D standard tree model in the picking stage; displaying picking snapshots of the 3D standard tree model after picking according to the picking parameters;
the 3D virtual sand table system is provided with a display of a touch display screen: receiving a user touch instruction, starting a 3D virtual sand table system, and entering a starting interface of a touch screen; receiving a touch instruction of selecting crop types by a user, and displaying corresponding crop 3D standard tree forms; receiving a growth stage selection instruction of a user for the selected crops; a dialog box for popping up whether the fruits in the growing stage can be picked up; if the picking tool can pick, the holding mode of the picking tool is displayed; controlling the picking tool to move to the fruits on the screen, prompting the position relation of the picking tool on the fruits, and controlling how the picking tool picks the fruits; inputting a fruit picking instruction by touch, picking fruits, and moving the fruits to a fruit basket of a screen; prompting how to put the fruits into the fruit basket; according to the prompt, moving fruits on the screen into the fruit basket;
the method comprises the steps of displaying collected crops, fruits on the crops and environmental images of the fruits on the crops in real time; identifying crop species; the 3D virtual sand table system is pre-stored with various crops and picking modes thereof, and the 3D standard tree model with approximate current growth conditions and the 3D sand table with similar environments; the 3D virtual sand table system demonstrates picking videos of the crops; prompting a user wearing the AR equipment to watch a picking video through a display screen of the AR equipment, guiding the user to conduct picking operation on crop fruits according to the video, collecting pictures picked on site by the user while demonstrating, and prompting the user to improve the operation until the picking action of the user meets the requirement if the picked pictures are greatly different from the picked pictures prestored in the 3D virtual sand table system.
9. A 3D model based crop picking presentation system comprising a server and the 3D model based crop picking presentation device of claim 8.
10. A storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010212188.1A CN111508059B (en) | 2020-03-24 | 2020-03-24 | Crop picking demonstration method, device, system and medium based on 3D model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010212188.1A CN111508059B (en) | 2020-03-24 | 2020-03-24 | Crop picking demonstration method, device, system and medium based on 3D model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111508059A CN111508059A (en) | 2020-08-07 |
| CN111508059B true CN111508059B (en) | 2023-09-12 |
Family
ID=71877828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010212188.1A Active CN111508059B (en) | 2020-03-24 | 2020-03-24 | Crop picking demonstration method, device, system and medium based on 3D model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111508059B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103105929A (en) * | 2013-01-05 | 2013-05-15 | 北京农业信息技术研究中心 | Interactive design and experience method of virtual agricultural park and system |
| CN104067781A (en) * | 2014-06-16 | 2014-10-01 | 华南农业大学 | Virtual robot and real robot integration based picking system and method |
| CN104966430A (en) * | 2015-06-09 | 2015-10-07 | 北京农业智能装备技术研究中心 | Grape vine binding simulation training system and method |
| CN106846954A (en) * | 2017-02-09 | 2017-06-13 | 广东小天才科技有限公司 | A kind of interactive digital method and apparatus |
| CN108961909A (en) * | 2018-09-06 | 2018-12-07 | 佛山三水芦苞吉星普照智能农业科技有限公司 | It is a kind of to the miniature farm of intelligence imparted knowledge to students and its management system |
| CN208521255U (en) * | 2018-06-26 | 2019-02-19 | 杭州书微信息科技有限公司 | A kind of man-machine interactive system based on virtual reality |
| CN109635347A (en) * | 2018-11-15 | 2019-04-16 | 江苏农林职业技术学院 | A kind of overhead Strawberry in Greenhouse production of intelligent facility virtual emulation platform |
| JP2019061453A (en) * | 2017-09-26 | 2019-04-18 | 株式会社Ｎｔｔドコモ | Information processing apparatus |
-
2020
- 2020-03-24 CN CN202010212188.1A patent/CN111508059B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103105929A (en) * | 2013-01-05 | 2013-05-15 | 北京农业信息技术研究中心 | Interactive design and experience method of virtual agricultural park and system |
| CN104067781A (en) * | 2014-06-16 | 2014-10-01 | 华南农业大学 | Virtual robot and real robot integration based picking system and method |
| CN104966430A (en) * | 2015-06-09 | 2015-10-07 | 北京农业智能装备技术研究中心 | Grape vine binding simulation training system and method |
| CN106846954A (en) * | 2017-02-09 | 2017-06-13 | 广东小天才科技有限公司 | A kind of interactive digital method and apparatus |
| JP2019061453A (en) * | 2017-09-26 | 2019-04-18 | 株式会社Ｎｔｔドコモ | Information processing apparatus |
| CN208521255U (en) * | 2018-06-26 | 2019-02-19 | 杭州书微信息科技有限公司 | A kind of man-machine interactive system based on virtual reality |
| CN108961909A (en) * | 2018-09-06 | 2018-12-07 | 佛山三水芦苞吉星普照智能农业科技有限公司 | It is a kind of to the miniature farm of intelligence imparted knowledge to students and its management system |
| CN109635347A (en) * | 2018-11-15 | 2019-04-16 | 江苏农林职业技术学院 | A kind of overhead Strawberry in Greenhouse production of intelligent facility virtual emulation platform |
Non-Patent Citations (1)
| Title |
|---|
| 基于 Unity 3D果树交互虚拟修剪技术及其实现;吕萌萌 等;《农机化研究》(第4期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111508059A (en) | 2020-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109117811A (en) | A kind of system and method based on low-altitude remote sensing measuring technique estimation urban vegetation coverage rate | |
| CN111582234B (en) | Large-scale oil tea tree forest fruit intelligent detection and counting method based on UAV and deep learning | |
| CN113657469B (en) | Automatic observation method and system for woody plant waiting period based on image recognition | |
| JP6273473B2 (en) | Forest information management device | |
| CN112116514B (en) | Plant planting recommendation method, device, equipment and computer readable storage medium | |
| Röder et al. | Application of optical unmanned aerial vehicle-based imagery for the inventory of natural regeneration and standing deadwood in post-disturbed spruce forests | |
| JPWO2013030965A1 (en) | Imaging apparatus, imaging support program, information providing method, and information providing program | |
| CN108633533B (en) | Tree automatic tending method and device, electronic equipment and storage medium | |
| CN116645321B (en) | Vegetation leaf inclination angle calculation statistical method and device, electronic equipment and storage medium | |
| CN111552762A (en) | Orchard planting digital map management method and system based on fruit tree coding | |
| CN112084989A (en) | Unmanned aerial vehicle and CNN-based large-range pine wood nematode withered vertical wood intelligent detection method | |
| CN115410192A (en) | Citrus huanglongbing image acquisition and identification method, storage medium and system | |
| CN106683092B (en) | Device and method for measuring and calculating crown canopy density of blueberries | |
| CN111508059B (en) | Crop picking demonstration method, device, system and medium based on 3D model | |
| CN110162812B (en) | Target sample generation method based on infrared simulation | |
| CN110415130A (en) | Agricultural insurance Claims Resolution method, apparatus, equipment and computer readable storage medium | |
| CN111508078A (en) | Crop pruning demonstration method, device and system based on 3D model | |
| CN105929938A (en) | Information processing method and electronic device | |
| CN112115885A (en) | Fruit tree bearing branch shearing point positioning method for picking based on deep convolutional neural network | |
| CN113033386A (en) | High-resolution remote sensing image-based transmission line channel hidden danger identification method and system | |
| CN115294472A (en) | Fruit yield estimation method, model training method, equipment and storage medium | |
| TWI640952B (en) | Virtual object collection method and portable electronic device | |
| CN113723833B (en) | Method, system, terminal equipment and storage medium for evaluating quality of forestation actual results | |
| CN114359599A (en) | Image processing method, storage medium and computer terminal | |
| CN109344188B (en) | Artificial intelligence statistical method for remote sensing satellite efficiency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |