CN115797444A - Pineapple eye positioning method and device and electronic equipment - Google Patents
Pineapple eye positioning method and device and electronic equipment Download PDFInfo
- Publication number
- CN115797444A CN115797444A CN202310068581.1A CN202310068581A CN115797444A CN 115797444 A CN115797444 A CN 115797444A CN 202310068581 A CN202310068581 A CN 202310068581A CN 115797444 A CN115797444 A CN 115797444A
- Authority
- CN
- China
- Prior art keywords
- pineapple
- point cloud
- eye
- cloud data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 241000234671 Ananas Species 0.000 title claims abstract description 315
- 235000007119 Ananas comosus Nutrition 0.000 title claims abstract description 315
- 238000000034 method Methods 0.000 title claims abstract description 55
- 235000013399 edible fruits Nutrition 0.000 claims abstract description 100
- 238000001914 filtration Methods 0.000 claims abstract description 25
- 230000011218 segmentation Effects 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 14
- 230000004807 localization Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 235000019640 taste Nutrition 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a pineapple eye positioning method and device and electronic equipment, and belongs to the technical field of data identification. The method comprises the following steps: shooting RGB images and depth images of peeled pineapples from multiple angles; obtaining a target RGB image and a target depth image of the pineapple part from the RGB image and the depth image; acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image; carrying out rotary splicing on the point cloud data of the pineapples at multiple angles to obtain a reconstructed point cloud model of the pineapples, wherein the reconstructed point cloud model comprises the integral point cloud data of the pineapples and the point cloud data of fruit eyes on the pineapples; filtering error points in the point cloud data of the fruit eyes on the pineapples to obtain point cloud data of target fruit eyes; and determining the three-dimensional coordinates of the fruit eyes on the pineapples according to the point cloud data of the target fruit eyes. The method can be used for positioning the fruit eyes on the pineapples, and is rapid in positioning and high in positioning accuracy.
Description
Technical Field
The invention relates to the technical field of data identification, in particular to a pineapple eye positioning method and device and electronic equipment.
Background
The pineapple is a common tropical fruit in daily life, although the pineapple tastes delicious, the pineapple is fussy in processing work before eating, particularly in an eye removing link because the skin of the pineapple is a bud-shaped hard skin and is spirally arranged, and each bud is provided with a deeper 'fruit eye' or 'black core'.
At present, the manual removal of pineapple eyes by a cutter is the most main and most common means for people, but the manual removal of the pineapple eyes by the cutter is troublesome, the eyes are removed along the spiral direction of the pineapple eyes by manually operating a V-shaped cutter, time and labor are wasted, and the hands are easily injured by the cutter. There are also many devices for removing the eyes of pineapples, but the positions of the eyes are different for different pineapples, so that the accurate determination is difficult, and thus the devices cause great waste when removing the eyes, and the removing effect of the eyes is poor.
Disclosure of Invention
The invention provides a pineapple eye positioning method, a pineapple eye positioning device and electronic equipment, which are used for solving the defects that the pineapple eye is difficult to position or inaccurate in positioning when a machine device is adopted to remove the pineapple eye in the prior art, and realize the pineapple eye positioning method with high accuracy and high positioning speed.
The invention provides a pineapple eye positioning method, which comprises the following steps:
shooting RGB images and depth images of peeled pineapples from multiple angles;
obtaining a target RGB image and a target depth image of a pineapple part from the RGB image and the depth image;
acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image;
carrying out rotary splicing on the point cloud data of the pineapple at multiple angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises the integral point cloud data of the pineapple and the point cloud data of fruit eyes on the pineapple;
filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye;
and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
According to the invention, the method for positioning the pineapple eye comprises the following steps of shooting RGB images and depth images of peeled pineapples from a plurality of angles:
peeling the pineapple;
taking the axis of the pineapple as an axis, and shooting RGB images and depth images of the peeled pineapple from a plurality of rotation angles by adopting a depth camera around the axis of the pineapple, wherein included angles among a plurality of angles are equal.
According to the pineapple eye positioning method provided by the invention, the target RGB image and the target depth image of the pineapple part are obtained from the RGB image and the depth image, and the method comprises the following steps:
inputting RGB images of the pineapple, which are shot from multiple angles, into a pre-trained example segmentation network model, so as to obtain a target RGB image of the pineapple part;
and obtaining a target depth image corresponding to the pineapple part from the depth image according to the target RGB image.
According to the pineapple eye positioning method provided by the invention, the training method of the example segmentation network model comprises the following steps:
establishing an initial instance segmentation network model;
establishing a training set, wherein the training set comprises an RGB image of the peeled pineapple and label information, and the label information comprises contour area label information of the pineapple and area label information of fruit eyes on the pineapple;
and training the initial example segmentation network model by adopting a training set to obtain an example segmentation network model.
According to the pineapple eye positioning method provided by the invention, the filtering of the error points in the point cloud data of the pineapple eyes on the pineapple to obtain the point cloud data of the target eyes comprises the following steps:
righting the reconstructed point cloud model of the pineapple;
projecting the mass center of the point cloud of the fruit eye on the pineapple to a two-dimensional polar coordinate system;
fitting a straight line projected by a spiral line of the pineapple on a two-dimensional polar coordinate according to the point cloud center of mass of the fruit eye on the pineapple;
acquiring the distance between the point cloud centroid of each fruit eye on the pineapple and the nearest straight line;
and when the distance between the point cloud centroid of the fruit eye and the nearest straight line is greater than a preset distance threshold value, determining that the fruit eye is an error point and filtering.
According to the pineapple eye positioning method provided by the invention, the alignment of the reconstructed point cloud model of the pineapple comprises the following steps:
calculating the axis direction of the reconstructed point cloud model based on the integral point cloud data of the pineapple;
determining the relative position of the reference surface of the pineapple and the fruit eye on the pineapple and the pose of the reconstructed point cloud model according to the axis direction of the reconstructed point cloud model;
and correcting the reconstructed point cloud model according to the relative position of the fruit eye on the pineapple and the reference surface of the pineapple and the pose of the reconstructed point cloud model.
According to the pineapple eye positioning method provided by the invention, before the point cloud data of a plurality of angles of the pineapple are subjected to rotation splicing to obtain the reconstructed point cloud model of the pineapple, the method further comprises the following steps:
and filtering out the point cloud of the pineapple eye outside the outline range of the pineapple in the point cloud data of the plurality of angles of the pineapple.
The invention also provides a pineapple eye positioning device, which comprises:
the shooting module is used for shooting RGB images and depth images of peeled pineapples from a plurality of angles;
the target acquisition module is used for acquiring a target RGB image and a target depth image of the pineapple part from the RGB image and the depth image;
the point cloud data acquisition module is used for acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image;
the splicing module is used for carrying out rotary splicing on the point cloud data of multiple angles of the pineapple to obtain a reconstructed point cloud model of the pineapple, and the reconstructed point cloud model comprises the whole point cloud data of the pineapple and the point cloud data of fruit eyes on the pineapple;
the filtering module is used for filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye;
and the determining module is used for determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the pineapple eye positioning method.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a pineapple eye localization method as described in any one of the above.
The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements any of the above-described pineapple eye positioning methods.
According to the method, the device and the electronic equipment for positioning the pineapple eyes, the reconstructed point cloud model of the pineapple is obtained by collecting the RGB images and the depth images of the peeled pineapple at all angles, the three-dimensional coordinate data of the pineapple eyes in the reconstructed point cloud model is obtained after the point cloud data of the wrong pineapple eyes are filtered, and therefore the pineapple eyes on the pineapple are positioned.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a pineapple eye positioning method provided by the present invention;
FIG. 2 is a point cloud effect diagram of pineapple eyes in the reconstructed point cloud model provided by the invention;
FIG. 3 is a schematic flow chart of a method of the present invention for capturing RGB images and depth images of peeled pineapples from multiple angles;
FIG. 4 is a schematic view of the present invention providing a peeled pineapple image taken at 6 rotation angles;
FIG. 5 is a schematic flow chart of a method for obtaining a target RGB image and a target depth image according to the present invention;
FIG. 6 is a schematic diagram of a target RGB image provided by the present invention;
FIG. 7 is a flowchart illustrating a method for training an example segmented network model provided by the present invention;
FIG. 8 is a schematic flow chart of a method for obtaining point cloud data of a target fruit eye according to the present invention;
FIG. 9 is a diagram of the centroid distribution of pineapple eyes in a two-dimensional polar coordinate system provided by the present invention;
FIG. 10 is a schematic flow chart of a method for rectifying a reconstructed point cloud model of a pineapple according to the present invention;
FIG. 11 is a schematic view of the device for positioning pineapple eye according to the present invention;
fig. 12 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A pineapple eye positioning method of the present invention is described below with reference to fig. 1 to 10, and the method includes:
s101: RGB images and depth images of peeled pineapples were taken from multiple angles.
Illustratively, since the pineapple is shaped like a cylinder, it is necessary to take RGB images and depth images of the peeled pineapple from multiple angles to obtain a complete RGB image and depth image of the peeled pineapple.
Depth Images (Depth Images), also known as Range Images (Range Images), refer to Images that take as pixel values the distance (Depth) values of points in a scene captured by an image capture device, which directly reflect the geometry of the visible surface of the scene.
When the rotation shooting is carried out by taking the axis of the pineapple core of the pineapple as an axis, the included angle between two adjacent shooting angles is preferably less than or equal to 60 degrees, so that images of the pineapple at all angles can be completely acquired.
S102: and obtaining a target RGB image and a target depth image of the pineapple part from the RGB image and the depth image.
Specifically, the RGB image and the depth image of the pineapple photographed at multiple angles include a background image in addition to the pineapple image. Therefore, it is necessary to obtain only the target RGB image and the target depth image of the pineapple portion from the RGB image and the depth image.
S103: and acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image.
Illustratively, the point cloud data of the pineapple at multiple angles can be obtained by respectively combining the target RGB image and the target depth image obtained at each angle. For example, combining a target RGB image obtained from a first angle with a target depth image to obtain point cloud data of the first angle; combining the target RGB image and the target depth image obtained from the second angle to obtain point cloud data of the second angle; and by analogy, point cloud data of a plurality of angles of the peeled pineapples are obtained.
S104: and rotationally splicing the point cloud data of the pineapple at multiple angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises the integral point cloud data of the pineapple and the point cloud data of the fruit eye on the pineapple.
Specifically, the obtained point cloud data of the peeled pineapples at a plurality of angles are spliced according to the sequence of the plurality of shooting angles and included angles among the plurality of shooting angles, so that a reconstructed point cloud model of the pineapples can be obtained, and the reconstructed point cloud model comprises integral point cloud data of the pineapples and point cloud data of fruit eyes on the pineapples. Referring to fig. 2, fig. 2 is a point cloud effect diagram of pineapple eye in the reconstructed point cloud model.
S105: and filtering error points in the point cloud data of the fruit eye on the pineapple to obtain the point cloud data of the target fruit eye.
It can be understood that some interference points may be mistakenly identified when reconstructing the point cloud model, so that the interference points are regarded as the fruit eyes of the pineapple. Therefore, after the point cloud data of the fruit eye on the pineapple is reconstructed, the mistakenly identified interference point needs to be filtered, and the point cloud data of the target fruit eye is obtained after filtering, so that the accuracy of the fruit eye on the pineapple is improved.
S106: and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
After the point cloud data of the target fruit eye is obtained, the three-dimensional coordinate of the fruit eye on the pineapple can be determined, and the positioning information of the fruit eye can be obtained.
According to the pineapple eye positioning method provided by the invention, the reconstructed point cloud model of the pineapple is obtained by collecting RGB images and depth images of all angles of the peeled pineapple, and three-dimensional coordinate data of the pineapple eye in the reconstructed point cloud model is obtained after filtering out the point cloud data of the wrong pineapple eye, so that the pineapple eye on the pineapple is positioned.
In one embodiment, the capturing of RGB images and depth images of peeled pineapples from a plurality of angles comprises:
s301: peeling the pineapple.
S302: taking the axis of the pineapple as an axis, and shooting RGB images and depth images of the peeled pineapple from a plurality of rotation angles by adopting a depth camera around the axis of the pineapple, wherein included angles among a plurality of angles are equal.
Specifically, the axis of the pineapple is also the axis of the pineapple core, and the depth camera is adopted to perform rotation shooting around the axis of the pineapple, so that the RGB images and the depth images of the peeled pineapple can be shot from a plurality of rotation angles.
Exemplarily, referring to fig. 4, the RGB image and the depth image of the peeled pineapple can be taken specifically from 6 rotation angles, and the included angles between the 6 angles are the same, each 60 °. When the shooting angle is smaller, it may be difficult to obtain the entire image information of the pineapple, and when the shooting angle is larger, it may increase the data processing difficulty in the whole process.
The algorithm for obtaining the reconstructed point cloud model of the pineapple by performing rotation splicing on the point cloud data of the 6 angles is as follows:
the point cloud centroids of all pineapple eyes corresponding to the nth (the value of n is 0-5) image are as follows: pcl: pointXYZ pic _ n _ group.
The number of pineapple eyes is as follows: num = pic _ n _ closed.
The reconstructed point cloud corresponding to the nth image is as follows:
pcl::PointXYZ pic_n_cloud_restruct。
after the point cloud model is reconstructed, the barycentric coordinate of the kth fruit eye of the pineapple is as follows:
in one embodiment, the obtaining a target RGB image and a target depth image of a pineapple portion from the RGB image and the depth image includes:
s501: inputting RGB images of the pineapple, which are shot from multiple angles, into a pre-trained example segmentation network model, so as to obtain a target RGB image of the pineapple part.
Specifically, the pre-trained example segmentation network model is input to RGB images of the pineapple photographed from multiple angles, and as shown in fig. 6, a target RGB image of the pineapple portion can be obtained, where the target RGB image includes an image of the outline of the pineapple and an image of black dots of the pineapple.
S502: and obtaining a target depth image corresponding to the pineapple part from the depth image according to the target RGB image.
Specifically, according to the target RGB image, a depth value corresponding to each pixel point on the target RGB image is screened from the depth image, so as to obtain the target depth image.
In one embodiment, the training method of the example segmentation network model comprises the following steps:
s701: and establishing an initial instance segmentation network model.
Specifically, the initial instance segmentation network model may be a blendmask based instance segmentation model of a deep neural network.
S702: establishing a training set, wherein the training set comprises RGB images of the peeled pineapples and label information, and the label information comprises contour area label information of the pineapples and area label information of fruit eyes on the pineapples.
And establishing a training set based on the established initial example segmentation network model, wherein the training set comprises the collected RGB images of the peeled pineapples, and the collected RGB images of the pineapples can be expanded by adopting image enhancement to obtain more training data. The label information may be manually labeled, and specifically, the outline area of the pineapple on the RGB image of the pineapple and the area of the fruit eye on the pineapple may be labeled by online labeling software.
S703: and training the initial example segmentation network model by adopting a training set to obtain an example segmentation network model.
Specifically, the initial instance segmentation network model is trained based on a training set to obtain an instance segmentation network model. And training the initial example segmentation network model based on a training set, wherein the RGB image of the pineapple is used as input during training, and the estimated RGB image of the pineapple part is output. And establishing a loss function according to the error between the estimated RGB image and the label information, optimizing the loss function, and updating the parameters of the initial instance segmentation network model until convergence to obtain the instance segmentation network model.
In one embodiment, the filtering out the error points in the point cloud data of the fruit eye on the pineapple to obtain the point cloud data of the target fruit eye comprises:
s801: and righting the reconstructed point cloud model of the pineapple.
S802: and projecting the mass center of the point cloud of the fruit eye on the pineapple to a two-dimensional polar coordinate system.
Specifically, the centroid of the three-dimensional point cloud of the pineapple eye on the reconstructed point cloud model is projected to the two-dimensional polar coordinate system, so as to obtain the pineapple eye centroid distribution map as shown in fig. 9.
For example, the algorithm for projecting the three-dimensional point cloud centroid of the pineapple eye on the coordinate of the z-theta two-dimensional polar coordinate system is as follows:
solving the point cloud of the ith pineapple eye: pcl: pointXYZ eye _ i _ group.
The number of points constituting the fruit eye point cloud size = eye _ i _ closed.
The x-coordinate of the centroid center _ i _ x = sum (eye _ i _ closed. Point [ j ]. X for j in range (0, size))/size;
the y-coordinate of the centroid center _ i _ y = sum (eye _ i _ closed. Point [ j ]. Y for j in range (0, size))/size;
the z-coordinate of the centroid center _ i _ z = sum (eye _ i _ closed. Point [ j ]. Z for j range (0, size))/size.
Then, the polar coordinates (center _ i _ θ, center _ i _ z) of the ith point cloud centroid are:
center_i_θ = arctan(center_i_y/center_i_x)；
center_i_z = center_i_z。
s803: and fitting a straight line projected by the spiral line of the pineapple on a two-dimensional polar coordinate according to the point cloud center of mass of the fruit eye on the pineapple.
Specifically, it is known that the fruit eye of the pineapple is basically located on the spiral line of the pineapple, and the spiral line is a straight line when projected to a two-dimensional polar coordinate. Therefore, the straight line projected by the spiral line of the pineapple on the two-dimensional polar coordinate can be fitted based on the approximate slope range of the straight line under the two-dimensional polar coordinate according to the position of the point cloud centroid of the fruit eye.
S804: and acquiring the distance between the point cloud centroid of each fruit eye on the pineapple and the nearest straight line.
It will be appreciated that depending on the configuration of the pineapple, the eyes of the pineapple should lie mostly on a line where the helix is projected on a two-dimensional polar coordinate, and the individual eyes not lying on a line should also be at a position close to the line.
And calculating the distance between the point cloud centroid of each fruit eye on the pineapple and the nearest straight line based on the properties of the fruit eye on the pineapple.
S805: and when the distance between the point cloud centroid of the fruit eye and the nearest straight line is greater than a preset distance threshold value, determining the fruit eye as an error point and filtering.
Specifically, the fruit eye is not wrong when the distance between the point cloud centroid of the fruit eye on the pineapple and the nearest straight line is smaller than a preset distance threshold, and the fruit eye is wrong when the distance between the point cloud centroid of the fruit eye on the pineapple and the nearest straight line is larger than the preset distance threshold, and the fruit eye needs to be filtered.
In one embodiment, the rectifying the reconstructed point cloud model of the pineapple comprises:
s1001: and calculating the axis direction of the reconstructed point cloud model based on the integral point cloud data of the pineapple.
And (4) reconstructing the axis direction of the point cloud model, namely the direction of the axis of the pineapple core on the point cloud model.
S1002: and determining the relative position of the fruit eye on the pineapple and the reference surface of the pineapple and the pose of the reconstructed point cloud model according to the axis direction of the reconstructed point cloud model.
Specifically, the reference surface of the pineapple can be the upper top surface or the lower top surface of the pineapple, and the relative position of the upper eye of the pineapple and the reference surface of the pineapple can be determined according to the axis direction of the reconstructed point cloud model. Based on the axis direction of the reconstructed point cloud model, the pose of the reconstructed point cloud model can be obtained through a Principal Component Analysis (PCA) algorithm.
S1003, carrying out: and correcting the reconstructed point cloud model according to the relative position of the fruit eye on the pineapple and the reference surface of the pineapple and the pose of the reconstructed point cloud model.
In one embodiment, before the rotating and splicing the point cloud data of multiple angles of the pineapple to obtain the reconstructed point cloud model of the pineapple, the method further comprises:
and filtering out the point cloud of the pineapple eye outside the outline range of the pineapple in the point cloud data of the plurality of angles of the pineapple.
Specifically, after point cloud data of multiple angles of the pineapple are obtained according to the target RGB image and the target depth image, the point cloud data of the multiple angles are subjected to rotation splicing in the step, and before a reconstructed point cloud model of the pineapple is obtained. Points out of the outline range of the pineapple in the multiple point cloud data of the pineapple can be screened out for filtering. It can be understood that the points outside the pineapple contour are outliers, and filtering the outliers can improve the accuracy of reconstructing the point cloud model.
The following describes the pineapple eye positioning device provided by the present invention, and the pineapple eye positioning device described below and the pineapple eye positioning method described above can be referred to correspondingly. This pineapple eye positioner includes:
a shooting module 1101 for shooting the RGB image and the depth image of the peeled pineapple from a plurality of angles.
A target obtaining module 1102, configured to obtain a target RGB image and a target depth image of the pineapple portion from the RGB image and the depth image.
A point cloud data obtaining module 1103, configured to obtain point cloud data of multiple angles of the pineapple according to the target RGB image and the target depth image.
And the splicing module 1104 is used for performing rotary splicing on the point cloud data of multiple angles of the pineapple to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises integral point cloud data of the pineapple and point cloud data of fruit eyes on the pineapple.
And a filtering module 1105, configured to filter out an error point in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of the target fruit eye.
A determining module 1106, configured to determine a three-dimensional coordinate of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
According to the pineapple eye positioning device, the reconstructed point cloud model of the pineapple is obtained by collecting RGB images and depth images of all angles of the peeled pineapple, three-dimensional coordinate data of the pineapple eye in the reconstructed point cloud model is obtained after point cloud data of the wrong pineapple eye is filtered, and therefore the pineapple eye on the pineapple is positioned.
Fig. 12 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 12: a processor (processor) 1210, a communication Interface (Communications Interface) 1220, a memory (memory) 1230, and a communication bus 1240, wherein the processor 1210, the communication Interface 1220, and the memory 1230 communicate with each other via the communication bus 1240. Processor 1210 may invoke logic instructions in memory 1230 to perform a pineapple eye location method comprising: shooting RGB images and depth images of peeled pineapples from multiple angles; obtaining a target RGB image and a target depth image of a pineapple part from the RGB image and the depth image; acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image; carrying out rotary splicing on the point cloud data of the multiple angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises the integral point cloud data of the pineapple and the point cloud data of fruit eyes on the pineapple; filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye; and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
Furthermore, the logic instructions in the memory 1230 described above can be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, a computer is capable of executing the pineapple eye positioning method provided by the above methods, the method includes: shooting RGB images and depth images of peeled pineapples from a plurality of angles; obtaining a target RGB image and a target depth image of a pineapple part from the RGB image and the depth image; acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image; carrying out rotary splicing on the point cloud data of the plurality of angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises integral point cloud data of the pineapple and point cloud data of fruit eyes on the pineapple; filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye; and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the pineapple eye positioning method provided by the above methods, the method including: shooting RGB images and depth images of peeled pineapples from a plurality of angles; obtaining a target RGB image and a target depth image of a pineapple part from the RGB image and the depth image; acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image; carrying out rotary splicing on the point cloud data of the plurality of angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises integral point cloud data of the pineapple and point cloud data of fruit eyes on the pineapple; filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye; and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on the understanding, the above technical solutions substantially or otherwise contributing to the prior art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A pineapple eye positioning method is characterized by comprising the following steps:
shooting RGB images and depth images of peeled pineapples from multiple angles;
obtaining a target RGB image and a target depth image of a pineapple part from the RGB image and the depth image;
acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image;
carrying out rotary splicing on the point cloud data of the pineapple at multiple angles to obtain a reconstructed point cloud model of the pineapple, wherein the reconstructed point cloud model comprises integral point cloud data of the pineapple and point cloud data of fruit eyes on the pineapple;
filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye;
and determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
2. The method of claim 1, wherein the capturing of RGB images and depth images of peeled pineapples from a plurality of angles comprises:
peeling the pineapple;
taking the axis of the pineapple as an axis, and shooting RGB images and depth images of the peeled pineapple from a plurality of rotation angles by adopting a depth camera around the axis of the pineapple, wherein included angles among a plurality of angles are equal.
3. The pineapple eye positioning method of claim 1, wherein the obtaining of the target RGB image and the target depth image of the pineapple portion from the RGB image and the depth image comprises:
inputting RGB images of the pineapple, which are shot from multiple angles, into a pre-trained example segmentation network model so as to obtain a target RGB image of the pineapple part;
and obtaining a target depth image corresponding to the pineapple part from the depth image according to the target RGB image.
4. The pineapple eye positioning method of claim 3, wherein the training method of the example segmentation network model comprises:
establishing an initial instance segmentation network model;
establishing a training set, wherein the training set comprises RGB images of peeled pineapples and label information, and the label information comprises contour area label information of the pineapples and area label information of fruit eyes on the pineapples;
and training the initial example segmentation network model by adopting a training set to obtain an example segmentation network model.
5. The method for locating the pineapple eye of claim 1, wherein said filtering out the error points in the point cloud data of the pineapple eye to obtain the point cloud data of the target pineapple eye comprises:
righting the reconstructed point cloud model of the pineapple;
projecting the mass center of the point cloud of the fruit eye on the pineapple to a two-dimensional polar coordinate system;
fitting a straight line projected by a spiral line of the pineapple on a two-dimensional polar coordinate according to the point cloud center of mass of the fruit eye on the pineapple;
acquiring the distance between the point cloud centroid of each fruit eye on the pineapple and the nearest straight line;
and when the distance between the point cloud centroid of the fruit eye and the nearest straight line is greater than a preset distance threshold value, determining the fruit eye as an error point and filtering.
6. The pineapple eye positioning method of claim 5, wherein the aligning the reconstructed point cloud model of the pineapple comprises:
calculating the axis direction of the reconstructed point cloud model based on the integral point cloud data of the pineapple;
determining the relative position of the fruit eye on the pineapple and the reference surface of the pineapple and the pose of the reconstructed point cloud model according to the axis direction of the reconstructed point cloud model;
and correcting the reconstructed point cloud model according to the relative position of the fruit eye on the pineapple and the reference surface of the pineapple and the pose of the reconstructed point cloud model.
7. The pineapple eye positioning method of any one of claims 1 to 6, wherein before the rotating and stitching the point cloud data of a plurality of angles of the pineapple to obtain the reconstructed point cloud model of the pineapple, the method further comprises:
and filtering out the point cloud of the pineapple eye outside the outline range of the pineapple in the point cloud data of the plurality of angles of the pineapple.
8. A pineapple eye positioning device, comprising:
the shooting module is used for shooting RGB images and depth images of peeled pineapples from a plurality of angles;
the target acquisition module is used for acquiring a target RGB image and a target depth image of the pineapple part from the RGB image and the depth image;
the point cloud data acquisition module is used for acquiring point cloud data of a plurality of angles of the pineapple according to the target RGB image and the target depth image;
the splicing module is used for carrying out rotary splicing on the point cloud data of the pineapple at multiple angles to obtain a reconstructed point cloud model of the pineapple, and the reconstructed point cloud model comprises integral point cloud data of the pineapple and point cloud data of fruit eyes on the pineapple;
the filtering module is used for filtering error points in the point cloud data of the fruit eye on the pineapple to obtain point cloud data of a target fruit eye;
and the determining module is used for determining the three-dimensional coordinates of the fruit eye on the pineapple according to the point cloud data of the target fruit eye.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the pineapple eye positioning method of any one of claims 1 to 7.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the pineapple eye localization method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310068581.1A CN115797444A (en) | 2023-02-06 | 2023-02-06 | Pineapple eye positioning method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310068581.1A CN115797444A (en) | 2023-02-06 | 2023-02-06 | Pineapple eye positioning method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115797444A true CN115797444A (en) | 2023-03-14 |
Family
ID=85430017
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310068581.1A Pending CN115797444A (en) | 2023-02-06 | 2023-02-06 | Pineapple eye positioning method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115797444A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830272A (en) * | 2018-08-03 | 2018-11-16 | 中国农业大学 | Potato image collecting device and bud eye based on RGB-D camera identify and position method |
| CN111955757A (en) * | 2020-07-24 | 2020-11-20 | 上海云喷餐饮管理有限公司 | Method and equipment for peeling fruits and vegetables |
| EP3809363A1 (en) * | 2019-10-15 | 2021-04-21 | Continental Automotive GmbH | Method and device for providing a surround view image, and vehicle |
| CN113412954A (en) * | 2021-08-25 | 2021-09-21 | 中国科学院自动化研究所 | Automatic pineapple eye removing device and automatic pineapple eye removing method |
| CN114638974A (en) * | 2022-03-29 | 2022-06-17 | 中冶赛迪重庆信息技术有限公司 | Target object identification method, system, medium and electronic terminal |
| CN115587943A (en) * | 2022-10-09 | 2023-01-10 | 中国科学院半导体研究所 | Method and device for denoising point cloud data, electronic device and storage medium |
-
2023
- 2023-02-06 CN CN202310068581.1A patent/CN115797444A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830272A (en) * | 2018-08-03 | 2018-11-16 | 中国农业大学 | Potato image collecting device and bud eye based on RGB-D camera identify and position method |
| EP3809363A1 (en) * | 2019-10-15 | 2021-04-21 | Continental Automotive GmbH | Method and device for providing a surround view image, and vehicle |
| CN111955757A (en) * | 2020-07-24 | 2020-11-20 | 上海云喷餐饮管理有限公司 | Method and equipment for peeling fruits and vegetables |
| CN113412954A (en) * | 2021-08-25 | 2021-09-21 | 中国科学院自动化研究所 | Automatic pineapple eye removing device and automatic pineapple eye removing method |
| CN114638974A (en) * | 2022-03-29 | 2022-06-17 | 中冶赛迪重庆信息技术有限公司 | Target object identification method, system, medium and electronic terminal |
| CN115587943A (en) * | 2022-10-09 | 2023-01-10 | 中国科学院半导体研究所 | Method and device for denoising point cloud data, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110349147B (en) | Model training method, fundus macular region lesion recognition method, device and equipment | |
| JP2022051873A (en) | Eyelid shape estimation | |
| CN110263755B (en) | Eye ground image recognition model training method, eye ground image recognition method and eye ground image recognition device | |
| CN110070610B (en) | Feature point matching method, and feature point matching method and device in three-dimensional reconstruction process | |
| CN110458855B (en) | Image extraction method and related product | |
| CN110956114A (en) | Face living body detection method, device, detection system and storage medium | |
| CN109447117B (en) | Double-layer license plate recognition method and device, computer equipment and storage medium | |
| CN113296542B (en) | Aerial photography shooting point acquisition method and system | |
| CN113296543B (en) | Method and system for planning aerial route | |
| CN114240978B (en) | Cell edge segmentation method and device based on adaptive morphology | |
| CN112488067B (en) | Face pose estimation method and device, electronic equipment and storage medium | |
| KR102433313B1 (en) | Matching method, apparatus, electronic device, computer readable storage medium, and computer program | |
| CN113706472A (en) | Method, device and equipment for detecting road surface diseases and storage medium | |
| CN110942511B (en) | Indoor scene model reconstruction method and device | |
| CN111127556B (en) | Target object identification and pose estimation method and device based on 3D vision | |
| CN113792718B (en) | Method for positioning face area in depth map, electronic device and storage medium | |
| CN107705279A (en) | Realize the view data real-time processing method and device, computing device of double exposure | |
| CN115797444A (en) | Pineapple eye positioning method and device and electronic equipment | |
| CN110276333B (en) | Eye ground identity recognition model training method, eye ground identity recognition method and equipment | |
| US11337432B1 (en) | Automated crab meat picking system and method | |
| CN111178307A (en) | Gaze direction identification method and device, electronic equipment and storage medium | |
| CN115775253A (en) | Blade flash cutting method, device, equipment, storage medium and robot | |
| CN113077021A (en) | Machine learning-based electronic medical record multidimensional mining method | |
| CN115238307A (en) | Face privacy protection method and device for intelligent driving and intelligent cabin images | |
| Zhang et al. | An External Contour Extraction Method for Pollen Images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |