CN109544572B - Method for acquiring near-large fruit target in orchard image - Google Patents
Method for acquiring near-large fruit target in orchard image Download PDFInfo
- Publication number
- CN109544572B CN109544572B CN201811373577.1A CN201811373577A CN109544572B CN 109544572 B CN109544572 B CN 109544572B CN 201811373577 A CN201811373577 A CN 201811373577A CN 109544572 B CN109544572 B CN 109544572B
- Authority
- CN
- China
- Prior art keywords
- image
- area
- fruit
- radius
- color difference
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Manipulator (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for acquiring a near-large fruit target in an orchard image, which comprises the following steps: 1. RGB image acquisition; 2. obtaining a color difference image: extracting an R-G color difference image from the RGB image; 3. image morphology operation: sequentially performing corrosion, primary hole filling, small area removal, expansion and secondary hole filling on the R-G color difference image; 4. fruit area acquisition: threshold segmentation is carried out based on object edge information in the image to obtain a fruit area; 5. equal area circle radius of each fruit area is obtained: calculating the area of each fruit area, and calculating the radius of an equal-area circle on the basis of the area of each fruit area; 6. obtaining a near-large fruit area: and performing iterative open operation on the disc-shaped structural elements with the radius changing on the fruit image, so as to obtain an image of a target area of the near-large fruit. The invention provides a simple and effective method for determining the picking targets of the single-arm picking robot.
Description
Technical Field
The invention belongs to the technical field of image processing, and relates to a method for acquiring a near-large fruit target in an orchard image, in particular to a method for acquiring a near-large fruit target of an orchard red fruit image.
Background
The fruit industry is the third largest industry after grain and vegetables in the planting industry. At present, in the fruit planting production process, except that spraying is semi-mechanized, fruit picking is an important link, and at present, manual work is basically performed, so that the labor intensity is high and the consumed time is long. In recent years, picking robots based on machine vision become research hotspots in the field of domestic and foreign agricultural engineering, and aim to realize mechanical automatic intelligent picking of fruits. For a single-arm picking robot based on machine vision, the near-large fruits (fruits which are near to the robot vision and larger) are the preferred targets for picking each time, so how to simply and effectively acquire the near-large fruits is naturally the primary task of the picking robot.
Disclosure of Invention
In order to solve the problems, the invention provides a method for acquiring the near-large fruit target in the orchard image, which enables a picking robot to acquire the near-large fruit target rapidly and efficiently in an image processing stage, lays a foundation for positioning and smooth picking of subsequent fruits, and promotes the practical process of the picking robot. The technical scheme for realizing the invention comprises the following steps:
a method for acquiring a near-large fruit target in an orchard image comprises the following steps:
(1) Acquiring RGB images based on a vision sensor;
(2) R-G operation is carried out based on the RGB three-channel image, and a color difference image is obtained;
(3) Sequentially carrying out corrosion, primary hole filling, small area removal, expansion and secondary hole filling morphological operation on the R-G color difference image;
(4) Performing threshold segmentation based on target edge information in the image after the operation (3) to obtain a fruit area;
(5) Calculating the area of each fruit area, and obtaining the radius of an equal-area circle;
(6) And performing iterative open operation on the disc-shaped structural elements with the radius changing on the fruit image, so as to obtain an image of a target area of the near-large fruit.
In a further preferred scheme, in the step (3), the R-G color difference image is subjected to a corrosion operation based on a disc-shaped structural element with a radius of 6; performing hole filling operation on the corroded R-G color difference image for one time based on a water-diffusion filling algorithm; marking the connected region in the R-G color difference image filled with the holes by an 8 neighborhood marking method, counting the total number, and removing the small region smaller than the threshold value of 2000 pixels total number; and then performing expansion operation on the R-G color difference image with the small area removed based on the disc-shaped structural element with the radius of 6, and performing secondary hole filling operation by using a water diffusion filling algorithm.
In the step (4), the outer contour of the object in the image is searched by adopting an area tracking method, gray values of all outer contour points are added and an average value is obtained, and the RGB original image is segmented by taking the average value as a threshold value which is 3 times, so that a fruit area is obtained.
In a further preferred scheme, in the step (6), iterative open operation is performed on the fruit image firstly by using disc-shaped structural elements with decreasing radius until the average gray value of the image is greater than 0, wherein the initial value of the radius of the disc-shaped structural elements is the maximum radius of each equal-area circle, and the decreasing radius change amount is 1/17 of the maximum radius of each equal-area circle; and then the radius change of the structural element is increased by 1 to perform an open operation until the average gray value of the image is equal to 0, and the image obtained by subtracting 1 from the radius value of the structural element and performing the open operation is the image of the near-large fruit target area.
The beneficial effects of the invention are as follows:
(1) For a picking robot, the method can effectively acquire near-large fruits in an orchard image, and lays a foundation for positioning and picking of subsequent fruits.
(2) The link of picking sequence planning again after all fruits are extracted from the orchard image in the past is abandoned, and only target fruits to be picked are given out in the image processing stage.
Drawings
FIG. 1 is a general flow of near-large fruit target acquisition in an orchard image;
fig. 2 is an effect diagram of the acquisition of near-large fruit targets in an orchard image.
Detailed Description
The invention is further described below with reference to the drawings and specific embodiments. The invention is described with respect to apples, but the invention is equally applicable to other fruits.
As shown in FIG. 1, the method for acquiring the near-large fruit target in the orchard image provided by the invention comprises the following steps:
(1) The acquisition of RGB images is based on a vision sensor for subsequent extraction of the target object, the acquired images being as in (a) of fig. 2.
(2) The difference between the apple fruits and the background is obvious, R-G operation is carried out based on the RGB three-channel image acquired in the step (1), and a color difference image is acquired as shown in (b) in fig. 2; as can be seen from fig. 2 (b), only fruits and branches remain in the image, and the other background is not black.
(3) In order to eliminate redundant branches in the color difference image, firstly, performing corrosion operation on the R-G color difference image based on a disc-shaped structural element with the radius of 6; the tail calyx of the fruit is greatly different from the main body color of the fruit, so that holes can be formed, if the tail calyx is not treated, the subsequent fruit area is lost and incomplete, and therefore, a flooding filling algorithm is adopted to carry out hole filling operation on the corroded R-G color difference image; the image filled with holes still has noise points or noise blocks except fruits, and in order to eliminate the noise points or the noise blocks, the connected areas in the image are marked and counted by an 8-neighborhood marking method, and small areas with the total number of pixels smaller than 2000 are removed; in order to ensure the integrity of the subsequent fruit image to the maximum extent, the R-G color difference image after the small area is removed is reversibly expanded based on a disc-shaped structural element with the radius of 6 relative to the corrosion operation; since the swelled image may form holes again, the swelled image is hole-filled again by using the flooding filling algorithm. The image for performing the above operation is as in (c) of fig. 2.
(4) (3) the object edge in the image after operation may still have background information of no fruit, for this purpose, the outline of the object in the image after operation is first searched for by adopting the area tracking method (3), then gray values of all outline points are added and averaged, and then the RGB original image is segmented by taking the 3 times of the average value as a threshold value, so as to obtain the final fruit area, as shown in (d) in fig. 2.
(5) To obtain near-large fruit targets in the fruit areas, the area of each fruit area is first calculated, and the radius of the equal area circle is obtained for each fruit area, as in (e) of fig. 2.
(6) Performing iterative open operation on the obtained fruit image by using disc-shaped structural elements with decreasing radius until the average gray value of the image is greater than 0, wherein the initial value of the radius of the disc-shaped structural elements is the maximum radius of each equal-area circle in (5), and the decreasing radius change amount is 1/17 of the maximum radius of each equal-area circle; then the radius change of the structural element is increased by 1 to perform an open operation until the average gray value of the image is equal to 0, and the image obtained by subtracting 1 from the radius value of the structural element and performing the open operation is the near-large fruit target area image, as shown in (f) of fig. 2.
The above embodiments are only for illustrating the technical solution of the present invention, but not for limiting the present invention, and those skilled in the relevant art can make various changes without departing from the spirit and scope of the present invention, so that all equivalent technical solutions are also within the scope of the protection of the present invention.
Claims (3)
1. The method for acquiring the near-large fruit target in the orchard image is characterized by comprising the following steps of:
(1) Acquiring RGB images based on a vision sensor;
(2) R-G operation is carried out based on the RGB three-channel image, and a color difference image is obtained;
(3) Sequentially carrying out corrosion, primary hole filling, small area removal, expansion and secondary hole filling morphological operation on the R-G color difference image;
(4) Performing threshold segmentation based on target edge information in the image after the operation (3) to obtain a fruit area;
(5) Calculating the area of each fruit area, and obtaining the radius of an equal-area circle;
(6) Performing iterative operation on disc-shaped structural elements with radius changing on the fruit image, so as to obtain an image of a target area of the near-large fruit;
in the step (6), iterative operation is firstly carried out on the disc-shaped structural elements with decreasing radius change on the fruit image until the average gray value of the image is more than 0, wherein the initial value of the radius of the disc-shaped structural elements is the maximum radius of each equal-area circle, and the decreasing radius change amount is 1/17 of the maximum radius of each equal-area circle; and then the radius change of the structural element is increased by 1 to perform an open operation until the average gray value of the image is equal to 0, and the image obtained by subtracting 1 from the radius value of the structural element and performing the open operation is the image of the near-large fruit target area.
2. The method for obtaining near-large fruit targets in an orchard image according to claim 1, wherein in the step (3), the R-G color difference image is subjected to a corroding operation based on a disc-shaped structural element with a radius of 6; performing hole filling operation on the corroded R-G color difference image for one time based on a water-diffusion filling algorithm; marking the connected region in the R-G color difference image filled with the holes by an 8 neighborhood marking method, counting the total number, and removing the small region smaller than the threshold value of 2000 pixels total number; and then performing expansion operation on the R-G color difference image with the small area removed based on the disc-shaped structural element with the radius of 6, and performing secondary hole filling operation by using a water diffusion filling algorithm.
3. The method for obtaining the near-large fruit target in the orchard image according to claim 1, wherein in the step (4), firstly, an area tracking method is adopted to search the outline of the target in the image, then all outline point gray values are added and averaged, and the RGB original image is segmented by taking 3 times of the average value as a threshold value to obtain the fruit area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811373577.1A CN109544572B (en) | 2018-11-19 | 2018-11-19 | Method for acquiring near-large fruit target in orchard image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811373577.1A CN109544572B (en) | 2018-11-19 | 2018-11-19 | Method for acquiring near-large fruit target in orchard image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109544572A CN109544572A (en) | 2019-03-29 |
| CN109544572B true CN109544572B (en) | 2023-07-25 |
Family
ID=65847993
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811373577.1A Active CN109544572B (en) | 2018-11-19 | 2018-11-19 | Method for acquiring near-large fruit target in orchard image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109544572B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361315B (en) * | 2021-02-23 | 2021-12-07 | 仲恺农业工程学院 | Banana string identification method based on background saturation compression and difference threshold segmentation fusion |
| CN114511619A (en) * | 2022-04-01 | 2022-05-17 | 石家庄市长安育才建材有限公司 | Concrete appearance evaluation method and device and computer readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103295018A (en) * | 2013-05-21 | 2013-09-11 | 常州大学 | Method for precisely recognizing fruits covered by branches and leaves |
| CN104636722A (en) * | 2015-01-26 | 2015-05-20 | 江苏大学 | Fast tracking recognition method for overlapped fruits by picking robot |
| CN106327467A (en) * | 2015-06-25 | 2017-01-11 | 吴海峰 | Method for quickly tracking and indentifying target fruits picked by apple picking robot |
| CN106780438A (en) * | 2016-11-11 | 2017-05-31 | 广东电网有限责任公司清远供电局 | Defects of insulator detection method and system based on image procossing |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105574514B (en) * | 2015-12-31 | 2019-03-22 | 上海交通大学 | The raw tomato automatic identifying method in greenhouse |
| CN105719282B (en) * | 2016-01-16 | 2018-06-08 | 常州大学 | A kind of orchard mcintosh image fruit branches and leaves area obtaining method |
-
2018
- 2018-11-19 CN CN201811373577.1A patent/CN109544572B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103295018A (en) * | 2013-05-21 | 2013-09-11 | 常州大学 | Method for precisely recognizing fruits covered by branches and leaves |
| CN104636722A (en) * | 2015-01-26 | 2015-05-20 | 江苏大学 | Fast tracking recognition method for overlapped fruits by picking robot |
| CN106327467A (en) * | 2015-06-25 | 2017-01-11 | 吴海峰 | Method for quickly tracking and indentifying target fruits picked by apple picking robot |
| CN106780438A (en) * | 2016-11-11 | 2017-05-31 | 广东电网有限责任公司清远供电局 | Defects of insulator detection method and system based on image procossing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109544572A (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Luo et al. | A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard | |
| Malik et al. | Mature tomato fruit detection algorithm based on improved HSV and watershed algorithm | |
| Awate et al. | Fruit disease detection using color, texture analysis and ANN | |
| CN103295018B (en) | A kind of branches and leaves block fruit precise recognition method | |
| Blok et al. | Machine vision for a selective broccoli harvesting robot | |
| CN110610506B (en) | Image processing technology-based agaricus blazei murill fruiting body growth parameter detection method | |
| CN108710840B (en) | Visual navigation path identification method for farmland pesticide spraying robot | |
| CN105719282B (en) | A kind of orchard mcintosh image fruit branches and leaves area obtaining method | |
| CN107527343B (en) | A kind of agaricus bisporus stage division based on image procossing | |
| CN112132153B (en) | Tomato fruit identification method and system based on clustering and morphological processing | |
| CN109544572B (en) | Method for acquiring near-large fruit target in orchard image | |
| CN113674226A (en) | Tea leaf picking machine tea leaf bud tip detection method based on deep learning | |
| CN111798470B (en) | Crop image entity segmentation method and system applied to intelligent agriculture | |
| CN107316043A (en) | A kind of stacking mushroom method for quickly identifying of picking robot | |
| CN110599507A (en) | Tomato identification and positioning method and system | |
| Fu et al. | Kiwifruit recognition method at night based on fruit calyx image | |
| CN111046782B (en) | Quick fruit identification method for apple picking robot | |
| CN103226709B (en) | A kind of network curtain image recognition method of fall webworm larvae | |
| Nisar et al. | Predicting yield of fruit and flowers using digital image analysis | |
| Abdelghafour et al. | Potential of on-board colour imaging for in-field detection and counting of grape bunches at early fruiting stages | |
| CN111401121A (en) | Method for realizing citrus segmentation based on super-pixel feature extraction | |
| CN115994921A (en) | Mature cherry fruit image segmentation method integrating HSV model and improving Otsu algorithm | |
| CN107437254B (en) | Orchard adjacent overlapping shape fruit distinguishing method | |
| CN113269750A (en) | Banana leaf disease image detection method and system, storage medium and detection device | |
| Yun et al. | Recognition of the position of Chinese wolfberry branches under the artificial background |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |