CN111046726B - Underwater sea cucumber identification and positioning method based on AI intelligent vision - Google Patents

Underwater sea cucumber identification and positioning method based on AI intelligent vision Download PDF

Info

Publication number
CN111046726B
CN111046726B CN201911021934.2A CN201911021934A CN111046726B CN 111046726 B CN111046726 B CN 111046726B CN 201911021934 A CN201911021934 A CN 201911021934A CN 111046726 B CN111046726 B CN 111046726B
Authority
CN
China
Prior art keywords
sea cucumber
image
underwater
thorn
outline
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911021934.2A
Other languages
Chinese (zh)
Other versions
CN111046726A (en
Inventor
李娟�
李文升
李波
王东伟
王峰
张维东
高洪伟
佟春明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LAIZHOU MINGBO AQUATIC CO LTD
Qingdao Agricultural University
Original Assignee
LAIZHOU MINGBO AQUATIC CO LTD
Qingdao Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LAIZHOU MINGBO AQUATIC CO LTD, Qingdao Agricultural University filed Critical LAIZHOU MINGBO AQUATIC CO LTD
Priority to CN201911021934.2A priority Critical patent/CN111046726B/en
Publication of CN111046726A publication Critical patent/CN111046726A/en
Application granted granted Critical
Publication of CN111046726B publication Critical patent/CN111046726B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
    • Y02A40/81—Aquaculture, e.g. of fish

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Image Processing (AREA)

Abstract

The invention provides an AI intelligent vision-based underwater sea cucumber identification and positioning method, which comprises the steps of acquiring an underwater sea cucumber image, preprocessing the image, detecting edges, expanding, enhancing the profile, filling the sea cucumber thorn profile, extracting the sea cucumber thorn centroid, carrying out ellipse fitting, obtaining ellipse center coordinates, taking the ellipse center coordinates as the capturing point coordinates of sea cucumbers, and the like, wherein the method suitable for the fusion of RGB three-channel gray level images of the underwater sea cucumbers to highlight the sea cucumbers is provided in the image preprocessing process, and the sea cucumbers can be highlighted in a targeted manner, so that the accurate positioning of the sea cucumbers is realized.

Description

Underwater sea cucumber identification and positioning method based on AI intelligent vision
Technical Field
The invention relates to the field of AI intelligent vision and fishery, in particular to an underwater sea cucumber identification method based on AI intelligent vision.
Background
Related data show that the development of the fishery industry in China is faster in recent years, the sea cucumber has obvious increase in yield and profit due to the special nutritional value and medicinal value, and the sea cucumber is the fishery product with the best economic benefit at present. The sea cucumber culture history in China is long, and large areas of Shandong, liaoning, hebei and Fujian are formed, wherein the sea cucumber in Shandong has the highest yield and is popular with high-quality stichopus japonicus. The regions such as Qingdao, weihai, smoke counter and the like greatly develop the sea cucumber culture industry by virtue of the unique geographic advantages of the regions, so that a complete industrial chain is formed at present, the economic driving effect is very obvious, and a plurality of well-known brands such as the caretaker, the old-fashioned alien family, the eastern ocean and the like are formed at present.
Along with the continuous increase of the sea cucumber demand in the world, the cultivation area of each large-yield area is gradually increased, but the sea cucumber industry still has related problems to restrict the development of the sea cucumber industry. Sea cucumber fishing is a key factor in a plurality of restriction sea cucumber industry development factors, and according to actual investigation, the current sea cucumber fishing mainly adopts manual fishing, and the fishing time is generally 5 months and 11 months each year, so that the problems of low manual fishing efficiency and high risk coefficient are particularly remarkable. When a diver catches sea cucumbers underwater, the diver is easily dangerous due to the influence of air temperature and pressure, and irrecoverable loss is caused to individuals and families. If the underwater sea cucumber identification algorithm and the underwater sea cucumber identification system can be invented, the development of the sea cucumber industry can be well promoted.
By combining the background analysis, the equipment capable of automatically capturing the underwater sea cucumbers is urgently needed at present, and the identification and the positioning of the sea cucumbers are needed to be realized firstly, so that the AI intelligent vision technology is widely applied to various fields of industrial production as an emerging technology, the labor intensity of people is greatly reduced, and a foundation is laid for identifying the underwater sea cucumber images by using the AI intelligent vision technology.
The research on the identification technology of the underwater sea cucumber image is always a key problem of research by related expert students, the underwater image contains larger noise due to the fact that the shooting environment of the underwater image is complex, the image is generally blurred and the information loss is serious, and the difficulty of sea cucumber identification is greatly increased. The noise processing of the underwater image is the key of sea cucumber identification, currently, a universally applicable underwater image denoising algorithm is not seen with our knowledge, and the existing related algorithm can achieve better effect in a laboratory but is applied to reality or is not ideal, so the invention is very necessary for a method suitable for denoising the underwater sea cucumber image.
And, sea cucumber identification and positioning are the precondition and key of automation and intellectualization of sea cucumber fishing. The invention provides an intelligent AI vision-based underwater sea cucumber identification method, which aims at solving the problems of complex living environment of sea cucumbers, low illumination intensity in water, change of body color of sea cucumbers along with the environment, high noise, low contrast, serious image distortion and the like of images of the underwater sea cucumbers, and aims at solving the problems of low image processing speed, unsatisfactory identification effect and the like of the conventional image processing method.
Disclosure of Invention
The invention mainly aims at: (1) The method for projecting sea cucumber thorns by using ten-position precision gray level conversion and fusion of RGB three-channel gray level images of underwater sea cucumber is provided, so that the operation efficiency is greatly improved, sea cucumber thorns can be projected in a targeted manner, and a sea cucumber thorn contour area is obtained by edge detection; (2) The method for identifying the sea cucumber based on direct least square sea cucumber thorn centroid ellipse fitting is provided; and taking the centroid of the elliptical outline of the sea cucumber obtained by fitting as the pixel coordinates of the capturing point image.
In order to achieve the technical purpose, the technical scheme of the invention provides an AI intelligent vision-based underwater sea cucumber identification and positioning method, which comprises the following steps:
a: acquiring an underwater sea cucumber image, wherein the underwater sea cucumber image is a color image;
b, image preprocessing, namely performing sea cucumber trunk image fusion processing on the underwater sea cucumber image, and performing image enhancement and de-drying processing on the fused image; wherein the sea cucumber trunk image fusion processing is based on formula (1);
wherein f is the gray value of the pixel after fusion processing, and R, G and B respectively represent RGB components of the color sea cucumber image;
c: edge detection, namely dividing sea cucumber thorns by adopting an edge detection algorithm to obtain a sea cucumber thorn outline binary image;
d: performing expansion operation, namely performing morphological expansion operation on the sea cucumber thorn profile binary image after edge detection;
e: profile enhancement, comprising:
e1, filling diagonal pixels in eight adjacent areas of edge pixel points in the image subjected to morphological dilation operation;
and E2, removing the isolated point pixels and removing the pixel points without adjacent pixels.
And F, removing small outlines, screening the outline area of the sea cucumber image in the image, setting a threshold value, removing outlines with outline areas smaller than the threshold value, and reserving more than the outline areas.
G, filling sea cucumber thorn contours, and performing contour filling operation on the sea cucumber thorn contour binary image after edge detection;
h: extracting barycenters of all outline areas, and obtaining coordinates of each barycenter;
i, ellipse fitting, namely fitting an ellipse shape by using a least square method based on the obtained centroid coordinates of each contour area;
and J, obtaining an ellipse center coordinate, and taking the ellipse center coordinate as a sea cucumber capturing point coordinate.
Drawings
FIG. 1 is a flow chart of the method of the present invention
FIG. 2 sea cucumber image under water
FIG. 3 sea cucumber image sample under water
FIG. 4 is a flow chart of an embodiment of image preprocessing
FIG. 5A is a gray scale converted underwater sea cucumber image
FIG. 6 CLAHE algorithm enhancement after gray level conversion
FIG. 7 Soft threshold wavelet transform after gray level transform to remove noise
FIG. 8 is a second flowchart of an image preprocessing embodiment
FIG. 9 RGB spatial distribution of a typical sample of an underwater sea cucumber image
FIG. 10a R passage sea cucumber gray scale map under water
FIG. 10b G passage sea cucumber gray scale map under water
FIG. 10c B passage sea cucumber gray scale map under water
FIG. 11 image fusion followed by salient sea cucumber thorn images
FIG. 12 CLAHE algorithm enhancement after image fusion
FIG. 13 image post-fusion soft threshold wavelet transform de-drying
FIG. 14 Canny edge detection results
FIG. 15 is an image of the underwater sea cucumber thorn profile after the inflation operation
FIG. 16 removes eight neighborhood background and contour enhancement
FIG. 17 image region contour screening
FIG. 18 is a view of a sea cucumber spine image after contour filling
FIG. 19 sea cucumber thorn image after inflation and filling
FIG. 20A sea cucumber thorn map with centroid marked
Ellipse fitting in binary image of FIG. 21a
Ellipse fitting in original figure 21b
FIG. 22 is an elliptical schematic view
FIG. 23 interface for acquiring sea cucumber center coordinate system
Detailed Description
The invention is further illustrated by the following examples in conjunction with the accompanying drawings:
FIG. 1 shows a flow chart of the method of the present invention.
A sea cucumber image picking up water under water
The underwater sea cucumber image water collection is a key factor for post-treatment, and in order to accurately acquire the underwater sea cucumber image in a real scene as much as possible, the image water collection equipment uses a GoPro HERO 6 BLACK underwater camera, the pixels of the camera are 1200 ten thousands, the frames are transmitted every second, and the maximum waterproof depth is 10 meters, so that the practical requirement of underwater sea cucumber image water collection is completely met. The image used in the invention is a sea cucumber image obtained by water in the underwater site, and the underwater sea cucumber image shot by the camera is shown in figure 2.
The underwater sea cucumber image photographed in practice is analyzed to clearly find that the living environment where the sea cucumber is located is very complex, and the selective absorption effect of water on light inevitably exists during underwater photographing, so that the definition of the image photographed by the underwater photographing is poor, sea weeds, oyster, shells and riverbed in the living environment of the sea cucumber can cause serious interference on the identification of the sea cucumber, and the difficulty of the identification of the sea cucumber is intangibly increased. A typical sea cucumber image sample is shown in fig. 3 by taking a sea cucumber image 2 photographed under water as an example.
Features of aquatic weeds, sea cucumber trunks, sand, shells, oyster and sea cucumber thorns are clearly marked in the underwater sea cucumber image sample, however, for a color chart, identification of sea cucumbers through colors is relatively difficult. The sea cucumber has strong self-protection capability, the trunk of the sea cucumber changes color along with the color change of the living environment, and the color of the trunk of the sea cucumber in fig. 2 is close to the background color, so that the sea cucumber is difficult to recognize by naked eyes. The statistics of the colors of the sea cucumber trunk, the sea cucumber thorns and the background in the underwater sea cucumber image sample are shown in table 1.
TABLE 1 image features of sea cucumber under water and background color statistics table
B, image preprocessing, wherein optional FIG. 4 is a flowchart of an embodiment of image preprocessing, wherein the sea cucumber image is converted into a gray image, and then the gray image is subjected to image enhancement and de-drying treatment;
b1 underwater sea cucumber image gray scale conversion
Currently, the photos taken by a camera mostly use RGB color mode, in which the color mode consists of R, G, B three channels, each channel having a luminance interval of [0, 255]. Where R represents red, G represents green, and B represents blue, the general colors can be expressed using a three-channel luminance combination. The underwater sea cucumber image acquired in the invention also adopts the mode, but the RGB color mode needs to be operated on R, G, B channels respectively when the processing is carried out, which brings great inconvenience to the image processing process, and the RGB color mode shows obvious disadvantages because the RGB color mode can not reflect morphological characteristics in the image, so how to solve the problems becomes the key of image preprocessing in the image processing.
In order to avoid the restriction of RGB color model on underwater sea cucumber image recognition in image processing, the invention firstly carries out graying operation on the image, wherein graying is the operation of taking the same value of R, G, B three channels of components in the color image. Currently, the graying of an image mainly comprises two main lines, one is a method for obtaining an average value of three RGB channels, and the other is to use a brightness value Y to represent an image gray value Grey according to the relation between RGB and YUV color components. The second method is adopted in the invention, the operation can convert the three-channel color image into a single-channel gray image, the gray values of different pixels are adopted for representing the pixel points in the gray image, the value interval is [0, 255], and the gray conversion of the underwater sea cucumber image is completed according to the formula (1).
Grey＝0.299*R+0.587*G+0.114*B (1)
The formula (1) can complete the required gray level conversion function, but considering that the operation speed of the sea cucumber identification system needs to be improved, the operation speed of the related operation mode in the computer Arithmetic Logic Unit (ALU) is sequenced from high to low as follows: bit manipulation, integer addition, integer multiplication, floating point arithmetic. The gray level conversion formula (2) with ten-bit precision is obtained by improving the formula (1) based on the above ideas.
Grey＝(306*R+601*G+117*B)>>10 (2)
The experiment shows that the running time is 0.2536s when the formula (1) is used, the program running time is 0.2013s when the formula (2) is used, the running efficiency is greatly improved, and the result of converting the color chart 2 into the gray chart by the formula (2) is shown in fig. 5.
B2 underwater sea cucumber image enhancement
When the HE algorithm is used for enhancing the underwater sea cucumber image, the contrast of background information in the image is enhanced at the same time, which is disadvantageous to the underwater sea cucumber image identification, so that the contrast-limiting self-adaptive histogram equalization CLAHE (Contrast Limited Adaptive Histongram Equalization) algorithm is analyzed according to the principle of optimizing the invention in the invention.
The CLAHE limits the amplitude of the contrast of the image when the image is enhanced, is an optimization invention for the HE algorithm, avoids the phenomenon that the background noise is amplified when the HE algorithm is realized, and is used for processing the underwater sea cucumber image 5 after gray level conversion to obtain the image 6.
B3 wavelet transform de-drying based on soft threshold
The sea cucumber image obtained by underwater water collection contains more noise, and how to process the noise is a key problem to be considered in the invention, and common filtering modes include Gaussian filtering, median filtering, low-pass filtering, mean filtering, wavelet filtering and the like. The mean filtering, the median filtering and the Gaussian filtering can remove Gaussian white noise, the median filtering can remove spiced salt noise, and the wavelet transformation can realize the treatment of various noises. In order to eliminate noise interference in the underwater sea cucumber image after image enhancement, a wavelet transformation denoising mode is introduced, and common wavelet transformation denoising modes comprise a soft threshold value, a hard threshold value, a soft threshold value, a hard threshold value and the like. The invention comprehensively compares the denoising modes and finds that the denoising of the current soft threshold wavelet transform is most suitable. Fig. 7 is a graph of a soft threshold wavelet transform based de-drying result.
B', image preprocessing, wherein an optional FIG. 8 shows an image preprocessing embodiment II, wherein the image fusion processing is carried out on the underwater sea cucumber image, and then the image enhancement and the drying processing are carried out;
in order to intuitively see the difference of sea participation backgrounds in the underwater sea cucumber image in the RGB color space, the invention takes brightness values of RGB three channels as coordinate axes according to a typical sample of the underwater sea cucumber image in fig. 9, and then draws a typical sample space distribution diagram after water-related data is acquired as shown in fig. 9. According to analysis of RGB space distribution map of typical sample of underwater sea cucumber image, the RGB color space distribution of sea cucumber thorns is obviously different from RGB color space distribution information of sea cucumber trunks, shells, oyster, sandy land and waterweed, the distribution characteristic distinction degree of sea cucumber trunks, shells, oyster, sandy land and waterweed in RGB color space is lower, sea cucumber thorns can be firstly segmented in sea cucumber identification, and then the identification of underwater sea cucumber image is realized through the corresponding algorithm of the invention.
B1 underwater sea cucumber image fusion processing
The image fusion is to re-synthesize a new image by converting the target multi-source image acquired in the same time, and the interested area in the new image is highlighted, which is beneficial to the segmentation processing of the image. The image fusion technology is widely applied to image segmentation because of having very important significance to image segmentation, in particular to image segmentation of agricultural products, such as tomatoes, medlar, cucumbers, honeysuckle and the like.
The image fusion process of the invention takes the spatial distribution of a typical sea cucumber image underwater sample as a basis, fully excavates RGB three-channel data information of sea cucumber thorns, sea cucumber trunks, shells, oyster, sandy lands and aquatic weeds in the underwater sea cucumber image, and respectively makes a gray level map under an R channel, a gray level map under a G channel and a gray level map under a B channel as shown in fig. 10a, 10B and 10c according to fig. 2.
In order to highlight sea cucumbers in the image, a plurality of arithmetic operations are carried out on an R-channel lower sea cucumber image gray level map, a G-channel lower sea cucumber gray level map and a B-channel lower sea cucumber gray level map in the chapter, a sea cucumber thorn protrusion type (3) in the underwater sea cucumber image is invented, the formula is used for completing calculation of the numerical value of the three channels, and the fusion algorithm can highlight sea cucumbers well through experiments. For the sea cucumber trunk, the sea cucumber trunk has higher similarity in color with waterweed, shell, oyster and the like in the background, and the sea cucumber trunk can be highlighted when the image fusion is carried out, but the waterweed, shell and oyster in the background are highlighted, so that the sea cucumber trunk image fusion mode with higher universality is not found out.
Re-fusing the decomposed single-channel underwater sea cucumber image to obtain an image 11 through the process of (3)
B'2 CLAHE algorithm image enhancement
The image fusion formula (3) well completes the function of image fusion to highlight sea cucumber thorns, but the background in the image after image fusion is still enhanced, the highlighted sample in the background is less, the highlighted background area is much smaller than the sea cucumber thorns, and the sea cucumber thorns can be removed through the subsequent processing process. The image 11 which is fused again by the formula (3) and highlights the sea cucumber thorns is fuzzy and is unfavorable for subsequent image processing, so that the image 11 is enhanced by using the CLAHE algorithm to obtain an image 12. Comparing fig. 11 and fig. 12 can find that the definition of the image is obviously improved, which is very beneficial to the segmentation of sea cucumber thorns in the subsequent images.
B'3 wavelet transform de-drying based on soft threshold
The present invention still selects the wavelet transform denoising operation based on the soft threshold, and the image after denoising by wavelet transform is shown in fig. 13.
C underwater sea cucumber image sea cucumber thorn edge detection
Edge detection is widely applied to image processing because the edge in the image always has the most obvious brightness information change, and alternative edge detection algorithms include a Robert edge detection algorithm, a Sobel and Prewitt edge detection algorithm, a Laplacian edge detection algorithm, a Log edge detection algorithm and a Canny edge detection algorithm, which are widely applied in practice, but have advantages and disadvantages. Fig. 14 shows the Canny edge detection result.
D sea cucumber thorn contour image after expansion edge detection
The sea cucumber thorn contour is detected after the Canny edge detection, but the detected sea cucumber thorn edge is not continuous, but is a few isolated points, so that the expansion operation is firstly required to be carried out on the image 14 after the Canny edge detection to connect adjacent elements in the image. After the expansion operation, fig. 15 is obtained. According to analysis of the outline image of the expanded underwater sea cucumber thorn edge, the edge of the expanded sea cucumber thorn outline is obviously protruded, the edges are well connected, and the sea cucumber thorn extracted through the algorithm is relatively complete. However, the interference factors in the background in fig. 14 are also inflated, so that the non-sea cucumber thorn contours in the image are then processed to eliminate the interference factors in the background and accurately extract sea cucumber thorns.
E contour enhancement
The expansion operation connects the sea cucumber thorn edges and makes the edges clearer, but the background objects which interfere with sea cucumber thorn detection in the image are also expanded, so that the outline is further enhanced in the invention for removing the interference background influence in the image. When the contour enhancement is carried out, the diagonal pixel filling is firstly carried out on eight adjacent areas of the edge pixel points in the image, namely, the diagonal filling of the eight adjacent areas of the current pixel Point is carried out, namely, X1, X3, X6 and X8 pixel points in the lower table are filled, then, the isolated Point pixels are removed, and the pixels without adjacent pixels are removed. Further contour enhancements are shown in fig. 16.
X_1 X_2 X_3
X_4 Piont X_5
X_6 X_7 X_8
Eight neighborhoods: x_1, x_2, x_3, … …, x_6, x_7, x_8.
F removing small outline
The sea cucumber thorn image after expansion and morphological treatment still has some interference of background noise, how to eliminate the contours becomes a key problem, and the noise treatment can remove corresponding noise in the underwater sea cucumber image, so that the accurate identification of the underwater sea cucumber is realized. By analysing the sea cucumber stab profile and other profiles in fig. 16, a very pronounced feature can be summarized, namely that the sea cucumber stab profile is closed, whereas the profile of the detected disturbing factors in the background mostly does not constitute a closed and smaller profile.
Through the analysis, the invention can firstly screen the outline area of the sea cucumber image in the image through an inventive algorithm. In the invention, a threshold is set during screening, the contour smaller than the threshold is removed, and the contour is reserved larger than the contour. Therefore, the interference of the underwater sea cucumber image background can be accurately removed. In order to delete sea cucumber thorn contours smaller than a certain size, a bwaseaopen function is introduced in the invention, and the function can delete contours with eight neighborhood inner contour areas smaller than a specified size in an image. Through a large number of experiments, the minimum contour area is set to be 700 in the invention, and contours with contour areas smaller than the set area in the image are deleted. The image after profile screening and deburring is shown in fig. 17, and the algorithm can be found by comparing fig. 16 to basically eliminate non-sea cucumber stab interference in the background. Although some smaller sea cucumber thorn contours are deleted in the process of underwater sea cucumber thorn contour screening, the retained sea cucumber thorn contours are enough to fit sea cucumber contours through the least square sea cucumber thorn centroid ellipse fitting algorithm of the invention.
The sea cucumber thorn contours in the screened underwater sea cucumber images are clear and are all sea cucumber thorn contours, and how to acquire the sea cucumber trunk areas through the corresponding algorithm of the invention becomes the key of the invention. The acquisition of the trunk area of the sea cucumber is realized by determining an ellipse fitting mode after a large amount of data is consulted, because the sea cucumber is in an elliptical shape when seen from the side in general, and the sea cucumber has strong self-protection capability, and the trunk color of the sea cucumber can be changed along with the living environment. For example, sea cucumbers living near reefs are generally brown or light blue in trunk color, while sea cucumbers living in seaweed and some aquatic weeds are generally light green in trunk color, so that it is difficult to simply identify sea cucumbers by trunk color. And (3) acquiring sea cucumber trunk areas in the underwater sea cucumber image through sea cucumber thorn area fitting, wherein the sea cucumber trunk areas are filled by sea cucumber thorn outlines, sea cucumber thorn centroid extraction and sea cucumber outline fitting.
G sea cucumber thorn contour filling
In fig. 17, the screening of the sea cucumber thorn outline in the underwater sea cucumber image is completed, and after the corresponding screening, the sea cucumber thorn outline in the image is all the sea cucumber thorn outline. In order to acquire the main contour of the underwater sea cucumber, firstly, filling the sea cucumber thorn contour, introducing an imfill function to fill the white contour inside the binary image, and processing the processed image as shown in fig. 18.
As can be seen from an analysis of fig. 18, the sea cucumber thorn contour in fig. 18 is substantially filled, and the left lower corner is still unfilled, and the reason why the region is unfilled is found after the enlargement is that the contour is not a closed region, so that the filling thereof is impossible. In order to avoid the situation of sea cucumber thorn filling caused by the non-closed outline as shown in fig. 18, an expansion operation is introduced again before the outline filling, the expansion operation is performed according to the matrix P in the formula (4), and the sea cucumber thorn image filled after the expansion operation is shown in fig. 19.
Compared with the sea cucumber spine image 14 without the second sea cucumber spine expansion operation, the expanded underwater sea cucumber spine image 19 is accurately filled with the unfilled outline in the sea cucumber spine image 18, which lays a foundation for the next acquisition of the centroid of the underwater sea cucumber spine image, and morphological expansion operation is very important for processing of the discontinuous outline.
H sea cucumber thorn centroid extraction
After the profile-filled sea cucumber thorn image is obtained, the extraction of the mass center of the sea cucumber thorn is started, a regionoprofs function is introduced for the extraction, all profiles in the image are traversed through programming, and the mass center coordinate of each profile in the image is calculated. After the barycenter coordinates of each region are obtained, a plot function is called to draw each barycenter coordinate into the sea cucumber thorn image filled with the outline, and the sea cucumber thorn image marked by the barycenter is shown in fig. 20. In fig. 20 each centroid coordinate is plotted accurately and each centroid is represented using a different color.
I ellipse fitting
Through analysis of morphological characteristics of underwater sea cucumber images, the sea cucumbers are mostly elliptical and have stable shape characteristics. Therefore, the invention adopts an ellipse fitting mode to obtain the sea cucumber trunk, uses a least square method to fit an ellipse shape according to the obtained sea cucumber thorn centroid coordinates in fig. 20 as shown in fig. 21, and adopts the least square method ellipse fitting algorithm principle as follows.
In the invention, an elliptic equation obtained by sea cucumber thorn fitting is assumed to be:
F(α,X)＝α·X＝Ax 2 +Bxy+Cy 2 +Dx+Ey+F＝0 (5)
each parameter in the formula (5) forms a parameter matrix alpha= [ A, B, C, D, E, F] T Each variable constitutes a matrixIs provided with->The barycenter coordinates of sea cucumber thorn (x) i ,y i ) The distance to the fitted curve F (α, X) can be expressed as F (α, X) i ) When the sea cucumber profile is obtained through the sea cucumber thorn fitting curve, the least algebraic distance square sum of all sea cucumber thorn centroids is solved, and then a corresponding curve equation is solved. When the number of the sea cucumber thorn centroid coordinates is n, a mathematical relationship can be established as shown in formula (6).
Limiting the curve equation obtained by fitting to ellipse to satisfy the constraint b of ellipse 2 -4ac<0, if the constraint fitting cannot be satisfied, the result will be parabolic or hyperbolic. And due to b 2 -4ac<0 is not a constraint of the equation, and it is known from the Country condition that there is no solution to solve the equation under the constraint, and the constraint is converted to b 2 -4ac= -1 and expressed in matrix form as:
α T Cα＝1 (7)
wherein:
the problem of ellipse fitting of the thorn centroid of the sea cucumber is converted into an optimization problem, namely:
H min ＝min||Lα|| 2 s.t.α T Cα＝1 (8)
the specific solving process introduces Lagrange operator according to Lagrange multiplier method and performs derivative calculation to obtain the following formula:
for ease of analysis let s=l T L, therefore, can be simplified to yield formula (9):
solving the equation sα=λcα can find the eigenvalue λ i Feature vector u corresponding to the feature value i Let the variable ρ i Is any real number, ρ i u i Is also a characteristic solution that satisfies the equation sα=λcα. Let α=ρ i u i Substituting the constraint alpha T After cα=1, a unique ρ can be obtained i Satisfy the following requirementsFrom this, the constant ρ can be obtained i 。
Then the following calculation results:
calculating the corresponding rho according to the formula (12) i >Feature vector of 0And the parameter matrix of an elliptic equation is obtained by using the ellipse fitting of the sea cucumber thorn centroid of the underwater sea cucumber image. In order to check the accuracy of the direct least square based sea cucumber thorn centroid ellipse fitting algorithm on the acquisition of sea cucumber trunks, firstly, an ellipse is fitted in a binary image according to the acquired sea cucumber thorn centroid coordinates, as shown in fig. 21a, and the ellipse obtained by fitting is further drawn in a sea cucumber image, as shown in fig. 21b. The elliptical area obtained by fitting in the figure 21b well surrounds the sea cucumber trunk, and the purpose of extracting the sea cucumber trunk in the underwater sea cucumber image is achieved.
J geometric calculation to obtain central coordinates of sea cucumber
The sea cucumber contours in the underwater sea cucumber images are fitted through the sea cucumber thorn centroid least square method ellipse fitting algorithm, and coordinates of sea cucumbers are extracted to realize underwater sea cucumber fishing. Therefore, in the stage, the center point coordinate is obtained for the fitted sea cucumber contour through a least square method sea cucumber thorn centroid ellipse fitting algorithm, and the coordinate is regarded as sea cucumber capturing coordinate.
The general formula for the elliptic equation is given according to formula (4.4):
F(α,X)＝Ax 2 +Bxy+Cy 2 +Dx+Ey+F＝0 (13)
and an elliptical schematic image is drawn as shown in fig. 22.
In the image 22, it is assumed that the origin coordinates of the ellipse are o (x 0 ,y 0 ) The long half axis length is a, the short half axis length is b, the included angle between the long axis and the x axis of the ellipse is theta, and the relation between each coefficient of the general ellipse and the long axis length a, the short axis length b and the included angle theta is obtained according to the geometric relation of each variable of the ellipse equation as follows:
by combining equation (13) and equation (14), the ellipse center coordinates o (x) can be found 0 ,y 0 ) In the invention, the coordinates are regarded as coordinates of the fishing point of the underwater sea cucumber.
Fig. 23 shows a sea cucumber contour center coordinate acquisition interface of the sea cucumber identification system, after the sea cucumber thorn is fitted to obtain an ellipse, the fitted ellipse is found to well surround a sea cucumber region, and the ellipse is basically consistent with the sea cucumber contour, so that the least square ellipse centroid fitting algorithm adopted by the invention is feasible to fit the sea cucumber contour region. In order to realize sea cucumber capturing, the coordinate information of sea cucumber is also required to be acquired, in this stage, the coordinate of the middle point of the elliptical sea cucumber profile is obtained by calculating the fitted elliptical sea cucumber profile, and the coordinate of the pixel coordinate system of the image is transmitted to a sea cucumber capturing device after being subjected to coordinate conversion of range measurement by a monocular camera, so that capturing is realized.
The present invention has been described above by way of example, but the present invention is not limited to the above-described embodiments, and any modifications or variations based on the present invention fall within the scope of the present invention.

Claims (6)

1. An AI intelligent vision-based underwater sea cucumber identification and positioning method is characterized by comprising the following steps:
a: acquiring an underwater sea cucumber image, wherein the underwater sea cucumber image is a color image;
b, image preprocessing, namely performing sea cucumber trunk image fusion processing on the underwater sea cucumber image, and performing image enhancement and de-drying processing on the fused image; wherein the sea cucumber trunk image fusion processing is based on formula (1);
wherein f is the gray value of the pixel after fusion processing, and R, G and B respectively represent RGB components of the color sea cucumber image;
c: edge detection, namely dividing sea cucumber thorns by adopting an edge detection algorithm to obtain a sea cucumber thorn outline binary image;
d: performing expansion operation, namely performing morphological expansion operation on the sea cucumber thorn profile binary image after edge detection;
e: profile enhancement, comprising:
e1, filling diagonal pixels in eight adjacent areas of edge pixel points in the image subjected to morphological dilation operation;
e2, removing isolated point pixels and removing pixel points without adjacent pixels;
f, removing small outline, screening the outline area of the sea cucumber image in the image, setting a threshold value which is 700, removing outline with outline area smaller than the threshold value, and reserving more than the outline area;
g, filling sea cucumber thorn contours, and performing contour filling operation on the sea cucumber thorn contour binary image after edge detection;
h: extracting barycenters of all outline areas, and obtaining coordinates of each barycenter;
i, ellipse fitting, namely fitting an ellipse shape by using a least square method based on the obtained centroid coordinates of each contour area;
and J, obtaining an ellipse center coordinate, and taking the ellipse center coordinate as a sea cucumber capturing point coordinate.
2. The method for identifying and positioning underwater sea cucumber according to claim 1, wherein the step B of image preprocessing further comprises the step of performing image enhancement on the fused image by using a CLAHE algorithm.
3. The underwater sea cucumber identification and localization method of claim 1, the step B image pre-processing further comprising, the de-desiccation process comprising, wavelet transform de-desiccation based on a soft threshold.
4. The method for identifying and locating underwater sea cucumber according to claim 1, wherein the step C edge detection further comprises a Robert edge detection algorithm, a Sobel edge detection algorithm, a Prewitt edge detection algorithm, a Laplacian edge detection algorithm, a Log edge detection algorithm, and a Canny edge detection algorithm.
5. The method for identifying and locating underwater sea cucumber according to claim 1, wherein the step F of sea cucumber thorn profile filling further comprises: and filling the whole interior of the outline in the sea cucumber thorn outline binary image by adopting an imfill function.
6. The method for identifying and locating underwater sea cucumber according to claim 1, wherein the step F of sea cucumber thorn profile filling further comprises: performing secondary filling on the interior of the outline in the sea cucumber thorn outline binary image, wherein the secondary filling comprises expanding according to a matrix P in a formula (2)
CN201911021934.2A 2019-10-25 2019-10-25 Underwater sea cucumber identification and positioning method based on AI intelligent vision Active CN111046726B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911021934.2A CN111046726B (en) 2019-10-25 2019-10-25 Underwater sea cucumber identification and positioning method based on AI intelligent vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911021934.2A CN111046726B (en) 2019-10-25 2019-10-25 Underwater sea cucumber identification and positioning method based on AI intelligent vision

Publications (2)

Publication Number Publication Date
CN111046726A CN111046726A (en) 2020-04-21
CN111046726B true CN111046726B (en) 2023-08-08

Family

ID=70232774

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911021934.2A Active CN111046726B (en) 2019-10-25 2019-10-25 Underwater sea cucumber identification and positioning method based on AI intelligent vision

Country Status (1)

Country Link
CN (1) CN111046726B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115953422B (en) * 2022-12-27 2023-12-19 北京小米移动软件有限公司 Edge detection method, device and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106097320A (en) * 2016-06-06 2016-11-09 中国农业大学 Stichopus japonicus image automatic segmentation method and device under water
CN107369140A (en) * 2017-06-29 2017-11-21 大连理工大学 High-accuracy target ball center extraction method under unstructured moving grids
CN108876855A (en) * 2018-05-28 2018-11-23 哈尔滨工程大学 A kind of sea cucumber detection and binocular visual positioning method based on deep learning
CN109166098A (en) * 2018-07-18 2019-01-08 上海理工大学 Work-piece burr detection method based on image procossing
CN109978869A (en) * 2019-03-29 2019-07-05 清华大学 A kind of sea horizon detection method and system based on gray level co-occurrence matrixes and Hough transform

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106097320A (en) * 2016-06-06 2016-11-09 中国农业大学 Stichopus japonicus image automatic segmentation method and device under water
CN107369140A (en) * 2017-06-29 2017-11-21 大连理工大学 High-accuracy target ball center extraction method under unstructured moving grids
CN108876855A (en) * 2018-05-28 2018-11-23 哈尔滨工程大学 A kind of sea cucumber detection and binocular visual positioning method based on deep learning
CN109166098A (en) * 2018-07-18 2019-01-08 上海理工大学 Work-piece burr detection method based on image procossing
CN109978869A (en) * 2019-03-29 2019-07-05 清华大学 A kind of sea horizon detection method and system based on gray level co-occurrence matrixes and Hough transform

Also Published As

Publication number Publication date
CN111046726A (en) 2020-04-21

Similar Documents

Publication Publication Date Title
CN111080696B (en) Underwater sea cucumber identification and positioning method based on computer vision
CN108596849B (en) Single image defogging method based on sky region segmentation
CN107862667B (en) Urban shadow detection and removal method based on high-resolution remote sensing image
CN108319973B (en) Detection method for citrus fruits on tree
CN107194937B (en) Traditional Chinese medicine tongue picture image segmentation method in open environment
Qiao et al. An automatic active contour method for sea cucumber segmentation in natural underwater environments
CN109978848B (en) Method for detecting hard exudation in fundus image based on multi-light-source color constancy model
Gui et al. A new method for soybean leaf disease detection based on modified salient regions
CN111027497B (en) Weak and small target rapid detection method based on high-resolution optical remote sensing image
CN108537239A (en) A kind of method of saliency target detection
WO2015092904A1 (en) Image-processing apparatus, image-processing method, and image-processing program
CN105405138B (en) Waterborne target tracking based on conspicuousness detection
CN113077486B (en) Method and system for monitoring vegetation coverage rate in mountainous area
CN109785321A (en) Meibomian gland method for extracting region based on deep learning and Gabor filter
CN115578660B (en) Land block segmentation method based on remote sensing image
CN111199195A (en) Pond state full-automatic monitoring method and device based on remote sensing image
CN114677525B (en) Edge detection method based on binary image processing
CN111046726B (en) Underwater sea cucumber identification and positioning method based on AI intelligent vision
Mehena et al. Brain tumor segmentation and extraction of MR images based on improved watershed transform
CN116912260B (en) Broiler chicken breeding health state detection method based on artificial intelligence
CN106372593B (en) Optic disk area positioning method based on vascular convergence
Hua et al. Image segmentation algorithm based on improved visual attention model and region growing
CN112101260A (en) Method, device, equipment and storage medium for identifying safety belt of operator
CN108717699B (en) Ultrasonic image segmentation method based on continuous minimum segmentation
CN115841618A (en) Remote sensing image coastline extraction method based on clustering and edge detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant