CN108090895B - Container lockhole contour extraction method based on image processing - Google Patents

Abstract

The invention discloses a container lock hole contour extraction method based on image processing, which realizes the accurate extraction of a container lock hole contour and improves the positioning accuracy of a container; the automatic container positioning device has the advantages of being high in precision, high in speed, simple in operation and capable of processing in real time, reducing the manual positioning process with low efficiency and complex operation, reducing error problems and time cost caused by manual positioning, and solving the problems of high labor intensity, low efficiency and poor reliability of manual operation.

Description

Container lockhole contour extraction method based on image processing

Technical Field

The invention relates to a container lock hole contour extraction method based on image processing, in particular to a method for processing container corner fittings images by utilizing an image recognition technology to obtain a lock hole contour.

Background

The shipping volume of shipping containers is increasing due to the continuous development of global commerce. From the economic accounting analysis of the voyage times, the residence time of the ship in the port is shortened, the ship berthing cost can be reduced, the cargo throughput of the port is improved, and the economic benefit is further improved. The traditional method of controlling the lifting appliance to grab the container by visual observation of workers seriously affects the loading and unloading speed of goods, and the problems of low efficiency, easy fatigue, low accuracy rate and the like exist in manual operation, so that the actual production and work requirements cannot be met. The method for extracting the contour of the container lock hole based on image processing becomes a better solution because the key problem lies in the accurate positioning of the position of the container.

In order to solve the problem of positioning the lock hole of the container, a plurality of schemes are provided in the academic circles and the industrial circles at home and abroad. The technical scheme which is closer to the invention comprises the following steps: the method detects and positions the container lockholes through a binocular stereo vision technology (based on an image container lockhole positioning technology [ J ]. hoisting and transporting machinery, 2015(10):67-69.), but the method mainly aims at the thought realization and mathematical demonstration of the binocular stereo vision technology, is difficult to apply in an actual scene, is not complete in the introduction of the lockhole positioning and identification related contents, and cannot be well applied to a scene with obvious external interference; and secondly, establishing a container lock hole recognition algorithm research [ J ] based on machine vision, the Chinese engineering machinery science, 2016,14(05), wherein the research identifies the approximate position of a container body through a color space histogram, obtains the specific edge of the container body through projection, further detects the lock hole position at the edge intersection according to the identified edge of the container body, and finally positions the container lock hole position through trapping. Under the condition of low running time, the algorithm can better identify the image of the container which occupies a large position in the graph and is newer in the graph; in an actual application scene, the container is affected by the working conditions of the container, more rusty spots and dust exist on the container, the position of the container is difficult to position through the color of the container body, and due to the problem of uneven illumination, the colors of part of corner fittings and lock holes are difficult to distinguish, so that the accuracy of positioning the lock holes of the container through trapped waves is not high; MIZhaosurpass (container lockhole recognition algorithm research [ C ]// national mechanical engineering Forum. 2015.) based on vision system) the method processes container images through gray histogram equalization, utilizes H channel information in HSV channel to carry out binarization operation to obtain approximate region of the box body, then according to the location of the notched position of the edge region of the box body, utilizes Hough algorithm to detect the circular edge of the notched region, and realizes the accurate location of lockholes; however, in an actual scene, the outline of the lock hole is a rounded rectangle, the Hough circular detection effect for the lock hole area is not obvious, the lock hole area is easily influenced by the external illumination condition, the identification precision is reduced, even the lock hole area cannot be identified, and the system cannot well meet the requirement of the container lock hole accurate identification

In summary, the following disadvantages exist in the current container keyhole outline extraction processing scheme:

(1) most methods firstly identify the container body, then estimate the approximate area of the lockhole according to the edge of the container, but the container is easily polluted due to long-time exposure to the sun, and the accurate positioning of the lockhole is easily influenced due to the incorrect identification of the container;

(2) the accurate outline of the container lock hole is not really recognized in the method, but the accuracy of a recognition system is insufficient, so that the container grabbing failure is easily caused, the container lifting appliance and the container are damaged, and unnecessary loss is caused;

(3) under the influence of illumination conditions, illumination at the positions of the lock holes of the container is uneven, so that part of areas in the lock holes are bright and are shielded by shadows of corner pieces, and the identification precision of the lock holes is reduced;

the container transportation plays an important role in the global trade industry, the construction of an automatic port is an important way for promoting the development of international trade, and the accurate positioning of a container lock hole is the important basis of the automatic grabbing function of the container. However, due to the influence of illumination conditions, part of the lock holes are not uniformly illuminated, which easily causes low positioning accuracy of the lock holes and causes large system errors. The lock hole contour extraction strategy can better identify the accurate contour of the lock hole in the image aiming at the images with uneven illumination and stained corner fitting surfaces, and realize the accurate positioning of the lock hole of the container.

Disclosure of Invention

In order to realize the positioning problem of the container lock hole, the invention provides a container lock hole contour extraction method based on image processing; the scheme adopted for solving the technical problem comprises the following steps:

step 1: collecting an image of a container below by using a camera arranged on a lifting appliance;

step 2: the method for roughly positioning the lock hole of the container based on HOG + SVM is utilized to obtain the rough positioning range of the upper lock hole and the lower lock hole, the width of the rough positioning area of the lock hole is width, the height is height, the unit is pixel, and the rough positioning area is marked as a rough positioning image F of the lock hole₁Image size of A_keyhole：

A_keyholeWidth × height (1) step 3: image F by region Segmentation method Graph-Based Segmentation₁Performing superpixel division to divide the image into n₁A super pixel region set S ═ R_i(x_i,y_i)|i＝1,2,3,…,n₁In which R is_iIs formed by m_iSuper-pixel area composed of pixels, x_iAnd y_iRespectively represent regions R_iAbscissa and ordinate of the center point, R_i＝{p_j|j＝1,2,3,…,m_i}，p_jRepresents a region R_iA pixel of (1);

and 4, step 4: calculating significance values of all super-pixel regions in the set S by using a histogram contrast significance calculation method HC to obtain a significant value set S of the super-pixels_sal＝{v_i|i＝1,2,3,…,n₁}，v_iIs a super pixel region R_iA significance value of; will S_salSuper-image with highest mean saliencyElement region, denoted as R_sal＝{p_j|j＝1,2,3,…,n_salIn which n is_salRepresents R_salThe number of pixels in;

and 5: calculating all superpixel center points to superpixel R in set S_salThe Manhattan distance of the central point is sorted from small to large according to the distance to obtain a super-pixel set S₁＝{R_w|w＝1,2,3,…,n₁}；

Step 6: performing lockhole region spelling and lockhole region aggregation

Degree of area filling c_oldWhen the value is 0, the following is concrete:

step 6.1: from the super-pixel set S₁Taking out the element R₁Is added to the set S₂In S₂Is marked with R_n(ii) a And reacting the element R₁From S₁Deletion in, for set S₁Reordering according to the distance from small to large;

step 6.2: computing a set S₂Number of pixels N:

wherein n is the set S₂Number of super pixels, m_nIs S₂The number of pixels of the super-middle pixel;

step 6.3: extracting the set S₂The minimum value and the maximum value in the abscissa of the middle pixel are respectively marked as x₁And x₂Set S₂The minimum value and the maximum value in the vertical coordinate of the middle pixel are respectively marked as y₁And y₂(ii) a Calculate the set S₂The middle pixel constitutes the minimum bounding rectangle R (w, h) of the image, where w represents the width of the rectangle R, h represents the height of the rectangle R, the area of the rectangle is a:

w＝x₂-x₁ (4)

h＝y₂-y₁ (5)

A＝w×h (6)

step 6.4: calculating the region filling degree c after region merging_new：

Step 6.5: if the condition c is satisfied_new≥c_oldThen c will be_newValue of (c) is given to_old(ii) a Otherwise, the element R is_nFrom S₂Deleting; repeating steps 6.1 to 6.5 until

Or the number of iterations is greater than λ₁Wherein λ is₁Representing a predetermined maximum number of iterations;

and 7: calculating to obtain a lockhole area set S₂＝{p_u|u＝1,2,3,…,n₂}, region set S₂The circumscribed rectangle of (A) is R_box(x_box,y_box,w_box,h_box) Wherein p is_uRepresentation set S₂Pixel of (2), n₂Denotes S₂Number of pixels in, x_boxAnd y_boxRespectively represent a rectangle R_boxAbscissa and ordinate of the upper left corner, w_boxAnd h_boxRespectively represent a rectangle R_boxWidth and height of (d);

and 8: calculating the circumscribed rectangle R of the lock hole by adopting the formula (8) or (9)_hole(x_hole,y_hole,w_hole,h_hole)，x_holeAnd y_holeRespectively showing the abscissa and ordinate of the upper left corner of the rectangle circumscribed by the keyhole, w_holeAnd h_holeRespectively representing the width and the height of a circumscribed rectangle of the lock hole;

and step 9: adopting GrubCut graph cut algorithm to cut image F₁Processing is carried out, and approximate foreground and background areas of the image are set; setting the area in the external rectangle as possible foreground area and the area outside the rectangle as background area, and meeting the condition n₂≥λ₂Wherein λ is₂A preset keyhole size threshold value is obtained; otherwise, set S is set₂Middle pixel being a possible foreground region, region R_salMiddle pixel is foreground region, image F₁The rest of the image is a background area;

step 10: iterating GrubCut graph cut algorithm lambda₃Second, lambda₃For a given number of image cutting iterations, a keyhole contour image F can be obtained₂。

THE ADVANTAGES OF THE PRESENT INVENTION

The accurate extraction of the outline of the lock hole of the container is realized, and the positioning accuracy of the container is improved; the automatic container positioning device has the advantages of being high in precision, high in speed, simple in operation and capable of processing in real time, reducing the manual positioning process with low efficiency and complex operation, reducing error problems and time cost caused by manual positioning, and solving the problems of high labor intensity, low efficiency and poor reliability of manual operation.

Drawings

Fig. 1 is a photograph of an upper portion of a container taken in accordance with an exemplary embodiment of the present invention.

FIG. 2 is a rough keyhole positioning image obtained by HOG + SVM processing in step 2 according to the present invention.

FIG. 3 is a super pixel region image after the region segmentation in step 3 according to the present invention.

Fig. 4 is a saliency region image after the saliency calculation in step 4 according to the present invention.

Fig. 5 is a keyhole area image obtained by stitching keyhole areas in step 6 according to the present invention.

Fig. 6 is an image of the precise contour of the keyhole obtained in step 10 according to the present invention.

Detailed Description

The following describes in detail a specific embodiment of the container keyhole contour extraction method according to the present invention with reference to an implementation example. The method comprises the following steps:

step 1: collecting an image of a container below by using a camera arranged on a lifting appliance;

step 2: the method for roughly positioning the lock hole of the container based on HOG + SVM is utilized to obtain the rough positioning range of the upper lock hole and the lower lock hole, the width of the rough positioning area of the lock hole is width, the height is height, the unit is pixel, and the rough positioning area is marked as a rough positioning image F of the lock hole₁Image size of A_keyhole：

A_keyhole＝width×height (1)

And step 3: image F by region Segmentation method Graph-Based Segmentation₁Performing superpixel division to divide the image into n₁A super pixel region set S ═ R_i(x_i,y_i)|i＝1,2,3,…,n₁In which R is_iIs formed by m_iSuper-pixel area composed of pixels, x_iAnd y_iRespectively represent regions R_iAbscissa and ordinate of the center point, R_i＝{p_j|j＝1,2,3,…,m_i}，p_jRepresents a region R_iA pixel of (1);

and 4, step 4: calculating significance values of all super-pixel regions in the set S by using a histogram contrast significance calculation method HC to obtain a significant value set S of the super-pixels_sal＝{v_i|i＝1,2,3,…,n₁}，v_iIs a super pixel region R_iA significance value of; will S_salThe super pixel region with the highest medium saliency value is marked as R_sal＝{p_j|j＝1,2,3,…,n_salIn which n is_salRepresents R_salThe number of pixels in;

and 5: calculating all superpixel center points to superpixel R in set S_salThe Manhattan distance of the central point is sorted from small to large according to the distance to obtain a super-pixel set S₁＝{R_w|w＝1,2,3,…,n₁}；

Step 6: performing lockhole region spelling and lockhole region aggregation

Degree of area filling c_oldWhen the value is 0, the following is concrete:

step 6.1: from the super-pixel set S₁Taking out the element R₁Is added to the set S₂In S₂Is marked with R_n(ii) a And reacting the element R₁From S₁Deletion in, for set S₁Reordering according to the distance length from small to large;

step 6.2: computing a set S₂Number of pixels N:

wherein n is the set S₂Number of super pixels, m_nIs S₂The number of pixels of the super-middle pixel;

step 6.3: extracting the set S₂The minimum value and the maximum value of the abscissa of the middle pixel are respectively marked as x₁And x₂Set S₂The minimum value and the maximum value of the vertical coordinate of the middle pixel are respectively marked as y₁And y₂(ii) a Calculate the set S₂The middle pixel constitutes the minimum bounding rectangle R (w, h) of the image, where w represents the width of the rectangle R, h represents the height of the rectangle R, the area of the rectangle is a:

w＝x₂-x₁ (4)

h＝y₂-y₁ (5)

A＝w×h (6)

step 6.4: calculating the region filling degree c after region merging_new：

Step 6.5: if the condition c is satisfied_new≥c_oldThen c will be_newValue of (c) is given to_old(ii) a Inverse directionWherein the element R is_nFrom S₂Deleting; repeating steps 6.1 to 6.5 until

Or the number of iterations is greater than λ₁Wherein λ is₁Denotes a predetermined maximum number of iterations, in this example λ₁10; and 7: calculating to obtain a lockhole area set S₂＝{p_u|u＝1,2,3,…,n₂}, region set S₂The circumscribed rectangle of (A) is R_box(x_box,y_box,w_box,h_box) Wherein p is_uRepresentation set S₂Pixel of (2), n₂Denotes S₂Number of pixels in, x_boxAnd y_boxRespectively represent a set S of regions₂Abscissa and ordinate of the top left corner of the circumscribed rectangle, w_boxAnd h_boxRespectively represent a set S of regions₂The width and height of the circumscribed rectangle;

and 8: calculating the circumscribed rectangle R of the lock hole by adopting the formula (8) or (9)_hole(x_hole,y_hole,w_hole,h_hole)，x_holeAnd y_holeRespectively showing the abscissa and ordinate of the upper left corner of the rectangle circumscribed by the keyhole, w_holeAnd h_holeRespectively representing the width and the height of a circumscribed rectangle of the lock hole;

and step 9: image F is cut by using GrubCut graph cut algorithm₁Processing is carried out, and approximate foreground and background areas of the image are set; setting the area in the external rectangle as possible foreground area and the area outside the rectangle as background area, and meeting the condition n₂≥λ₂Wherein λ is₂For a predetermined keyhole size threshold, in this example, λ₂1000; otherwise, the reverse is carried outSet S₂Middle pixel being a possible foreground region, region R_salMiddle pixel is foreground region, image F₁The rest of the image is a background area;

step 10: iterating GrubCut graph cut algorithm lambda₃Next, a keyhole contour image F can be obtained₂，λ₃For a predetermined number of graph cut iterations, in this example, λ₃＝3。

The embodiments described in this specification are merely illustrative of implementations of the inventive concept and the scope of the present invention should not be considered limited to the specific forms set forth in the embodiments but rather by the equivalents thereof as may occur to those skilled in the art upon consideration of the present inventive concept.

Claims (1)

1. A container lock hole contour extraction method based on image processing comprises the following steps:

step 1: collecting an image of a container below by using a camera arranged on a lifting appliance;

step 2: the method for roughly positioning the lock hole of the container based on HOG + SVM is utilized to obtain the rough positioning range of the upper lock hole and the lower lock hole, the width of the rough positioning area of the lock hole is width, the height is height, the unit is pixel, and the rough positioning area is marked as a rough positioning image F of the lock hole₁Image size of A_keyhole：

A_keyhole＝width×height (1)；

And step 3: image F by region Segmentation method Graph-Based Segmentation₁Performing superpixel division to divide the image into n₁A super pixel region set S ═ R_i(x_i,y_i)|i＝1,2,3,…,n₁In which R is_iIs formed by m_iSuper-pixel area composed of pixels, x_iAnd y_iRespectively represent regions R_iAbscissa and ordinate of the center point, R_i＝{p_j|j＝1,2,3,…,m_i}，p_jRepresents a region R_iA pixel of (1);

and 4, step 4: calculating significance values of all super-pixel regions in the set S by using a histogram contrast significance calculation method HC to obtain a significant value set S of the super-pixels_sal＝{v_i|i＝1,2,3,…,n₁}，v_iIs a super pixel region R_iA significance value of; will S_salThe super pixel region with the highest medium saliency value is marked as R_sal＝{p_j|j＝1,2,3,…,n_salIn which n is_salRepresents R_salThe number of pixels in;

and 5: calculating all superpixel center points to superpixel R in set S_salThe Manhattan distance of the central point is sorted from small to large according to the distance to obtain a super-pixel set S₁＝{R_w|w＝1,2,3,…,n₁}；

Step 6: performing lockhole region spelling and lockhole region aggregation

Degree of area filling c_old＝0；

And 7: calculating to obtain a lockhole area set S₂＝{p_u|u＝1,2,3,…,n₂}, region set S₂The circumscribed rectangle of (A) is R_box(x_box,y_box,w_box,h_box) Wherein p is_uRepresentation set S₂Pixel of (2), n₂Denotes S₂Number of pixels in, x_boxAnd y_boxRespectively represent a set S of regions₂Abscissa and ordinate of the top left corner of the circumscribed rectangle, w_boxAnd h_boxRespectively represent a set S of regions₂The width and height of the circumscribed rectangle;

and 8: calculating the circumscribed rectangle R of the lock hole by adopting the formula (8) or (9)_hole(x_hole,y_hole,w_hole,h_hole)，x_holeAnd y_holeRespectively showing the abscissa and ordinate of the upper left corner of the rectangle circumscribed by the keyhole, w_holeAnd h_holeRespectively representing the width and the height of a circumscribed rectangle of the lock hole;

and step 9: image F is cut by using GrubCut graph cut algorithm₁Processing is carried out, and foreground and background areas of the image are set; setting the area in the external rectangle as possible foreground area and the area outside the rectangle as background area, and meeting the condition n₂≥λ₂Wherein λ is₂A preset keyhole size threshold value is obtained; if the condition n is not satisfied₂≥λ₂Set S₂Middle pixel being a possible foreground region, region R_salMiddle pixel is foreground region, image F₁The rest of the image is a background area;

step 10: iterating GrubCut graph cut algorithm lambda₃Next, a keyhole contour image F can be obtained₂，λ₃Cutting iteration times for a preset graph;

the step 6 is as follows:

step 6.1: from the super-pixel set S₁Taking out the element R₁Is added to the set S₂In S₂Is marked with R_n(ii) a And reacting the element R₁From S₁Deletion in, for set S₁Reordering according to the distance length from small to large;

step 6.2: computing a set S₂Number of pixels N:

wherein n is the set S₂Number of super pixels, m_nIs S₂The number of pixels of the super-middle pixel;

step 6.3: extracting the set S₂The minimum value and the maximum value of the abscissa of the middle pixel are respectively marked as x₁And x₂Set S₂The minimum value and the maximum value of the vertical coordinate of the middle pixel are respectively marked as y₁And y₂(ii) a Calculate the set S₂The middle pixel constitutes the minimum bounding rectangle R (w, h) of the image, where w represents the width of the rectangle R, h represents the height of the rectangle R, the area of the rectangle is a:

w＝x₂-x₁ (4)

h＝y₂-y₁ (5)

A＝w×h (6)

step 6.4: calculating the region filling degree c after region merging_new：

Step 6.5: if the condition c is satisfied_new≥c_oldThen c will be_newValue of (c) is given to_old(ii) a Otherwise, the element R is_nFrom S₂Deleting; repeating steps 6.1 to 6.5 until

Or the number of iterations is greater than λ₁Wherein λ is₁Representing a predetermined maximum number of iterations.

Images