CN112381834A

CN112381834A - Labeling method for image interactive instance segmentation

Info

Publication number: CN112381834A
Application number: CN202011145197.XA
Authority: CN
Inventors: 李融
Original assignee: Zhejiang Lab
Current assignee: Zhejiang Lab
Priority date: 2021-01-08
Filing date: 2021-01-08
Publication date: 2021-02-19
Anticipated expiration: 2041-01-08
Also published as: CN112381834B

Abstract

The invention discloses an annotation method for image interactive instance segmentation, which comprises the step of S1, constructing a to-be-processed set E and an annotated set C_iAnd initializing; s2, the set E is divided by an interactive image division algorithm to form a subdivided region R_kAnd E is set to be empty; s3, dividing the region R into sub-regions_kPut into the set E and the set C respectively_iEach subdivided region is placed in only one set; if the region R is subdivided_kIs completely contained by the ith target instance object area, then R is added_kPut into set C_iPerforming the following steps; if the region R is subdivided_kHaving intersecting regions with any one of the target instance regions that are not empty and the intersecting regions not equal to R_kThen R is_kPutting the mixture into a set E; s4, when the set E is not empty, repeating the steps S2 and S3 untilThe set E is an empty set; s5, collecting the set C_iAll the subdivided areas in the target are merged, and each pixel in the merged area is marked as the ith target instance.

Description

Labeling method for image interactive instance segmentation

Technical Field

The invention relates to the technical field of computer image processing, in particular to an annotation method for image interactive instance segmentation.

Background

Image instance segmentation is an important research effort in the field of artificial intelligence computer vision. This work provides pixel-level instance differentiation for image frames, enabling understanding of image content. In the current stage, the artificial intelligence algorithm based on deep learning needs a large amount of image labels at pixel level when an instance/semantic segmentation model is trained, and has wide application requirements in numerous fields including unmanned driving, intelligent matting and the like. The traditional labeling method for carrying out example segmentation on the image mainly comprises the step of manually outlining a pixel-level outline of a specific individual in the image to finally form full-image pixel-level labeling. The method requires a long-time and high-precision operation of a marking person. When the image content is complex and the data size is large, a large amount of manual operation time is consumed.

Disclosure of Invention

In order to solve the defects of the prior art and achieve the purpose of reducing the labor cost of outline delineation, the invention adopts the following technical scheme:

an annotation method for image interactive instance segmentation comprises the following steps:

s1, setting the input image as I, setting N target examples to be marked on the image, and constructing a set E and a set C_iN, each element of the set E representing an image area to be processed on the image I, the set C_iEach element in the set represents an image area which is marked and belongs to the ith target instance area, the initial set E has only one element and is the whole image area of the image I, and the initial set C_iIs an empty set;

s2, dividing all the image areas to be processed in the set E through an interactive image division algorithm to form K subdivided areas R_kK =1,2, …, K, after which E is left empty;

s3, dividing the region R into sub-regions_kPut into the set E and the set C respectively_iThe conditions for putting into the collection are as follows: each subdivided region is placed in only one set; if the region R is subdivided_kIs completely contained by the ith target instance object area, then R is added_kPut into set C_iPerforming the following steps; if the region R is subdivided_kHaving intersecting regions with any one of the target instance regions that are not empty and the intersecting regions not equal to R_kThen R is_kPutting the mixture into a set E;

s4, when the set E is not empty, repeating the steps S2 and S3 until the set E is an empty set;

s5, for each i e { 1.. N }, a set C is collected_iAnd merging all the subdivided regions in the image, wherein the merged region represents the region of the ith target instance, and labeling each pixel in the merged region as the ith target instance.

And step S2, when the area of the region to be processed is greater than or equal to the preset threshold, subdividing the region by using an automatic image segmentation algorithm, where the automatic image segmentation algorithm has a set of subdivision parameters, the number of the output subdivided regions is controlled by setting the subdivision parameters, and when the area of the region to be processed is smaller than the preset threshold, the boundary of the target instance in the region to be processed is identified by interactive operation, so as to segment the subdivided region belonging to the target instance.

The subdivision parameters are adjusted until the local boundary error of the boundary of the subdivided region from the target instance is less than a pre-specified error threshold.

In the step S3, the region R is subdivided₁,R₂,...,R_KPut into the sets C respectively₁，C₂,...,C_NThe method comprises the following steps:

s311, selecting the ith target instance;

s312, putting the subdivided regions positioned in the ith target instance into a set C_iIn the method, a group of tracks are interactively drawn in the ith target instance region, and the subdivided regions which are intersected with the tracks and completely fall in the target instance region are put into a set C_iPerforming the following steps;

s313, if all the subdivided areas located inside the ith target instance area are put into the set C_iThen, the process is ended, otherwise, the step S312 is repeated.

Step S312, selecting a pixel in the target instance region, setting a growth threshold, performing region growth from the pixel according to a region growth algorithm, where the generated region includes a plurality of subdivided regions, and setting the plurality of subdivided regions as candidate subdivided regions, and if the candidate subdivided region is completely in the ith target instance region, setting the candidate subdivided region as a candidate subdivided regionThese regions are put into the set C_i. If the candidate subdivision regions exceed or are not completely inside the ith target instance, interactively identifying the regions, and putting the candidate subdivision regions which are not identified into the set C_i。

In the step S3, the region R is subdivided₁,R₂,...,R_KRespectively putting the sets E into the sets E, and comprising the following steps:

s321, selecting the ith target instance;

s322, drawing a group of tracks, enabling the tracks to pass through the subdivision area where the ith target instance area outline is located, and putting the subdivision areas intersected with the tracks into a set E;

and S323, when all the subdivided regions intersected with the ith target instance are in the set E and the intersected parts are not equal to the subdivided regions, ending, otherwise, repeating the step S322.

The invention has the advantages and beneficial effects that:

the method has the advantages that through an interactive image segmentation algorithm, set classification is combined, circulation operation is carried out, and required instance segmentation labeling is completed in an auxiliary mode by utilizing the computing power of a computer, so that the labor cost of traditional labeling is reduced, the degree of uncertainty is reduced, the labeling precision is improved, and the working efficiency is improved.

Drawings

FIG. 1 is a flow chart of the method of the present invention.

Fig. 2 is an exemplary diagram of forming candidate inner subdivided regions in the present invention.

FIG. 3 is an exemplary diagram of finding the interior regions of several example objects from a drawn trajectory in accordance with the present invention.

FIG. 4 is an exemplary diagram of a subdivided region to be segmented in which the outlines of a plurality of example objects are obtained according to a drawn track.

FIG. 5 is an exemplary diagram of an internal segment map of several target objects obtained after adjusting parameters in the present invention.

FIG. 6 is an exemplary diagram of the results of an example segmentation annotation in accordance with the present invention.

Detailed Description

The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.

As shown in fig. 1, an annotation method for interactive instance segmentation of an image includes the following steps:

step one, an input image is set as I, as shown in figure 2, 2 target sheep examples to be labeled are arranged on the image, a set E and a set C are constructed_i(where I =1, 2), where each element in the set E represents an image area to be processed on the image I, the set C₁Including the subdivided regions of the left-hand annotated image, set C₂Including the image subdivision areas in the right sheep where labeling has been completed. The initial set E has only one element and is the whole image area of the image I, and the initial set C₁，C₂Is an empty set;

secondly, all the image areas to be processed in the set E are segmented through an interactive image segmentation algorithm to form K subdivided areas R₁,R₂,...,R_KAnd thereafter, E is left empty. As shown in fig. 2, an automatic image segmentation algorithm is adopted to segment the image area to be processed, segmentation parameters are manually set, the number of the output segmented areas is controlled by setting the segmentation parameters, and the matching degree of the edge contour and the segmentation algorithm of the local parts of the two sheep is observed. When the edge of the subdivided region divided by the algorithm is observed to be matched with the local part of the sheep, the subdivision is stopped, K subdivided regions are formed at the moment, and the subdivided regions are schematically divided by using a dividing line in the figure.

Third, the subdivided regions R are interactively subdivided₁,R_2,...,R_KPut into corresponding sets C respectively_iIn (1). As shown in fig. 2, the left sheep is selected, a pixel is selected in the left sheep, a growth threshold is set, region growth is performed from the pixel according to a region growth algorithm, the generated region includes a plurality of subdivided regions (highlighted portions in fig. 2), and the subdivided regions are set as candidate subdivided regions. Observing these candidate subdivided regions, having been completely inside the left sheep region, these regions are put into the set C₁. However, the device is not suitable for use in a kitchenThen, as shown in fig. 3, a track is interactively drawn in the left sheep region, a subdivided region which is intersected with the track and completely falls into the left sheep is selected, and the subdivided region is continuously put into the set C₁In (1). These two operations are repeated until the inner subdivided area of the left sheep is filled with C₁In (1). Selecting right sheep, and performing similar operation until the inner subdivision region is placed in C₂In (1).

Interactively putting the remaining subdivided regions on into the set E. Specifically, as shown in fig. 4, the left sheep is selected first, and a trajectory is drawn. The trajectory crosses the left sheep partial contour, and the subdivided regions intersecting the trajectory contain partial sheep contour inner pixels and partial sheep contour outer pixels. These regions are added to set E. Similar operations are repeated until the subdivided areas which are locally intersected with any sheep are in the set E, and the subdivided areas need to satisfy the following conditions: there is an intersection with the sheep and the intersection is not equal to the subdivided region itself.

And fourthly, repeating the second step and the third step n times when the E is not empty until the set E is an empty set. Specifically, as shown in fig. 5, in the area E, the left leg is further subdivided locally, and by drawing a track, some detail internal areas of the sheep are precisely selected and placed in the area C₁In (1). Similar operations continue until E is empty.

Fifthly, for each i E { 1.. N }, a set C is collected_iAll the areas in the image are merged, the merged result area is the area of the ith target instance, and each pixel in the area is marked as the ith target instance. Specifically, as shown in FIG. 6, for C₁The inner areas are merged, and the merged pixel area is used for segmenting and labeling the example of the left sheep; to C₂The inner regions are merged, and the merged pixel region is used for segmenting and labeling the examples of the right sheep.

The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An annotation method for interactive instance segmentation of images, characterized by comprising the steps of:

2. The method as claimed in claim 1, wherein in step S2, when the area of the region to be processed is greater than or equal to the preset threshold, the region is subdivided by using an automatic image segmentation algorithm, the automatic image segmentation algorithm has a set of segmentation parameters, the number of the subdivided regions is controlled and output by setting the segmentation parameters, and when the area of the region to be processed is smaller than the preset threshold, the boundary of the target instance in the region to be processed is identified by interactive operation, so as to segment the subdivided region belonging to the target instance.

3. The method of claim 2, wherein the subdivision parameters are adjusted until the local boundary error between the boundary of the subdivided region and the target instance is less than a pre-specified error threshold.

4. The method for labeling interactive instance segmentation of images as claimed in claim 1, wherein said step S3 is to subdivide the region R₁,R₂,...,R_KPut into the sets C respectively₁，C₂,...,C_NThe method comprises the following steps:

s311, selecting the ith target instance;

5. The method as claimed in claim 4, wherein in step S312, a pixel is selected in the target instance region, a growing threshold is set, region growing is performed from the pixel according to a region growing algorithm, and the generated region comprises a plurality of subdivisionsSetting a plurality of subdivided areas as candidate subdivided areas, and putting the candidate subdivided areas into the set C if the candidate subdivided areas are completely inside the ith target instance area_i。

6. The method for labeling interactive instance segmentation of images as claimed in claim 1, wherein said step S3 is to subdivide the region R₁,R₂,...,R_KRespectively putting the sets E into the sets E, and comprising the following steps:

s321, selecting the ith target instance;