CN114066788B - Balanced instance segmentation data synthesis method - Google Patents
Balanced instance segmentation data synthesis method Download PDFInfo
- Publication number
- CN114066788B CN114066788B CN202111245120.4A CN202111245120A CN114066788B CN 114066788 B CN114066788 B CN 114066788B CN 202111245120 A CN202111245120 A CN 202111245120A CN 114066788 B CN114066788 B CN 114066788B
- Authority
- CN
- China
- Prior art keywords
- image
- label
- size
- pasting
- foreground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 14
- 238000001308 synthesis method Methods 0.000 title claims abstract description 12
- 238000000034 method Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 abstract description 3
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a balanced instance segmentation data synthesis method, which comprises the following steps: 1) Constructing an object instance library by using the image and the label of the original data set; 2) Reading an image and a label in the original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10 multiplied by 10 candidate points according to the size of the image; 3) Setting a pasting size list, calculating a region with 10 multiplied by 10 candidate points as the center and a foreground and background mask map according to the set pasting size list, and selecting a region which is not overlapped with the foreground and adding the region into a pasting region; 4) And selecting the object from the object instance library through class balancing, scaling, pasting the object to a pasting area, and updating the label. The method for realizing data enhancement by using the image synthesis method has better applicability and diversity, can be applied to instance segmentation tasks with higher difficulty, has very little calculated amount and high operation speed, and basically does not increase the time of a training network.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a balanced instance segmentation data synthesis method.
Background
Object detection and instance segmentation are important research directions in deep learning, and are used for detecting and identifying objects in images. Specifically, the object detection is to frame the object in the image by a rectangular frame, and the instance segmentation is further to realize the object separation at the pixel level. Whether it is object detection or instance segmentation, the data set is an essential part required for training the network, but in practical application, the data set is very limited, not only in number but also in quality, there is an imbalance problem: scale imbalance, category imbalance, distribution imbalance.
The main method for solving the deficiency of the data set is data enhancement, the research and the realization of the data enhancement are mainly color transformation and geometric transformation, no more remarkable innovation is developed until now, a certain stagnation is trapped, and the data synthesis is an emerging method in the data enhancement and provides more feasible solutions.
By combining the above discussion, the balanced example segmentation data synthesis method has higher practical application value.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a balanced instance segmentation data synthesis method, which mainly utilizes images and labels of an original data set to construct an object instance library, synthesizes the original data set and the object instance library through an image processing technology, synthesizes a new image and generates a new label, thereby achieving the effect of data enhancement and relieving the defects of the data set.
In order to achieve the above purpose, the technical scheme provided by the invention is as follows: a balanced instance segmentation data synthesis method, comprising the steps of:
1) Constructing an object instance library by using the image and the label of the original data set;
2) Reading an image and a label in the original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10 multiplied by 10 candidate points according to the size of the image;
3) Setting a pasting size list, calculating the region with 10 multiplied by 10 candidate points as the center in the step 2) and a foreground background mask image according to the set pasting size list, and selecting a region which is not overlapped with the foreground and adding the region into a pasting region;
4) Selecting an object from the object instance library in the step 1) through class balancing, scaling, pasting the object to the pasting area in the step 3), and updating the label.
Further, in step 1), an object instance library is constructed by using the image and the label of the original dataset, a mask corresponding to each object in the image is obtained by parsing the label, the objects are separated from the original image by the mask, and the objects are classified according to the category to which the objects belong.
Further, in step 2), reading an image and a label in the original data set, generating a foreground-background mask image for the image according to the label, and uniformly generating 10×10 candidate points according to the size of the image, including the steps of:
2.1 Generating foreground-background mask patterns
Obtaining an original image and a corresponding label, generating a single-channel image with the length and width dimensions of the original image, setting each pixel value to be 0, namely a foreground background mask image, obtaining a mask of an object in the label, setting the pixel value of a position corresponding to the foreground background mask image to be 1, setting a region with the final pixel value of 1 as the foreground, and setting a region with the pixel value of 0 as the background;
2.2 Generating 10×10 candidate points
The length and width values of the original image are obtained, the length and width are respectively and uniformly divided into 11 sections, namely 10 equally divided points are obtained, the 10 equally divided points of the length and width are respectively combined, and finally 10 multiplied by 10 candidate points are obtained.
Further, in step 3), a paste size list is set, and a region meeting the requirements is selected from the region centered on the 10×10 candidate points of step 2) to be added to the paste region according to the set paste size list, comprising the steps of:
3.1 Setting a paste size list
For the relative equalization of paste sizes, a paste size list is set to 150, 120, 90, 60, 30;
3.2 Generating a paste area
Sequentially selecting the sizes from the pasting size list, taking the selected sizes as square side lengths, taking the candidate points as centers, judging whether the candidate points meet the condition of no prospect, adding a pasting area and selecting the next size if no prospect exists, and selecting the next candidate points until the pasting size list and all the candidate points are traversed.
Further, in step 4), the pasting area in step 3) is sequentially processed, a category is randomly selected from the object instance library in step 1), then an object is randomly selected from the category, if the object size is smaller than the size of the pasting area, an object is randomly selected again, after the object size meets the size requirement, the object size is scaled to a size which is suitable for the size of the pasting area, and then the pasting area is pasted, and a new label is added on the original label.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the invention uses the data synthesis method to realize data enhancement, and has better applicability and diversity compared with the traditional data enhancement method.
2. The data synthesized by the method can be applied to not only the target detection task, but also the instance segmentation task with higher difficulty.
3. The data synthesis method can be continuously used on the basis of the traditional data enhancement method, and a better data enhancement effect is achieved.
4. The invention uses the image processing method, the calculated amount is very little, the operation speed is fast, and the time for training the network is not increased basically.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is an original image schematic diagram.
Fig. 3 is a schematic diagram of an original image and a label.
Fig. 4 is a schematic diagram of a composite image.
Fig. 5 is a schematic diagram of a composite image and label.
Detailed Description
The present invention will be described in further detail with reference to examples and drawings, but embodiments of the present invention are not limited thereto.
As shown in fig. 1, the present embodiment provides a balanced example segmentation data synthesis method, which includes the following steps:
1) The dataset uses the Microsoft COCO dataset (COCO for short) and the object instance library is constructed using the images and labels of the COCO dataset.
2) Reading an image and a label in the COCO data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10×10 candidate points according to the size of the image, as shown in fig. 2 and 3, wherein the method comprises the following steps:
2.1 Generating foreground-background mask patterns
The method comprises the steps of obtaining an original image and a corresponding label, generating a single-channel image with the length and width dimensions of the original image, enabling each pixel value to be 0, namely, a foreground background mask image, obtaining a mask of an object in the label, enabling the pixel value of a position corresponding to the foreground background mask image to be 1, enabling a region with the final pixel value of 1 to be the foreground, and enabling a region with the pixel value of 0 to be the background.
2.2 Generating 10×10 candidate points
The length and width values of the original image are obtained, the length and width are respectively and uniformly divided into 11 sections, namely 10 equally divided points are obtained, the 10 equally divided points of the length and width are respectively combined, and finally 10 multiplied by 10 candidate points are obtained.
3) Setting a paste size list, calculating from the region with 10 multiplied by 10 candidate points as the center in the step 2) and the foreground and background mask map according to the set paste size list, and selecting a region which does not overlap with the foreground to add into the paste region, wherein the method comprises the following steps:
3.1 Setting a paste size list
For the relative equalization of paste sizes, a paste size list is set of 150, 120, 90, 60, 30.
3.2 Generating a paste area
Sequentially selecting the sizes from the pasting size list, taking the selected sizes as square side lengths, taking the candidate point as a center, judging whether the candidate point meets the condition of no prospect, adding a pasting area and selecting the next size if no prospect exists, and selecting the next candidate point if the prospect exists. Until the paste size list and all candidate points are traversed.
4) Sequentially processing the pasting areas in the step 3), randomly selecting a category from the object instance library in the step 1), randomly selecting an object from the category, randomly selecting an object again if the size of the object is smaller than the size of the pasting area, scaling the size of the object to a size which is suitable for the size of the pasting area after the size requirement is met, pasting the object to the pasting area, and adding a new label on the original label, wherein the new label is shown in fig. 4 and 5.
In summary, after the scheme is adopted, the invention provides a new method for data enhancement, and the data synthesis is used as an effective method for data enhancement, so that the problems of insufficient data quantity and unbalanced data can be effectively solved, the development of deep learning technology is effectively promoted, and the method has practical popularization value and is worthy of popularization.
The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.
Claims (1)
1. A balanced instance segmentation data synthesis method, comprising the steps of:
1) Constructing an object instance library by using the image and the label of the original dataset, analyzing the label to obtain a mask corresponding to each object in the image, separating the object from the original image by the mask, and classifying the object according to the category to which the object belongs;
2) Reading an image and a label in the original data set, generating a foreground background mask image for the image according to the label, and uniformly generating 10×10 candidate points according to the size of the image, wherein the method comprises the following steps:
2.1 Generating foreground-background mask patterns
Obtaining an original image and a corresponding label, generating a single-channel image with the length and width dimensions of the original image, setting each pixel value to be 0, namely a foreground background mask image, obtaining a mask of an object in the label, setting the pixel value of a position corresponding to the foreground background mask image to be 1, setting a region with the final pixel value of 1 as the foreground, and setting a region with the pixel value of 0 as the background;
2.2 Generating 10×10 candidate points
Acquiring the length and width values of an original image, uniformly dividing the length and width into 11 sections respectively, namely, 10 equally dividing points are acquired, and combining the 10 equally dividing points respectively to finally acquire 10 multiplied by 10 candidate points;
3) Setting a paste size list, calculating the region with 10 multiplied by 10 candidate points as the center in the step 2) and a foreground background mask map according to the set paste size list, and selecting a region which does not overlap with the foreground to be added into the paste region, wherein the method comprises the following steps:
3.1 Setting a paste size list
For the relative equalization of paste sizes, a paste size list is set to 150, 120, 90, 60, 30;
3.2 Generating a paste area
Sequentially selecting the sizes from the pasting size list, taking the selected sizes as square side lengths, taking the candidate points as centers, judging whether the candidate points meet the condition of no prospect, adding a pasting area and selecting the next size if no prospect exists, and selecting the next candidate points until the pasting size list and all the candidate points are traversed;
4) Selecting an object from the object instance library in the step 1) through class balancing, scaling, pasting the object to the pasting area in the step 3), and updating the label, wherein the method specifically comprises the following steps:
sequentially processing the pasting areas in the step 3), randomly selecting a category from the object instance library in the step 1), randomly selecting an object from the category, randomly selecting an object again if the size of the object is smaller than a threshold value compared with the size of the pasting area, scaling the size of the object to a size which is suitable for the size of the pasting area after the size requirement is met, pasting the object to the pasting area, and adding a new label on the original label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245120.4A CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111245120.4A CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114066788A CN114066788A (en) | 2022-02-18 |
| CN114066788B true CN114066788B (en) | 2024-03-29 |
Family
ID=80235484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111245120.4A Active CN114066788B (en) | 2021-10-26 | 2021-10-26 | Balanced instance segmentation data synthesis method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114066788B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114418898B (en) * | 2022-03-21 | 2022-07-26 | 南湖实验室 | Data enhancement method based on target overlapping degree calculation and self-adaptive adjustment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111768415A (en) * | 2020-06-15 | 2020-10-13 | 哈尔滨工程大学 | Image instance segmentation method without quantization pooling |
| CN111832745A (en) * | 2020-06-12 | 2020-10-27 | 北京百度网讯科技有限公司 | Data augmentation method and device and electronic equipment |
| CN112132832A (en) * | 2020-08-21 | 2020-12-25 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for enhancing image instance segmentation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8396328B2 (en) * | 2001-05-04 | 2013-03-12 | Legend3D, Inc. | Minimal artifact image sequence depth enhancement system and method |
-
2021
- 2021-10-26 CN CN202111245120.4A patent/CN114066788B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111832745A (en) * | 2020-06-12 | 2020-10-27 | 北京百度网讯科技有限公司 | Data augmentation method and device and electronic equipment |
| CN111768415A (en) * | 2020-06-15 | 2020-10-13 | 哈尔滨工程大学 | Image instance segmentation method without quantization pooling |
| CN112132832A (en) * | 2020-08-21 | 2020-12-25 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for enhancing image instance segmentation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114066788A (en) | 2022-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111461114B (en) | Multi-scale feature pyramid text detection method based on segmentation | |
| CN111126472B (en) | SSD (solid State disk) -based improved target detection method | |
| CN111709420B (en) | Text detection method, electronic device and computer readable medium | |
| CN110378222B (en) | Method and device for detecting vibration damper target and identifying defect of power transmission line | |
| CN111640125B (en) | Aerial photography graph building detection and segmentation method and device based on Mask R-CNN | |
| CN108288075A (en) | A kind of lightweight small target detecting method improving SSD | |
| CN113609896A (en) | Object-level remote sensing change detection method and system based on dual-correlation attention | |
| CN109948593A (en) | Based on the MCNN people counting method for combining global density feature | |
| CN107506792B (en) | Semi-supervised salient object detection method | |
| CN110263877B (en) | Scene character detection method | |
| CN114998337B (en) | Scratch detection method, device, equipment and storage medium | |
| CN115131797A (en) | Scene text detection method based on feature enhancement pyramid network | |
| CN109523558A (en) | A kind of portrait dividing method and system | |
| CN110598698A (en) | Natural scene text detection method and system based on adaptive regional suggestion network | |
| CN114612872A (en) | Target detection method, target detection device, electronic equipment and computer-readable storage medium | |
| CN114066788B (en) | Balanced instance segmentation data synthesis method | |
| Li et al. | Gated auxiliary edge detection task for road extraction with weight-balanced loss | |
| CN110232726A (en) | The generation method and device of intention material | |
| CN111080723B (en) | Image element segmentation method based on Unet network | |
| CN115171183A (en) | Mask face detection method based on improved yolov5 | |
| CN114120148A (en) | Method for detecting changing area of remote sensing image building | |
| CN114612971A (en) | Face detection method, model training method, electronic device, and program product | |
| CN113313108A (en) | Saliency target detection method based on super-large receptive field characteristic optimization | |
| CN116541549B (en) | Subgraph segmentation method, subgraph segmentation device, electronic equipment and computer readable storage medium | |
| Ren et al. | Context Aware Edge-Enhanced GAN for Remote Sensing Image Super-Resolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |