CN113628287A - Deep learning-based single-stage garment color recognition system and method - Google Patents
Deep learning-based single-stage garment color recognition system and method Download PDFInfo
- Publication number
- CN113628287A CN113628287A CN202110941044.4A CN202110941044A CN113628287A CN 113628287 A CN113628287 A CN 113628287A CN 202110941044 A CN202110941044 A CN 202110941044A CN 113628287 A CN113628287 A CN 113628287A
- Authority
- CN
- China
- Prior art keywords
- color
- clothing
- image
- region image
- garment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013135 deep learning Methods 0.000 title claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims abstract description 60
- 238000013145 classification model Methods 0.000 claims abstract description 35
- 239000003086 colorant Substances 0.000 claims abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 5
- 238000013507 mapping Methods 0.000 claims abstract description 4
- 238000012549 training Methods 0.000 claims description 21
- 238000012360 testing method Methods 0.000 claims description 12
- 238000005259 measurement Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 9
- 238000002372 labelling Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 229910021532 Calcite Inorganic materials 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 239000000843 powder Substances 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 241000282373 Panthera pardus Species 0.000 description 1
- 206010070834 Sensitisation Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000006071 cream Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000013016 learning Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000008313 sensitization Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The disclosure provides a deep learning-based single-stage garment color recognition system and method, and aims to solve the problem of low recognition speed in the prior art. The system comprises: the clothing segmentation model branch performs clothing segmentation on the clothing image to obtain a clothing region image; the color classification model branch carries out RGB color recognition on the clothing image to obtain RGB values of pixels, carries out pixel classification on the clothing image according to the RGB values of the pixels, and classifies the pixels into corresponding color clusters; the fusion unit obtains the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image; the mapping matching unit represents the colors of the clothing region image color cluster in Lab space and measures the distance between the colors and the target colors, and the target color closest to the colors is used as the color category of the clothing region image color cluster. The system disclosed by the invention is high in identification speed and small in calculation amount.
Description
Technical Field
The disclosure belongs to the technical field of garment color recognition, and particularly relates to a deep learning-based single-stage garment color recognition system and method.
Background
Color identification is a classic technology in computer vision, and can be applied to many aspects of life such as traffic, security and the like. In the field of clothing filled with popular visual elements, color is undoubtedly an important ring. The accurate color accurate identification result of the clothing picture can provide convenience in various links of the clothing industry. For example, in the aspect of garment design, the color of the garment is analyzed on a fine granularity for design; for example, in the aspect of trend analysis, clothing pictures of street photos and other multi-source channels can be used for analyzing the fashion trend of colors; in addition, in the aspect of clothing sales, the E-commerce platform can provide accurate clothing search with similar money and similar color, and the style search experience of buyers is improved.
Color recognition of garments generally requires two steps: the first step is to extract the clothing region in a picture, the main method of the step is 2 types, and the first step is to detect the clothing edge through some traditional digital image processing operators so as to plan the clothing region through the edge; in addition, the other method is an image segmentation method based on deep learning, which segments the clothing region and irrelevant regions such as the background and the human body. The second step is to extract the color of the clothing part, the realization mode of the step is many, the input pixels are classified by the traditional machine learning method, and the color proportion of the clothing area is obtained by counting statistics of the classification result; extracting a plurality of center clusters from pixels of the clothing region by a clustering method, and determining the color of the clothing according to the distance relationship of the center clusters in a Lab color space; and combining with deep learning, setting the background as a white background by utilizing the clothing region image obtained in the first step, manually marking data to train a classifier based on the deep learning, and carrying out color recognition and proportion calculation on the input clothing white background image to obtain a clothing color result.
Garment color identification currently faces a number of major problems:
1. the clothing image is generally a picture in an RGB format, any pixel is composed of different r, g and b channel numerical values, the color of the clothing image is not changed linearly in an RGB color space, the colors of the same clothing image are different due to different conditions (such as brightness, sensitization of shooting equipment and the like) when the picture is obtained, and the color of the clothing image is changed in the subjective impression of people due to the change of any channel numerical value of the pixel, so that the problem faced by the clustering method is that the color of the clothing image is not accurate easily to be seen in eyes of an observer.
2. The problem of the speed of color identification is that two-stage identification is mostly adopted, namely, a clothing region is obtained first, and then color classification of clothing region pixels is obtained through other classification methods, so that the problem is complicated, and the identification speed needs to be improved.
Disclosure of Invention
The disclosure provides a deep learning-based single-stage garment color recognition system and method, and aims to solve the problems that in the prior art, the garment color recognition speed is low and the calculation amount is large.
In order to solve the technical problem, the technical scheme adopted by the disclosure is as follows:
in a first aspect, the present disclosure provides a deep learning-based single-stage garment color recognition system, including:
the clothing segmentation model branch is used for carrying out clothing segmentation on the clothing image to obtain a clothing region image and an image category of the clothing region image;
the color classification model branch is used for carrying out RGB color identification on the clothing image to obtain RGB values of pixels, carrying out pixel classification on the clothing image according to the RGB values of the pixels and classifying the pixels into corresponding color clusters;
the fusion unit is used for obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
and the mapping matching unit is used for expressing the colors of the clothing region image color cluster in Lab space and measuring the distance between the colors and the target colors, and taking the target color with the closest distance as the color category of the clothing region image color cluster.
According to a further improved scheme, in the process of carrying out pixel classification on the clothing image according to the RGB value of the pixel, the pixels with the distance between the RGB value and the preset RGB value of the color cluster being smaller than the preset value are classified into the same color cluster.
According to the further improved scheme, in the process of obtaining the color representation of each color cluster in the clothing region image according to the RGB values of the pixels in the color clusters of the clothing region image; and taking the average value of the RGB values of the pixels in the color cluster as the color representation of the color cluster.
Based on the technical scheme, the average value of the RGB values of the pixels in the color cluster is used as the color representation of the color cluster, and when the average value represents the RGB values of the pixels in the whole color cluster, the error is small.
In a further improved scheme, before the color representation of the clothing region image color cluster is subjected to distance measurement with the target color in the Lab space and the target color with the closest distance is taken as the color category of the clothing region image color cluster, the method further comprises the step of counting each color cluster in the clothing region image:
counting the proportion of the number of pixels in each color cluster in the clothing region image in all pixels of the whole clothing region image;
judging whether the proportion of the pixels of each color cluster is greater than a threshold value, if so, expressing the color of the color cluster in a Lab space to measure the distance between the color of the color cluster and the target color, and taking the target color closest to the color cluster as the color category of the image color cluster of the clothing area; if not, the color cluster is deleted.
Based on the technical scheme, the proportion of the pixels of each color cluster in the clothing region image is counted, so that data with small influence on the whole color of the clothing can be eliminated, and the calculation amount is reduced.
According to the further improved scheme, in the process of obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image;
and taking the segmentation result of the clothing image by the clothing segmentation model branch as a mask, and multiplying the segmentation result by the pixel classification result of the clothing image to obtain the pixel classification of each clothing region image.
In a further improved scheme, the clothing segmentation model adopts a depllabv 3+ segmentation model with MobilenetV2 as a basic frame.
In a further improved scheme, the training step of the clothing segmentation model comprises the following steps:
collecting different types of garment images and forming a garment image data set;
marking the clothing image in the clothing image data set, wherein the marked information comprises the segmentation information mark and the clothing category mark of the clothing;
training the clothing segmentation model through the marked clothing image data until the clothing segmentation model achieves stable training;
and testing the trained clothing segmentation model to obtain a clothing segmentation model qualified in testing.
In a further refinement, the color classification model employs a 1 × 1 convolution kernel and is a model built from a stack of several convolution layers.
In a further improved scheme, the training step of the color classification model comprises:
collecting different types of garment images and forming a garment image data set;
training the color classification model by using the clothing image in the clothing image data set; in the training process, a self-supervision method is adopted to train the color classification model until the color classification model is stably trained;
and testing the trained color classification model to obtain a color classification model qualified in testing.
Based on the scheme, the color classification model is trained by adopting a self-supervision method, on one hand, a large amount of workload of manual labeling can be saved, and on the other hand, the self-supervision is only related to the input picture, so that the picture can be enhanced on various pixel levels, and a data set is expanded to a certain extent.
In a second aspect, the present disclosure provides a deep learning-based single-stage garment color identification method, including:
receiving a clothing image;
clothing segmentation is carried out on the clothing image to obtain a clothing region image and an image category of the clothing region image;
performing RGB color recognition on the clothing image to obtain RGB values of pixels, performing pixel classification on the clothing image according to the RGB values of the pixels, and classifying the pixels into corresponding color clusters;
obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
and expressing the color of the clothing region image color cluster in Lab space to measure the distance between the color of the clothing region image color cluster and the target color, and taking the target color with the closest distance as the color category of the clothing region image color cluster.
The beneficial effect of this disclosure does:
in the disclosure, on one hand, the garment segmentation model and the color classification model are fused into one model to complete garment segmentation and color identification in one step, so that color identification of a garment image is achieved, and the calculation speed is increased. On the other hand, in the clothing segmentation model and the color classification model, clothing segmentation and RGB color recognition are carried out on the clothing image to obtain pixel classification of the clothing region image, then the target color with higher fineness in the clothing region image is obtained through calculation in the Lab space according to the classification result, and compared with the method that the color class with higher fineness is directly obtained through the color classification model, the calculation amount is reduced, and the calculation efficiency is improved. The optimized superposition of the operation speeds in multiple aspects improves the operation speed by 3-4 times compared with the prior color identification of the clothing region; in addition, the memory occupation ratio of the whole system is smaller.
On the basis of improving the operation speed, the color output fineness of the final clothing region is higher, and the fineness of color identification and the identification amount of colors are improved.
The RGB values in the same range can be classified into the same color cluster through classification, and data interference is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present disclosure and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings may be obtained from the drawings without inventive effort.
Fig. 1 is a schematic view of a recognition process of a deep learning-based single-stage garment color recognition system in the present disclosure.
Fig. 2 is a schematic flow chart of a deep learning-based single-stage garment color identification method in the present disclosure.
Fig. 3 is a schematic diagram of the recognition time of the deep learning-based single-stage garment color recognition system in the present disclosure compared with the recognition time of the existing model.
Detailed Description
The technical solution in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. It should be understood that the specific embodiments described herein are merely illustrative of the disclosure and are not intended to limit the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without inventive step, are within the scope of the disclosure.
The first embodiment is as follows:
as shown in fig. 1, the present embodiment provides a deep learning-based single-stage garment color identification system, including:
the clothing segmentation model branch is used for carrying out clothing segmentation on the clothing image to obtain a clothing region image and an image category of the clothing region image;
the color classification model branch is used for carrying out RGB color identification on the clothing image to obtain RGB values of pixels, carrying out pixel classification on the clothing image according to the RGB values of the pixels and classifying the pixels into corresponding color clusters;
the fusion unit is used for obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
and the mapping matching unit is used for expressing the colors of the clothing region image color cluster in Lab space and measuring the distance between the colors and the target colors, and taking the target color with the closest distance as the color category of the clothing region image color cluster.
In the clothing segmentation model branch, clothing segmentation is performed on the clothing image to obtain a clothing region image, namely the clothing image is segmented into a background part and a clothing region image part, wherein more than one clothing region image may be in the same image, and the clothing region image only comprises clothing images.
Wherein the pixel classification of the clothing image is a coarse-grained color classification, for example, classifying the pixels in the clothing image on similar pixel classes with coarse granularity, such as puff color, calcite, cream, etc.; while the match in Lab color space is a more refined match, for example: the puff color in the target color is further subdivided into puff powder, pollen color, white color and the like; calcite is further subdivided into calcite powder, light violet, pink leopard powder, etc., and then precisely matched according to the color representation of the color cluster.
In the fusion unit, the segmentation result of the clothing image by the clothing segmentation model branch is used as a mask (whether the label is the label of the clothing region, if so, the mask is 1, otherwise, the mask is 0) and multiplied by the pixel classification result (the color classification label of the pixel) of the clothing image to obtain the pixel classification of each clothing region image.
The color classification model branch is used for performing RGB color identification on the clothing image to obtain RGB values of pixels, performing pixel classification on the clothing image according to the RGB values of the pixels, and classifying the pixels into corresponding color clusters. One specific way is as follows: in the process of classifying the clothing image according to the RGB value of the pixel, the pixels with the distance between the RGB value and the preset RGB value of the color cluster being smaller than the preset value are classified into the same color cluster. For example: the color cluster is preset with RGB values (120, 130, 140), and the RGB values (122, 129, 139) and the RGB values (121, 132, 141) are assigned to the color cluster.
In the process of representing the colors of the clothing region image color clusters in the Lab space and measuring the distance between the colors and the target colors, and taking the target color closest to the colors as the color categories of the clothing region image color clusters, the adopted target color data covers color contrast data which is fine in granularity and comprehensive in types and is commonly used in the clothing industry, and the target color data totally accounts for 900 or more color categories.
On the basis of the scheme, in the process of obtaining the color representation of each color cluster in the clothing region image according to the RGB values of the pixels in the color clusters of the clothing region image; and taking the average value of the RGB values of the pixels in the color cluster as the color representation of the color cluster. Where a color is represented as an RGB value that can represent all pixels within the entire color cluster. The average value of the RGB values of the pixels in the color cluster is taken as the color representation of the color cluster, and the error is small when representing the RGB values of the pixels in the whole color cluster.
In order to eliminate data which has small influence on the overall color of the garment, on the basis of any scheme, before the color representation of the color cluster of the garment region image is subjected to distance measurement in a Lab space and the target color with the closest distance is taken as the color category of the color cluster of the garment region image, the method further comprises the step of counting each color cluster in the garment region image: counting the proportion of the number of pixels in each color cluster in the clothing region image in all pixels of the whole clothing region image; judging whether the proportion of the pixels of each color cluster is greater than a threshold value, if so, expressing the color of the color cluster in a Lab space to measure the distance between the color of the color cluster and the target color, and taking the target color closest to the color cluster as the color category of the image color cluster of the clothing area; if not, the color cluster is deleted. The threshold may be set to 5% or 10%, and the like, and may be set according to different situations. By counting the proportion of pixels of each color cluster in the clothing region image, data with small influence on the whole clothing color can be eliminated, and the calculation amount is reduced.
On the basis of any scheme, the clothing segmentation model adopts a deplab v3+ segmentation model with MobilenetV2 as a basic frame. The garment segmentation model adopts a supervised learning method, utilizes cross entropy loss as a loss function, measures the classification accuracy of a certain pixel class, and judges whether the certain pixel class belongs to the background or the garment and the class of the garment. The clothing segmentation model has the advantages of light weight, high reasoning speed and the like, and is characterized by adopting an anti-bottleneck structure and using separable convolution, so that the model parameters of a conventional convolution layer are greatly reduced. In the segmentation process, the input picture is firstly subjected to 2-time down-sampling for 3 times, and then 8-time up-sampling is carried out to restore the original picture size.
Wherein the training step of the clothing segmentation model comprises the following steps:
collecting different types of garment images and forming a garment image data set; in the process of collecting the clothing image, various types of clothing are required to be contained, the clothing is a more prominent picture element in the picture, and the occasions, postures and styles are not limited so as to ensure the sufficiency of data and the representativeness of the actual life. In addition, certain quality filtering is carried out on the pictures of the clothing, firstly, the theme or main elements of the pictures are ensured to be related to the clothing, the clothing accounts for a higher proportion in the pictures, the pictures have higher quality, the types and scenes are rich, and a high-quality clothing image data set is established;
marking the clothing image in the clothing image data set, wherein the marked information comprises the segmentation information mark and the clothing category mark of the clothing; when the segmentation data set is marked, the segmentation information marking of the clothing is mainly marked, the clothing, background information and human body information are separated from an original picture, and the separated clothing areas are marked in categories, such as coats, trousers, skirts and the like, wherein the subsequent color identification of different types of clothing is mainly performed; and after the labeling is finished, a training data set is obtained and can be used for training a clothing segmentation model. In addition, strict labeling requirements are required, the boundary between the garment and the background is ensured to have higher labeling quality on the segmentation labeling information, and the labeled garment regions are labeled correspondingly and are divided into several large categories, namely, coats, trousers, skirts, coats and half-skirts. In the labeling process, the data of the related clothes are counted, the relative balance of the data sets is ensured, and the missing data is additionally collected;
training the clothing segmentation model through the marked clothing image data until the clothing segmentation model achieves stable training;
and testing the trained clothing segmentation model to obtain a clothing segmentation model qualified in testing.
On the basis of any scheme, the color classification model adopts a model which is formed by stacking a plurality of convolution layers and adopts a 1 x 1 convolution kernel. The color classification model employs a multi-layer convolutional layer stack of conventional convolution kernels of 1 × 1 size, and uses a layer normalization method to perform classification of each pixel into a preset color class on the whole image. The color identification is based on the principle that a fully-connected neural network can fit any function, firstly, the clothes color library is constructed to be classified in a multi-layer mode according to the similarity of colors, the class (marked as N) of the middle layer is selected as a preset color classification class label, a function which is mapped to the N dimension from 16777216 dimensions (the three primary color values are 0-255 respectively and are 256 respectively, and the combined colors are 256 x 256), is constructed in a fitting mode, and the function of performing pixel parallel classification on a picture is achieved by means of the characteristic that the convolutional neural network can be parallel.
Wherein the training of the color classification model comprises:
collecting different types of garment images and forming a garment image data set;
training the color classification model by using the clothing image in the clothing image data set; in the training process, a self-supervision method is adopted to train the color classification model until the color classification model is stably trained;
and testing the trained color classification model to obtain a color classification model qualified in testing.
The training data set can use a clothing image data set of the clothing segmentation model, can also additionally collect other clothing pictures, has low requirements on quality, and requires rich picture sources, styles and clothing categories. The color classification model is trained by adopting a self-supervision method, on one hand, a large amount of workload of manual marking can be saved, and on the other hand, the self-supervision is only related to the input picture, so that the picture can be enhanced on various pixel levels, and a data set is expanded to a certain extent.
The present disclosure is further illustrated below in conjunction with FIG. 1:
inputting a clothing image in a model formed by fusing a clothing segmentation model and a color classification model, obtaining a pixel classification result of the whole image through color classification model branching, segmenting the clothing image through the clothing segmentation model, segmenting the background and the clothing to obtain a clothing image segmentation result, multiplying the clothing segmentation model branching output result and the color classification model output result to obtain a clothing region color classification result (clothing region image pixel classification result), and expressing the pixel classification result of each clothing region image by using an RGB mean value; and finally, matching the target color with the closest matching distance in the Lab color space by using the mean value as a recognition result.
The operation speed of the deep learning-based single-stage garment color identification system in the present disclosure is further described below with reference to fig. 3:
in fig. 3, New _ model (New model) on the left side is a deep learning based single stage garment color recognition system time diagram in the present disclosure; the old model on the right side means that irrelevant regions such as a clothing region, a background, a human body and the like are firstly segmented by an image segmentation method based on deep learning in the prior art; then extracting a plurality of center clusters from the pixels of the clothing region by a clustering method, and determining the color of the clothing according to the distance relationship of the center clusters in the Lab color space.
Calculating the operation time of the New _ model and old _ model by adopting 5000 costume images; the vertical axis represents the time required to identify a garment image; since the recognition time required for each garment image may be different, the width of the comparison graph represents the amount of recognition of the garment image at a certain time in the recognition time.
As can be seen from the New _ model on the left side, the time of each garment image does not exceed 0.06S, and most of the identification time is concentrated in the range of 0.035S-0.04S (corresponding to the time period with the widest transverse width); as can be seen by the old _ model on the right, the time of each garment image exceeds 0.05S, and most of the recognition time is concentrated in the range of 0.08S-0.23S (corresponding to the time period with the widest transverse width).
By counting 5000 garment images, the mean values of the New _ model when a single garment image is identified are 0.037 seconds respectively; the average value of the old _ model for identifying a single garment map is 0.12 seconds.
In summary, it can be seen that the operation speed of the deep learning-based single-stage garment color identification system in the present disclosure is significantly higher than that of the prior art.
Example two:
referring to fig. 2, the present embodiment provides a deep learning-based single-stage garment color identification method, including:
s100, receiving a clothing image;
s200, performing clothing segmentation on the clothing image to obtain a clothing region image and an image category of the clothing region image;
s300, performing RGB color recognition on the clothing image to obtain RGB values of pixels, performing pixel classification on the clothing image according to the RGB values of the pixels, and classifying the pixels into corresponding color clusters;
s400, obtaining pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
s500, representing the colors of the clothing region image color cluster in a Lab space, measuring the distance between the colors and the target colors, and taking the target color closest to the colors as the color category of the clothing region image color cluster.
The number before each step is not limited to the sequence of the logic steps, and can be adjusted arbitrarily under the condition of no conflict; for example: step S200 and step S300 are not in a specific sequence, and may be performed simultaneously.
The present disclosure is not limited to the above alternative embodiments, and any other various forms of products may be obtained by anyone in the light of the present disclosure, but any changes in shape or structure thereof fall within the scope of the present disclosure, which is defined by the claims of the present disclosure.
Claims (10)
1. A deep learning-based single-stage garment color recognition system, comprising:
the clothing segmentation model branch is used for carrying out clothing segmentation on the clothing image to obtain a clothing region image and an image category of the clothing region image;
the color classification model branch is used for carrying out RGB color identification on the clothing image to obtain RGB values of pixels, carrying out pixel classification on the clothing image according to the RGB values of the pixels and classifying the pixels into corresponding color clusters;
the fusion unit is used for obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
and the mapping matching unit is used for expressing the colors of the clothing region image color cluster in Lab space and measuring the distance between the colors and the target colors, and taking the target color with the closest distance as the color category of the clothing region image color cluster.
2. The deep learning-based single-stage garment color identification system according to claim 1, wherein in the process of pixel classification of the garment image according to the RGB values of the pixels, the pixels with the distance between the RGB values and the preset RGB values of the color clusters smaller than the preset value are classified as the same color cluster.
3. The deep learning-based single-stage garment color identification system according to claim 1, wherein in the process of obtaining the color representation of each color cluster in the garment region image according to the RGB values of the pixels in the color cluster of the garment region image; and taking the average value of the RGB values of the pixels in the color cluster as the color representation of the color cluster.
4. The deep learning-based single-stage garment color identification system according to claim 1, further comprising a step of counting each color cluster in the garment region image before distance measurement is performed on the color representation of the garment region image color cluster in Lab space and the target color, and the target color with the closest distance is taken as the color class of the garment region image color cluster:
counting the proportion of the number of pixels in each color cluster in the clothing region image in all pixels of the whole clothing region image;
judging whether the proportion of the pixels of each color cluster is greater than a threshold value, if so, expressing the color of the color cluster in a Lab space to measure the distance between the color of the color cluster and the target color, and taking the target color closest to the color cluster as the color category of the image color cluster of the clothing area; if not, the color cluster is deleted.
5. The deep learning-based single-stage garment color recognition system according to claim 1, wherein in the process of obtaining the pixel classification of the garment region image according to the pixel classification of the garment image and the garment region image;
and taking the segmentation result of the clothing image by the clothing segmentation model branch as a mask, and multiplying the segmentation result by the pixel classification result of the clothing image to obtain the pixel classification of each clothing region image.
6. The deep learning-based single-stage garment color recognition system as claimed in claim 1, wherein the garment segmentation model adopts a deplabv 3+ segmentation model with MobilenetV2 as a basic frame.
7. The deep learning-based single-stage garment color recognition system according to claim 1 or 6, wherein the training step of the garment segmentation model comprises:
collecting different types of garment images and forming a garment image data set;
marking the clothing image in the clothing image data set, wherein the marked information comprises the segmentation information mark and the clothing category mark of the clothing;
training the clothing segmentation model through the marked clothing image data until the clothing segmentation model achieves stable training;
and testing the trained clothing segmentation model to obtain a clothing segmentation model qualified in testing.
8. The deep learning-based single-stage garment color identification system according to claim 1, wherein the color classification model is a model constructed by stacking a number of convolutional layers with a 1 x 1 convolutional kernel.
9. The deep learning-based single-stage garment color recognition system according to claim 1 or 8, wherein the training step of the color classification model comprises:
collecting different types of garment images and forming a garment image data set;
training the color classification model by using the clothing image in the clothing image data set; in the training process, a self-supervision method is adopted to train the color classification model until the color classification model is stably trained;
and testing the trained color classification model to obtain a color classification model qualified in testing.
10. A deep learning-based single-stage garment color identification method is characterized by comprising the following steps:
receiving a clothing image;
clothing segmentation is carried out on the clothing image to obtain a clothing region image and an image category of the clothing region image;
performing RGB color recognition on the clothing image to obtain RGB values of pixels, performing pixel classification on the clothing image according to the RGB values of the pixels, and classifying the pixels into corresponding color clusters;
obtaining the pixel classification of the clothing region image according to the pixel classification of the clothing image and the clothing region image; obtaining color representation of each color cluster in the clothing region image according to the RGB values of pixels in the color clusters of the clothing region image;
and expressing the color of the clothing region image color cluster in Lab space to measure the distance between the color of the clothing region image color cluster and the target color, and taking the target color with the closest distance as the color category of the clothing region image color cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110941044.4A CN113628287B (en) | 2021-08-16 | 2021-08-16 | Single-stage clothing color recognition system and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110941044.4A CN113628287B (en) | 2021-08-16 | 2021-08-16 | Single-stage clothing color recognition system and method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113628287A true CN113628287A (en) | 2021-11-09 |
| CN113628287B CN113628287B (en) | 2024-07-09 |
Family
ID=78385948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110941044.4A Active CN113628287B (en) | 2021-08-16 | 2021-08-16 | Single-stage clothing color recognition system and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113628287B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350485A (en) * | 2023-09-27 | 2024-01-05 | 广东电网有限责任公司 | Power market control method and system based on data mining model |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080144943A1 (en) * | 2005-05-09 | 2008-06-19 | Salih Burak Gokturk | System and method for enabling image searching using manual enrichment, classification, and/or segmentation |
| CN105069466A (en) * | 2015-07-24 | 2015-11-18 | 成都市高博汇科信息科技有限公司 | Pedestrian clothing color identification method based on digital image processing |
| CN106250874A (en) * | 2016-08-16 | 2016-12-21 | 东方网力科技股份有限公司 | A kind of dress ornament and the recognition methods of carry-on articles and device |
| CN107358242A (en) * | 2017-07-11 | 2017-11-17 | 浙江宇视科技有限公司 | Target area color identification method, device and monitor terminal |
| CN107909580A (en) * | 2017-11-01 | 2018-04-13 | 深圳市深网视界科技有限公司 | A kind of pedestrian wears color identification method, electronic equipment and storage medium clothes |
| CN109614508A (en) * | 2018-12-12 | 2019-04-12 | 杭州知衣科技有限公司 | A kind of image of clothing searching method based on deep learning |
| CN110825899A (en) * | 2019-09-18 | 2020-02-21 | 武汉纺织大学 | Clothing image retrieval method integrating color features and residual network depth features |
| CN111160350A (en) * | 2019-12-23 | 2020-05-15 | Oppo广东移动通信有限公司 | Portrait segmentation method, model training method, device, medium and electronic equipment |
| CN112001251A (en) * | 2020-07-22 | 2020-11-27 | 山东大学 | Pedestrian re-identification method and system based on combination of human body analysis and clothing color |
| CN112232173A (en) * | 2020-10-12 | 2021-01-15 | 济南博观智能科技有限公司 | Pedestrian attribute identification method, deep learning model, equipment and medium |
| CN112784728A (en) * | 2021-01-18 | 2021-05-11 | 山东省人工智能研究院 | Multi-granularity clothes changing pedestrian re-identification method based on clothing desensitization network |
| CN112949648A (en) * | 2021-03-12 | 2021-06-11 | 上海眼控科技股份有限公司 | Method and equipment for acquiring training sample data set of image segmentation model |
| CN113420797A (en) * | 2021-06-08 | 2021-09-21 | 杭州知衣科技有限公司 | Online learning image attribute identification method and system |
-
2021
- 2021-08-16 CN CN202110941044.4A patent/CN113628287B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080144943A1 (en) * | 2005-05-09 | 2008-06-19 | Salih Burak Gokturk | System and method for enabling image searching using manual enrichment, classification, and/or segmentation |
| CN105069466A (en) * | 2015-07-24 | 2015-11-18 | 成都市高博汇科信息科技有限公司 | Pedestrian clothing color identification method based on digital image processing |
| CN106250874A (en) * | 2016-08-16 | 2016-12-21 | 东方网力科技股份有限公司 | A kind of dress ornament and the recognition methods of carry-on articles and device |
| CN107358242A (en) * | 2017-07-11 | 2017-11-17 | 浙江宇视科技有限公司 | Target area color identification method, device and monitor terminal |
| CN107909580A (en) * | 2017-11-01 | 2018-04-13 | 深圳市深网视界科技有限公司 | A kind of pedestrian wears color identification method, electronic equipment and storage medium clothes |
| CN109614508A (en) * | 2018-12-12 | 2019-04-12 | 杭州知衣科技有限公司 | A kind of image of clothing searching method based on deep learning |
| CN110825899A (en) * | 2019-09-18 | 2020-02-21 | 武汉纺织大学 | Clothing image retrieval method integrating color features and residual network depth features |
| CN111160350A (en) * | 2019-12-23 | 2020-05-15 | Oppo广东移动通信有限公司 | Portrait segmentation method, model training method, device, medium and electronic equipment |
| CN112001251A (en) * | 2020-07-22 | 2020-11-27 | 山东大学 | Pedestrian re-identification method and system based on combination of human body analysis and clothing color |
| CN112232173A (en) * | 2020-10-12 | 2021-01-15 | 济南博观智能科技有限公司 | Pedestrian attribute identification method, deep learning model, equipment and medium |
| CN112784728A (en) * | 2021-01-18 | 2021-05-11 | 山东省人工智能研究院 | Multi-granularity clothes changing pedestrian re-identification method based on clothing desensitization network |
| CN112949648A (en) * | 2021-03-12 | 2021-06-11 | 上海眼控科技股份有限公司 | Method and equipment for acquiring training sample data set of image segmentation model |
| CN113420797A (en) * | 2021-06-08 | 2021-09-21 | 杭州知衣科技有限公司 | Online learning image attribute identification method and system |
Non-Patent Citations (2)
| Title |
|---|
| MARCO MANFREDI等: "A complete system for garment segmentation and color classification", 《 MACHINE VISION AND APPLICATIONS 》, vol. 25, no. 04, 26 November 2013 (2013-11-26), pages 955, XP035368583, DOI: 10.1007/s00138-013-0580-3 * |
| 胡江华: "静态图像的行人分割及其应用", 《中国优秀硕士学位论文全文数据库（电子期刊）》（信息科技辑）, 15 August 2014 (2014-08-15), pages 138 - 1472 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117350485A (en) * | 2023-09-27 | 2024-01-05 | 广东电网有限责任公司 | Power market control method and system based on data mining model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113628287B (en) | 2024-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102102161B1 (en) | Method, apparatus and computer program for extracting representative feature of object in image | |
| CN108491835B (en) | Two-channel convolutional neural network for facial expression recognition | |
| CN108520226B (en) | Pedestrian re-identification method based on body decomposition and significance detection | |
| CN103514456B (en) | Image classification method and device based on compressed sensing multi-core learning | |
| Mahapatra et al. | Retinal image quality classification using saliency maps and CNNs | |
| CN106599854B (en) | Automatic facial expression recognition method based on multi-feature fusion | |
| CN111160249A (en) | Multi-class target detection method of optical remote sensing image based on cross-scale feature fusion | |
| CN108647625A (en) | A kind of expression recognition method and device | |
| CN108830237B (en) | Facial expression recognition method | |
| CN109886161B (en) | Road traffic identification recognition method based on likelihood clustering and convolutional neural network | |
| CN103106265B (en) | Similar image sorting technique and system | |
| CN102194108B (en) | Smile face expression recognition method based on clustering linear discriminant analysis of feature selection | |
| CN106682601A (en) | Driver violation conversation detection method based on multidimensional information characteristic fusion | |
| CN111126240B (en) | Three-channel feature fusion face recognition method | |
| CN108629319B (en) | Image detection method and system | |
| CN111160194B (en) | Static gesture image recognition method based on multi-feature fusion | |
| CN107169508A (en) | A kind of cheongsam Image emotional semantic method for recognizing semantics based on fusion feature | |
| CN112446340A (en) | Pedestrian searching method and system combining pedestrian local feature and decorative attribute classification and storage medium | |
| CN114239754B (en) | Pedestrian attribute identification method and system based on attribute feature learning decoupling | |
| CN111815582A (en) | Two-dimensional code area detection method for improving background prior and foreground prior | |
| CN113628287A (en) | Deep learning-based single-stage garment color recognition system and method | |
| Mahapatra | Retinal image quality classification using neurobiological models of the human visual system | |
| CN110910497B (en) | Method and system for realizing augmented reality map | |
| CN114067185A (en) | Film evaluation system based on facial expression recognition | |
| CN106096527A (en) | A kind of recognition methods of real-time high-precision online bank note face amount |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |