CN115511775A - Light-weight ceramic tile surface defect detection method based on semantic segmentation - Google Patents
Light-weight ceramic tile surface defect detection method based on semantic segmentation Download PDFInfo
- Publication number
- CN115511775A CN115511775A CN202110695481.2A CN202110695481A CN115511775A CN 115511775 A CN115511775 A CN 115511775A CN 202110695481 A CN202110695481 A CN 202110695481A CN 115511775 A CN115511775 A CN 115511775A
- Authority
- CN
- China
- Prior art keywords
- image
- semantic segmentation
- defect detection
- defect area
- detected
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 138
- 238000001514 detection method Methods 0.000 title claims abstract description 65
- 230000011218 segmentation Effects 0.000 title claims abstract description 65
- 239000000919 ceramic Substances 0.000 title claims abstract description 43
- 238000012549 training Methods 0.000 claims abstract description 19
- 238000012805 post-processing Methods 0.000 claims abstract description 11
- 238000007781 pre-processing Methods 0.000 claims abstract description 11
- 238000013507 mapping Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 33
- 238000011176 pooling Methods 0.000 claims description 19
- 230000004913 activation Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000009432 framing Methods 0.000 claims 1
- 238000010606 normalization Methods 0.000 claims 1
- 230000006870 function Effects 0.000 description 19
- 238000004519 manufacturing process Methods 0.000 description 15
- 230000000694 effects Effects 0.000 description 5
- 230000004927 fusion Effects 0.000 description 5
- 238000007689 inspection Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000002950 deficient Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 235000017166 Bambusa arundinacea Nutrition 0.000 description 1
- 235000017491 Bambusa tulda Nutrition 0.000 description 1
- 241001330002 Bambuseae Species 0.000 description 1
- 235000015334 Phyllostachys viridis Nutrition 0.000 description 1
- BQCADISMDOOEFD-UHFFFAOYSA-N Silver Chemical compound [Ag] BQCADISMDOOEFD-UHFFFAOYSA-N 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 239000011425 bamboo Substances 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 239000011449 brick Substances 0.000 description 1
- 239000004566 building material Substances 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 230000018044 dehydration Effects 0.000 description 1
- 238000006297 dehydration reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 238000000227 grinding Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007641 inkjet printing Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000005498 polishing Methods 0.000 description 1
- 239000000843 powder Substances 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 229910052709 silver Inorganic materials 0.000 description 1
- 239000004332 silver Substances 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Geometry (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a light-weight ceramic tile surface defect detection method based on semantic segmentation, which is used for detecting the defects of the surface of a ceramic tile and is characterized by comprising the following steps: acquiring the surface of the ceramic tile by using a high-resolution camera to obtain an image to be detected, and preprocessing the image to be detected to obtain a preprocessed image; establishing a light-weight semantic segmentation network for detecting the surface defects of the ceramic tiles and training the light-weight semantic segmentation network to obtain a ceramic tile surface defect detection network; inputting the preprocessed image into a tile surface defect detection network for defect detection, thereby detecting a suspected defect area; carrying out post-processing on the suspected defect area, judging whether the area of the suspected defect area is larger than a given threshold value or not, and marking the suspected defect area as a defect area if the area of the suspected defect area is larger than the given threshold value; and acquiring a minimum external positive rectangle of the defect area, and mapping the minimum external positive rectangle back to the image to be detected and marking the image frame.
Description
Technical Field
The invention relates to a light-weight ceramic tile surface defect detection method based on semantic segmentation.
Background
The ceramic tile has the advantages of corrosion resistance, wear resistance, water resistance, rich texture and the like, and is a popular decoration building material product. With the improvement of living standard, people demand beauty, and various artistic ceramic tiles with different styles and complex textures are gradually popular in the market. These colorful tiles put higher demands on the production process.
The production process of the ceramic tile generally comprises the steps of raw material mixing and grinding, dehydration, blank pressing, ink-jet printing, glaze pouring, firing, polishing, and finally quality detection and packaging. Most production links of the existing ceramic tiles are basically unmanned due to the development of industrial automation, but the final quality inspection link is mainly completed manually. Generally, 2-6 quality inspection workers are required to be arranged on one production line, the production line works for a long time under highlight to observe the surface of a ceramic tile and search flaws, visual fatigue is easily caused, and the quality of manual quality inspection is directly reduced. The manual quality inspection has low efficiency and high cost, and the quality varies from person to person and the quality is uneven. Therefore, the realization of automation of tile surface detection is an important link of production and quality management in the tile industry and a technical bottleneck which puzzles the industry for many years. Common flaws in the ceramic tile production line include powder lumps, corner cracks, glaze drops, ink breaks, ink drops, B holes, dirt falls, edge cracks, unfilled corners, brick residues, white edges, guy wires and the like. Among the defects, the width of the stay wire is usually 1-2 mm, the height of the stay wire is consistent with the height of the ceramic tile, and the stay wire is not easy to identify in a complex ceramic tile background pattern, so that the manual quality inspection difficulty is higher, and no effective scheme specially aiming at the defect detection of the stay wire exists at present.
The patent application No. 201910062683.6 discloses a complex tile surface defect detection method based on a difference image method and a local variance measurement operator. The method comprises the steps of firstly segmenting a salient region in an image by using a clustering method, then obtaining a rough defect region of the image by using a difference image method, then extracting the outline of a defective ceramic tile by calculating the local variance value of a defect edge pixel point and the surrounding pixel points, and finally obtaining an accurate and complete defect region after morphological operations such as smoothing, filling and the like. The method can effectively and stably detect defects from a complex tile background, but the method needs to prepare a defect-free standard tile image in advance, which not only increases the complexity of detection but also limits the variety of tile surface patterns.
The patent with the application number of 202010222390.2 discloses a mosaic tile defect detection method and device. The method comprises the steps of preprocessing a picture to be detected, inputting the picture to be detected into a GoogleNet convolutional neural network for detection, and judging whether a defect exists when the probability value is larger than or equal to 50%. The input size of the GoogleNet convolutional neural network in the method is 224x224x3, the number of parameters and the calculated amount of the network are large and are respectively 6.99M and 1.58G, and the method is not suitable for a rapid pipeline production scene. In addition, the method performs downsampling processing on the picture to be detected during preprocessing, which can cause loss of detail features of a part of the picture, thereby reducing the accuracy of detection.
The patent with the application number of 202010469022.8 discloses a deep neural network surface defect detection method based on feature fusion. The method adopts a coder-decoder network structure, firstly a characteristic extraction module adopts ResNet50 to extract surface defect characteristics, then a coding characteristic fusion module is utilized to fuse the characteristics of each convolution layer, and finally a two-stage cascade decoding module is utilized to realize the detection of the surface defects. This method also has the disadvantage of being computationally expensive.
Other methods for detecting different objects, such as a rail surface defect detection method and device based on few-sample learning disclosed in application No. 202011059937.8, a hot-rolled strip steel surface defect classification method based on a convolutional neural network disclosed in application No. 201910676299.5, a long bamboo strip surface defect detection method and system based on a YOLOv3 improved network disclosed in application No. 202011558112.0, an airplane engine blade surface defect detection method based on deep learning disclosed in patent No. 202010674099.9, and the like, are poor in detection effect under a complex background, large in network parameters and calculated amount, and long in detection time.
It is summarized that the method applied to the stay wire defect detection in the prior art mainly has the following defects:
(1) The traditional machine vision algorithm mainly comprises a difference threshold value method and a template matching method. Both methods need a fixed layout, namely, a defect-free standard image is prepared in advance and compared with an image detected in real time in the production process, so that the diversity of the tile patterns is sacrificed, the threshold value is not easy to determine, and the robustness is poor.
(2) Classic target detection networks such as VGG-NET, YOLO-V3 and the like, and the improved networks on the basis are not suitable for defect detection under a complex background;
(3) Classic semantic segmentation networks such as FCN (fuzzy probabilistic networks), U-NET and the like, and networks improved on the basis of the classic semantic segmentation networks are high in detection accuracy, but due to the fact that model parameters are large, the computation amount is large, and the requirement of flow production line instantaneity cannot be met.
Disclosure of Invention
In order to solve the problems, the invention provides a light-weight ceramic tile surface defect detection method based on semantic segmentation, which adopts the following technical scheme:
the invention provides a semantic segmentation-based light-weight ceramic tile surface defect detection method, which is characterized by comprising the following steps: acquiring the surface of the ceramic tile by using a high-resolution camera to obtain an image to be detected, and preprocessing the image to be detected to obtain a preprocessed image; establishing a light-weight semantic segmentation network for detecting the surface defects of the ceramic tiles and training the light-weight semantic segmentation network to obtain a ceramic tile surface defect detection network; inputting the preprocessed image into a tile surface defect detection network for defect detection, thereby detecting a suspected defect area; performing post-processing on the suspected defect area, judging whether the area of the suspected defect area is larger than a given threshold value or not, and marking the suspected defect area as a defect area if the area of the suspected defect area is larger than the given threshold value; the method comprises the steps of obtaining a minimum circumscribed regular rectangle of a defect area, mapping the minimum circumscribed regular rectangle back to an image to be detected and marking a picture frame, wherein the lightweight semantic segmentation network comprises 4 convolutional layers, a first pooling layer, 8 convolutional layers, a second pooling layer, 8 convolutional layers, a third pooling layer, 2Inception V1 modules and a finishing convolutional layer which are sequentially arranged, and the convolutional layers adopt convolution kernels with the size of 3 x 3.
The light-weight ceramic tile surface defect detection method based on semantic segmentation provided by the invention can also have the technical characteristics that an image to be detected is converted into a gray image; carrying out binarization on the gray level image to form a binary image; respectively carrying out horizontal projection and vertical projection on the binary image, counting the number of white pixel points on each row and each column, and taking the counting result as two discrete number columns; taking the median of the discrete number series as a threshold, respectively judging whether the number of white pixel points on each row and each column of the binary image is less than the threshold, and if so, removing the corresponding row and column and forming a processed image; scaling the processed image so that the adjusted size of the processed image satisfies the following expression:
H new =[H/1408]*1408
W new =[W/512]*512
wherein H, W is the height and width of the image to be detected, respectively]To round the fractional part, H new 、W new The adjusted processed image is cropped into a number of small images of 1408 × 512 size as the pre-processed image for the height and width of the adjusted processed image, respectively.
The semantic segmentation based lightweight tile surface defect detection method provided by the invention can also have the technical characteristics that the resolution of the first pooling layer, the resolution of the second input image and the resolution of the third input image are respectively reduced by half, a Batchnormal layer and a Relu layer are added after all the convolution layers, and the function formula of the Batchnormal layer is as follows:
in the formula, x i ∈{x 1…m Is a batch of input values and is,is the average value of the batch,is the variance of the batch, e is a minimum value to prevent the denominator from being 0,for normalized input, gamma, beta are two parameters to be learned, y i For normalized output, the Relu function is formulated as:
where f (x) is the output of the activation function Relu and x is the input of the activation function.
The semantic segmentation based light-weight ceramic tile surface defect detection method provided by the invention can also have the technical characteristics that in the step S2, an adam optimizer is used for training a light-weight semantic segmentation network, in addition, positive and negative samples are alternately trained in the training process, the samples are disordered after each iteration, and the loss function BCELoss formula is as follows:
in the formula, o is an output value, t is a tag value, and N is the number of samples.
The method for detecting the defects on the surface of the light-weight ceramic tile based on the semantic segmentation can also have the technical characteristics that the defect detection in the step S3 is to use a plurality of convolution layers and pooling layers to perform feature extraction on an input preprocessed image, and then a segmentation mask with the resolution of 1/8 of the preprocessed image is output by using a Sigmoid function, wherein each pixel value in the segmentation mask is the probability that the corresponding pixel is a defect, and the Sigmoid function formula is as follows:
in the formula, S (x) is the sigmoid function output, and x is the sigmoid function input.
The method for detecting the surface defects of the light-weight ceramic tiles based on the semantic segmentation, provided by the invention, can also have the technical characteristics that the post-processing in the step S4 comprises the following steps: multiplying each segmentation mask by 255 and performing splicing processing to obtain a pre-input image; carrying out binaryzation on a pre-input image; and vertically projecting the binarized image, counting the number of white pixel points on each column, judging whether the number of the white pixel points is less than a given threshold value, and if so, setting all the pixel values on the corresponding columns to be 0.
The light-weight ceramic tile surface defect detection method based on semantic segmentation provided by the invention can also have the technical characteristics that the outline coordinates of a defect area are calculated; calculating the coordinate of the minimum circumscribed regular rectangle wrapping the outline according to the coordinates of the outline so as to obtain a plurality of minimum circumscribed regular rectangles corresponding to each defect area; combining a plurality of minimum circumscribed positive rectangles on each defect area into a vertical rectangle; and calculating a rectangular coordinate corresponding to the vertical rectangle in the image to be detected according to the mapping relation between the image to be detected and the defect area, and drawing a rectangular frame on the image to be detected according to the rectangular coordinate to mark the defect area.
Action and Effect of the invention
According to the light-weight ceramic tile surface defect detection method based on semantic segmentation, the light-weight semantic segmentation network is trained to obtain the ceramic tile surface defect detection network, and the preprocessed image is subjected to defect detection, so that a suspected defect area is detected. And carrying out post-processing on the suspected defect area, and judging whether the area of the suspected defect area is larger than a given threshold value to mark whether the suspected defect area is a defect area. And acquiring a minimum external positive rectangle of the defect area, mapping the minimum external positive rectangle back to the image to be detected, and finally marking the picture frame. Therefore, the light-weight semantic segmentation network specially used for detecting the ceramic tile defects has the characteristics of high detection precision, few network parameters, small calculation amount, high robustness and the like. The speed of flow production can be kept up to on the actual production line, and the detection result can be given in real time; meanwhile, the method can replace manual work to automatically detect the stay wire defect on the surface of the ceramic tile, so that the labor cost is saved, and the reliability and the efficiency of defect detection are improved; in addition, the embodiment can also detect the surface defects of other types of tiles, and only different defect samples need to be collected for model training.
Drawings
FIG. 1 is a flow chart of a semantic segmentation based method for detecting defects on a surface of a lightweight tile in an embodiment of the invention;
FIG. 2 is a flow chart of image pre-processing in an embodiment of the present invention;
FIG. 3 is a block diagram of a lightweight semantic segmentation network in an embodiment of the invention;
FIG. 4 is a flow diagram of training a lightweight semantic segmentation network in an embodiment of the invention;
FIG. 5 is a flow chart of image post-processing in an embodiment of the present invention;
fig. 6 is a flowchart of marking a defective area in an embodiment of the present invention.
Detailed Description
In order to make the technical means, the creation features, the achievement objects and the effects of the present invention easy to understand, the following describes the method for detecting surface defects of lightweight ceramic tiles based on semantic segmentation in detail with reference to the embodiments and the accompanying drawings.
< example >
FIG. 1 is a flow chart of a semantic segmentation based method for detecting defects on a surface of a lightweight tile in an embodiment of the invention.
As shown in fig. 1, the method for detecting defects on a surface of a lightweight tile based on semantic segmentation specifically includes steps S1 to S5.
S1, acquiring the surface of the ceramic tile by using a high-resolution camera to obtain an image to be detected, and preprocessing the image to be detected to obtain a preprocessed image.
Fig. 2 is a flow chart of image preprocessing in an embodiment of the present invention.
As shown in fig. 2, in step S1 of this embodiment, a 300dpi high definition industrial camera is used to collect an image of a tile surface on a production line, and before inputting the collected image into a convolution network for defect detection, image preprocessing needs to be performed, where the preprocessing process specifically includes the following sub-steps:
s1-1, converting an image to be detected into a gray image;
s1-2, carrying out binarization on the gray level image to form a binary image;
s1-3, respectively carrying out horizontal projection and vertical projection on the binary image, counting the number of white pixel points on each row and each column, and taking the counting result as two discrete number columns;
s1-4, taking the median of the discrete number series as a threshold, respectively judging whether the number of white pixel points on each row and each column of the binary image is less than the threshold, and if so, removing the corresponding row and column and forming a processed image;
step S1-5, scaling the processed image so that the adjusted size of the processed image satisfies the following expression:
H new =[H/1408]*1408
W new =[W/512]*512
wherein H, W are eachIs the height and width of the image to be detected]To round the fractional part, H new 、W new The adjusted processed image is cropped into a number of small images of 1408 × 512 size as the pre-processed image for the height and width of the adjusted processed image, respectively.
And S2, establishing a light-weight semantic segmentation network for detecting the surface defects of the ceramic tiles, and training the light-weight semantic segmentation network to obtain the surface defect detection network of the ceramic tiles.
Fig. 3 is a block diagram of a lightweight semantic segmentation network in an embodiment of the invention.
As shown in fig. 3, the present embodiment uses a deep learning framework Pytorch to construct a lightweight semantic segmentation network, which includes 21 convolutional layers, 2 innovative v1 modules, and 3 maximum pooling layers, each pooling layer halving the resolution of the input image (i.e., image) to increase the receptive field, and a batcnormal layer and a Relu layer are added after all convolutional layers except the last convolutional layer. Wherein, the front of the first pooling layer is provided with 4 convolution layers, and the front of the second pooling layer and the front of the third pooling layer are respectively provided with 8 convolution layers. And two Inception V1 modules are continuously used for constructing sparse connection after the third pooling layer, multi-scale receptive field and feature fusion is introduced into the network, and finally a 1 x 1 convolution layer is used for channel fusion of output. The structural design can obtain a large receptive field in a high-resolution input image and can capture fine image features. In addition, convolution layers in the network almost adopt convolution kernels with the size of 3 x3, and the stacking of a plurality of layers of 3 x3 convolutions not only enhances the nonlinear expression capability of the network, but also can obtain the same feature extraction capability as that of the convolution layers with large convolution kernels while reducing network parameters.
Specifically, the Batchnormal function is formulated as:
in the formula, x i ∈{x 1…m Is a batch of input values and is,is the average value of the batch,is the variance of the batch, e is a minimum value to prevent the denominator from being 0,for normalized input, gamma, beta are two parameters to be learned, y i Is the normalized output.
The Relu function is formulated as:
where f (x) is the output of the activation function Relu and x is the input of the activation function.
The detailed parameters of each layer of the lightweight semantic segmentation network are shown in table 1 below.
TABLE 1 light weight semantically segmented network parameter table
Specific parameters of two IncepotionV 1 modules are shown in Table 2 below.
TABLE 2Inception V1 Module parameter Table
Wherein the activation function of the 26 th convolutional layer is Sigmoid. Compared with the classic semantic segmentation network FCN and the U-NET network, the network has small parameter and calculation amount, is a light-weight semantic segmentation network, and is shown in the following table 3.
TABLE 3 comparison of parameters and calculated quantities of this network with FCN and U-Net networks
The above is a specific structure of the lightweight semantic segmentation network, and in this embodiment, the lightweight semantic segmentation network further needs to be trained in advance through a training process, specifically:
FIG. 4 is a flow diagram of training a lightweight semantic segmentation network according to an embodiment of the present invention.
As shown in fig. 4, a data set (i.e., a training set prepared in advance, where the training set includes a plurality of images, and each image is preprocessed through the preprocessing process in step S1) is read to perform iterative training, a verification set is read to perform model evaluation after the number of iterations is reached, if the evaluation result reaches an index, the training is ended and the model is saved, otherwise, the training is continued. In addition, an adam optimizer is used for training a lightweight semantic segmentation network, positive and negative samples are alternately trained in a training process, sample scrambling is performed after each iteration, and a loss function BCELoss formula is as follows:
in the formula, o is an output value, t is a tag value, and N is the number of samples.
And S3, inputting the preprocessed image into a tile surface defect detection network for defect detection, thereby detecting a suspected defect area.
The tile image to be detected is input into a trained lightweight semantic segmentation network after being preprocessed in the step S1, is subjected to feature extraction of a plurality of convolution layers and pooling layers, and is output into a segmentation mask with the resolution of 1/8 of the preprocessed image by using a Sigmoid function, wherein each pixel value in the segmentation mask is the probability that the corresponding pixel is a defect, and the Sigmoid function formula is as follows:
in the formula, S (x) is the sigmoid function output, and x is the sigmoid function input.
Compared with the FCN and the U-Net network, the network greatly reduces the parameter amount and the calculated amount under the condition of not reducing the detection precision. The comparison of the network performance indexes is shown in table 4 below, in which the calculation of the parameters is performed in the following experimental environment: ubuntu 18.04.5LTS, intel (R) Xeon (R) Silver [email protected], 64G for memory, 11G for memory, and 2080Ti for video card model.
TABLE 4 comparison of Performance indicators for this network with FCN and U-Net networks
And S4, carrying out post-processing on the suspected defect area, judging whether the area of the suspected defect area is larger than a given threshold value or not, and marking the suspected defect area as a defect area if the area of the suspected defect area is judged to be larger than the given threshold value.
Fig. 5 is a flowchart of image post-processing in an embodiment of the present invention.
As shown in fig. 5, the post-processing specifically proceeds as follows:
multiplying each segmentation mask by 255 and performing splicing processing to obtain a pre-input image;
carrying out binarization on a pre-input image;
and vertically projecting the binarized image, counting the number of white pixel points on each column, judging whether the number of the white pixel points is less than a given threshold value, and if so, setting all the pixel values on the corresponding columns to be 0.
And S5, acquiring the minimum external positive rectangle of the defect area, and mapping the minimum external positive rectangle back to the image to be detected and carrying out frame marking.
Fig. 6 is a flowchart of marking a defective area in an embodiment of the present invention.
As shown in fig. 6, step S5 includes the following sub-steps:
s5-1, calculating the outline coordinates of the defect area;
s5-2, calculating the coordinate of the minimum circumscribed regular rectangle wrapping the outline according to the coordinates of the outline so as to obtain a plurality of minimum circumscribed regular rectangles corresponding to each defect area;
s5-3, combining a plurality of minimum external positive rectangles on each defect area into a vertical rectangle;
and S5-4, calculating a rectangular coordinate corresponding to the vertical rectangle in the image to be detected according to the mapping relation between the image to be detected and the defect area, and drawing a rectangular frame on the image to be detected according to the rectangular coordinate to mark the defect area.
Through the steps, the detection of the surface defects of the ceramic tiles can be completed.
Examples effects and effects
According to the light-weight tile surface defect detection method based on semantic segmentation provided by the embodiment, the light-weight semantic segmentation network is trained to obtain a tile surface defect detection network, and the preprocessed image is subjected to defect detection, so that a suspected defect area is detected. And carrying out post-processing on the suspected defect area, and judging whether the area of the suspected defect area is larger than a given threshold value to mark whether the suspected defect area is a defect area. And acquiring a minimum external positive rectangle of the defect area, mapping the minimum external positive rectangle back to the image to be detected, and finally marking the picture frame. Therefore, the light-weight semantic segmentation network specially used for detecting the ceramic tile defects has the characteristics of high detection precision, few network parameters, small calculation amount, high robustness and the like. The speed of flow production can be kept up to on the actual production line, and the detection result can be given in real time; meanwhile, the method can replace manual work to automatically detect the stay wire defect on the surface of the ceramic tile, so that the labor cost is saved, and the reliability and the efficiency of defect detection are improved; in addition, the embodiment can also detect the surface defects of other types of tiles, and only different defect samples need to be collected for model training.
In an embodiment, the lightweight semantic segmentation network includes 4 convolutional layers, a first pooling layer, 8 convolutional layers, a second pooling layer, 8 convolutional layers, a third pooling layer, 2 inclusion v1 modules, and a termination convolutional layer, which are sequentially arranged, where the convolutional layers adopt convolutional kernels of 3 × 3 size, and finally, one 1 × 1 convolutional layer is used to perform channel fusion on outputs. The structural design can obtain a large receptive field in a high-resolution input image and can capture fine image features. In addition, convolution layers in the network almost adopt convolution kernels with the size of 3 x3, and the stacking of a plurality of layers of 3 x3 convolutions not only enhances the nonlinear expression capability of the network, but also can obtain the same feature extraction capability as the convolution layers with the large convolution kernels while reducing the network parameters.
The above-described embodiments are merely illustrative of specific embodiments of the present invention, and the present invention is not limited to the scope of the description of the above-described embodiments.
Claims (7)
1. A light-weight ceramic tile surface defect detection method based on semantic segmentation is used for detecting defects on the surface of a ceramic tile, and is characterized by comprising the following steps:
s1, acquiring the surface of the ceramic tile by using a high-resolution camera to obtain an image to be detected, and preprocessing the image to be detected to obtain a preprocessed image;
s2, establishing a light-weight semantic segmentation network for detecting the surface defects of the ceramic tiles and training the light-weight semantic segmentation network to obtain a ceramic tile surface defect detection network;
s3, inputting the preprocessed image into the tile surface defect detection network for defect detection, so as to detect a suspected defect area;
step S4, carrying out post-processing on the suspected defect area, judging whether the area of the suspected defect area is larger than a given threshold value or not, and marking the suspected defect area as a defect area if the area of the suspected defect area is judged to be larger than the given threshold value;
s5, acquiring the minimum circumscribed positive rectangle of the defect area, mapping the minimum circumscribed positive rectangle back to the image to be detected and framing a frame mark,
wherein the lightweight semantic segmentation network comprises 4 convolutional layers, a first pooling layer, 8 convolutional layers, a second pooling layer, 8 convolutional layers, a third pooling layer, 2InceptionV1 modules and a termination convolutional layer which are arranged in sequence,
the convolutional layer employs a convolution kernel of size 3 × 3.
2. A semantic segmentation based lightweight tile surface defect detection method according to claim 1, characterized in that:
wherein the step S1 comprises the following substeps:
s1-1, converting the image to be detected into a gray image;
s1-2, carrying out binarization on the gray level image to form a binary image;
s1-3, respectively carrying out horizontal projection and vertical projection on the binary image, counting the number of white pixel points on each row and each column, and taking the counting result as two discrete number columns;
s1-4, taking the median of the discrete number columns as a threshold, respectively judging whether the number of white pixel points on each row and each column of the binary image is smaller than the threshold, if so, removing the corresponding row and the corresponding column and forming a processed image;
step S1-5, scaling the processed image so that the adjusted size of the processed image satisfies the following expression:
H new =[H/1408]*1408
W hew =[W/512]*512
wherein H, W is the height and width of the image to be detected, respectively]To round the fractional part, H new 、W new Respectively the adjusted height and width of the processed image,
and cutting the adjusted processed image into a plurality of small images with the size of 1408 multiplied by 512 as the pre-processed image.
3. A semantic segmentation based lightweight tile surface defect detection method according to claim 1, characterized in that:
wherein,
halving the resolution of the first, second and third input images respectively,
all the convolutional layers were followed by a Batchnormal layer and a Relu layer,
the functional formula of the Batchnormal layer is as follows:
in the formula, x i ∈{x 1...m Is a batch of input values and is,is the average value of the batch in question,for the variance of the batch, e is a minimum value to prevent the denominator from being 0,for normalized input, gamma, beta are two parameters to be learned, y i In order to be the output after the normalization,
the Relu function is formulated as:
where f (x) is the output of the activation function Relu and x is the input of the activation function.
4. A semantic segmentation based lightweight tile surface defect detection method according to claim 1, characterized in that:
wherein, in the step S2, an adam optimizer is used to train the lightweight semantic segmentation network, and in the training process, positive and negative samples are alternately trained, and sample scrambling is performed after each iteration,
the loss function BCELoss is formulated as:
in the formula, o is an output value, t is a tag value, and N is the number of samples.
5. A semantic segmentation based lightweight tile surface defect detection method according to claim 1, characterized in that:
the defect detection in step S3 is to perform feature extraction on an input preprocessed image by using a plurality of convolutional layers and pooling layers, and then output a segmentation mask with a resolution of 1/8 of the preprocessed image by using a Sigmoid function, where each pixel value in the segmentation mask is a probability that a corresponding pixel is a defect, and the Sigmoid function formula is as follows:
in the formula, S (x) is the sigmoid function output, and x is the sigmoid function input.
6. A semantic segmentation based lightweight tile surface defect detection method according to claim 5, characterized in that:
wherein the post-processing comprises:
multiplying 255 by each segmentation mask and splicing to obtain a pre-input image;
carrying out binarization on the pre-input image;
and vertically projecting the binarized image, counting the number of white pixel points on each column, judging whether the number of the white pixel points is less than a given threshold value, and if so, setting all the pixel values on the corresponding columns to be 0.
7. A semantic segmentation based lightweight tile surface defect detection method according to claim 1, characterized in that:
wherein the step S5 comprises the following substeps:
s5-1, calculating the outline coordinates of the defect area;
s5-2, calculating the coordinates of the minimum circumscribed regular rectangles wrapping the outline according to the coordinates of the outline, thereby obtaining a plurality of minimum circumscribed regular rectangles corresponding to each defect area;
s5-3, combining a plurality of minimum external positive rectangles on each defect area into a vertical rectangle;
and S5-4, calculating a rectangular coordinate corresponding to the vertical rectangle in the image to be detected according to the mapping relation between the image to be detected and the defect area, and drawing a rectangular frame on the image to be detected according to the rectangular coordinate to mark the defect area.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110695481.2A CN115511775A (en) | 2021-06-23 | 2021-06-23 | Light-weight ceramic tile surface defect detection method based on semantic segmentation |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110695481.2A CN115511775A (en) | 2021-06-23 | 2021-06-23 | Light-weight ceramic tile surface defect detection method based on semantic segmentation |
Publications (1)
Publication Number | Publication Date |
---|---|
CN115511775A true CN115511775A (en) | 2022-12-23 |
Family
ID=84500311
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110695481.2A Pending CN115511775A (en) | 2021-06-23 | 2021-06-23 | Light-weight ceramic tile surface defect detection method based on semantic segmentation |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115511775A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116580026A (en) * | 2023-07-12 | 2023-08-11 | 深圳市致尚科技股份有限公司 | Automatic optical detection method, equipment and storage medium for appearance defects of precision parts |
CN117250201A (en) * | 2023-11-08 | 2023-12-19 | 深圳市洪发建筑工程有限公司 | Swimming pool brick layer defect detection method and system |
-
2021
- 2021-06-23 CN CN202110695481.2A patent/CN115511775A/en active Pending
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116580026A (en) * | 2023-07-12 | 2023-08-11 | 深圳市致尚科技股份有限公司 | Automatic optical detection method, equipment and storage medium for appearance defects of precision parts |
CN116580026B (en) * | 2023-07-12 | 2023-09-12 | 深圳市致尚科技股份有限公司 | Automatic optical detection method, equipment and storage medium for appearance defects of precision parts |
CN117250201A (en) * | 2023-11-08 | 2023-12-19 | 深圳市洪发建筑工程有限公司 | Swimming pool brick layer defect detection method and system |
CN117250201B (en) * | 2023-11-08 | 2024-02-13 | 深圳市洪发建筑工程有限公司 | Swimming pool brick layer defect detection method and system |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108960245B (en) | Tire mold character detection and recognition method, device, equipment and storage medium | |
CN111325203B (en) | American license plate recognition method and system based on image correction | |
CN108562589B (en) | Method for detecting surface defects of magnetic circuit material | |
CN110544251B (en) | Dam crack detection method based on multi-migration learning model fusion | |
CN104992449B (en) | Information identification and surface defect online test method based on machine vision | |
CN112233067A (en) | Hot rolled steel coil end face quality detection method and system | |
CN109444169B (en) | Bearing defect detection method and system | |
CN107543828A (en) | A kind of Surface Flaw Detection method and system | |
CN110033431B (en) | Non-contact detection device and detection method for detecting corrosion area on surface of steel bridge | |
CN108181316B (en) | Bamboo strip defect detection method based on machine vision | |
CN111582294A (en) | Method for constructing convolutional neural network model for surface defect detection and application thereof | |
CN109215009B (en) | Continuous casting billet surface image defect detection method based on deep convolution neural network | |
CN112613097A (en) | BIM rapid modeling method based on computer vision | |
CN112907519A (en) | Metal curved surface defect analysis system and method based on deep learning | |
CN115511775A (en) | Light-weight ceramic tile surface defect detection method based on semantic segmentation | |
CN108985337A (en) | A kind of product surface scratch detection method based on picture depth study | |
CN109376740A (en) | A kind of water gauge reading detection method based on video | |
CN110473201A (en) | A kind of automatic testing method and device of disc surface defect | |
CN111612747B (en) | Rapid detection method and detection system for product surface cracks | |
CN113869300A (en) | Workpiece surface defect and character recognition method and system based on multi-vision fusion | |
CN113393426A (en) | Method for detecting surface defects of rolled steel plate | |
CN113393438B (en) | Resin lens defect detection method based on convolutional neural network | |
CN109101976A (en) | A kind of detection method of arc extinguishing grid pieces surface defect | |
CN113240623A (en) | Pavement disease detection method and device | |
CN116824347A (en) | Road crack detection method based on deep learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |