CN113642662A - Lightweight classification model-based classification detection method and device - Google Patents
Lightweight classification model-based classification detection method and device Download PDFInfo
- Publication number
- CN113642662A CN113642662A CN202110973138.XA CN202110973138A CN113642662A CN 113642662 A CN113642662 A CN 113642662A CN 202110973138 A CN202110973138 A CN 202110973138A CN 113642662 A CN113642662 A CN 113642662A
- Authority
- CN
- China
- Prior art keywords
- sample image
- evaluated
- model
- stage
- basic unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013145 classification model Methods 0.000 title claims abstract description 100
- 238000001514 detection method Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 claims abstract description 9
- 238000011176 pooling Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004804 winding Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 abstract description 4
- 238000003466 welding Methods 0.000 description 58
- 238000013136 deep learning model Methods 0.000 description 12
- 238000000605 extraction Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000007547 defect Effects 0.000 description 4
- 230000007423 decrease Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The application provides a classification detection method and device based on a lightweight classification model. The method comprises the steps of obtaining a training sample image and determining the category of the training sample image; classifying and marking the training sample images according to the categories to obtain sample marked images; inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model; and inputting the sample image to be evaluated into the trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated. The utility model provides a model elementary cell in lightweight classification model trunk adopts succinct two branch structure, remains the transmission of waiting to evaluate sample image data when increasing model elementary cell quantity, and the convolution layer that uses output channel number to be equal to convolution group number can lightweight parameter space, and the initialization width of model is less, and the model is lighter for the speed that this model was handled and is waited to evaluate sample image data is faster, and detection effect is better, and detection efficiency is higher.
Description
Technical Field
The application relates to the technical field of deep learning models, in particular to a classification detection method and device based on a lightweight classification model.
Background
In the field of industrial inspection, products have various surface defect types and forms. When the classification detection of the surface defects of the product is performed, although the traditional recognition algorithm can be adopted to perform feature extraction on the product image, the surface defects and the classes of the surface defects are determined according to the result of the feature extraction. However, by adopting traditional recognition algorithms such as a gray level extraction algorithm, an area extraction algorithm, a contour extraction algorithm and the like, all effective features in the product image are difficult to extract, and the detection effect on the product image is poor.
The deep learning model can automatically learn the characteristics of the product image, the product image is detected by adopting a detection method based on the deep learning model, namely a classification model, and the product can be divided into a qualified class and an unqualified class according to a detection result. Therefore, the use of deep learning models for classification detection of products is becoming more common in the prior art.
Although it is more and more common to adopt a deep learning model to classify and detect products, and the deep learning model, i.e. the detection method of the classification model, is adopted to detect product images with a good effect, because the existing deep learning model, i.e. the network structure of the classification model, is complex and the parameters of the model are many, the adoption of the above scheme has a low detection efficiency on the product images, and also has a certain influence on the production efficiency of the products.
Disclosure of Invention
The application provides a classification detection method and device based on a lightweight classification model, and aims to solve the problem that the detection efficiency of a detection method adopting a deep learning model in the prior art on a product image is low, so that the production efficiency of a product is low.
In one aspect, the application provides a classification detection method based on a lightweight classification model, comprising the following steps:
acquiring a training sample image and determining the category of the training sample image;
classifying and marking the training sample images according to the categories to obtain sample marked images;
inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model;
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated;
the basic structure of the lightweight classification model comprises a trunk, wherein the trunk is used for extracting the features of the preprocessed sample image to be evaluated at different levels;
the main frame comprises at least one stage, the stage is composed of at least two model basic units, the resolution of the image processed by the first model basic unit in the stage is reduced, and the resolution of the image processed by the second model basic unit is unchanged; the first model basic unit is used for extracting the characteristics of the preprocessed sample image to be evaluated, superposing the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated and outputting the superposed image, and the second model basic unit is used for carrying out convolution operation on the superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage;
the model basic unit comprises convolution layers, and the initial width of each stage is equal to the number of output channels of the convolution layers in the stage.
In the technical scheme, the basic model units adopt a simple double-branch structure, and the transmission of the image data of the sample to be evaluated is reserved while the number of the basic model units is increased; the convolution layer with the number of output channels equal to the number of convolution groups can reduce parameter space, so that the model is lighter and the processing speed of the sample image to be evaluated is higher.
In a preferred embodiment of the present application, inputting a sample image to be evaluated into a trained lightweight classification model, and determining and outputting a category to which the sample image to be evaluated belongs includes:
preprocessing the sample image to be evaluated;
sequentially extracting features of different levels in the preprocessed sample image to be evaluated to obtain the sample image to be evaluated containing the highest-level features;
calculating the class probability value of the sample image to be evaluated containing the highest-level features;
and comparing the category probability value, and outputting the category of the sample image to be evaluated according to the comparison result.
In a preferred embodiment of the present application, sequentially extracting features of different levels in the preprocessed sample image to be evaluated to obtain a sample image to be evaluated containing the highest-level features, includes:
extracting the first-stage features of the preprocessed sample image to be evaluated in a first stage to obtain the sample image to be evaluated containing the first-stage features;
extracting second-level features of the sample image to be evaluated containing the first-level features through a second stage to obtain the sample image to be evaluated containing the second-level features;
extracting the third-level features of the sample image to be evaluated containing the second-level features through a third stage to obtain the sample image to be evaluated containing the third-level features;
and extracting the highest-level feature of the sample image to be evaluated containing the third-level feature through a fourth stage to obtain the sample image to be evaluated containing the highest-level feature.
In a preferred embodiment of the present application, the model base unit is composed of a convolutional layer and a residual connection convolutional layer, and the model base unit includes a first model base unit, a second model base unit, a third model base unit, a fourth model base unit, a fifth model base unit, a sixth model base unit, a seventh model base unit, and an eighth model base unit;
the first model basic unit and the second model basic unit form a first stage and are used for extracting first-level features of the sample image to be evaluated;
the third model basic unit and the fourth model basic unit form a second stage for extracting second-stage features of the sample image to be evaluated;
the fifth model basic unit and the sixth model basic unit form a third stage for extracting third-stage features of the sample image to be evaluated;
and the seventh model basic unit and the eighth model basic unit form a fourth stage for extracting the highest-level features of the sample image to be evaluated.
In a preferred embodiment of the present application, the seventh model basic unit includes:
a first winding layer: 1 × 1cov, 512/2, wherein 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 2 denotes a 2-fold reduction in image resolution of the first convolution layer;
a second convolution layer: 3 × 3group cov, 512/1, where 3 × 3 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the second convolution layer, and 1 denotes a 1-fold reduction in image resolution of the second convolution layer;
a third convolutional layer: 1 × 1cov, 512/1, where 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 1 denotes a 1-fold reduction in image resolution of the third convolution layer;
side branch convolutional layer: and if the image resolution of the first convolution layer is reduced by 1 time, the first convolution layer is connected with the convolution layer through a residual error and is directly communicated with a third convolution layer for superposition output.
In a preferred embodiment of the present application, the side branch convolution layer further comprises:
if the image resolution of the first convolution layer is reduced by 2 times, adding 1 × 1 convolution kernel, processing branch convolution layers with 512 output channels, and then superposing the branch convolution layers to a third convolution layer for output.
On the other hand, the present application provides a classification detection device based on a lightweight classification model, the classification detection device includes:
the device comprises a training module and a classification detection module;
wherein the training module is to:
acquiring a training sample image and determining the category of the training sample image;
classifying and marking the training sample images according to the categories to obtain sample marked images;
inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model;
the classification detection module is configured to:
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated;
the basic structure of the lightweight classification model comprises a trunk, wherein the trunk is used for extracting the features of the preprocessed sample image to be evaluated at different levels;
the main frame comprises at least one stage, the stage is composed of at least two model basic units, the resolution of an image processed by a first model basic unit in the stage is reduced, the resolution of an image processed by a second model basic unit is unchanged, the first model basic unit is used for extracting the characteristics of a preprocessed sample image to be evaluated and outputting the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated after superposition, the second model basic unit is used for performing convolution operation on a superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage;
the model basic unit comprises convolution layers, and the initial width of each stage is equal to the number of output channels of the convolution layers in the stage.
In a third aspect, the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the classification detection method when executing the computer program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the classification detection method.
Compared with the prior art, the classification detection method and device based on the lightweight classification model have the following beneficial effects:
(1) according to the method and the device, the lightweight classification model is built, the features of the sample image to be evaluated in different levels are sequentially extracted step by using the built lightweight classification model, the class probability value of the sample image to be evaluated is output, and the class of the sample image to be evaluated can be accurately and efficiently judged.
(2) The model basic units in the light-weight classification model trunk adopt a simple double-branch structure, the number of the model basic units is increased, meanwhile, the transmission of sample image data to be evaluated is reserved, and the number of the convolution layers with the output channels equal to the number of the convolution groups can be used for realizing light-weight parameter space.
(3) The lightweight classification model used in the application has the advantages of small parameter space, small initialization width and lighter weight, so that the speed of processing the image data of the sample to be evaluated by the model is higher, the detection effect is better, and the detection efficiency is higher.
Drawings
In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a classification detection method based on a lightweight classification model in this embodiment 1;
FIG. 2 is a schematic diagram of the overall structure of a lightweight classification model according to the present application;
fig. 3 is a schematic structural diagram of a seventh model basic unit in this embodiment 1;
fig. 4 is a schematic structural diagram of an eighth model basic unit in this embodiment 1.
Detailed Description
To make the objects, embodiments and advantages of the present application clearer, the following description of exemplary embodiments of the present application will clearly and completely describe the exemplary embodiments of the present application with reference to the accompanying drawings in the exemplary embodiments of the present application, and it is to be understood that the described exemplary embodiments are only a part of the embodiments of the present application, and not all of the embodiments.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
All other embodiments, which can be derived by a person skilled in the art from the exemplary embodiments described herein without inventive step, are intended to be within the scope of the claims appended hereto. In addition, while the disclosure herein has been presented in terms of one or more exemplary examples, it should be appreciated that aspects of the disclosure may be implemented solely as a complete embodiment.
It should be noted that the brief descriptions of the terms in the present application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of the present application. These terms should be understood in their ordinary and customary meaning unless otherwise indicated.
The term "module," as used herein, refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and/or software code that is capable of performing the functionality associated with that element.
All orientation-based words in this application, such as "upper", "lower", etc., are described with reference to the figure positions in this application.
Example 1
As shown in fig. 1, embodiment 1 provides a classification detection method based on a lightweight classification model, which includes the following steps:
s101, acquiring a training sample image and determining the category of the training sample image.
In step S101, the training sample images are training set images of a sufficient number required to ensure the accuracy of product category classification of a lightweight classification model constructed according to actual detection conditions in different industrial detection fields. Inputting enough training set images, namely training sample images, into the constructed lightweight classification model, so that the lightweight classification model can learn enough training set images; the method comprises the steps of detecting the characteristics of training sample images, and judging the training set images, namely the categories of the training sample images according to the characteristics of each training set image, namely the training sample images, extracted through detection; and storing enough training set images, namely training sample images, through the training learning results of the lightweight classification model, so that when the class detection judgment is performed on the sample images to be evaluated, namely the evaluation set images, the lightweight classification model after the training through the training set images, namely the training sample images, can be directly adopted for carrying out the classification detection.
It should be noted that, in this embodiment 1, the categories of the training sample images need to be divided according to different industrial detection fields, that is, according to actual use, common general knowledge of a person skilled in the art or conventional technical means divide specific categories in each different industrial detection field, and the number of the divided categories also needs to be determined by the person skilled in the art according to specific use cases.
For example: during welding detection, training sample images, namely a training set is taken as welding sample images, and sufficient welding sample images are collected to meet the training and learning requirements of a constructed lightweight classification model applied to welding detection, so that the detection categories of the welding sample images are required to be determined after sufficient welding sample images are obtained. The categories in the weld inspection can be divided into two categories of weld correctness and weld offset according to the common general knowledge of the skilled person.
And S102, carrying out classification marking on the training sample image according to the category to obtain a sample marked image.
In step S102, the sample label image is a training sample set image containing the belonging category; the sample labeled image is obtained by labeling enough training sample images, namely training set images, according to specific categories marked by common general knowledge and the like of the technicians in the field, and the labeling can be carried out according to practical experience of the technicians in the field and can also be carried out by other methods. The marking of the categories is to distinguish different categories to which each training sample image belongs, so that the constructed lightweight classification model can accurately learn and extract the characteristics of the sample marked images and the corresponding categories when the sample marked images are adopted for training and learning.
For example: in the welding detection, each obtained training sample image, namely the welding sample image, is classified and marked according to two classes of the correct welding seam and the deviation welding seam, so that a plurality of welding mark images are obtained, wherein the welding mark images not only contain the characteristics of the welding sample image, but also contain the class information of the correct welding seam or the deviation welding seam corresponding to the welding sample image, so that the constructed lightweight classification model applied to the welding detection can simultaneously extract and learn the characteristics of the welding sample image and the class to which the welding sample image belongs.
And S103, inputting the sample labeled image into a lightweight classification model for training to obtain the trained lightweight classification model.
In step S103, the sample labeled image including the class information to which the training sample image belongs is input to the constructed lightweight classification model, that is, the lightweight classification model is used to train and learn the features of the sample labeled image and the corresponding class to which the sample labeled image belongs; and the lightweight classification model stores the training learning results of all the sample labeled images containing the class information of the training sample images, so that the trained lightweight classification model can be obtained.
For example: in the welding detection, a plurality of welding mark images containing the category information corresponding to the welding sample images are input into a constructed lightweight classification model for training and learning, the lightweight classification model is used for extracting and learning a plurality of welding mark images containing the category information corresponding to the welding sample images and the characteristics of the welding mark images, and after the training and learning process is completed, the characteristics of the welding mark images obtained through training and learning and the category information of the welding mark images are stored, so that the trained lightweight classification model applied to the welding detection can be obtained.
In a specific implementation manner of this embodiment 1, the lightweight classification model in step S103 includes, from top to bottom, a branch, a trunk, and a head, where the branch is mainly composed of branch convolution layers; the skeleton comprises at least one stage, the stage is composed of at least two model basic units, the resolution of an image processed by a first model basic unit in the stage is reduced by m times, and the resolution of an image processed by a second model basic unit is unchanged; the first model basic unit is used for extracting the characteristics of the preprocessed sample image to be evaluated, superposing the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated and outputting the superposed image, and the second model basic unit is used for carrying out convolution operation on the superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage. The initial width of the first model basic unit is equal to the number W of output channels of the branch convolution layer, the head part mainly comprises a global pooling layer and a linear layer and is used for extracting and outputting each category probability value of a sample image to be evaluated, and the set value of the related numerical values of the global pooling layer and the linear layer can be set according to the actual situation.
In a specific implementation manner of this embodiment 1, the number of output channels of the branch convolution layer is W, the size of a convolution kernel in the branch convolution layer is a × a, and the number of times of image resolution reduction is m, where specific values of W and a may be specifically set according to a reduction situation of a network structure simplification situation and a parameter space of a lightweight classification model that is required in an actual situation, m is a positive integer, and m may be set according to the number of times of image resolution reduction that is actually required.
Further, the structure of the model basic unit is as follows:
a first winding layer: b × B cov, W × n/m, wherein B × B represents the width × height of a convolution kernel, W × n represents the number of output channels of the first convolution layer, m represents the image resolution reduction m times of the first convolution layer, n and m are positive integers, and n and m can be set according to the times of the image resolution reduction actually required;
a second convolution layer: a multiplied by A group cov, W multiplied by n/m, wherein A multiplied by A represents the width multiplied by the height of a convolution kernel, W multiplied by n represents the number of output channels of the second convolution layer, m represents the image resolution reduction of the second convolution layer by m times, n and m are positive integers, and n and m can be set according to the times of the image resolution reduction actually required; the second convolution layer uses group convolution, and the group number of the group convolution is equal to the output channel number of the second convolution layer; the parameters of the lightweight classification model can be reduced through the processing of the second convolution layer, and the processing efficiency of the lightweight classification model can be improved through grouping and parallel processing by the group convolution;
a third convolutional layer: b × B cov, W × n/m, wherein B × B represents the width × height of a convolution kernel, W × n represents the number of output channels of the first convolution layer, m represents the image resolution reduction m times of the first convolution layer, n and m are positive integers, and n and m can be set according to the times of the image resolution reduction actually required;
side branch convolutional layer: if the image resolution of the first convolution layer is not reduced, namely m is 1, the first convolution layer is connected with the convolution layer through a residual error and directly communicated with a third convolution layer for superposition output; the effective transmission of the image data of the sample to be evaluated can be ensured;
and if the image resolution of the first convolution layer is reduced by m times, namely m is not equal to 1, adding a convolution kernel of BxB, performing branch convolution layer processing on the side with the number of output channels of W x n, and superposing the branch convolution layer processing to the output of a third convolution layer.
It should be noted that, the number of the specific convolutional layer hierarchy structures of each model basic unit and the total number of the model units can be set and selected according to the requirement of simplifying the existing classification model, that is, the degree of the network structure of the deep learning model and the requirement of reducing the number of parameters of the classification model, in this embodiment 1, only the general structure of the lightweight classification model and the setting of the most critical structure hierarchy and each layer and structure specific value in the lightweight classification model that can achieve the optimal simplified network complexity and number of parameters are shown. Therefore, those skilled in the art cannot consider that the lightweight classification model of the present application has only one structure.
In the best embodiment of this embodiment 1, the lightweight classification model in step S103 has a specific structure as shown in fig. 2, and includes a branch, a trunk, and a head in sequence from top to bottom, wherein the branch is formed by a branch convolution layer; the main body consists of a first stage, a second stage, a third stage and a fourth stage, each stage comprises two model basic units, each model basic unit consists of a convolution layer and a residual connecting convolution layer, the convolution layers and the residual connecting convolution layers form two branch structures of the model basic units, and the double-branch structure is adopted, so that image data transmission can be kept while the number of the model basic units is increased; the header includes a global pooling layer and a linear layer.
As shown in fig. 2, the model basic units constituting the light-weight classification model skeleton of embodiment 1 include, in order from top to bottom, a first model basic unit, a second model basic unit, a third model basic unit, a fourth model basic unit, a fifth model basic unit, a sixth model basic unit, a seventh model basic unit, and an eighth model basic unit; the first model basic unit and the second model basic unit form a first stage and are used for extracting first-level features of the preprocessed sample image to be evaluated; the third model basic unit and the fourth model basic unit form a second stage for extracting second-stage features of the sample image to be evaluated; the fifth model basic unit and the sixth model basic unit form a third stage for extracting third-stage features of the sample image to be evaluated; and the seventh model basic unit and the eighth model basic unit form a fourth stage for extracting the highest-level features of the sample image to be evaluated. Note that the levels from the first level feature to the highest level feature increase layer by layer.
Further, as shown in fig. 3, a specific structure of a seventh model basic unit constituting the fourth stage includes:
a first winding layer: 1 × 1cov, 512/2, wherein 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 2 denotes a 2-fold reduction in image resolution of the first convolution layer;
a second convolution layer: 3 × 3group cov, 512/1, where 3 × 3 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the second convolution layer, and 1 denotes a 1-fold reduction in image resolution of the second convolution layer; the second convolution layer uses group convolution, and the group number of the group convolution is equal to the output channel number of the second convolution layer; for example: when the parameter size of the standard convolution is equal to 512 × 3 × 512, the number of output channels is also equal to 512, and the group convolution is divided into 512 groups for processing, the parameter size is equal to 512/512 (divided into 512 groups) × 3 × 512/512 (divided into 512 groups) × (multiplied by 512 groups), so that the parameter number of basic units of the model can be reduced, the existing deep learning model, namely the classification model, is lighter, and meanwhile, the processing efficiency of the lightweight classification model can be improved by grouping and parallel processing;
a third convolutional layer: 1 × 1cov, 512/1, where 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 1 denotes a 1-fold reduction in image resolution of the third convolution layer;
side branch convolutional layer: if the image resolution of the first convolution layer is reduced by 2 times, adding a convolution kernel of 1 multiplied by 1, processing branch convolution layers at the side with the number of output channels of 512, and then superposing the branch convolution layers to a third convolution layer for output; the upper arrow in fig. 3 receives the image data processed by the sixth model base unit in the third stage, and the lower arrow sends the image data processed by the seventh model base unit to the eighth model base unit.
Further, as shown in fig. 4, a specific structure of the eighth model basic unit forming the fourth stage includes:
a first winding layer: 1 × 1cov, 512/2, wherein 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 2 denotes a 2-fold reduction in image resolution of the first convolution layer;
a second convolution layer: 3 × 3group cov, 512/1, where 3 × 3 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the second convolution layer, and 1 denotes a 1-fold reduction in image resolution of the second convolution layer; the second convolution layer uses group convolution, and the group number of the group convolution is equal to the output channel number of the second convolution layer; for example: when the parameter size of the standard convolution is equal to 512 × 3 × 512, the number of output channels is also equal to 512, and the group convolution is divided into 512 groups for processing, the parameter size is equal to 512/512 (divided into 512 groups) × 3 × 512/512 (divided into 512 groups) × (multiplied by 512 groups), so that the parameter number of basic units of the model can be reduced, the existing deep learning model, namely the classification model, is lighter, and meanwhile, the processing efficiency of the lightweight classification model can be improved by grouping and parallel processing;
a third convolutional layer: 1 × 1cov, 512/1, where 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 1 denotes a 1-fold reduction in image resolution of the third convolution layer;
side branch convolutional layer: if the image resolution of the first convolution layer is reduced by 1 time, the first convolution layer is connected with the convolution layer through a residual error and is directly communicated with a third convolution layer for superposition output; the upper arrow in fig. 4 receives the image data processed by the seventh model basic unit, and the lower arrow transmits the image data processed by the eighth model basic unit to the head.
Furthermore, in the specific structure of the seventh model basic unit constituting the fourth stage, after the seventh model basic unit is processed from the first convolution layer to the side branch convolution layer, the image resolution is reduced by 2 times, after the eighth model basic unit constituting the fourth stage is processed from the first convolution layer to the side branch convolution layer, the image resolution is not reduced, and the eighth model basic unit is disposed below the seventh model basic unit.
In addition, it should be noted that the composition and structure of the model basic units in the first stage, the second stage and the third stage are similar to those in the fourth stage, but the number of output channels in each stage is different, and the number of output channels decreases from the fourth stage to the first stage, that is, the number of output channels decreases from bottom to top in fig. 2, and each stage decreases by 2 times.
In the best embodiment of example 1, as shown in fig. 2, the lightweight classification model stem in step S103 is designed to have a structure:
branch convolution layer: 3 × 3cov, 64/2, where 3 × 3 denotes the width × height of the convolution kernel, 64 denotes the number of output channels of the branch convolutional layer, and 2 denotes a 2-fold reduction in image resolution of the branch convolutional layer; using 64 as the initialization width of the lightweight classification model, the initialization width is smaller than that of the existing deep learning model, i.e., the classification model.
In a specific embodiment of this embodiment 1, as shown in fig. 2, the structure of the lightweight classification model head in step S103 is designed to:
global pooling layer: avg-pool;
linear layer: fc class, wherein the global pooling layer is used for reducing the image resolution, and each class probability value of the sample image to be evaluated can be extracted and output through the global pooling layer and the linear layer; the set values of the related values of the global pooling layer and the linear layer can be set according to actual conditions.
And S104, inputting the sample image to be evaluated into the trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated.
In a specific implementation manner of this embodiment 1, the specific processing procedure of the trained lightweight classification model in step S104 on the sample image to be evaluated is as follows:
preprocessing a sample image to be evaluated through a branch in the constructed lightweight classification model, namely, performing noise elimination, primary feature extraction and other processing on the sample image to be evaluated through the setting of a branch convolution layer;
the features of different levels in the sample image to be evaluated are sequentially and gradually extracted through the trunk in the lightweight classification model constructed in fig. 2, so as to obtain the sample image to be evaluated containing the highest level features, namely, the features of the sample image to be evaluated are respectively and sequentially and gradually extracted through the first stage, the second stage, the third stage and the fourth stage of the trunk, the low-level features of the sample image to be evaluated are extracted in the first stage, the levels of the features are increased along with the gradual extraction of each stage, and therefore the highest-level features of the sample image to be evaluated are extracted in the fourth stage. And (3) extracting from low-level features to high-level features, namely, the features of the sample image to be evaluated extracted in each stage are clearer along with higher levels, and the image resolution of the sample image to be evaluated is smaller along with the increment of the stages. Therefore, when the highest-level features of the sample image to be evaluated are extracted, the image resolution of the sample image to be evaluated is small enough, and the features in the sample image to be evaluated, namely the highest-level features, are clear enough;
extracting a class probability value of a sample image to be evaluated containing highest-level features through a head part in a constructed lightweight classification model, judging and outputting the class of the sample image to be evaluated according to the class probability value, wherein the class probability value is that the sample image to be evaluated belongs to the highest-level features, calculating a probability value of the sample image with evaluation belonging to a certain class or a probability value of the certain class in the classes divided according to common knowledge in the field, if a plurality of class probability values are obtained through calculation, comparing the class probability values, and outputting the class with the largest class probability value as the class of the sample image to be evaluated.
The following are exemplary: in the welding detection, collecting a welding sample image to be detected, namely a sample image to be evaluated, inputting the welding sample image to a trained lightweight classification model applied to the welding detection, and performing preprocessing processes such as noise elimination, preliminary feature extraction and the like on the welding sample image to be detected by the lightweight classification model applied to the welding detection; secondly, after sequentially and gradually extracting the features of the preprocessed welding sample image to be detected until the highest-level features of the welding sample image to be detected are extracted, namely the features are clear enough, and after the image resolution of the welding sample image to be detected is small enough, calculating and outputting the class probability values of the welding sample image to be detected through a global pooling layer and a linear layer, namely the specific probability values of two classes of correct welding seams and deviation welding seams, and if only the class probability value of one of the classes of correct welding seams or deviation welding seams is obtained, directly outputting the class of the welding sample image to be detected; if the class probability values of the correct welding seam and the offset welding seam are obtained, the class probability value of the correct welding seam and the class probability value of the offset welding seam are required to be compared, if the class probability value of the correct welding seam is larger than the class probability value of the offset welding seam, a lightweight classification model applied to welding detection judges and outputs the class of the welding sample image to be detected as the correct welding seam; and if the correct class probability value of the welding seam is smaller than the class probability value of the welding seam deviation, judging by a lightweight classification model applied to welding detection and outputting the class of the welding sample image to be detected as the welding seam deviation.
The above is only one of the welding detection fields to which the technical solution of this embodiment 1 is applicable, and the technical solution of this embodiment 1 is also applicable to other industrial detection application fields. And the specific classification setting of the categories is determined according to the industrial detection application field in the actual situation.
Example 2
Corresponding to embodiment 1 of the classification detection method based on the lightweight classification model, the application also provides a classification detection device based on the lightweight classification model. The classification detection device includes:
the device comprises a training module and a classification detection module;
wherein the training module is to:
acquiring a training sample image and determining the category of the training sample image;
classifying and marking the training sample images according to the categories to obtain sample marked images;
inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model;
the classification detection module is configured to:
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated;
the basic structure of the lightweight classification model comprises a trunk, wherein the trunk is used for extracting the features of the preprocessed sample image to be evaluated at different levels;
the main frame comprises at least one stage, the stage is composed of at least two model basic units, the resolution of an image processed by a first model basic unit in the stage is reduced, the resolution of an image processed by a second model basic unit is unchanged, the first model basic unit is used for extracting the characteristics of a preprocessed sample image to be evaluated and outputting the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated after superposition, the second model basic unit is used for performing convolution operation on a superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage;
the model basic unit comprises convolution layers, and the initial width of each stage is equal to the number of output channels of the convolution layers in the stage.
The embodiments provided in the present application are only a few examples of the general concept of the present application, and do not limit the scope of the present application. Any other embodiments extended according to the scheme of the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.
Claims (10)
1. A classification detection method based on a lightweight classification model is characterized by comprising the following steps:
acquiring a training sample image and determining the category of the training sample image;
classifying and marking the training sample images according to the categories to obtain sample marked images;
inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model;
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated;
the basic structure of the lightweight classification model comprises a trunk, wherein the trunk is used for extracting the features of the preprocessed sample image to be evaluated at different levels;
the main frame comprises at least one stage, the stage is composed of at least two model basic units, the resolution of the image processed by the first model basic unit in the stage is reduced, and the resolution of the image processed by the second model basic unit is unchanged; the first model basic unit is used for extracting the characteristics of the preprocessed sample image to be evaluated, superposing the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated and outputting the superposed image, and the second model basic unit is used for carrying out convolution operation on the superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage;
the model basic unit comprises convolution layers, and the initial width of each stage is equal to the number of output channels of the convolution layers in the stage.
2. The classification detection method based on the lightweight classification model according to claim 1,
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated, wherein the method comprises the following steps:
preprocessing the sample image to be evaluated;
sequentially extracting features of different levels in the preprocessed sample image to be evaluated to obtain the sample image to be evaluated containing the highest-level features;
calculating the class probability value of the sample image to be evaluated containing the highest-level features;
and comparing the category probability value, and outputting the category of the sample image to be evaluated according to the comparison result.
3. The classification detection method based on the lightweight classification model according to claim 2,
sequentially extracting the features of different levels in the preprocessed sample image to be evaluated to obtain the sample image to be evaluated containing the highest-level features, wherein the method comprises the following steps:
extracting the first-stage features of the preprocessed sample image to be evaluated in a first stage to obtain the sample image to be evaluated containing the first-stage features;
extracting second-level features of the sample image to be evaluated containing the first-level features through a second stage to obtain the sample image to be evaluated containing the second-level features;
extracting the third-level features of the sample image to be evaluated containing the second-level features through a third stage to obtain the sample image to be evaluated containing the third-level features;
and extracting the highest-level feature of the sample image to be evaluated containing the third-level feature through a fourth stage to obtain the sample image to be evaluated containing the highest-level feature.
4. The classification detection method based on the lightweight classification model according to any one of claims 1 to 3,
the model basic unit consists of a convolutional layer and a residual connecting convolutional layer, and comprises a first model basic unit, a second model basic unit, a third model basic unit, a fourth model basic unit, a fifth model basic unit, a sixth model basic unit, a seventh model basic unit and an eighth model basic unit;
the first model basic unit and the second model basic unit form a first stage and are used for extracting first-level features of the sample image to be evaluated;
the third model basic unit and the fourth model basic unit form a second stage for extracting second-stage features of the sample image to be evaluated;
the fifth model basic unit and the sixth model basic unit form a third stage for extracting third-stage features of the sample image to be evaluated;
and the seventh model basic unit and the eighth model basic unit form a fourth stage for extracting the highest-level features of the sample image to be evaluated.
5. The classification detection method based on the lightweight classification model according to claim 4, wherein the seventh model basic unit comprises:
a first winding layer: 1 × 1cov, 512/2, wherein 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 2 denotes a 2-fold reduction in image resolution of the first convolution layer;
a second convolution layer: 3 × 3group cov, 512/1, where 3 × 3 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the second convolution layer, and 1 denotes a 1-fold reduction in image resolution of the second convolution layer;
a third convolutional layer: 1 × 1cov, 512/1, where 1 × 1 denotes the width × height of the convolution kernel, 512 denotes the number of output channels of the first convolution layer, and 1 denotes a 1-fold reduction in image resolution of the third convolution layer;
side branch convolutional layer: and if the image resolution of the first convolution layer is reduced by 1 time, the first convolution layer is connected with the convolution layer through a residual error and is directly communicated with a third convolution layer for superposition output.
6. The classification detection method based on the lightweight classification model according to claim 5, wherein the side branch convolution layer further comprises:
if the image resolution of the first convolution layer is reduced by 2 times, adding 1 × 1 convolution kernel, processing branch convolution layers with 512 output channels, and then superposing the branch convolution layers to a third convolution layer for output.
7. A classification detection apparatus based on a lightweight classification model, the classification detection apparatus comprising:
the device comprises a training module and a classification detection module;
wherein the training module is to:
acquiring a training sample image and determining the category of the training sample image;
classifying and marking the training sample images according to the categories to obtain sample marked images;
inputting the sample labeled image into a lightweight classification model for training to obtain a trained lightweight classification model;
the classification detection module is configured to:
inputting a sample image to be evaluated into a trained lightweight classification model, and judging and outputting the category of the sample image to be evaluated;
the basic structure of the lightweight classification model comprises a trunk, wherein the trunk is used for extracting the features of the preprocessed sample image to be evaluated at different levels;
the main frame comprises at least one stage, the stage is composed of at least two model basic units, the resolution of an image processed by a first model basic unit in the stage is reduced, the resolution of an image processed by a second model basic unit is unchanged, the first model basic unit is used for extracting the characteristics of a preprocessed sample image to be evaluated and outputting the sample image to be evaluated after the characteristics are extracted and the preprocessed sample image to be evaluated after superposition, the second model basic unit is used for performing convolution operation on a superposed image output by the first model basic unit again and directly inputting the superposed image after the convolution operation to the next stage;
the model basic unit comprises convolution layers, and the initial width of each stage is equal to the number of output channels of the convolution layers in the stage.
8. The classification detection apparatus based on a lightweight classification model according to claim 7,
the basic structure of the lightweight classification model further comprises a branch and a head;
the branch adopts a branch convolution layer to preprocess the sample image to be evaluated, and the preprocessed sample image to be evaluated is input into the trunk;
the main stem comprises a first stage, a second stage, a third stage and a fourth stage, the features of different levels of the preprocessed sample image to be evaluated are sequentially extracted step by step through the first stage, the second stage, the third stage and the fourth stage, and the sample image to be evaluated containing the highest level features is input to the head;
the head comprises a global pooling layer and a linear layer, and the category of the sample image to be evaluated is output by calculating and comparing the category probability value of the sample image to be evaluated containing the highest-level features.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the classification detection method according to any one of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the classification detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973138.XA CN113642662B (en) | 2021-08-24 | 2021-08-24 | Classification detection method and device based on lightweight classification model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110973138.XA CN113642662B (en) | 2021-08-24 | 2021-08-24 | Classification detection method and device based on lightweight classification model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113642662A true CN113642662A (en) | 2021-11-12 |
| CN113642662B CN113642662B (en) | 2024-02-20 |
Family
ID=78423534
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110973138.XA Active CN113642662B (en) | 2021-08-24 | 2021-08-24 | Classification detection method and device based on lightweight classification model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113642662B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116627789A (en) * | 2023-07-19 | 2023-08-22 | 支付宝(杭州)信息技术有限公司 | Model detection method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287849A (en) * | 2019-06-20 | 2019-09-27 | 北京工业大学 | A kind of lightweight depth network image object detection method suitable for raspberry pie |
| US20200065606A1 (en) * | 2018-08-24 | 2020-02-27 | Petrochina Company Limited | Method and apparatus for automatically extracting image features of electrical imaging well logging |
| CN112561910A (en) * | 2020-12-28 | 2021-03-26 | 中山大学 | Industrial surface defect detection method based on multi-scale feature fusion |
| CN113033547A (en) * | 2021-02-27 | 2021-06-25 | 北京工业大学 | Welding state classification method based on MobileNet V2 |
| CN113052246A (en) * | 2021-03-30 | 2021-06-29 | 北京百度网讯科技有限公司 | Method and related device for training classification model and image classification |
-
2021
- 2021-08-24 CN CN202110973138.XA patent/CN113642662B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200065606A1 (en) * | 2018-08-24 | 2020-02-27 | Petrochina Company Limited | Method and apparatus for automatically extracting image features of electrical imaging well logging |
| CN110287849A (en) * | 2019-06-20 | 2019-09-27 | 北京工业大学 | A kind of lightweight depth network image object detection method suitable for raspberry pie |
| CN112561910A (en) * | 2020-12-28 | 2021-03-26 | 中山大学 | Industrial surface defect detection method based on multi-scale feature fusion |
| CN113033547A (en) * | 2021-02-27 | 2021-06-25 | 北京工业大学 | Welding state classification method based on MobileNet V2 |
| CN113052246A (en) * | 2021-03-30 | 2021-06-29 | 北京百度网讯科技有限公司 | Method and related device for training classification model and image classification |
Non-Patent Citations (1)
| Title |
|---|
| 蒋梦莹;林小竹;柯岩;: "基于优化分类的数据增广方法", 计算机工程与设计, no. 11 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116627789A (en) * | 2023-07-19 | 2023-08-22 | 支付宝(杭州)信息技术有限公司 | Model detection method and device, electronic equipment and storage medium |
| CN116627789B (en) * | 2023-07-19 | 2023-11-03 | 支付宝(杭州)信息技术有限公司 | Model detection method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113642662B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113674247B (en) | X-ray weld defect detection method based on convolutional neural network | |
| CN112966684B (en) | Cooperative learning character recognition method under attention mechanism | |
| US20210174149A1 (en) | Feature fusion and dense connection-based method for infrared plane object detection | |
| CN110533086B (en) | Semi-automatic image data labeling method | |
| CN110413319B (en) | Code function taste detection method based on deep semantics | |
| US20230360390A1 (en) | Transmission line defect identification method based on saliency map and semantic-embedded feature pyramid | |
| CN106895975A (en) | Bearing fault diagnosis method based on Stacked SAE deep neural network | |
| CN111397902B (en) | Rolling bearing fault diagnosis method based on feature alignment convolutional neural network | |
| CN113657383A (en) | Defect region detection method and device based on lightweight segmentation model | |
| CN112756759B (en) | Spot welding robot workstation fault judgment method | |
| CN111382785A (en) | GAN network model and method for realizing automatic cleaning and auxiliary marking of sample | |
| US20210096530A1 (en) | System and method for identifying manufacturing defects | |
| CN112381763A (en) | Surface defect detection method | |
| CN114882340A (en) | Weak supervision target detection method based on bounding box regression | |
| CN111161224A (en) | Casting internal defect grading evaluation system and method based on deep learning | |
| CN115631186B (en) | Industrial element surface defect detection method based on double-branch neural network | |
| CN113034483A (en) | Cigarette defect detection method based on deep migration learning | |
| CN111598854B (en) | Segmentation method for small defects of complex textures based on rich robust convolution feature model | |
| CN114627106A (en) | Weld defect detection method based on Cascade Mask R-CNN model | |
| CN111177135B (en) | Landmark-based data filling method and device | |
| CN107545281B (en) | Single harmful gas infrared image classification and identification method based on deep learning | |
| CN113642662B (en) | Classification detection method and device based on lightweight classification model | |
| CN108932471B (en) | Vehicle detection method | |
| CN114187272A (en) | Industrial part surface defect detection method based on deep learning | |
| CN112215301B (en) | Image straight line detection method based on convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |