CN116721302B - Ice and snow crystal particle image classification method based on lightweight network - Google Patents
Ice and snow crystal particle image classification method based on lightweight network Download PDFInfo
- Publication number
- CN116721302B CN116721302B CN202311002748.0A CN202311002748A CN116721302B CN 116721302 B CN116721302 B CN 116721302B CN 202311002748 A CN202311002748 A CN 202311002748A CN 116721302 B CN116721302 B CN 116721302B
- Authority
- CN
- China
- Prior art keywords
- ice
- convolution
- network
- image
- crystal particle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000013078 crystal Substances 0.000 title claims abstract description 77
- 239000002245 particle Substances 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000012360 testing method Methods 0.000 claims abstract description 36
- 238000012549 training Methods 0.000 claims abstract description 23
- 238000000605 extraction Methods 0.000 claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 9
- 238000003860 storage Methods 0.000 claims abstract description 7
- 230000008521 reorganization Effects 0.000 claims description 25
- 230000007246 mechanism Effects 0.000 claims description 16
- 238000010586 diagram Methods 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000008707 rearrangement Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 241000533950 Leucojum Species 0.000 claims description 3
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims 3
- 230000002779 inactivation Effects 0.000 claims 2
- 230000000694 effects Effects 0.000 abstract description 5
- 238000009826 distribution Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 239000005427 atmospheric aerosol Substances 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an ice and snow crystal particle image classification method based on a lightweight network, which belongs to the technical field of computer vision and comprises the following steps: training the built network model by using a training sample data set containing a plurality of classified ice and snow crystal particle images, classifying, testing and verifying the ice and snow crystal particle images of the network model by using a testing data set of the ice and snow crystal particle images, and selecting model parameters with the best performance for storage; inputting the obtained ice and snow crystal particle image into a trained and tested network model, and carrying out feature extraction and category reasoning calculation on the ice and snow crystal particle sub-image to obtain a predicted category so as to realize automatic classification of the ice and snow crystal particle image. The depth separable cavity convolution layer can effectively fuse global and local characteristics, and has a good classifying effect on detail characteristics of ice and snow crystal grains, which have small differences in scale and structure.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to an ice and snow crystal particle image classification method based on a lightweight network.
Background
Ice and snow crystal particles are important atmospheric aerosol and have important influence on climate and weather. The correct classification of the ice and snow crystal particle images is beneficial to researchers to better understand and research weather patterns, so that weather changes are predicted more accurately, and the accuracy of weather forecast is improved. Secondly, the ice and snow crystal particles can change optical properties, sky colors, human visual feelings and the like in the atmosphere, and the information influences the observation capability and the feeling experience of people. The correct classification of ice and snow crystal particles is helpful for relevant workers to better understand the influencing factors, so that more reasonable weather policies are formulated to ensure the visual experience and health of the public.
In recent years, with the rapid development of deep learning, the artificial intelligence technology has a great application potential in the processing and analysis of images and big data, and the application in the atmospheric science is also valued by researchers, and the artificial intelligence, especially the deep learning technology, is successively applied to the automatic classification research of the ice and snow grain shape. The classical is that the pre-trained residual network ResNet152 is used for identifying the shape of ice and snow crystal particles, and the ICDC (Ice Crystals Database in China, ICDC) ice and snow crystal data set containing 7282 images in 10 categories is used for realizing 96% of identification accuracy, but the prediction effect on small sample data is poor. In order to improve the possible small sample prediction problem of the residual convolution network, an embedded hypergraph convolution layer based deep learning method is proposed, and the embedded hypergraph structure is used for carrying out characteristic relation construction from local and global graph structure, so that the method is improved in the small sample expression, and a better classification effect is obtained under the condition of unbalanced sample distribution.
However, the following problems exist in the prior art: 1. the method is lack of effective extraction of morphological characteristics of ice and snow crystal grains: the existing ice and snow grain sub-classification method based on the deep convolution network has limited capability when processing the problems of large intra-class difference, small inter-class difference fine granularity and unbalanced samples of ice and snow grain sub-, is difficult to fully extract and express characteristic information in ice and snow grain images, causes errors, and reduces classification accuracy and reliability. 2. The reasoning speed is slow: the existing ice and snow grain sub-classification methods based on the deep convolutional network have the problems of over-deep depth, excessive IO (input/output) expenditure calculation and the like, so that the reasoning speed is low, and the methods have a large optimization space in terms of the reasoning speed for rapidly identifying and classifying ice and snow grains. 3. The parameter number is large: the existing ice and snow crystal particle classification method based on the deep convolution network has large-scale parameter, which directly leads to the difficulty in embedding the algorithm into the terminal equipment and limits the practical application of the algorithm.
It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, provides an ice and snow crystal particle image classification method based on a lightweight network, and solves the defects of the prior art.
The aim of the invention is achieved by the following technical scheme: an ice and snow crystal particle image classification method based on a lightweight network, the classification method comprises the following steps:
training the built lightweight network UDiNet by using a training sample data set containing a plurality of classified ice and snow crystal particle images, classifying, testing and verifying the ice and snow crystal particle images of the lightweight network UDiNet by using a testing data set of the ice and snow crystal particle images, and selecting model parameters with the best performance for storage;
inputting the obtained ice and snow crystal particle image into a lightweight network UDiNet which is trained and tested, and carrying out feature extraction and category reasoning calculation on ice and snow crystal particle sub-images to obtain a predicted category so as to realize automatic classification of the ice and snow crystal particle image.
The backbone network of the lightweight network UDiNet is a channel reorganization network, and the lightweight network UDiNet comprises a channel reorganization convolution unit, a lightweight attention mechanism layer and a depth separable cavity convolution layer;
the channel reorganization convolution unit is used for constructing a channel reorganization network, and rearranging information of each channel group in the feature map during grouping convolution to form a channel rearrangement mechanism, so that information fusion among channels is increased; the lightweight attention mechanism layer is used for weighting importance of the feature map, and the attention weight obtained through learning enables the lightweight network UDiNet to automatically select and pay attention to the features which are more important for the current task;
constructing a channel reorganization network by using basic units formed by group convolution and point-by-point convolution, and introducing a group channel rearrangement mechanism into the group convolution to take an output characteristic diagram of a basic unit of the last layer of the channel reorganization network as an input of a depth separable cavity convolution layer.
The depth separable cavity convolution layer uses a residual network structure and comprises depth separable cavity convolution branches and short connection analysis, wherein the two branches respectively take a feature image and the number of channels generated by a basic unit of the last layer of a channel reorganization network as standards, global feature extraction is carried out on an input feature image through the depth separable cavity convolution branches, local information acquisition is carried out on the input feature image through the short connection branches, and the two branch output feature images are connected in the channel dimension to realize channel expansion.
The global feature extraction is carried out on the input feature map through the depth separable cavity convolution branch, and the local information acquisition is carried out on the input feature map through the short connection branch specifically comprises the following steps:
a1, learning characteristic information of the detail of the ice and snow grain sub-image on a single characteristic image through depth convolution with the convolution kernel size of 3 multiplied by 3, and then integrating information of a plurality of characteristic images of the ice and snow grain particle image through point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
a2, carrying out ascending dimension through point-by-point convolution with the convolution kernel size of 1×1 to obtain characteristic information of more different channels of the ice and snow crystal particle image, and then filling two layers of pixel points with the value of 0 along the periphery of the characteristic image so as to not interfere the arrival noise of the characteristic information while expanding the ice and snow crystal grain sub-characteristic image;
a3, learning the characteristic information of the whole ice and snow crystal grain sub-characteristic image on a single characteristic image through the depth convolution with the convolution kernel size of 3 multiplied by 3, and integrating the information of a plurality of characteristic images of the ice and snow crystal grain image through the point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
and A4, connecting the calculation results of the step A1 and the step A2 along the channel dimension to be used as an output characteristic diagram of the depth separable cavity convolution layer.
The training of the lightweight network UDiNet comprises the following contents:
parameters of a convolutional neural network pre-trained on a large image dataset ImageNet are used for initializing a convolutional layer in a backbone network of a lightweight network UDiNet;
using the Pytorch framework, epochs set to 100 and batch Size set to 128;
using AdamW optimization algorithm, the learning rate was set to 0.0001;
in the public data set ICDC including ice and snow crystal particle images of a plurality of categories, 80% of the image data are randomly selected for training for each category.
The test data set of the ice and snow crystal particle image is used for classifying, testing and verifying the ice and snow crystal particle image of the lightweight network UDiNet, and the model parameters with the best performance are selected for storage, which specifically comprises the following contents:
when each Epoch training is finished, testing the lightweight network UDiNet by using the same test sample data set, and evaluating a classification result obtained by the test;
by the formulaCalculating classification accuracy of lightweight network UDiNet for all test sample data sets, ŷ i Representing class, y, of ith image predicted by lightweight network UDiNet i Is the corresponding true class value, M is the total number of images input into the network model at the time of testing, eq (y i, ŷ i ) Is an equal function if and only if y i Approach to ŷ i When the lightweight network UDiNet has the advantages that the better the prediction performance is, the closer the accuracy is to 1;
the data used for testing are all the remaining 20% of image data except a training sample data set in a public data set ICDC containing ice and snow crystal particle images of a plurality of categories;
and saving a lightweight network UDiNet with highest accuracy calculated in multiple tests.
The plurality of classifications of the ice and snow crystal particle image comprise rosettes, long columns, short columns, hollow columns, spheres, small irregularities, plates, hexagonal snowflakes and complex shapes; and (3) inputting the depth separable cavity convolution layer output feature map into a convolution classifier, and normalizing the output probability by using a softmax function to finish the classification task of the ice and snow grain sub-images of 10 classes.
The invention has the following advantages: the ice and snow crystal particle image classification method based on the lightweight network can effectively integrate global and local characteristics through the depth separable cavity convolution layer, has good classification effect on detail characteristics of ice and snow crystal particles with small difference in scale and structure, and can greatly reduce the calculation amount and the parameter amount of a model by dividing a common roll into two steps through the depth separable convolution; the depth separable cavity volume integral branch uses an inverse bottleneck structure, so that a low-dimensional feature map can be mapped to a high dimension under the condition of fewer input feature map channels, and feature extraction is performed on data in the high dimension, thereby reducing loss in the process of extracting feature map information; the lightweight model architecture design is used, so that the model parameter is greatly reduced, the model reasoning speed is accelerated, and the deployment application in various edge computing terminals is expected; by designing the convolution classifier, the space information is reserved, and meanwhile, the convolution classifier has certain robustness on translational invariance of input, and compared with a full-connection classifier, the convolution classifier not only improves the parameter efficiency of a model, but also enhances the generalization capability of the model.
Drawings
FIG. 1 is a schematic flow chart of the present invention;
fig. 2 is a schematic structural diagram 1 of a lightweight network UDiNet of the present invention;
fig. 3 is a schematic structural diagram of a lightweight network UDiNet of the present invention 2;
fig. 4 is a schematic structural diagram of a lightweight network UDiNet of the present invention, fig. 3;
fig. 5 is a schematic diagram of the architecture of the lightweight network UDiNet of the present invention 4;
FIG. 6 is a schematic diagram of a depth separable void convolution layer.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Accordingly, the following detailed description of the embodiments of the present application, provided in connection with the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, are intended to be within the scope of the present application. The invention is further described below with reference to the accompanying drawings.
As shown in fig. 1, the invention specifically relates to an ice and snow crystal particle image classification method based on a lightweight network UDiNet. The network greatly reduces the parameter quantity and the calculated quantity required by the network through the depth separable rolls, and solves the pain points with overlarge parameter template and overlarge calculated quantity of the existing ice and snow grain sub-classification algorithm. Secondly, UDiNet uses a kind of deep separable cavity convolution to realize the quick fusion of space information and position information in the feature map, has increased the receptive field of network, makes the network still possess more accurate classification ability when the quick classification information of integrating. The method specifically comprises the following steps:
training the built lightweight network UDiNet by using a training sample data set containing a plurality of classified ice and snow crystal particle images, classifying, testing and verifying the ice and snow crystal particle images of the lightweight network UDiNet by using a testing data set of the ice and snow crystal particle images, and selecting model parameters with the best performance for storage;
inputting the obtained ice and snow crystal particle image into a lightweight network UDiNet which is trained and tested, and carrying out feature extraction and category reasoning calculation on ice and snow crystal particle sub-images to obtain a predicted category so as to realize automatic classification of the ice and snow crystal particle image.
The input of the lightweight network UDiNet is 224×224×3 RGB color ice and snow crystal particle image, the backbone network is channel reorganization network, the lightweight network UDiNet comprises a channel reorganization convolution unit, a lightweight attention mechanism layer and a depth separable cavity convolution layer;
wherein, the depth separable hole convolution layer is the last layer convolution network of the network, and each lightweight attention mechanism layer of the lightweight network UDiNet is activated by using ReLU6, and the batch normalization Batchnormal acceleration training with an index smoothing of momentum=0.9 improves the performance.
As shown in fig. 2-5, first, a general convolution with a convolution kernel size of 3×3 is used to perform preliminary feature extraction in shallow layers, then a channel reorganization unit and an attention mechanism module are combined to perform channel reorganization feature learning with an attention mechanism, then a depth separable hole convolution is used in the last layer of a feature extractor part of a network to perform full feature extraction and integration on all features between channels and in channels, and finally, based on the output of the network feature extractor, a full connection layer is used to predict the category of a model.
The channel reorganization convolution unit is used for constructing a channel reorganization network, and rearranges the information of each channel group in the feature map during grouping convolution to form a channel rearrangement mechanism, so that information fusion among channels is increased. In order to further increase the recognition accuracy, as shown in fig. 3 and fig. 4, a lightweight attention mechanism layer is introduced after the step 2 and the step 3 of the channel reorganization network to realize the importance weighting of the feature map, and the model can automatically select and pay attention to the features which are more important to the current task through the attention weight obtained by learning. The method is beneficial to improving the perception capability of the model to key features and reducing the interference of redundant information, so that the performance and generalization capability of the model are improved. And finally, taking the output characteristic diagram of the final layer of channel reorganization convolution unit as the input of the depth separable cavity convolution layer.
Constructing a channel reorganization network by using basic units formed by group convolution and point-by-point convolution, and introducing a group channel rearrangement mechanism into the group convolution to take an output characteristic diagram of a basic unit of the last layer of the channel reorganization network as an input of a depth separable cavity convolution layer.
The depth separable cavity convolution layer uses a residual network structure and comprises depth separable cavity convolution branches and short connection analysis, wherein the two branches respectively take a feature image and the number of channels generated by a basic unit of the last layer of a channel reorganization network as standards, global feature extraction is carried out on an input feature image through the depth separable cavity convolution branches, local information acquisition is carried out on the input feature image through the short connection branches, and the two branch output feature images are connected in the channel dimension to realize channel expansion.
As shown in fig. 6, global feature extraction is performed on an input feature map through a depth separable cavity convolution branch, and local information acquisition is performed on the input feature map through a short connection branch, which specifically includes:
a1, learning characteristic information of the detail of the ice and snow grain sub-image on a single characteristic image through depth convolution with the convolution kernel size of 3 multiplied by 3, and then integrating information of a plurality of characteristic images of the ice and snow grain particle image through point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
a2, carrying out ascending dimension through point-by-point convolution with the convolution kernel size of 1×1 to obtain characteristic information of more different channels of the ice and snow crystal particle image, and then filling two layers of pixel points with the value of 0 along the periphery of the characteristic image so as to not interfere the arrival noise of the characteristic information while expanding the ice and snow crystal grain sub-characteristic image;
a3, learning the characteristic information of the whole ice and snow crystal grain sub-characteristic image on a single characteristic image through the depth convolution with the convolution kernel size of 3 multiplied by 3, and integrating the information of a plurality of characteristic images of the ice and snow crystal grain image through the point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
and A4, connecting the calculation results of the step A1 and the step A2 along the channel dimension to be used as an output characteristic diagram of the depth separable cavity convolution layer.
The loss function of the lightweight network UDiNet selects a cross entropy loss function, which is defined as shown in the following formula:
wherein N is the number of samples, x i Is an inputted ice and snow crystal particle image, and the probability distribution p (x i ) For desired output, i.e. ice and snow crystal particle image x i Probability distribution q (x i ) For actually outputting, i.e. the network model is used for imaging the ice and snow crystal particles x i The smaller the losses of the predicted category, the closer the distribution of the two is, the better the classification result is.
The training of the lightweight network UDiNet comprises the following contents:
parameters of a convolutional neural network pre-trained on a large image dataset ImageNet are used for initializing a convolutional layer in a backbone network of a lightweight network UDiNet;
using the Pytorch framework, epochs set to 100 and batch Size set to 128;
using AdamW optimization algorithm, the learning rate was set to 0.0001;
in the public data set ICDC including ice and snow crystal particle images of a plurality of categories, 80% of the image data are randomly selected for training for each category. Meanwhile, as the number of samples of the existing data set is small, in order to improve the generalization capability of the model, preprocessing operations such as random aspect ratio cutting, random horizontal overturning and the like are performed on the data to enhance the data, so that the data set is expanded to a certain extent; the image is standardized, the image data is subjected to centering through mean removal, and according to the convex optimization theory and the data probability distribution related knowledge, the data centering accords with the data distribution rule, so that the generalization effect after training is easier to obtain; furthermore, to accommodate the input size of the UDiNet model, random cropping is used to unify the image size to 224×224. The specific pretreatment operation is as follows: random horizontal overturn and vertical overturn of the image are realized with probability of 0.8; filling 0 according to the size of the image to enable the size of the image to be square, and then unifying the image to 256×256 size by using a bilinear interpolation method of a traditional interpolation algorithm; 224×224 random clipping is performed again for the model UDiNet input; and performing pixel-by-pixel standardization processing, namely subtracting the average value of the pixel values of the corresponding channels from the pixel value of each channel in the image and dividing the average value by the standard deviation to realize data centralization. When the pixel-by-pixel standardization processing is performed, the average value of the three RGB channels in the training set is 0.056,0.331,0.666, and the standard deviation is 0.080,0.218,0.303.
The test data set of the ice and snow crystal particle image is used for classifying, testing and verifying the ice and snow crystal particle image of the lightweight network UDiNet, and the model parameters with the best performance are selected for storage, which specifically comprises the following contents:
when each Epoch training is finished, testing the lightweight network UDiNet by using the same test sample data set, and evaluating a classification result obtained by the test;
by the formulaCalculating classification accuracy of lightweight network UDiNet for all test sample data sets, ŷ i Representing class, y, of ith image predicted by lightweight network UDiNet i Is the corresponding true class value, M is the total number of images input into the network model at the time of testing, eq (y i, ŷ i ) Is an equal function if and only if y i Approach to ŷ i When the lightweight network UDiNet has the advantages that the better the prediction performance is, the closer the accuracy is to 1;
the data used for testing are all the remaining 20% of image data except a training sample data set in a public data set ICDC containing ice and snow crystal particle images of a plurality of categories; and the following preprocessing operation is carried out on the network before the network is input: scaling the image to 256 x 256 using bilinear interpolation; randomly cropping the image to 224×224; and performing pixel-by-pixel standardization processing, namely subtracting the average value of the pixel values of the corresponding channels from the pixel value of each channel in the image and dividing the average value by the standard deviation to realize data centralization. When pixel-by-pixel standardization processing is carried out, the average value of RGB channels in a test set is 0.056,0.331,0.666, and the standard deviation is 0.080,0.218,0.303 respectively;
and saving a lightweight network UDiNet with highest accuracy calculated in multiple tests.
The plurality of classifications of the ice and snow crystal particle image comprise rosettes, long columns, short columns, hollow columns, spheres, small irregularities, plates, hexagonal snowflakes and complex shapes; and (3) inputting the depth separable cavity convolution layer output feature map into a convolution classifier, and normalizing the output probability by using a softmax function to finish the classification task of the ice and snow grain sub-images of 10 classes.
The foregoing is merely a preferred embodiment of the invention, and it is to be understood that the invention is not limited to the form disclosed herein but is not to be construed as excluding other embodiments, but is capable of numerous other combinations, modifications and environments and is capable of modifications within the scope of the inventive concept, either as taught or as a matter of routine skill or knowledge in the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.
Claims (3)
1. A method for classifying ice and snow crystal particle images based on a lightweight network is characterized by comprising the following steps of: the classification method comprises the following steps:
training the built lightweight network UDiNet by using a training sample data set containing a plurality of classified ice and snow crystal particle images, classifying, testing and verifying the ice and snow crystal particle images of the lightweight network UDiNet by using a testing data set of the ice and snow crystal particle images, and selecting model parameters with the best performance for storage;
inputting the obtained ice and snow crystal particle image into a lightweight network UDiNet which is trained and tested, and carrying out feature extraction and category reasoning calculation on ice and snow crystal particle sub-images to obtain a predicted category so as to realize automatic classification of the ice and snow crystal particle image;
the backbone network of the lightweight network UDiNet is a channel reorganization network, and the lightweight network UDiNet comprises a channel reorganization convolution unit, a lightweight attention mechanism layer and a depth separable cavity convolution layer;
the channel reorganization convolution unit is used for constructing a channel reorganization network, and rearranging information of each channel group in the feature map during grouping convolution to form a channel rearrangement mechanism, so that information fusion among channels is increased; the lightweight attention mechanism layer is used for weighting importance of the feature map, and the attention weight obtained through learning enables the lightweight network UDiNet to automatically select and pay attention to the features which are more important for the current task; the lightweight attention mechanism layer comprises the operations of depth separable convolution, maximum pooling, depth separable convolution and expansion normalization indexes on each group of feature images, attention force diagrams of the current group of feature images are obtained, cross product operation is carried out on the obtained attention force diagrams and the feature images, and an obtained operation result is added with the feature images to realize feature reuse;
constructing a channel reorganization network by using basic units formed by grouping convolution and point-by-point convolution, and introducing a grouping channel rearrangement mechanism into the grouping convolution to take an output characteristic diagram of a basic unit of the last layer of the channel reorganization network as an input of a depth separable cavity convolution layer;
the depth separable cavity convolution layer uses a residual network structure and comprises depth separable cavity convolution branches and short connection analysis, wherein the two branches respectively take a feature image and the number of channels generated by a basic unit of the last layer of a channel reorganization network as standards, global feature extraction is carried out on an input feature image through the depth separable cavity convolution branches, local information acquisition is carried out on the input feature image through the short connection branches, and the two branch output feature images are connected in the channel dimension to realize channel expansion;
the global feature extraction is carried out on the input feature map through the depth separable cavity convolution branch, and the local information acquisition is carried out on the input feature map through the short connection branch specifically comprises the following steps:
a1, learning characteristic information of the detail of the ice and snow grain sub-image on a single characteristic image through depth convolution with the convolution kernel size of 3 multiplied by 3, and then integrating information of a plurality of characteristic images of the ice and snow grain particle image through point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
a2, carrying out ascending dimension through point-by-point convolution with the convolution kernel size of 1×1 to obtain characteristic information of more different channels of the ice and snow crystal particle image, and then filling two layers of pixel points with the value of 0 along the periphery of the characteristic image so as to not interfere the arrival noise of the characteristic information while expanding the ice and snow crystal grain sub-characteristic image;
a3, learning the characteristic information of the whole ice and snow crystal grain sub-characteristic image on a single characteristic image through the depth convolution with the convolution kernel size of 3 multiplied by 3, and integrating the information of a plurality of characteristic images of the ice and snow crystal grain image through the point-by-point convolution with the convolution kernel size of 1 multiplied by 1;
a4, connecting the calculation results of the step A1 and the step A2 along the channel dimension to be used as an output characteristic diagram of the depth separable cavity convolution layer;
the plurality of classifications of the ice and snow crystal particle image comprise rosettes, long columns, short columns, hollow columns, spheres, small irregularities, plates, hexagonal snowflakes and complex shapes; inputting the depth separable cavity convolution layer output feature map into a convolution classifier, and normalizing the output probability by using a softmax function to finish the classification task of the ice and snow grain subimage of 10 classes;
the convolution classifier comprises an input channel, a random inactivation unit, point-by-point convolution, average pooling and an output channel, wherein the input channel, the random inactivation unit, the point-by-point convolution, the average pooling and the output channel are sequentially connected.
2. The method for classifying ice and snow crystal particle images based on the lightweight network according to claim 1, wherein the method is characterized by comprising the following steps of: the training of the lightweight network UDiNet comprises the following contents:
parameters of a convolutional neural network pre-trained on a large image dataset ImageNet are used for initializing a convolutional layer in a backbone network of a lightweight network UDiNet;
using the Pytorch framework, epochs set to 100 and batch Size set to 128;
using AdamW optimization algorithm, the learning rate was set to 0.0001;
in the public data set ICDC including ice and snow crystal particle images of a plurality of categories, 80% of the image data are randomly selected for training for each category.
3. The method for classifying ice and snow crystal particle images based on the lightweight network according to claim 1, wherein the method is characterized by comprising the following steps of: the test data set of the ice and snow crystal particle image is used for classifying, testing and verifying the ice and snow crystal particle image of the lightweight network UDiNet, and the model parameters with the best performance are selected for storage, which specifically comprises the following contents:
when each Epoch training is finished, testing the lightweight network UDiNet by using the same test sample data set, and evaluating a classification result obtained by the test;
by the formulaCalculating the classification accuracy of the lightweight network UDiNet for all test sample data sets, wherein +.>Representing class, y, of ith image predicted by lightweight network UDiNet i Is the corresponding true class value, M is the total number of images input into the network model at test, +.>Is an equal function if and only if y i Approach->When the lightweight network UDiNet has the advantages that the better the prediction performance is, the closer the accuracy is to 1;
the data used for testing are all the remaining 20% of image data except a training sample data set in a public data set ICDC containing ice and snow crystal particle images of a plurality of categories;
and saving a lightweight network UDiNet with highest accuracy calculated in multiple tests.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311002748.0A CN116721302B (en) | 2023-08-10 | 2023-08-10 | Ice and snow crystal particle image classification method based on lightweight network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311002748.0A CN116721302B (en) | 2023-08-10 | 2023-08-10 | Ice and snow crystal particle image classification method based on lightweight network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116721302A CN116721302A (en) | 2023-09-08 |
| CN116721302B true CN116721302B (en) | 2024-01-12 |
Family
ID=87864761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311002748.0A Active CN116721302B (en) | 2023-08-10 | 2023-08-10 | Ice and snow crystal particle image classification method based on lightweight network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116721302B (en) |
Citations (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019071754A1 (en) * | 2017-10-09 | 2019-04-18 | 哈尔滨工业大学深圳研究生院 | Method for sensing image privacy on the basis of deep learning |
| CN110427990A (en) * | 2019-07-22 | 2019-11-08 | 浙江理工大学 | A kind of art pattern classification method based on convolutional neural networks |
| CN111539314A (en) * | 2020-04-21 | 2020-08-14 | 上海海事大学 | Cloud and fog shielding-oriented sea surface target significance detection method |
| CN111553193A (en) * | 2020-04-01 | 2020-08-18 | 东南大学 | Visual SLAM closed-loop detection method based on lightweight deep neural network |
| WO2020222985A1 (en) * | 2019-04-30 | 2020-11-05 | The Trustees Of Dartmouth College | System and method for attention-based classification of high-resolution microscopy images |
| CN113077444A (en) * | 2021-04-01 | 2021-07-06 | 复旦大学 | CNN-based ultrasonic nondestructive detection image defect classification method |
| CN113420643A (en) * | 2021-06-21 | 2021-09-21 | 西北工业大学 | Lightweight underwater target detection method based on depth separable cavity convolution |
| CN113627281A (en) * | 2021-07-23 | 2021-11-09 | 中南民族大学 | SK-EfficientNet-based lightweight crop disease identification method |
| CN113642390A (en) * | 2021-07-06 | 2021-11-12 | 西安理工大学 | Street view image semantic segmentation method based on local attention network |
| CN114266898A (en) * | 2022-01-11 | 2022-04-01 | 辽宁石油化工大学 | Liver cancer identification method based on improved EfficientNet |
| CN114445430A (en) * | 2022-04-08 | 2022-05-06 | 暨南大学 | Real-time image semantic segmentation method and system for lightweight multi-scale feature fusion |
| CN114724019A (en) * | 2022-03-31 | 2022-07-08 | 湘潭大学 | Remote sensing image sea ice intelligent monitoring method based on wavelet transformation and separable convolution semantic segmentation |
| CN114757904A (en) * | 2022-04-07 | 2022-07-15 | 河南大学 | Surface defect detection method based on AI deep learning algorithm |
| CN114882234A (en) * | 2022-04-29 | 2022-08-09 | 西安理工大学 | Construction method of multi-scale lightweight dense connected target detection network |
| CN114937151A (en) * | 2022-05-06 | 2022-08-23 | 西安电子科技大学 | Lightweight target detection method based on multi-receptive-field and attention feature pyramid |
| CN115049941A (en) * | 2022-06-27 | 2022-09-13 | 无锡学院 | Improved ShuffleNet convolutional neural network and remote sensing image classification method thereof |
| CN115409846A (en) * | 2022-05-30 | 2022-11-29 | 福州大学 | Colorectal cancer focus region lightweight segmentation method based on deep learning |
| CN115496658A (en) * | 2022-09-25 | 2022-12-20 | 桂林理工大学 | Lightweight image super-resolution reconstruction method based on double attention mechanism |
| CN115496951A (en) * | 2022-10-09 | 2022-12-20 | 淮阴工学院 | Security equipment detection method based on improved YOLOv5s |
| CN115690479A (en) * | 2022-05-23 | 2023-02-03 | 安徽理工大学 | Remote sensing image classification method and system based on convolution Transformer |
| CN115830466A (en) * | 2022-11-19 | 2023-03-21 | 山东科技大学 | Glacier change remote sensing detection method based on deep twin neural network |
| CN116029946A (en) * | 2023-03-29 | 2023-04-28 | 中南大学 | Heterogeneous residual error attention neural network model-based image denoising method and system |
| CN116229283A (en) * | 2023-03-10 | 2023-06-06 | 江西师范大学 | Remote sensing image change detection system and method based on depth separable convolution module |
| WO2023101564A1 (en) * | 2021-12-02 | 2023-06-08 | Kahu.Ai Limited | Skin lesion classification system and method |
| CN116310273A (en) * | 2023-01-10 | 2023-06-23 | 河南工业大学 | Unmanned aerial vehicle detection method based on multi-cavity convolution and SE attention residual error |
| WO2023123108A1 (en) * | 2021-12-29 | 2023-07-06 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Methods and systems for enhancing qualities of images |
| CN116434002A (en) * | 2023-03-24 | 2023-07-14 | 国网河北省电力有限公司电力科学研究院 | Smoke detection method, system, medium and equipment based on lightweight neural network |
| CN116452900A (en) * | 2023-04-24 | 2023-07-18 | 西安电子科技大学 | Target detection method based on lightweight neural network |
| CN116524432A (en) * | 2023-04-18 | 2023-08-01 | 中交华控(天津)建筑集团有限公司 | Application of small target detection algorithm in traffic monitoring |
| CN116524253A (en) * | 2023-04-19 | 2023-08-01 | 杭州电子科技大学 | Thyroid cancer pathological image classification method based on lightweight transducer |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210279505A1 (en) * | 2020-03-09 | 2021-09-09 | Shenzhen Malong Technologies Co., Ltd. | Progressive verification system and methods |
-
2023
- 2023-08-10 CN CN202311002748.0A patent/CN116721302B/en active Active
Patent Citations (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019071754A1 (en) * | 2017-10-09 | 2019-04-18 | 哈尔滨工业大学深圳研究生院 | Method for sensing image privacy on the basis of deep learning |
| WO2020222985A1 (en) * | 2019-04-30 | 2020-11-05 | The Trustees Of Dartmouth College | System and method for attention-based classification of high-resolution microscopy images |
| CN110427990A (en) * | 2019-07-22 | 2019-11-08 | 浙江理工大学 | A kind of art pattern classification method based on convolutional neural networks |
| CN111553193A (en) * | 2020-04-01 | 2020-08-18 | 东南大学 | Visual SLAM closed-loop detection method based on lightweight deep neural network |
| CN111539314A (en) * | 2020-04-21 | 2020-08-14 | 上海海事大学 | Cloud and fog shielding-oriented sea surface target significance detection method |
| CN113077444A (en) * | 2021-04-01 | 2021-07-06 | 复旦大学 | CNN-based ultrasonic nondestructive detection image defect classification method |
| CN113420643A (en) * | 2021-06-21 | 2021-09-21 | 西北工业大学 | Lightweight underwater target detection method based on depth separable cavity convolution |
| CN113642390A (en) * | 2021-07-06 | 2021-11-12 | 西安理工大学 | Street view image semantic segmentation method based on local attention network |
| CN113627281A (en) * | 2021-07-23 | 2021-11-09 | 中南民族大学 | SK-EfficientNet-based lightweight crop disease identification method |
| WO2023101564A1 (en) * | 2021-12-02 | 2023-06-08 | Kahu.Ai Limited | Skin lesion classification system and method |
| WO2023123108A1 (en) * | 2021-12-29 | 2023-07-06 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Methods and systems for enhancing qualities of images |
| CN114266898A (en) * | 2022-01-11 | 2022-04-01 | 辽宁石油化工大学 | Liver cancer identification method based on improved EfficientNet |
| CN114724019A (en) * | 2022-03-31 | 2022-07-08 | 湘潭大学 | Remote sensing image sea ice intelligent monitoring method based on wavelet transformation and separable convolution semantic segmentation |
| CN114757904A (en) * | 2022-04-07 | 2022-07-15 | 河南大学 | Surface defect detection method based on AI deep learning algorithm |
| CN114445430A (en) * | 2022-04-08 | 2022-05-06 | 暨南大学 | Real-time image semantic segmentation method and system for lightweight multi-scale feature fusion |
| CN114882234A (en) * | 2022-04-29 | 2022-08-09 | 西安理工大学 | Construction method of multi-scale lightweight dense connected target detection network |
| CN114937151A (en) * | 2022-05-06 | 2022-08-23 | 西安电子科技大学 | Lightweight target detection method based on multi-receptive-field and attention feature pyramid |
| CN115690479A (en) * | 2022-05-23 | 2023-02-03 | 安徽理工大学 | Remote sensing image classification method and system based on convolution Transformer |
| CN115409846A (en) * | 2022-05-30 | 2022-11-29 | 福州大学 | Colorectal cancer focus region lightweight segmentation method based on deep learning |
| CN115049941A (en) * | 2022-06-27 | 2022-09-13 | 无锡学院 | Improved ShuffleNet convolutional neural network and remote sensing image classification method thereof |
| CN115496658A (en) * | 2022-09-25 | 2022-12-20 | 桂林理工大学 | Lightweight image super-resolution reconstruction method based on double attention mechanism |
| CN115496951A (en) * | 2022-10-09 | 2022-12-20 | 淮阴工学院 | Security equipment detection method based on improved YOLOv5s |
| CN115830466A (en) * | 2022-11-19 | 2023-03-21 | 山东科技大学 | Glacier change remote sensing detection method based on deep twin neural network |
| CN116310273A (en) * | 2023-01-10 | 2023-06-23 | 河南工业大学 | Unmanned aerial vehicle detection method based on multi-cavity convolution and SE attention residual error |
| CN116229283A (en) * | 2023-03-10 | 2023-06-06 | 江西师范大学 | Remote sensing image change detection system and method based on depth separable convolution module |
| CN116434002A (en) * | 2023-03-24 | 2023-07-14 | 国网河北省电力有限公司电力科学研究院 | Smoke detection method, system, medium and equipment based on lightweight neural network |
| CN116029946A (en) * | 2023-03-29 | 2023-04-28 | 中南大学 | Heterogeneous residual error attention neural network model-based image denoising method and system |
| CN116524432A (en) * | 2023-04-18 | 2023-08-01 | 中交华控(天津)建筑集团有限公司 | Application of small target detection algorithm in traffic monitoring |
| CN116524253A (en) * | 2023-04-19 | 2023-08-01 | 杭州电子科技大学 | Thyroid cancer pathological image classification method based on lightweight transducer |
| CN116452900A (en) * | 2023-04-24 | 2023-07-18 | 西安电子科技大学 | Target detection method based on lightweight neural network |
Non-Patent Citations (3)
| Title |
|---|
| End-to-end aerial scenes classification with depthwise separable convolution and an attention mechanism;Wang, X等;《IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing》;第14卷;10484-10499 * |
| 基于双流CNN 与Bi－LSTM的施工人员不安全行为轻量级识别模型;马莉等;《西安科技大学学报》;第42卷(第4期);809-817 * |
| 基于改进YOLOX的接触网U型抱箍开口销检测算法;石强等;《国外电子测量技术》;第42卷(第7期);154-163 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116721302A (en) | 2023-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113159051B (en) | Remote sensing image lightweight semantic segmentation method based on edge decoupling | |
| CN110929602A (en) | Foundation cloud picture cloud shape identification method based on convolutional neural network | |
| CN108229550B (en) | Cloud picture classification method based on multi-granularity cascade forest network | |
| CN114092832B (en) | High-resolution remote sensing image classification method based on parallel hybrid convolutional network | |
| CN109272500B (en) | Fabric classification method based on adaptive convolutional neural network | |
| CN112381097A (en) | Scene semantic segmentation method based on deep learning | |
| CN108446616B (en) | Road extraction method based on full convolution neural network ensemble learning | |
| CN114092815B (en) | Remote sensing intelligent extraction method for large-range photovoltaic power generation facility | |
| CN111652273B (en) | Deep learning-based RGB-D image classification method | |
| CN109741341A (en) | A kind of image partition method based on super-pixel and long memory network in short-term | |
| CN112528934A (en) | Improved YOLOv3 traffic sign detection method based on multi-scale feature layer | |
| CN112347970A (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| CN112232328A (en) | Remote sensing image building area extraction method and device based on convolutional neural network | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN112766283B (en) | Two-phase flow pattern identification method based on multi-scale convolution network | |
| CN110599502A (en) | Skin lesion segmentation method based on deep learning | |
| CN114973011A (en) | High-resolution remote sensing image building extraction method based on deep learning | |
| CN112001293A (en) | Remote sensing image ground object classification method combining multi-scale information and coding and decoding network | |
| CN115393293A (en) | Electron microscope red blood cell segmentation and positioning method based on UNet network and watershed algorithm | |
| CN113298817A (en) | High-accuracy semantic segmentation method for remote sensing image | |
| CN104504391B (en) | A kind of hyperspectral image classification method based on sparse features and markov random file | |
| CN111738052A (en) | Multi-feature fusion hyperspectral remote sensing ground object classification method based on deep learning | |
| CN111310820A (en) | Foundation meteorological cloud chart classification method based on cross validation depth CNN feature integration | |
| CN112818777B (en) | Remote sensing image target detection method based on dense connection and feature enhancement | |
| CN116721302B (en) | Ice and snow crystal particle image classification method based on lightweight network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |