CN114662567A - Convolutional neural network image classification method based on different model feature fusion - Google Patents

Convolutional neural network image classification method based on different model feature fusion Download PDF

Info

Publication number
CN114662567A
CN114662567A CN202210204664.4A CN202210204664A CN114662567A CN 114662567 A CN114662567 A CN 114662567A CN 202210204664 A CN202210204664 A CN 202210204664A CN 114662567 A CN114662567 A CN 114662567A
Authority
CN
China
Prior art keywords
neural network
feature
crnet
model
image classification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210204664.4A
Other languages
Chinese (zh)
Inventor
张玉存
臧运瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yanshan University
Original Assignee
Yanshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yanshan University filed Critical Yanshan University
Priority to CN202210204664.4A priority Critical patent/CN114662567A/en
Publication of CN114662567A publication Critical patent/CN114662567A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • G06F18/253Fusion techniques of extracted features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a convolutional neural network image classification method based on different model feature fusion, which comprises the following steps: preparing a data set, dividing the data set into a training set and a verification set, preprocessing the training set, constructing a cross neural network CRNet, setting a loss function and a network hyper-parameter, training the cross neural network CRNet, inputting the preprocessed test set into the trained cross neural network CRNet, and judging an image classification result; the cross neural network CRNet comprises a shallow layer feature extraction network, a different model feature transfer network and a prediction network. According to the method, the characteristic information is transmitted among the models, so that the characteristic reuse is realized, the loss of the key information of the characteristic is reduced, and the image classification precision is improved; meanwhile, redundant layers in the network are reduced, the number of parameters is reduced, the training speed is improved, and the practicability is high.

Description

Convolutional neural network image classification method based on different model feature fusion
Technical Field
The invention belongs to the technical field of image classification, and particularly relates to a convolutional neural network image classification method based on different model feature fusion.
Background
Computer vision technology is widely applied and mainly used for detecting and classifying images, and contents in the images are various objects on the ground. The image identification classification is a reference problem of image processing, and the improvement of the identification precision has great practical value in military application, medicine, life and other scenes.
With the great success of deep learning algorithms in machine vision in recent years, they have been considered as the first method of image processing. In the deep learning field, the performance of the neural network has a direct relation with the structure of the neural network model, the structure comprises the depth of the neural network and an internal path connection mode, namely a characteristic transmission path, theoretically, the depth of the neural network is increased, the complexity of the network is higher, and more accurate characteristics can be extracted, so that the performance of the neural network is improved. However, with the increase of the depth of the neural network, the problems of the gradient disappearance phenomenon, the neural network function degradation, the low training speed and the like become more obvious, and in order to solve the problems, the improvement of the feature transfer mode is particularly important.
The existing widely used deep neural network still has a lot of redundant information when performing feature extraction on input information, and when deep network features are transmitted, there is a possibility that part of detail information is discarded, and the performance of the algorithm is not sufficiently limited by feature extraction in the neural network training process. Therefore, in order to improve the efficiency of neural network image classification, it is urgent and necessary to find a convolutional neural network image classification method based on different model feature fusion to improve the image classification accuracy and speed from the aspect of feature reuse.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a convolutional neural network image classification method based on different model feature fusion. Preparing a data set, dividing the data set into a training set and a verification set, preprocessing the training set and the verification set, constructing a cross neural network CRNet, setting a loss function and a network hyper-parameter, training the cross neural network CRNet, inputting the preprocessed test set into the trained cross neural network CRNet, and judging an image classification result; the cross neural network CRNet comprises a shallow layer feature extraction network, a different model feature transfer network and a prediction network. According to the method, the characteristic information is transmitted among the models, so that the characteristic reuse is realized, the loss of the key information of the characteristic is reduced, and the image classification precision is improved; meanwhile, redundant layers in the network are reduced, the number of parameters is reduced, the training speed is improved, and the practicability is high.
The invention provides a convolutional neural network image classification method based on different model feature fusion, which comprises the following steps:
s1, preparing a data set: dividing a data set into a training set and a verification set, and carrying out preprocessing operation including normalization and order disorder on the data set;
s2, constructing a cross neural network CRNet: when different models are used for forward transmission of feature information, a feature fusion method is used for feature reuse; the cross neural network CRNet comprises a shallow layer feature extraction network, a different model feature transmission network and a prediction network; the shallow feature extraction network specifically comprises the following steps:
s211, inputting the preprocessed training set into a first convolution layer for shallow feature map extraction to obtain a first feature matrix;
s212, obtaining a second feature matrix by the first feature matrix through batch normalization and a first activation function;
s213, inputting the second feature matrix into the maximum pooling layer compression feature size to obtain a third feature matrix;
the different model feature transfer network specifically comprises the following steps:
s221, constructing a first neural network model and a second neural network model, wherein the input of the first neural network model and the input of the second neural network model are both the third feature matrix and both comprise a plurality of basic modules, and each basic module comprises a plurality of second convolution layers;
s222, establishing connection between the first neural network model and the second neural network model, and performing feature fusion;
and S223, performing feature fusion on the outputs of the first neural network model and the second neural network model to obtain a fourth feature matrix.
The prediction network specifically comprises the following steps:
s231, inputting the fourth feature matrix into a pooling layer for global weighted average pooling to obtain a fifth feature matrix, wherein the expression of the global weighted average pooling is as follows:
Figure BDA0003530955830000021
Figure BDA0003530955830000022
wherein,
Figure BDA0003530955830000023
represents a weighted average pixel value; p is a radical ofiRepresenting the probability occupied by the ith pixel value; a isiRepresents the ith pixel value; a isjRepresents traversing the jth pixel value; r is a real number;
s232, classifying the fifth feature matrix through a full connection layer by using a second activation function softmax, and outputting a prediction target image result;
s3, setting a loss function and a network hyper-parameter: setting a loss function, a learning rate, a minimum value of a batch and training times in a cross neural network CRNet;
s4, training a cross neural network CRNet: training the training set subjected to the preprocessing operation in the step S1 based on the cross neural network CRNet constructed in the step S2, and outputting a trained cross neural network CRNet;
s41, initializing neural network parameters and weights;
s42, performing forward propagation and backward propagation, and updating the weight matrix;
s43, judging whether the training times reach the maximum training times, if so, outputting a trained cross neural network CRNet, otherwise, executing a step S42;
and S5, inputting the test set preprocessed in the step S1 into a trained cross neural network CRNet, and judging the image classification result.
Further, in step S221, each of the first neural network model and the second neural network model has 8 basic modules, a structure of the first neural network model is {3,3,2}, a structure of the second neural network model is {2,2,2,2}, and each basic module has 2 second convolutional layers; in the step S222, the 3 rd basic module of the first neural network model and the 4 th basic module of the second neural network model perform feature matrix fusion, and the 2 nd basic module of the second neural network model and the 6 th basic module of the first neural network model perform feature matrix fusion.
Preferably, dropout operation is added to the second convolution layer in step S221 to prevent overfitting, the prediction target image result in step S232 is output in a one-hot manner, and initialization of the weights in step S41 is penalized by using L2 regularization to improve generalization capability of the model.
Preferably, the activation function in step S212 selects the modified linear unit ReLu, and the loss function in step S3 selects the cross entropy loss.
Preferably, the basic module in step S221 adopts a residual structure.
Preferably, the convolution kernel of the first convolution layer in step S211 is set to 7, the convolution kernel of the maximum pooling layer in step S213 is set to 3, and the convolution kernel of the second convolution layer in step S221 is set to 3.
Compared with the prior art, the invention has the technical effects that:
1. the convolutional neural network image classification method based on different model feature fusion is designed, feature reuse is realized through feature information transmission among models, loss of feature key information is reduced, and image classification precision is improved.
2. The convolutional neural network image classification method based on different model feature fusion uses relatively low network depth, reduces redundant layers in the network, accordingly reduces parameters, improves the speed of training the neural network, and has strong practicability.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a flow chart of the convolutional neural network image classification method based on different model feature fusion according to the present invention;
FIG. 2 is an exemplary diagram of a cross neural network CRNet of the present invention;
FIG. 3 is an exemplary diagram of the basic modules of the present invention;
FIG. 4 is a CIFAR10 image classification accuracy chart based on the cross neural network CRNet of the invention.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 shows a convolutional neural network image classification method based on different model feature fusion, which comprises the following steps:
s1, preparing a data set: and dividing the data set into a training set and a verification set, and carrying out preprocessing operation including normalization and disordering of the Image in the data set. In a particular embodiment, a CIFAR10 image classification dataset is selected as the study object.
S2, constructing a cross neural network CRNet, as shown in figure 2: when different models are used for forward transmission of feature information, a feature fusion method is used for feature reuse; the cross neural network CRNet comprises a shallow layer feature extraction network, a different model feature transfer network and a prediction network.
The shallow feature extraction network specifically comprises the following steps:
s211, inputting the preprocessed training set into a first convolution layer with convolution kernel of 7 to perform shallow feature map extraction, and obtaining a first feature matrix.
S212, the first feature matrix is subjected to batch normalization and first activation function correction linear unit ReLu, and a second feature matrix is obtained.
S213, inputting the second feature matrix into the largest pooling layer Pool with convolution kernel of 3 to compress the feature size, and obtaining a third feature matrix.
The different model feature transfer network specifically comprises the following steps:
s221, a first neural network model and a second neural network model are constructed, the input of the first neural network model and the input of the second neural network model are both third feature matrices and both comprise 8 basic modules blocks, the structure of the first neural network model is {3,3,2}, the structure of the second neural network model is {2,2,2,2}, each basic module Block comprises 2 second convolutional layers with convolution kernels of 3, a residual error structure is adopted, and as shown in FIG. 3, a dropout operation is added in each second convolutional layer to prevent overfitting.
S222, establishing connection between the first neural network model and the second neural network model, and performing feature fusion. And the 3 rd basic module Block of the first neural network model and the 4 th basic module Block of the second neural network model are subjected to feature matrix fusion, and the 2 nd basic module Block of the second neural network model and the 6 th basic module Block of the first neural network model are subjected to feature matrix fusion.
And S223, performing feature fusion on the outputs of the first neural network model and the second neural network model to obtain a fourth feature matrix.
The prediction network specifically comprises the following steps:
s231, inputting the fourth feature matrix into the pooling layer for global weighted average pooling wavg pool to obtain a fifth feature matrix, wherein an expression of the global weighted average pooling wavg pool is as follows:
Figure BDA0003530955830000051
Figure BDA0003530955830000052
wherein,
Figure BDA0003530955830000053
represents a weighted average pixel value; p is a radical ofiRepresenting the probability occupied by the ith pixel value; a isiRepresents the ith pixel value; a isjRepresents traversing the jth pixel value; r is a real number.
And S232, classifying the fifth feature matrix through the full connection layer Fc by using a second activation function softmax, outputting a predicted target image result, and outputting the result in a one-hot mode.
In a particular embodiment, the cross neural network CRNet is written based on the TensorFlow framework.
S3, setting a loss function and a network hyper-parameter: and setting a loss function, a learning rate, a minimum value of a batch and training times in the cross neural network CRNet, wherein the loss function selects cross entropy loss.
S4, training a cross neural network CRNet: training the training set subjected to the preprocessing operation in the step S1 based on the cross neural network CRNet constructed in the step S2, and outputting the trained cross neural network CRNet.
And S41, initializing neural network parameters and weights, and performing punishment on the initialization of the weights by utilizing L2 regularization so as to improve the generalization capability of the model.
And S42, carrying out forward propagation and backward propagation, and updating the weight matrix.
And S43, judging whether the training times reach the maximum training times, if so, outputting the trained cross neural network CRNet, otherwise, executing the step S42.
And S5, inputting the test set preprocessed in the step S1 into a trained cross neural network CRNet, and judging the image classification result.
In a specific embodiment, the accuracy of the trained CIFAR10 image classification is shown in fig. 4, which shows that the convolutional neural network image classification method based on different model feature fusion can improve the image classification accuracy.
According to the convolutional neural network image classification method based on different model feature fusion, feature reuse is realized through feature information transmission among models, loss of feature key information is reduced, and image classification precision is improved; the relatively low network depth is used, the redundant layers in the network are reduced, the parameter quantity is reduced, the speed of training the neural network is improved, and the practicability is high.
Finally, it should be noted that: although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that: modifications and equivalents may be made thereto without departing from the spirit and scope of the invention and it is intended to cover in the claims the invention as defined in the appended claims.

Claims (6)

1. A convolutional neural network image classification method based on different model feature fusion is characterized by comprising the following steps:
s1, preparing a data set: dividing a data set into a training set and a verification set, and carrying out preprocessing operations including normalization and disordering on the data set;
s2, constructing a cross neural network CRNet: when different models are used for forward transmission of feature information, a feature fusion method is used for feature reuse; the cross neural network CRNet comprises a shallow layer feature extraction network, a different model feature transmission network and a prediction network; the shallow feature extraction network specifically comprises the following steps:
s211, inputting the preprocessed training set into a first convolution layer for shallow feature map extraction to obtain a first feature matrix;
s212, obtaining a second feature matrix by the first feature matrix through batch normalization and an activation function;
s213, inputting the second feature matrix into the maximum pooling layer compression feature size to obtain a third feature matrix;
the different model feature transfer network specifically comprises the following steps:
s221, constructing a first neural network model and a second neural network model, wherein the input of the first neural network model and the input of the second neural network model are both the third feature matrix and both comprise a plurality of basic modules, and each basic module comprises a plurality of second convolution layers;
s222, establishing connection between the first neural network model and the second neural network model, and performing feature fusion;
and S223, performing feature fusion on the outputs of the first neural network model and the second neural network model to obtain a fourth feature matrix.
The prediction network specifically comprises the following steps:
s231, inputting the fourth feature matrix into a pooling layer for global weighted average pooling to obtain a fifth feature matrix, wherein the expression of the global weighted average pooling is as follows:
Figure FDA0003530955820000011
Figure FDA0003530955820000012
wherein,
Figure FDA0003530955820000013
represents a weighted average pixel value; p is a radical ofiRepresenting the probability occupied by the ith pixel value; a isiRepresents the ith pixel value; a isjRepresents traversing the jth pixel value; r is a real number;
s232, classifying the fifth feature matrix through a full connection layer by using a second activation function softmax, and outputting a prediction target image result;
s3, setting a loss function and a network hyper-parameter: setting a loss function, a learning rate, a minimum value of a batch and training times in a cross neural network CRNet;
s4, training a cross neural network CRNet: training the training set subjected to the preprocessing operation in the step S1 based on the cross neural network CRNet constructed in the step S2, and outputting a trained cross neural network CRNet;
s41, initializing neural network parameters and weights;
s42, performing forward propagation and backward propagation, and updating the weight matrix;
s43, judging whether the training times reach the maximum training times, if so, outputting a trained cross neural network CRNet, otherwise, executing a step S42;
and S5, inputting the test set preprocessed in the step S1 into a trained cross neural network CRNet, and judging the image classification result.
2. The convolutional neural network image classification method based on different model feature fusion of claim 1, wherein in step S221, each of the first neural network model and the second neural network model has 8 basic modules, the structure of the first neural network model is {3,3,2}, the structure of the second neural network model is {2,2,2,2}, and each basic module has 2 second convolutional layers; in the step S222, the 3 rd basic module of the first neural network model and the 4 th basic module of the second neural network model perform feature matrix fusion, and the 2 nd basic module of the second neural network model and the 6 th basic module of the first neural network model perform feature matrix fusion.
3. The convolutional neural network image classification method based on different model feature fusion as claimed in claim 1, wherein dropout operation is added to the second convolutional layer in step S221 to prevent overfitting, the prediction target image result in step S232 is output in a one-hot manner, and initialization of the weights in step S41 is punished by using L2 regularization to improve generalization capability of the model.
4. The method for classifying the image of the convolutional neural network based on fusion of different model features of claim 1, wherein the activation function in step S212 selects a modified linear unit ReLu, and the loss function in step S3 selects cross entropy loss.
5. The convolutional neural network image classification method based on different model feature fusion of claims 1 and 2, wherein the basic module in the step S221 adopts a residual structure.
6. The convolutional neural network image classification method based on different model feature fusion as claimed in claims 1 and 2, wherein the convolution kernel of the first convolutional layer in step S211 is set to 7, the convolution kernel of the maximum pooling layer in step S213 is set to 3, and the convolution kernel of the second convolutional layer in step S221 is set to 3.
CN202210204664.4A 2022-03-03 2022-03-03 Convolutional neural network image classification method based on different model feature fusion Pending CN114662567A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210204664.4A CN114662567A (en) 2022-03-03 2022-03-03 Convolutional neural network image classification method based on different model feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210204664.4A CN114662567A (en) 2022-03-03 2022-03-03 Convolutional neural network image classification method based on different model feature fusion

Publications (1)

Publication Number Publication Date
CN114662567A true CN114662567A (en) 2022-06-24

Family

ID=82027206

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210204664.4A Pending CN114662567A (en) 2022-03-03 2022-03-03 Convolutional neural network image classification method based on different model feature fusion

Country Status (1)

Country Link
CN (1) CN114662567A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115049885A (en) * 2022-08-16 2022-09-13 之江实验室 Storage and calculation integrated convolutional neural network image classification device and method
CN117853926A (en) * 2024-01-17 2024-04-09 南京北斗创新应用科技研究院有限公司 Building detection method and system based on artificial neural network classification

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115049885A (en) * 2022-08-16 2022-09-13 之江实验室 Storage and calculation integrated convolutional neural network image classification device and method
CN115049885B (en) * 2022-08-16 2022-12-27 之江实验室 Storage and calculation integrated convolutional neural network image classification device and method
CN117853926A (en) * 2024-01-17 2024-04-09 南京北斗创新应用科技研究院有限公司 Building detection method and system based on artificial neural network classification
CN117853926B (en) * 2024-01-17 2024-07-02 南京北斗创新应用科技研究院有限公司 Building detection method and system based on artificial neural network classification

Similar Documents

Publication Publication Date Title
CN111126488B (en) Dual-attention-based image recognition method
CN112541409B (en) Attention-integrated residual network expression recognition method
CN114897779B (en) Cervical cytology image abnormal region positioning method and device based on fusion attention
CN109063719B (en) Image classification method combining structure similarity and class information
CN114662567A (en) Convolutional neural network image classification method based on different model feature fusion
US12039440B2 (en) Image classification method and apparatus, and image classification model training method and apparatus
CN111612008A (en) Image segmentation method based on convolution network
CN111008639B (en) License plate character recognition method based on attention mechanism
CN114358188A (en) Feature extraction model processing method, feature extraction model processing device, sample retrieval method, sample retrieval device and computer equipment
CN116468740A (en) Image semantic segmentation model and segmentation method
CN116580322A (en) Unmanned aerial vehicle infrared small target detection method under ground background
Luan et al. Sunflower seed sorting based on convolutional neural network
WO2022127333A1 (en) Training method and apparatus for image segmentation model, image segmentation method and apparatus, and device
CN117237704A (en) Multi-label image classification method based on two-dimensional dependence
CN112132207A (en) Target detection neural network construction method based on multi-branch feature mapping
CN116758610A (en) Attention mechanism and feature fusion-based light-weight human ear recognition method and system
CN115861595A (en) Multi-scale domain self-adaptive heterogeneous image matching method based on deep learning
CN115527253A (en) Attention mechanism-based lightweight facial expression recognition method and system
CN111652079B (en) Expression recognition method and system applied to mobile crowd and storage medium
CN114648760A (en) Image segmentation method, image segmentation device, electronic device, and storage medium
CN114120406A (en) Face feature extraction and classification method based on convolutional neural network
CN113283530A (en) Image classification system based on cascade characteristic blocks
CN112364193A (en) Image retrieval-oriented method for fusing multilayer characteristic deep neural network model
Xu et al. Research and verification of convolutional neural network lightweight in BCI
Song et al. Lightweight small target detection algorithm based on YOLOv5

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication