CN112132278A - Model compression method and device, computer equipment and storage medium - Google Patents
Model compression method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN112132278A CN112132278A CN202011007728.9A CN202011007728A CN112132278A CN 112132278 A CN112132278 A CN 112132278A CN 202011007728 A CN202011007728 A CN 202011007728A CN 112132278 A CN112132278 A CN 112132278A
- Authority
- CN
- China
- Prior art keywords
- model
- backbone network
- feature map
- channel
- test result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007906 compression Methods 0.000 title claims abstract description 71
- 230000006835 compression Effects 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012360 testing method Methods 0.000 claims abstract description 140
- 238000012549 training Methods 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 23
- 238000010586 diagram Methods 0.000 claims description 44
- 238000004364 calculation method Methods 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 12
- 238000013473 artificial intelligence Methods 0.000 abstract description 6
- 238000004821 distillation Methods 0.000 abstract description 3
- 238000013140 knowledge distillation Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000002829 reductive effect Effects 0.000 description 6
- 210000002569 neuron Anatomy 0.000 description 4
- 241001465754 Metazoa Species 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000005457 optimization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 101100400452 Caenorhabditis elegans map-2 gene Proteins 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 101150064138 MAP1 gene Proteins 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, in particular to a model compression method, a model compression device, model compression equipment and a storage medium. The model compression method comprises the steps of obtaining an image recognition model trained in advance and a second backbone network to be trained; inputting an image to be tested into an image recognition model for testing to obtain a model test result; calculating a channel weight vector according to the model test result; respectively inputting the training images into a first backbone network and a second backbone network for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network; calculating model loss based on the first feature map, the second feature map and the channel weight vector; and updating and optimizing the second backbone network according to the model loss to obtain a compressed image identification model. The model compression method solves the problems that the efficiency of distillation compression on the whole artificial intelligent model is low and the universality is not high in the conventional model compression.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a model compression method, a model compression device, computer equipment and a storage medium.
Background
In the field of artificial intelligence, the life cycle of a model can be generally divided into two links of model training and model reasoning. In a model training link, redundancy exists in the model, which is inevitable for pursuing higher recognition accuracy of the model. In the model reasoning link, due to the influence of different reasoning application environments, the model is required to have the high performance characteristics of high reasoning speed, small resource occupation, small file size and the like while the accuracy of the model is concerned.
At present, model compression is a common optimization means for converting a model from a model training link to a model reasoning link. However, the current model compression is distillation compression for the whole artificial intelligence model, and different models have complex and various application scenarios, so that a customized compression scheme needs to be developed when different models are compressed, and the efficiency is low and the universality is low.
Disclosure of Invention
The embodiment of the invention provides a model compression method, a model compression device, computer equipment and a storage medium, and aims to solve the problems of low efficiency and low universality of the existing model compression.
A method of model compression, comprising:
acquiring an image recognition model which is trained in advance according to a training image and a second backbone network to be trained; wherein the image recognition model comprises a first backbone network;
inputting a plurality of images to be tested into the image recognition model for testing to obtain model test results corresponding to the plurality of images to be tested;
calculating a channel weight vector according to the model test result; the channel weight vector is used for describing the importance of a characteristic channel corresponding to a characteristic diagram output by the first backbone network;
inputting the training image into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network;
calculating a model loss based on the first feature map, the second feature map, and the channel weight vector;
and updating and optimizing the second backbone network according to the model loss to obtain a compressed image identification model.
A pattern compression apparatus comprising:
the backbone network acquisition module is used for acquiring a pre-trained image recognition model and a second backbone network to be trained; wherein the image recognition model comprises a first backbone network;
the model testing module is used for inputting a plurality of images to be tested into the image recognition model for testing to obtain model testing results corresponding to the plurality of images to be tested;
the channel weight calculation module is used for calculating a channel weight vector according to the model test result; the channel weight vector is used for describing the importance of a characteristic channel corresponding to a characteristic diagram output by the first backbone network;
the model training module is used for inputting the training images into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network;
a model loss calculation module for calculating a model loss based on the first feature map, the second feature map and the channel weight vector;
and the model updating module is used for updating and optimizing the second backbone network according to the model loss so as to obtain a compressed image identification model.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the above model compression method when executing the computer program.
A computer storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned model compression method.
In the model compression method, the device, the computer equipment and the storage medium, the pre-trained image recognition model and the second backbone network to be trained are obtained, so that knowledge distillation is performed according to the first backbone network in the image recognition model, the second backbone network is trained, model compression is not required to be performed on the whole artificial intelligent model, knowledge distillation is performed on a local network of an original model, display memory occupation and calculation amount are reduced, the model compression process is accelerated, the problem of model compression limitation caused by more applied scenes can be effectively solved, the universality is higher, the tool is facilitated, and the repeated investment can be effectively reduced. Then, the image to be tested is input into the image recognition model for testing to obtain a model test result, so that a channel weight vector is calculated according to the model test result, the importance of each characteristic channel is determined according to the model test result, and the accuracy and the practicability of the channel weight vector are ensured. And then, inputting the training images into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network, so as to calculate the model loss based on the first feature map, the second feature map and the channel weight vector. And finally, updating and optimizing the second backbone network according to the model loss to obtain a compressed image recognition model, so that the second backbone network can learn the characteristics which have great influence on the model accuracy, the recognition accuracy of the target recognition model obtained after the subsequent model compression is improved, and the problem that important information is lost due to the model compression is avoided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a diagram illustrating an application environment of a model compression method according to an embodiment of the present invention;
FIG. 2 is a flow chart of a model compression method according to an embodiment of the present invention;
FIG. 3 is a schematic structural view of a compression method of a mold in the present embodiment;
FIG. 4 is a detailed flowchart of step S202 in FIG. 3;
FIG. 5 is a detailed flowchart of step S203 in FIG. 2;
FIG. 6 is a detailed flowchart of step S203 in FIG. 2;
FIG. 7 is a detailed flowchart of step S205 in FIG. 2;
FIG. 8 is a schematic view of a model compression apparatus in accordance with an embodiment of the present invention;
FIG. 9 is a schematic diagram of a computer device according to an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The model compression method can be applied in an application environment as in fig. 1, where a computer device communicates with a server over a network. The computer device may be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server may be implemented as a stand-alone server.
In an embodiment, as shown in fig. 2, a model compression method is provided, which is described by taking the server in fig. 1 as an example, and includes the following steps:
s201: and acquiring a pre-trained image recognition model and a second backbone network to be trained, wherein the image recognition model comprises a first backbone network.
The first backbone network refers to a feature extraction backbone network in an image recognition model and can be understood as a Teacher network in a traditional knowledge distillation method. The image-to-recognition model is a model trained in advance for recognizing an object in an image, such as an animal, a person, or the like in the image. The second backbone network is a pre-created characteristic backbone network with a smaller scale (e.g., a smaller number of neurons or a smaller number of network layers) than the first backbone network, and may be understood as a Student network in a conventional knowledge distillation method. It should be noted that the feature channels of the feature maps output by the first backbone network and the second backbone network are kept consistent, so as to unify the feature channel dimensions, thereby facilitating subsequent calculation.
It can be understood that, when the image recognition model is trained, the scale of the model is huge due to pursuit of higher model accuracy, and when the model is deployed, high-cost storage space and consumption of computing resources are required due to higher complexity, so that the model is difficult to implement in each hardware platform. It is therefore necessary to perform an indicated distillation of the original model in order to minimize the consumption of computation space and time by the model.
Current artificial intelligence models include a feature extraction backbone network and other parts related to the application. The number of neurons of the feature extraction backbone network is large, and the calculation complexity is highest, so that knowledge distillation is only needed to be performed on the feature extraction backbone network in the embodiment, model compression is not needed to be performed on the whole artificial intelligence model, the compression workload can be reduced, and the problem of model compression limitation caused by more applied scenes can be effectively solved.
Specifically, as shown in the schematic structure diagram of model compression of fig. 3, the image recognition model includes a mask layer (mask layer) connected to the first backbone network (i.e., Teacher network) and a recognition network (i.e., Head network) connected to the mask layer. The mask layer is used for carrying out channel shielding processing on the characteristic channels in the first characteristic diagram so as to calculate an importance evaluation parameter (namely a channel weight vector) of each characteristic channel; the recognition network is used for recognizing the characteristic diagram output by the mask layer to obtain a corresponding recognition result.
As shown in fig. 3, a test image is input into the image recognition model for testing, so as to obtain a channel weight vector corresponding to a feature map output by the first backbone network, then a training image for training the image recognition model is simultaneously input into the first backbone network and the second backbone network, so as to obtain a first feature map and a second feature map, so as to calculate Loss (i.e., model Loss) based on the first feature map, the second feature map and the channel weight vector, and then the model Loss is transmitted to the second backbone network (i.e., Student network) for model optimization, so as to implement model compression.
S202: and inputting the plurality of images to be tested into the image recognition model for testing to obtain model test results corresponding to the plurality of images to be tested.
Specifically, a plurality of images to be tested can be input into the image recognition model for batch testing according to a preset batch size (representing the number of samples to be tested) parameter, so as to count the recognition accuracy of the plurality of images to be tested as a model test result.
S203: calculating a channel weight vector according to the model test result; the channel weight vector is used for describing the importance of the feature channel in the first feature map.
Specifically, a channel weight vector is calculated according to a model test result, namely the model identification accuracy, so that the importance of the feature channel in the first feature map is directly evaluated through the channel weight vector, the importance of each feature channel in the feature map output by the first backbone network or the second backbone network is evaluated through the model identification accuracy, and the accuracy is high and the practicability is high.
S204: and respectively inputting the training images into a first backbone network and a second backbone network for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network.
The first feature map is a feature map output by feature extraction of the training image through the first backbone network. The second feature map is a feature map output by feature extraction of the training image through a second backbone network. Specifically, the training images are respectively input into the first backbone network and the second backbone network for feature extraction, so as to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network, and further facilitate subsequent model loss calculation. It should be noted that the execution sequence of step S203 and step S204 is not in order, and may be executed simultaneously, which is not limited herein.
S205: and calculating the model loss based on the first feature map, the second feature map and the channel weight vector.
In this embodiment, the loss of the feature map is calculated according to the first feature map and the second feature map through a predefined loss function, and then the second backbone network performs weighting with the channel weight vector, so that the second backbone network can learn the features that have a large influence on the accuracy of the model, which is beneficial to improving the recognition accuracy of the target recognition model obtained after the subsequent model compression, and can avoid the problem that important information is lost due to the model compression.
S206: and updating and optimizing the second backbone network according to the model loss to obtain a compressed image identification model.
Specifically, the model parameters (such as model weights) of each neuron in the second backbone network are subjected to partial derivation by a model updating algorithm preset in the image prediction model
The model parameters of each neuron in the second backbone network can be optimized, and when the prediction accuracy of the second backbone network reaches a preset value, the compressed image can be obtainedAnd (4) identifying the model.
In the embodiment, the pre-trained image recognition model and the second backbone network to be trained are obtained, so that knowledge distillation is performed according to the first backbone network in the image recognition model, the second backbone network is trained, model compression is not required to be performed on the whole artificial intelligent model, knowledge distillation is performed on a local network of an original model, the display memory occupation amount and the calculated amount are reduced, the model compression process is accelerated, the problem of model compression limitation caused by more applied scenes can be effectively solved, the universality is higher, the implementation is facilitated, and the repeated investment can be effectively reduced. Then, the image to be tested is input into the image recognition model for testing to obtain a model test result, so that a channel weight vector is calculated according to the model test result, the importance of each characteristic channel is determined according to the model test result, and the accuracy and the practicability of the channel weight vector are ensured. And then, inputting the training images into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network, so as to calculate the model loss based on the first feature map, the second feature map and the channel weight vector. And finally, updating and optimizing the second backbone network according to the model loss to obtain a compressed image recognition model, so that the second backbone network can learn the characteristics which have great influence on the model accuracy, the recognition accuracy of the target recognition model obtained after the subsequent model compression is improved, and the problem that important information is lost due to the model compression is avoided.
In an embodiment, as shown in fig. 4, in step 202, a plurality of images to be tested are input into the image recognition model for testing, so as to obtain model test results corresponding to the plurality of images to be tested, which specifically includes the following steps:
s301: extracting the characteristics of each image to be tested by adopting a first backbone network, and outputting a test characteristic diagram corresponding to each image to be tested; wherein the image to be tested comprises a plurality of characteristic channels.
The test characteristic graph refers to a characteristic graph corresponding to an image to be tested, which is output by the image to be tested through feature extraction of the first backbone network. Specifically, each image to be tested is subjected to feature extraction through a first backbone network, namely, nonlinear transformation such as multilayer convolution, activation, pooling and the like is performed, so that a test feature graph corresponding to each image to be tested is output. The test feature map includes a plurality of feature channels, with different feature channels reflecting different image features.
S302: and carrying out channel shielding treatment on the same characteristic channel in each test characteristic diagram by adopting the mask layer to obtain a third characteristic diagram corresponding to each image to be tested.
The mask layer is a layer of mask covering the original tensor so as to shield or select some specific elements to obtain the image of the target area. The third characteristic diagram is the characteristic diagram after the same characteristic channel is shielded from each first characteristic diagram.
Specifically, a mapping matrix of 0 and 1 is constructed through a mask layer to retain the features of the target feature channel and remove the features of the non-target channel, for example, each test feature map includes a feature channel a and a feature channel b, and if the feature channel 1 in each test feature map needs to be shielded, the mapping matrix corresponding to the feature channel a is filled with 0 and the mapping matrix corresponding to the feature channel b is filled with 1, that is, the feature channel a can be shielded, and the feature channel b is retained.
S303: and identifying each third feature map by adopting an identification network to obtain an identification result corresponding to each third feature map.
Specifically, each third feature map is input into the identification network for identification, so that an identification result corresponding to each third feature map can be obtained.
S304: and obtaining a test result component corresponding to each characteristic channel according to the identification result and the real result corresponding to the image to be tested.
The real result refers to an image classification result corresponding to the pre-labeled image to be tested, for example, if the model application scene of the image recognition model is the animal type in the recognized image, the real result is the real type of the animal in the image to be tested.
Specifically, the identification result is compared with the real result corresponding to the image to be tested, so that the test result component corresponding to the round of test can be obtained. By comparing each recognition result with the real result, the recognition accuracy of a plurality of third feature maps which simultaneously shield the same feature channel can be counted, and the third feature maps can be used as the test result component corresponding to the feature channel.
Exemplarily, assuming that an image to be tested is input into a first backbone network for feature extraction, and output results are a feature map 1 and a feature map 2, each feature map includes 2 feature channels a and b, and by simultaneously shielding the feature channels a in the two first feature maps, a corresponding third feature map a can be obtained1' and a2', respectively, a1' and a2' input as input data of the identification network into the identification network for identification, and obtain corresponding identification result a1"(denoted as cat) and a2"(denoted as dog), comparing each recognition result with a real result (such as dog), and taking the recognition accuracy rate of the test of the current round, namely 50%, as a test result component corresponding to the characteristic channel a. By performing multiple testing rounds, namely, simultaneously shielding each characteristic channel in the two characteristic graphs, the testing result component corresponding to each characteristic channel can be obtained.
S305: and taking the data set containing the test result component corresponding to each characteristic channel as a model test result corresponding to a plurality of images to be tested.
Specifically, a data set of the test result component corresponding to each feature channel is used as a model test result, so that when loss is calculated in the subsequent process, data in the data set is called for calculation.
In an embodiment, as shown in fig. 5, in step S203, that is, according to the model test result, calculating the channel weight vector includes the following steps:
s401: and taking the difference value between the maximum value of the test result component and each test result component in the model test result as a first difference value.
S402: and taking the difference value of the maximum value and the minimum value of the test result component in the model test result as a second difference value.
S403: and calculating the ratio of the first difference value to the second difference value, and adding the ratio to a predefined constant term to obtain a channel weight component corresponding to each test feature map.
Specifically, for intuitive expression, the calculation process of steps S401 to S403 is expressed by the following formula, where Wi is 1+ (Amax-Ai)/(Amax-Amin), where 1 represents a predefined constant term, Amax represents the maximum value of the test result component, Amin represents the minimum value of the test result component, Ai represents the test result component corresponding to the feature channel i, i is the channel identifier, and Wi represents the channel weight component corresponding to the test feature map, and is used to characterize the importance of the feature channel i.
S404: and taking a data set containing the channel weight component corresponding to each test feature map as a channel weight vector.
Specifically, the channel weight component corresponding to each test feature map is stored in a data set, so that the data set is used as a channel weight vector, and when loss is calculated subsequently, data in the data set is called for calculation.
In an embodiment, as shown in fig. 6, in step S203, that is, according to the model test result, the channel weight vector is calculated, which specifically includes the following steps:
s501: and taking the difference value between the maximum value of the test result component and each test result component in the model test result as a first difference value.
S502: and taking the difference value of the maximum value and the minimum value of the test result component in the model test result as a second difference value.
S503: calculating a ratio of the first difference to the second difference, and calculating a product of the ratio and a preset scaling factor.
S504: and adding the product and a predefined constant term to obtain a channel weight component corresponding to each test characteristic diagram.
Specifically, for intuitive expression, the calculation process of steps S501 to S503 is expressed by the following formula, where Wi is 1+ α (Amax-Ai)/(Amax-Amin), where 1 represents a predefined constant term, Amax represents a maximum value of the test result component, Amin represents a minimum value of the test result component, Ai represents a test result component corresponding to the feature channel i, i is a channel identifier, α is a preset scaling factor (default set to 1, and may be set by a user as needed), and Wi represents a channel weight component corresponding to the test feature map, and is used to characterize the importance of the feature channel i.
It can be understood that, in order to further make the difference between the channel weight components obvious, that is, to amplify the difference between the channel weight components, the second backbone network can learn the features that are important to the accuracy of the model, and ensure the accuracy of model compression, in this embodiment, the difference between the channel weight components is amplified by presetting the scaling factor.
S504: and taking a data set containing the channel weight component corresponding to each test feature map as a channel weight vector.
Specifically, the channel weight component corresponding to each test feature map is stored in a data set, so that the data set is used as a channel weight vector, and when loss is calculated subsequently, data in the data set is called for calculation.
In an embodiment, as shown in fig. 7, in step S205, that is, based on the first feature map, the second feature map and the channel weight vector, the calculating of the model loss specifically includes the following steps:
s601: and calculating the first characteristic diagram and the second characteristic diagram by adopting a predefined loss function to obtain the characteristic diagram loss.
S602: and weighting the loss of the characteristic diagram based on the channel weight vector to obtain the model loss.
Specifically, for intuitive expression, the calculation procedures of steps S601 to S602 are expressed here by the following formulas,
wherein Loss represents model Loss, Ft represents a first feature diagram, Fs represents a second feature diagram, W represents a channel weight vector, n represents the number of images to be tested, c represents the number of feature channels in the first feature diagram, and Ft represents the number of feature channels in the first feature diagramiIs shown asIth feature channel map, Fs in a feature mapiThe ith eigen-channel map in the second eigen-channel map is shown, and f represents a predefined loss function, such as L1 loss, MSE loss, etc., which is not limited herein. It should be noted that the number of feature channels of the first feature map and the second feature map is consistent.
In this embodiment, when the model loss is calculated, the weighting processing is performed on the feature loss and the channel weight vector, so that the calculation of the model loss integrates the influence of the feature channel importance parameter, and the accuracy loss of the model compression can be effectively reduced while the model is compressed.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In one embodiment, a model compression apparatus is provided, and the model compression apparatus corresponds to the model compression method in the above embodiments one to one. As shown in fig. 8, the model compression apparatus includes a backbone network obtaining module 10, a model testing module 20, a channel weight calculating module 30, a model training module 40, a model loss calculating module 50, and a model updating module 60. The functional modules are explained in detail as follows:
the backbone network obtaining module 10 is configured to obtain a pre-trained image recognition model and a second backbone network to be trained, where the image recognition model includes a first backbone network.
The model testing module 20 is configured to input the image to be tested into the image recognition model for testing, to obtain a model testing result and a first feature map output by the first backbone network, and input the training image into the second backbone network for feature extraction, to obtain a second feature map output by the second backbone network.
The channel weight calculation module 30 is configured to calculate a channel weight vector corresponding to the first feature map according to the model test result; the channel weight vector is used for describing the importance of the feature channel in the first feature map.
And the model training module 40 is configured to input the training images into the first backbone network and the second backbone network respectively to perform feature extraction, so as to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network.
And a model loss calculation module 50, configured to calculate a model loss based on the first feature map, the second feature map, and the channel weight vector.
And a model updating module 60, configured to update and optimize the second backbone network according to the model loss to obtain a compressed image recognition model.
Specifically, the image recognition model includes a mask layer connected to the first backbone network and a recognition network connected to the mask layer.
Specifically, the model testing module comprises a feature extraction unit, a channel shielding unit, an image identification unit, a result statistics unit and a testing result acquisition unit.
The characteristic extraction unit is used for extracting the characteristics of each image to be tested by adopting a first backbone network and outputting a test characteristic diagram corresponding to each image to be tested; wherein the test feature map includes a plurality of feature channels.
The channel shielding unit is used for carrying out channel shielding treatment on the same characteristic channel in each test characteristic diagram by adopting the mask layer to obtain a third characteristic diagram corresponding to each image to be tested;
and the image identification unit is used for identifying each third feature map by adopting an identification network to obtain an identification result corresponding to each third feature map.
And the result counting unit is used for obtaining the test result component corresponding to each characteristic channel according to the identification result and the real result corresponding to the image to be tested.
And the test result acquisition unit is used for taking the data set containing the test result component corresponding to each characteristic channel as the model test result corresponding to the plurality of images to be tested.
Specifically, the channel weight calculation module includes a first difference calculation unit, a second difference calculation unit, a channel weight component calculation unit, and a channel weight vector acquisition unit.
And the first difference calculating unit is used for taking the difference between the maximum value of the test result components and each test result component in the model test result as a first difference.
And the second difference calculation unit is used for taking the difference between the maximum value and the minimum value of the test result component in the model test result as a second difference.
And the channel weight component calculation unit is used for calculating the ratio of the first difference value and the second difference value, and summing the ratio and a predefined constant term to obtain the channel weight component corresponding to each test characteristic diagram.
And the channel weight vector acquisition unit is used for taking a data set containing the channel weight component corresponding to each test characteristic graph as a channel weight vector.
Specifically, the channel weight calculation module includes a first difference calculation unit, a second difference calculation unit, a scaling unit, a channel weight component calculation unit, and a channel weight vector acquisition unit.
And the first difference calculating unit is used for taking the difference between the maximum value of the test result components and each test result component in the model test result as a first difference.
And the second difference calculation unit is used for taking the difference between the maximum value and the minimum value of the test result component in the model test result as a second difference.
And the scaling unit is used for calculating the ratio of the first difference value and the second difference value and calculating the product of the ratio and a preset scaling factor.
And the channel weight component calculation unit is used for adding the product and a predefined constant term to obtain a channel weight component corresponding to each test characteristic graph.
And the channel weight vector acquisition unit is used for taking a data set containing the channel weight component corresponding to each test characteristic graph as a channel weight vector.
Specifically, the model updating module comprises a feature map loss calculation unit and a model loss calculation unit.
And the characteristic diagram loss calculating unit is used for calculating the first characteristic diagram and the second characteristic diagram by adopting a predefined loss function to obtain the characteristic diagram loss.
And the model loss calculation unit is used for weighting the characteristic diagram loss based on the channel weight vector to obtain the model loss.
For the specific definition of the model compression device, reference may be made to the above definition of the model compression method, which is not described herein again. The modules in the model compression apparatus can be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 9. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a computer storage medium and an internal memory. The computer storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the computer storage media. The database of the computer device is used for storing data generated or acquired during execution of the model compression method, such as an image recognition model. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a model compression method.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the steps of the model compression method in the above embodiments are implemented, for example, steps S201 to S206 shown in fig. 2 or steps shown in fig. 3 to 7. Alternatively, the functions of each module/unit in this embodiment of the model compression apparatus, for example, the functions of each module/unit shown in fig. 8, are implemented when the processor executes the computer program, and are not described here again to avoid repetition.
In an embodiment, a computer storage medium is provided, and a computer program is stored on the computer storage medium, and when being executed by a processor, the computer program implements the steps of the model compression method in the foregoing embodiments, such as steps S201 to S206 shown in fig. 2 or steps shown in fig. 3 to fig. 7, which are not repeated herein for avoiding repetition. Alternatively, the computer program, when executed by the processor, implements the functions of each module/unit in the embodiment of the model compression apparatus, for example, the functions of each module/unit shown in fig. 8, and are not described herein again to avoid repetition.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A method of model compression, comprising:
acquiring an image recognition model which is trained in advance according to a training image and a second backbone network to be trained; wherein the image recognition model comprises a first backbone network;
inputting a plurality of images to be tested into the image recognition model for testing to obtain model test results corresponding to the plurality of images to be tested;
calculating a channel weight vector according to the model test result; the channel weight vector is used for describing the importance of a characteristic channel corresponding to a characteristic diagram output by the first backbone network;
inputting the training image into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network;
calculating a model loss based on the first feature map, the second feature map, and the channel weight vector;
and updating and optimizing the second backbone network according to the model loss to obtain a compressed image identification model.
2. The model compression method of claim 1, wherein the image recognition model includes a mask layer coupled to the first backbone network and a recognition network coupled to the mask layer.
3. The model compression method of claim 2, wherein the inputting a plurality of images to be tested into the image recognition model for testing to obtain model test results corresponding to the plurality of images to be tested comprises:
extracting the characteristics of each image to be tested by adopting the first backbone network, and outputting a test characteristic diagram corresponding to each image to be tested; wherein the test feature map comprises a plurality of feature channels;
carrying out channel shielding treatment on the same characteristic channel in each test characteristic diagram by adopting the mask layer to obtain a third characteristic diagram corresponding to each image to be tested;
identifying each third feature map by using the identification network to obtain an identification result corresponding to each third feature map;
obtaining a test result component corresponding to each characteristic channel according to the identification result and a real result corresponding to the image to be tested;
and taking a data set containing the test result component corresponding to each characteristic channel as a model test result corresponding to the plurality of images to be tested.
4. The model compression method of claim 3, wherein said computing a channel weight vector based on said model test results comprises:
taking the difference value between the maximum value of the test result component and each test result component in the model test result as a first difference value;
taking the difference value between the maximum value and the minimum value of the test result component in the model test result as a second difference value;
calculating a ratio of the first difference value to the second difference value, and summing the ratio and a predefined constant term to obtain a channel weight component corresponding to each test feature map;
and using a data set containing the channel weight component corresponding to each test feature map as the channel weight vector.
5. The model compression method of claim 3, wherein said computing a channel weight vector based on said model test results comprises:
taking the difference value between the maximum value of the test result component and each test result component in the model test result as a first difference value;
taking the difference value between the maximum value and the minimum value of the test result component in the model test result as a second difference value;
calculating a ratio of the first difference to the second difference and calculating a product of the ratio and a preset scaling factor;
adding the product and a predefined constant term to obtain a channel weight component corresponding to each test feature map;
and using a data set containing the channel weight component corresponding to each test feature map as the channel weight vector.
6. The model compression method of claim 1, wherein model penalties are calculated based on the first feature map, the second feature map, and the channel weight vector, including;
calculating the first characteristic diagram and the second characteristic diagram by adopting a predefined loss function to obtain characteristic diagram loss;
and weighting the loss of the characteristic diagram based on the channel weight vector to obtain the model loss.
7. A pattern compression apparatus, comprising:
the backbone network acquisition module is used for acquiring a pre-trained image recognition model and a second backbone network to be trained; wherein the image recognition model comprises a first backbone network;
the model testing module is used for inputting a plurality of images to be tested into the image recognition model for testing to obtain model testing results corresponding to the plurality of images to be tested;
the channel weight calculation module is used for calculating a channel weight vector according to the model test result; the channel weight vector is used for describing the importance of a characteristic channel corresponding to a characteristic diagram output by the first backbone network;
the model training module is used for inputting the training images into the first backbone network and the second backbone network respectively for feature extraction to obtain a first feature map output by the first backbone network and a second feature map output by the second backbone network;
a model loss calculation module for calculating a model loss based on the first feature map, the second feature map and the channel weight vector;
and the model updating module is used for updating and optimizing the second backbone network according to the model loss so as to obtain a compressed image identification model.
8. The pattern compression apparatus of claim 7, wherein the pattern testing module comprises:
the characteristic extraction unit is used for extracting the characteristics of each image to be tested by adopting the first backbone network and outputting a test characteristic diagram corresponding to each image to be tested; wherein the test feature map comprises a plurality of feature channels;
the channel shielding unit is used for carrying out channel shielding treatment on the same characteristic channel in each test characteristic diagram by adopting a mask layer to obtain a third characteristic diagram corresponding to each image to be tested;
the image identification unit is used for identifying each third feature map by adopting an identification network to obtain an identification result corresponding to each third feature map;
the result counting unit is used for obtaining a test result component corresponding to each characteristic channel according to the identification result and the real result corresponding to the image to be tested;
and the test result acquisition unit is used for taking a data set containing a test result component corresponding to each characteristic channel as a model test result corresponding to the plurality of images to be tested.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the model compression method according to any one of claims 1 to 6 when executing the computer program.
10. A computer storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the model compression method according to any one of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011007728.9A CN112132278A (en) | 2020-09-23 | 2020-09-23 | Model compression method and device, computer equipment and storage medium |
| PCT/CN2020/124813 WO2021159748A1 (en) | 2020-09-23 | 2020-10-29 | Model compression method and apparatus, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011007728.9A CN112132278A (en) | 2020-09-23 | 2020-09-23 | Model compression method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112132278A true CN112132278A (en) | 2020-12-25 |
Family
ID=73842781
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011007728.9A Pending CN112132278A (en) | 2020-09-23 | 2020-09-23 | Model compression method and device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112132278A (en) |
| WO (1) | WO2021159748A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990296A (en) * | 2021-03-10 | 2021-06-18 | 中科人工智能创新技术研究院(青岛)有限公司 | Image-text matching model compression and acceleration method and system based on orthogonal similarity distillation |
| WO2023113693A3 (en) * | 2021-12-17 | 2023-10-05 | Lemon Inc. | Optimal knowledge distillation scheme |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113936295A (en) * | 2021-09-18 | 2022-01-14 | 中国科学院计算技术研究所 | Character detection method and system based on transfer learning |
| CN115757745B (en) * | 2022-12-01 | 2023-09-15 | 甘肃省招标咨询集团有限责任公司 | Business scene control method and system based on artificial intelligence and cloud platform |
| CN117218580A (en) * | 2023-09-13 | 2023-12-12 | 杭州像素元科技有限公司 | Expressway cross-camera multi-vehicle tracking method and system combining multiple models |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10223635B2 (en) * | 2015-01-22 | 2019-03-05 | Qualcomm Incorporated | Model compression and fine-tuning |
| CN111695375B (en) * | 2019-03-13 | 2021-04-20 | 上海云从企业发展有限公司 | Face recognition model compression method based on model distillation, medium and terminal |
| CN110880036B (en) * | 2019-11-20 | 2023-10-13 | 腾讯科技（深圳）有限公司 | Neural network compression method, device, computer equipment and storage medium |
| CN111461212B (en) * | 2020-03-31 | 2023-04-07 | 中国科学院计算技术研究所 | Compression method for point cloud target detection model |
| CN111488985B (en) * | 2020-04-08 | 2023-11-14 | 华南理工大学 | Deep neural network model compression training method, device, equipment and medium |
-
2020
- 2020-09-23 CN CN202011007728.9A patent/CN112132278A/en active Pending
- 2020-10-29 WO PCT/CN2020/124813 patent/WO2021159748A1/en active Application Filing
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112990296A (en) * | 2021-03-10 | 2021-06-18 | 中科人工智能创新技术研究院(青岛)有限公司 | Image-text matching model compression and acceleration method and system based on orthogonal similarity distillation |
| WO2023113693A3 (en) * | 2021-12-17 | 2023-10-05 | Lemon Inc. | Optimal knowledge distillation scheme |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021159748A1 (en) | 2021-08-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112132278A (en) | Model compression method and device, computer equipment and storage medium | |
| CN109063742B (en) | Butterfly identification network construction method and device, computer equipment and storage medium | |
| JP7167306B2 (en) | Neural network model training method, apparatus, computer equipment and storage medium | |
| CN109241903B (en) | Sample data cleaning method, device, computer equipment and storage medium | |
| CN111191791B (en) | Picture classification method, device and equipment based on machine learning model | |
| WO2019228122A1 (en) | Training method for model, storage medium and computer device | |
| CN111126668B (en) | Spark operation time prediction method and device based on graph convolution network | |
| CN111667010A (en) | Sample evaluation method, device and equipment based on artificial intelligence and storage medium | |
| CN112183295A (en) | Pedestrian re-identification method and device, computer equipment and storage medium | |
| WO2022012668A1 (en) | Training set processing method and apparatus | |
| CN111898735A (en) | Distillation learning method, distillation learning device, computer equipment and storage medium | |
| CN113705685A (en) | Disease feature recognition model training method, disease feature recognition device and disease feature recognition equipment | |
| CN112308825A (en) | SqueezeNet-based crop leaf disease identification method | |
| CN114168318A (en) | Training method of storage release model, storage release method and equipment | |
| CN116403019A (en) | Remote sensing image quantum identification method and device, storage medium and electronic device | |
| CN113283388B (en) | Training method, device, equipment and storage medium of living body face detection model | |
| US20230342626A1 (en) | Model processing method and related apparatus | |
| CN117056721A (en) | Model parameter adjusting method and device, model prediction method, device and medium | |
| CN108256575A (en) | Image-recognizing method, device, computer equipment and storage medium | |
| CN113743448B (en) | Model training data acquisition method, model training method and device | |
| CN114372539B (en) | Machine learning framework-based classification method and related equipment | |
| CN114445692B (en) | Image recognition model construction method and device, computer equipment and storage medium | |
| CN113139590B (en) | Dimension reduction method and device for time series data, computer equipment and storage medium | |
| CN113139447B (en) | Feature analysis method, device, computer equipment and storage medium | |
| US20230059976A1 (en) | Deep neural network (dnn) accelerator facilitating quantized inference |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40041439 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201225 |
|
| RJ01 | Rejection of invention patent application after publication |