CN111612139B - Neural network model training method, system, equipment and computer medium - Google Patents
Neural network model training method, system, equipment and computer medium Download PDFInfo
- Publication number
- CN111612139B CN111612139B CN201910132624.1A CN201910132624A CN111612139B CN 111612139 B CN111612139 B CN 111612139B CN 201910132624 A CN201910132624 A CN 201910132624A CN 111612139 B CN111612139 B CN 111612139B
- Authority
- CN
- China
- Prior art keywords
- value
- neural network
- position value
- network model
- particle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003062 neural network model Methods 0.000 title claims abstract description 115
- 238000012549 training Methods 0.000 title claims abstract description 98
- 238000000034 method Methods 0.000 title claims abstract description 80
- 239000002245 particle Substances 0.000 claims abstract description 121
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 111
- 230000008569 process Effects 0.000 claims abstract description 32
- 238000013528 artificial neural network Methods 0.000 claims abstract 3
- 238000004590 computer program Methods 0.000 claims description 10
- 230000001133 acceleration Effects 0.000 claims description 8
- 244000300477 Gardenia carinata Species 0.000 claims description 4
- BSJGASKRWFKGMV-UHFFFAOYSA-L ammonia dichloroplatinum(2+) Chemical compound N.N.Cl[Pt+2]Cl BSJGASKRWFKGMV-UHFFFAOYSA-L 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 3
- 230000000875 corresponding effect Effects 0.000 description 95
- 238000013527 convolutional neural network Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000010845 search algorithm Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a neural network model training method, a system, equipment and a computer medium, wherein the method comprises the following steps: acquiring a weight value of a target neural network model in the current iterative training process; determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm; determining a second position value corresponding to the particle by adopting a PSO algorithm through the first speed value; determining a second speed value corresponding to the particle by adopting a GSA algorithm through the first position value and the second position value; determining a third position value corresponding to the particle by adopting a GSA algorithm through the second velocity value; the weight value is updated according to the first position value and the third position value. The neural network model training method, the system, the equipment and the computer readable storage medium have the advantages of high information sharing capability and high global searching capability, can improve the convergence rate of the neural network model, and can improve the recognition accuracy of the neural network.
Description
Technical Field
The present disclosure relates to the field of neural network model technologies, and in particular, to a neural network model training method, system, device, and computer medium.
Background
With the application of the neural network model, in order to make the neural network model more accurate, the training process of the neural network model is required to be more accurate.
The existing neural network model training method is as follows: in the training process of the neural network model, a filter group with poor effect and a filter group with good effect are selected, partial coefficients in the poor filter group are replaced by partial coefficients in the good filter group, so that parameter updating among convolution layer filters is realized, the neural network model after modification is used for training, after a certain number of times of training, the error size generated by the network after modification and before modification is compared, whether the parameter updating of the filter which is realized before is effective is judged, selection is carried out from the two neural network models, the neural network model with good performance is reserved for subsequent training, and finally, the neural network model with excellent performance can be trained by continuously repeating the process.
However, in the existing neural network model training method, the accuracy and convergence speed of the neural network model are difficult to be improved mainly by artificially replacing part of coefficients in the better filter bank with part of coefficients in the worse filter bank.
In summary, how to improve the accuracy and convergence rate of the neural network model is a problem to be solved by those skilled in the art.
Disclosure of Invention
The purpose of the application is to provide a neural network model training method, which can solve the technical problem of how to improve the accuracy and convergence rate of the neural network model to a certain extent. The application also provides a neural network model training system, equipment and a computer readable storage medium.
In order to achieve the above object, the present application provides the following technical solutions:
a neural network model training method, comprising:
acquiring a weight value of a target neural network model in the current iterative training process;
determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm;
determining a second position value corresponding to the particle by adopting a PSO algorithm through the first speed value;
determining a second speed value corresponding to the particle by adopting the GSA algorithm through the first position value and the second position value;
determining a third position value corresponding to the particle by adopting the GSA algorithm through the second speed value;
and updating the weight value according to the first position value and the third position value.
Preferably, the updating the weight value according to the first position value and the third position value includes:
calculating a first fitness value of the first position value;
calculating a second fitness value of the third position value;
and judging whether the second fitness value is larger than or equal to the first fitness value, if so, updating the weight value based on the third position value, and if not, updating the weight value based on the first position value.
Preferably, the determining, by using the GSA algorithm, a first velocity value and a first position value corresponding to the particle includes:
calculating a first speed value and a first position value corresponding to the particles by adopting a first GSA operation formula;
the first GSA operational formula includes:
wherein,representing the first speed value; />Representing the first position value; alpha represents a falling coefficient; g 0 Representing an initial gravitational constant; t represents the number of iterations, t max Representing a maximum number of iterations; x is X i (t) represents the position of the i-th particle corresponding to the weight value; x is X j (t) represents the position of the jth particle corresponding to the weight value; />Representing the position of the jth particle corresponding to the weight value on the d-th dimensional space; />Representing the position of the ith particle corresponding to the weight value on the d-th dimensional space; m is M i (t) represents the inertial mass of the i-th particle corresponding to the weight value; m is M j (t) represents the inertial mass of the jth particle corresponding to the weight value; epsilon is a constant; rand of j Is in [0,1 ]]Random numbers between the ranges; m represents the number of output nodes; q represents the number of output samples; g i Representing the predicted output after training; sigma represents training sample target output; best (t) represents the maximum fitness value of the ith particle corresponding to the weight value; the word (t) represents the minimum fitness value of the ith particle corresponding to the weight value; rand of i The expression ranges are [0,1 ]]A random number; and N represents the particle size corresponding to the weight value.
Preferably, the determining, by using the first velocity value and the PSO algorithm, a second position value corresponding to the particle includes:
determining a home position value corresponding to the particle by adopting the PSO algorithm;
calculating a fourth position value corresponding to the particle by adopting a first PSO operation formula through the first speed value and the home position value;
calculating the fitness value of the fourth position value and the home position value, and taking the position value corresponding to the fitness value with a large value as the second position value;
the first PSO operation formula includes:
wherein g' i,best Representing the fourth position value;representing the home position value; psize represents the total number of particles corresponding to the weight value; rand represents [0,1 ]]Random numbers in between.
Preferably, the determining, by using the GSA algorithm, the second velocity value corresponding to the particle according to the first position value and the second position value includes:
calculating a second speed value corresponding to the particle by adopting a second GSA operation formula through the first position value and the second position value;
the second GSA operational formula includes:
V i (t+1)＝w`V i (t)+c 1 randa i (t)+c 2 rand(g i,best -x i (t))+c 3 rand(g best -x i (t))；
wherein V is i Representing the second velocity value; w' represents inertial weight; g best A position value representing a global extremum calculated using the PSO algorithm; rand represents [0,1 ]]Random numbers in between; c 1 、c 2 、c 3 Representing an acceleration factor; g i,best Representing the second position value.
Preferably, after updating the weight value according to the first position value and the third position value, the method further includes:
calculating a real-time error value of the updated weight value;
and judging whether the real-time error value is smaller than a preset error value, if so, ending, and if not, returning to the step of executing the weight value of the obtained target neural network model in the current iterative training process.
Preferably, before the obtaining the weight value of the target neural network model in the current iterative training process, the method further includes:
initializing parameters of the GSA algorithm and the PSO algorithm.
A neural network model training system, comprising:
the first acquisition module is used for acquiring a weight value of the target neural network model in the current iterative training process;
the first determining module is used for determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm;
the second determining module is used for determining a second position value corresponding to the particle by adopting a PSO algorithm through the first speed value;
a third determining module, configured to determine, by using the GSA algorithm, a second velocity value corresponding to the particle according to the first position value and the second position value;
a fourth determining module, configured to determine, by using the second velocity value and using the GSA algorithm, a third position value corresponding to the particle;
and the first updating module is used for updating the weight value according to the first position value and the third position value.
A neural network model training device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the neural network model training method as described in any one of the above when executing the computer program.
A computer readable storage medium having stored therein a computer program which when executed by a processor performs the steps of the neural network model training method as described in any of the above.
According to the neural network model training method, the weight value of the target neural network model in the current iterative training process is obtained; determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm; determining a second position value corresponding to the particle by adopting a PSO algorithm through the first speed value; determining a second speed value corresponding to the particle by adopting a GSA algorithm through the first position value and the second position value; determining a third position value corresponding to the particle by adopting a GSA algorithm through the second velocity value; the weight value is updated according to the first position value and the third position value. According to the neural network model training method, the GSA algorithm and the PSO algorithm are combined to update the weight value of the neural network model, and the neural network model training method has the advantages of being high in information sharing capability and global searching capability, and can improve the accuracy and convergence rate of the neural network model. The neural network model training system, the neural network model training equipment and the computer readable storage medium also solve the corresponding technical problems.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.
Fig. 1 is a flowchart of a neural network model training method provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of error rate of a conventional CNN algorithm and the neural network model training method provided herein;
fig. 3 is a schematic structural diagram of a neural network model training system according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a neural network model training device according to an embodiment of the present application;
fig. 5 is another schematic structural diagram of a neural network model training device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
Referring to fig. 1, fig. 1 is a flowchart of a neural network model training method according to an embodiment of the present application.
The neural network model training method provided by the embodiment of the application can comprise the following steps:
step S101: and acquiring a weight value of the target neural network model in the current iterative training process.
In practical application, the weight value of the target neural network model in the current iterative training process can be acquired first, and the effect of the target neural network model can be determined according to practical requirements, for example, the target neural network model can be a license plate recognition neural network model; the type of the target neural network model can also be determined according to actual needs, for example, the target neural network model can be CNN (Convolutional Neural Network ) and the like, and specifically, the target neural network model can be a LeNet-5 network model and the like. In the process of determining the weight value, the neural network model needs to be subjected to iterative training for a plurality of times, and the current iterative training process refers to the iterative process currently executed by the neural network model.
Step S102: and determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm.
In practical application, after the weight value is obtained, the particle corresponding to the weight value in the GSA (Gravitational Search Algorithm, gravity search algorithm) algorithm can be determined, the GSA algorithm is adopted to determine the first speed value and the first position value corresponding to the particle, namely, the weight value is converted into the particle of the GSA algorithm, and the first speed value and the first position value corresponding to the particle are calculated through the GSA algorithm. The GSA algorithm related to the application refers to a novel global intelligent search algorithm which is inspired by the law of universal gravitation.
In practical application, the process of determining the first speed value and the first position value corresponding to the particle by adopting the GSA algorithm may specifically be:
calculating a first speed value and a first position value corresponding to the particles by adopting a first GSA operation formula;
the first GSA operation formula comprises:
wherein,representing a first speed value; />Representing a first position value; alpha represents a falling coefficient; g 0 Representing an initial gravitational constant; t represents the number of iterations, t max Representing a maximum number of iterations; x is X i (t) represents the position of the ith particle corresponding to the weight value; x is X j (t) represents the position of the jth particle corresponding to the weight value; />Representing the position of the jth particle corresponding to the weight value on the d-th dimensional space; />Representing the position of the ith particle corresponding to the weight value on the d-th dimensional space; m is M i (t) represents the inertial mass of the i-th particle corresponding to the weight value; m is M j (t) representing the rightsInertial mass of the j-th particle corresponding to the heavy value; epsilon is a constant; rand of j Is in [0,1 ]]Random numbers between the ranges; m represents the number of output nodes; q represents the number of output samples; g i Representing the predicted output after training; sigma represents training sample target output; best (t) represents the maximum fitness value of the ith particle corresponding to the weight value; the word (t) represents the minimum fitness value of the ith particle corresponding to the weight value; rand of i The expression ranges are [0,1 ]]Random numbers. Specifically, G (t) represents an gravitational constant; r is R ij (t) represents the Euclidean distance; f (F) i d (t) represents the attractive force resultant force of the ith particle corresponding to the weight value under other particles on the d-th dimensional space in the t-th iteration; f (f) i (t) represents the fitness function of the ith particle corresponding to the weight value in the t-th iteration; m is M i (t) represents the mass of the ith particle corresponding to the weight value; />Representing acceleration of the ith particle corresponding to the weight value in the d-th dimensional space; n represents the particle size corresponding to the weight value. It will be appreciated that when the GSA algorithm is used to determine the first velocity value and the first position value corresponding to the weight value, the weight value needs to be converted into the value of the corresponding particle in the GSA algorithm.
In a specific application scenario, before the weight value of the target neural network model in the current iterative training process is obtained, parameters of a GSA algorithm and a PSO algorithm can be initialized. For example, when initializing parameters of PSO algorithm and GSA algorithm, alpha value can be set to 20, G 0 The value is 1, the mass and the acceleration are 0, the particle size N is 25, the iteration number is 1000, the acceleration coefficient is c 1 、c 2 、c 3 All are 1.49, the initial mass and the acceleration are 0, the inertia weight w' is linearly increased from 0.4 to 0.9, and the initial speed is 0,1]With random values of the interval in between.
Step S103: and determining a second position value corresponding to the particle by using a PSO algorithm through the first speed value.
In practical application, after the first speed value is calculated, the second position value corresponding to the particle can be determined by adopting a PSO (Particle Swarm Optimization ) algorithm, namely, the first speed value calculated by the GSA algorithm is applied to the PSO algorithm to calculate the second position value corresponding to the particle, and it is easy to understand that the GSA algorithm and the PSO algorithm are combined together through the first speed value to be commonly used for updating the weight value of the neural network model, namely, the second position value is the position value obtained by combining the GSA algorithm and the PSO algorithm. The PSO algorithm referred to in the present application refers to a biological evolutionary algorithm commonly proposed by Kennedy and Electrical Engineers Eberhart, the society psychologist.
In practical applications, the process of determining the second position value corresponding to the particle by using the PSO algorithm through the first velocity value may specifically be:
determining a home position value corresponding to the particle by adopting a PSO algorithm;
calculating a fourth position value corresponding to the particle by adopting a first PSO operation formula through the first speed value and the home position value;
calculating the fitness value of the fourth position value and the original position value, and taking the position value corresponding to the fitness value with a large value as a second position value, namely the position value of the local extremum in the PSO algorithm;
the first PSO operation formula includes:
wherein g' i,best Representing a fourth position value;representing a home value; psize represents the total number of particles corresponding to the weight value; rand represents [0,1 ]]Random numbers in between.
Specifically, in calculating the fitness value of the fourth position value and the home position value, the fitness value may be calculated by the formulaTo calculate a corresponding fitness value.
Step S104: and determining a second speed value corresponding to the particle by adopting a GSA algorithm through the first position value and the second position value.
In practical application, after the second position value is obtained, the second speed value corresponding to the particle can be determined by adopting the GSA algorithm through the first position value and the second position value, and the second speed value is the speed value corresponding to the optimal particle in the GSA algorithm.
In a specific application scenario, the process of determining the second speed value corresponding to the particle by using the GSA algorithm through the first position value and the second position value may specifically be:
calculating a second speed value corresponding to the particle by adopting a second GSA operation formula through the first position value and the second position value;
the second GSA operation formula comprises:
V i (t+1)＝w`V i (t)+c 1 randa i (t)+c 2 rand(g i,best -x i (t))+c 3 rand(g best -x i (t))；
wherein V is i Representing a second velocity value; w' represents inertial weight; g best A position value representing the global extremum calculated using the PSO algorithm; rand represents [0,1 ]]Random numbers in between; c 1 、c 2 、c 3 Representing an acceleration factor; g i,best Representing a second position value.
Step S105: and determining a third position value corresponding to the particle by adopting a GSA algorithm through the second velocity value.
In practical application, after the second speed value is obtained, a third position value corresponding to the particle can be determined by adopting a GSA algorithm through the second speed value, and the second speed value is the speed value corresponding to the optimal particle in the GSA algorithm, so that the third position value is the position value corresponding to the optimal particle in the GSA algorithm. The third position value is obtained by combining the GSA algorithm and the PSO algorithm.
In a specific application scenario, the process of determining the third position value corresponding to the particle by using the GSA algorithm through the second velocity value may specifically be:
by a second velocity valueUsing formula X i (t+1)＝X i (t)+V i (t+1) calculating a third position value corresponding to the particle, wherein X i Representing a third position value.
Step S106: the weight value is updated according to the first position value and the third position value.
In practical application, after the third position value is obtained, the position value with the large performance value can be selected from the first position value and the third position value to update the weight value.
In a specific application scenario, the process of updating the weight value according to the first position value and the third position value may specifically be:
calculating a first fitness value of the first position value, e.g. by the formulaCalculating a first fitness value of the first position value;
calculating a second fitness value of the third position value, e.g. by the formulaCalculating a second fitness value of the third position value;
and judging whether the second fitness value is larger than or equal to the first fitness value, if so, updating the weight value based on the third position value, and if not, updating the weight value based on the first position value. The principle of updating the weight value based on the third position value or the first position value is the same as that of the prior art, and the weight value is updated by the position value according to the mapping relation between the position value and the weight value.
In a specific application scenario, when the error value of the neural network model is smaller than the preset error value, training of the neural network model can be considered to be completed, and after updating the weight value according to the first position value and the third position value, the method may further include:
calculating a real-time error value of the updated weight value, e.g. by the formulaCalculating real-time errors of updated weight valuesThe difference, i.e., the fitness value;
and judging whether the real-time error value is smaller than a preset error value, if so, ending, and if not, returning to the step of acquiring the weight value of the target neural network model in the current iterative training process.
In a specific application scenario, the training process may also be ended when the number of iterations is equal to the maximum number of iterations.
According to the neural network model training method, the weight value of the target neural network model in the current iterative training process is obtained; determining particles corresponding to the weight values in a GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm; determining a second position value corresponding to the particle by adopting a PSO algorithm through the first speed value; determining a second speed value corresponding to the particle by adopting a GSA algorithm through the first position value and the second position value; determining a third position value corresponding to the particle by adopting a GSA algorithm through the second velocity value; the weight value is updated according to the first position value and the third position value. According to the neural network model training method, the GSA algorithm and the PSO algorithm are combined to update the weight value of the neural network model, and the neural network model training method has the advantages of being high in information sharing capability and global searching capability, can improve the convergence rate of the neural network model, and meanwhile improves the recognition accuracy of the neural network model.
The performance of a neural network model training method provided by the application is verified by combining a specific embodiment. In the application, the LeNet-5 of CNN is taken as a target neural network model for verification. Firstly, sample data can be prepared, for example, a sample set of an MNIST handwriting digital library is used as sample data, the data volume of the handwriting digital library contains 60000 training samples, 10000 tens of thousands of test samples, each sample is a gray level diagram of 28 x 28 pixels, 5000 training samples can be extracted as the training set in the experiment, and 1000 test samples are used as the test set to be used as sample data; initializing LeNet-5 network parameters, wherein a LeNet-5 network model comprises 1 LBP (local=value mode) pretreatment layer, and then alternately connecting 2 convolution layers and 2 downsampling layers, converting all two-dimensional characteristic images into one-dimensional vectors through a full connection layer, outputting classification results through an output layer, initializing network connection weights among neurons when the network parameters are initialized, setting the network connection weights as uniform distribution with the average value of 0, setting an error threshold value as 1E-5, setting the maximum iteration number as 1000, enabling an activation function as a sigmoid function, and enabling the learning rate to be 0.01; then initializing parameters of a PSO algorithm and a GSA algorithm; finally training the LeNet-5 network model according to the neural network model training method. Referring to fig. 2 and table 1, fig. 2 is a schematic diagram of error rate of a conventional CNN algorithm and a neural network model training method provided in the present application; table 1 is an accuracy table of a conventional CNN algorithm and the neural network model training method provided in the present application; the PSO-GSA-CNN is the neural network model training method provided by the application. As can be seen from fig. 2 and table 1, the neural network model training method provided by the application has the advantages of low error rate, high accuracy and better performance.
Table 1 accuracy Table of conventional CNN algorithm and neural network model training method provided herein
Referring to fig. 3, fig. 3 is a schematic structural diagram of a neural network model training system according to an embodiment of the present application.
The neural network model training system provided in the embodiment of the application may include:
the first obtaining module 101 is configured to obtain a weight value of the target neural network model in a current iterative training process;
a first determining module 102, configured to determine a particle corresponding to the weight value in the GSA algorithm, and determine a first velocity value and a first position value corresponding to the particle by adopting the GSA algorithm;
a second determining module 103, configured to determine, by using the first speed value and using a PSO algorithm, a second position value corresponding to the particle;
a third determining module 104, configured to determine, according to the first position value and the second position value, a second velocity value corresponding to the particle by using a GSA algorithm;
a fourth determining module 105, configured to determine, according to the second velocity value, a third position value corresponding to the particle by using a GSA algorithm;
the first updating module 106 is configured to update the weight value according to the first location value and the third location value.
The embodiment of the application provides a neural network model training system, and a first update module may include:
a first calculation unit for calculating a first fitness value of the first position value;
a second calculation unit for calculating a second fitness value of the third position value;
the first judging unit is used for judging whether the second fitness value is larger than or equal to the first fitness value, if so, updating the weight value based on the third position value, and if not, updating the weight value based on the first position value.
The embodiment of the application provides a neural network model training system, and a first determination model may include:
the first determining unit is used for calculating a first speed value and a first position value corresponding to the particles by adopting a first GSA operation formula;
the first GSA operation formula comprises:
wherein,representing a first speed value; />Representing a first position value; alpha represents a falling coefficient; g 0 Representing an initial gravitational constant; t represents the number of iterations, t max Representing a maximum number of iterations; x is X i (t) represents the position of the i-th weight value; x is X j (t) represents the position of the jth weight value; />Representing the position of the jth weight value in the d-th dimensional space; />Representing the position of the ith weight value on the d-th dimensional space; m is M i (t) the inertial mass of the ith weight value; m is M j (t) inertial mass representing the jth weight value; epsilon is a constant; rand of j Is in [0,1 ]]Random numbers between the ranges; m represents the number of output nodes; q represents the number of output samples; g i Representing the predicted output after training; sigma represents training sample target output; best (t) represents the maximum fitness value of the ith weight value; the word (t) represents the minimum fitness value of the ith weight value; rand of i The expression ranges are [0,1 ]]A random number; n represents the particle size corresponding to the weight value.
The neural network model training system provided in the embodiment of the present application, the second determining module may include:
the second determining unit is used for determining a home position value corresponding to the particle by adopting a PSO algorithm;
the third calculation unit is used for calculating a fourth position value corresponding to the particle by adopting a first PSO operation formula through the first speed value and the home position value;
a fourth calculation unit, configured to calculate an fitness value of the fourth position value and the original position value, and use a position value corresponding to the fitness value with a large value as a second position value;
the first PSO operation formula includes:
wherein g' i,best Representing a fourth position value;representing a home value; psize represents the total number of weight values; rand represents [0,1 ]]Random numbers in between.
The neural network model training system provided in the embodiment of the present application, the third determining module may include:
a fifth calculating unit, configured to calculate, according to the first position value and the second position value, a second velocity value corresponding to the particle by using a second GSA operation formula;
the second GSA operational formula includes:
V i (t+1)＝w`V i (t)+c 1 randa i (t)+c 2 rand(g i,best -x i (t))+c 3 rand(g best -x i (t))；
wherein V is i Representing the second velocity value; w' represents inertial weight; g best A position value representing a global extremum calculated using the PSO algorithm; rand represents [0,1 ]]Random numbers in between; c 1 、c 2 、c 3 Representing an acceleration factor; g i,best Representing the second position value.
The neural network model training system provided in the embodiment of the application may further include:
the first calculating module is used for calculating a real-time error value of the updated weight value after the weight value is updated by the first updating module according to the first position value and the third position value;
the first judging module is used for judging whether the real-time error value is smaller than a preset error value, if yes, ending, otherwise prompting the first acquiring module to execute the step of acquiring the weight value of the target neural network model in the current iterative training process.
The neural network model training system provided in the embodiment of the application may further include:
the first initializing module is used for initializing parameters of the GSA algorithm and the PSO algorithm before the first acquiring module acquires the weight value of the target neural network model in the current iterative training process.
The application also provides a neural network model training device and a computer readable storage medium, which have the corresponding effects of the neural network model training method provided by the embodiment of the application. Referring to fig. 4, fig. 4 is a schematic structural diagram of a neural network model training device according to an embodiment of the present application.
The neural network model training device provided in the embodiment of the application may include:
a memory 201 for storing a computer program;
a processor 202 for implementing the steps of the neural network model training method as described in any of the embodiments above when executing a computer program.
Referring to fig. 5, another neural network model training apparatus provided in an embodiment of the present application may further include: an input port 203 connected to the processor 202 for transmitting an externally input command to the processor 202; a display unit 204 connected to the processor 202, for displaying the processing result of the processor 202 to the outside; and the communication module 205 is connected with the processor 202 and is used for realizing communication between the neural network model training device and the outside. The display unit 204 may be a display panel, a laser scanning display, or the like; communication means employed by the communication module 205 include, but are not limited to, mobile high definition link technology (HML), universal Serial Bus (USB), high Definition Multimedia Interface (HDMI), wireless connection: wireless fidelity (WiFi), bluetooth communication, bluetooth low energy communication, ieee802.11s based communication.
The embodiment of the application provides a computer readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the neural network model training method described in any embodiment above are implemented.
The computer readable storage medium referred to in this application includes Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The description of the related parts in the neural network model training system, the device and the computer readable storage medium provided in the embodiments of the present application refers to the detailed description of the corresponding parts in the neural network model training method provided in the embodiments of the present application, and will not be repeated here. In addition, the parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of the corresponding technical solutions in the prior art, are not described in detail, so that redundant descriptions are avoided.
It is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A neural network model training method, comprising:
preparing sample data, wherein the sample data is a two-dimensional image sample; initializing target neural network parameters, and initializing parameters of a PSO algorithm and a GSA algorithm;
acquiring a weight value of the target neural network model in the current iterative training process, wherein the target neural network model comprises a license plate recognition neural network model;
determining particles corresponding to the weight values in the GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm;
determining a second position value corresponding to the particle by adopting the PSO algorithm through the first speed value;
determining a second speed value corresponding to the particle by adopting the GSA algorithm through the first position value and the second position value;
determining a third position value corresponding to the particle by adopting the GSA algorithm through the second speed value;
updating the weight value according to the first position value and the third position value; all the two-dimensional characteristic images are converted into one-dimensional vectors through the full-connection layer of the target neural network model, and the classification result is output through the output layer of the target neural network model.
2. The method of claim 1, wherein the updating the weight value based on the first location value and the third location value comprises:
calculating a first fitness value of the first position value;
calculating a second fitness value of the third position value;
and judging whether the second fitness value is larger than or equal to the first fitness value, if so, updating the weight value based on the third position value, and if not, updating the weight value based on the first position value.
3. The method of claim 1, wherein determining a first velocity value and a first position value for the particle using the GSA algorithm comprises:
calculating a first speed value and a first position value corresponding to the particles by adopting a first GSA operation formula;
the first GSA operational formula includes:
R ij (t)＝||X i (t)X j (t)|| 2 ；
wherein,representing the first speed value; />Representing the first position value; alpha represents a falling coefficient; g 0 Representing an initial gravitational constant; t represents the number of iterations, t max Representing a maximum number of iterations; x is X i (t) represents the position of the i-th particle corresponding to the weight value; x is X j (t) represents the position of the jth particle corresponding to the weight value; />Representing the position of the jth particle corresponding to the weight value on the d-th dimensional space; />Representing the position of the ith particle corresponding to the weight value on the d-th dimensional space; m is M i (t) represents the inertial mass of the i-th particle corresponding to the weight value; m is M j (t) represents the inertial mass of the jth particle corresponding to the weight value; epsilon is a constant; rand of j Is in [0,1 ]]Random numbers between the ranges; m represents the number of output nodes; q represents the number of output samples; g i Representing the predicted output after training; sigma represents training sample target output; best (t) represents the maximum fitness value of the ith weight value; the word (t) represents the minimum fitness value of the ith particle corresponding to the weight value; rand of i The expression ranges are [0,1 ]]A random number; and N represents the particle size corresponding to the weight value.
4. A method according to claim 3, wherein said determining a second position value for said particle by said first velocity value using said PSO algorithm comprises:
determining a home position value corresponding to the particle by adopting the PSO algorithm;
calculating a fourth position value corresponding to the particle by adopting a first PSO operation formula through the first speed value and the home position value;
calculating the fitness value of the fourth position value and the home position value, and taking the position value corresponding to the fitness value with a large value as the second position value;
the first PSO operation formula includes:
wherein g' i,best Representing the fourth positionA value;representing the home position value; psize represents the total number of particles corresponding to the weight value; rand represents [0,1 ]]Random numbers in between.
5. The method of claim 4, wherein the determining a second velocity value corresponding to the particle using the GSA algorithm from the first location value, the second location value, comprises:
calculating a second speed value corresponding to the particle by adopting a second GSA operation formula through the first position value and the second position value;
the second GSA operational formula includes:
V i (t+1)＝w`V i (t)+c 1 randa i (t)+c 2 rand(g i,best -X i (t))+c 3 rand(g best -X i (t))；
wherein V is i Representing the second velocity value; w' represents inertial weight; g best A position value representing a global extremum calculated using the PSO algorithm; rand represents [0,1 ]]Random numbers in between; c 1 、c 2 、c 3 Representing an acceleration factor; g i,best Representing the second position value.
6. The method according to any one of claims 1-5, wherein after updating the weight value according to the first position value and the third position value, further comprising:
calculating a real-time error value of the updated weight value;
and judging whether the real-time error value is smaller than a preset error value, if so, ending, and if not, returning to the step of executing the weight value of the obtained target neural network model in the current iterative training process.
7. The method of claim 1, wherein the acquiring the weight value of the target neural network model prior to the current iterative training process further comprises:
initializing parameters of the GSA algorithm and the PSO algorithm.
8. A neural network model training system, comprising:
the first acquisition module is used for preparing sample data, wherein the sample data is a two-dimensional image sample; initializing target neural network parameters, and initializing parameters of a PSO algorithm and a GSA algorithm; acquiring a weight value of the target neural network model in the current iterative training process, wherein the target neural network model comprises a license plate recognition neural network model;
the first determining module is used for determining particles corresponding to the weight values in the GSA algorithm, and determining a first speed value and a first position value corresponding to the particles by adopting the GSA algorithm;
the second determining module is used for determining a second position value corresponding to the particle by adopting the PSO algorithm through the first speed value;
a third determining module, configured to determine, by using the GSA algorithm, a second velocity value corresponding to the particle according to the first position value and the second position value;
a fourth determining module, configured to determine, by using the second velocity value and using the GSA algorithm, a third position value corresponding to the particle;
a first updating module, configured to update the weight value according to the first position value and the third position value; all the two-dimensional characteristic images are converted into one-dimensional vectors through the full-connection layer of the target neural network model, and the classification result is output through the output layer of the target neural network model.
9. A neural network model training device, comprising:
a memory for storing a computer program;
processor for implementing the steps of the neural network model training method according to any one of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the steps of the neural network model training method according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910132624.1A CN111612139B (en) | 2019-02-22 | 2019-02-22 | Neural network model training method, system, equipment and computer medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910132624.1A CN111612139B (en) | 2019-02-22 | 2019-02-22 | Neural network model training method, system, equipment and computer medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111612139A CN111612139A (en) | 2020-09-01 |
| CN111612139B true CN111612139B (en) | 2024-04-09 |
Family
ID=72201899
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910132624.1A Active CN111612139B (en) | 2019-02-22 | 2019-02-22 | Neural network model training method, system, equipment and computer medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111612139B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102364501A (en) * | 2011-09-14 | 2012-02-29 | 哈尔滨工程大学 | Method for reproducing two-dimensional defect of petroleum pipeline PSO-BP (Particle Swarm Optimization-Back-Propagation) neural network |
| CN107506868A (en) * | 2017-08-31 | 2017-12-22 | 广东工业大学 | A kind of method and device of temporary electricity load prediction |
| CN108020759A (en) * | 2016-11-03 | 2018-05-11 | 云南电网有限责任公司普洱供电局 | A kind of XLPE cable wave of oscillation fault recognition method based on PSOGSA neutral nets |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8346712B2 (en) * | 2009-11-24 | 2013-01-01 | King Fahd University Of Petroleum And Minerals | Method for identifying hammerstein models |
-
2019
- 2019-02-22 CN CN201910132624.1A patent/CN111612139B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102364501A (en) * | 2011-09-14 | 2012-02-29 | 哈尔滨工程大学 | Method for reproducing two-dimensional defect of petroleum pipeline PSO-BP (Particle Swarm Optimization-Back-Propagation) neural network |
| CN108020759A (en) * | 2016-11-03 | 2018-05-11 | 云南电网有限责任公司普洱供电局 | A kind of XLPE cable wave of oscillation fault recognition method based on PSOGSA neutral nets |
| CN107506868A (en) * | 2017-08-31 | 2017-12-22 | 广东工业大学 | A kind of method and device of temporary electricity load prediction |
Non-Patent Citations (2)
| Title |
|---|
| Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm;SeyedAli Mirjalili et al;Applied Mathematics and Computation;第11125-11137页 * |
| 基于PSO和GSA的神经网络轴承故障诊断;郭文强，等;计算机仿真;第第35卷卷(第第3期期);第279-302页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111612139A (en) | 2020-09-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109948029B (en) | Neural network self-adaptive depth Hash image searching method | |
| CN109118564B (en) | Three-dimensional point cloud marking method and device based on fusion voxels | |
| CN107273936B (en) | GAN image processing method and system | |
| CN109522942B (en) | Image classification method and device, terminal equipment and storage medium | |
| US20160224903A1 (en) | Hyper-parameter selection for deep convolutional networks | |
| CN111627065A (en) | Visual positioning method and device and storage medium | |
| CN110728295B (en) | Semi-supervised landform classification model training and landform graph construction method | |
| CN110929080B (en) | Optical remote sensing image retrieval method based on attention and generation countermeasure network | |
| CN111414953B (en) | Point cloud classification method and device | |
| CN113570029A (en) | Method for obtaining neural network model, image processing method and device | |
| CN111382868A (en) | Neural network structure search method and neural network structure search device | |
| CN113807399A (en) | Neural network training method, neural network detection method and neural network detection device | |
| CN110245683B (en) | Residual error relation network construction method for less-sample target identification and application | |
| CN115071762B (en) | Pedestrian trajectory prediction method, model and storage medium under urban scene | |
| CN111428854A (en) | Structure searching method and structure searching device | |
| WO2021062050A1 (en) | Icospherical gauge convolutional neural network | |
| CN111310821A (en) | Multi-view feature fusion method, system, computer device and storage medium | |
| CN114565092A (en) | Neural network structure determining method and device | |
| CN111612139B (en) | Neural network model training method, system, equipment and computer medium | |
| CN110866866A (en) | Image color-matching processing method and device, electronic device and storage medium | |
| CN114972725B (en) | Model training method, readable medium and electronic device | |
| JP5969836B2 (en) | Model application apparatus, model application method, and program | |
| CN115115016A (en) | Method and device for training neural network | |
| Lu et al. | Action recognition based on adaptive region perception | |
| CN116030212B (en) | Picture construction method, equipment, vehicle and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |