CN108764455A - Parameter adjustment method, device and storage medium - Google Patents
Parameter adjustment method, device and storage medium Download PDFInfo
- Publication number
- CN108764455A CN108764455A CN201810473912.9A CN201810473912A CN108764455A CN 108764455 A CN108764455 A CN 108764455A CN 201810473912 A CN201810473912 A CN 201810473912A CN 108764455 A CN108764455 A CN 108764455A
- Authority
- CN
- China
- Prior art keywords
- value
- hyper parameter
- repetitive exercise
- hyper
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Feedback Control In General (AREA)
Abstract
The invention discloses a kind of parameter adjustment method, device and storage medium, the method includes：According to preset repetitive exercise frequency n, the repetitive exercise of convolutional neural networks model is carried out to the value of hyper parameter；Determine hyper parameter value of the value of the hyper parameter in each repetitive exercise；The value corresponding to maximum hyper parameter value in the hyper parameter value that the n times repetitive exercise is obtained is as the optimal value of the hyper parameter.The present invention effectively improves the speed of decision of optimal hyper parameter in convolutional neural networks model.
Description
Technical field
The present invention relates to nerual network technique fields, more particularly to a kind of parameter adjustment method, device and storage medium.
Background technology
Deep learning is originated from neural network, and core is feature learning, i.e., is more abstracted by combining low-level feature formation
High-level characteristic, to find the distribution characteristics of data.Convolutional neural networks model is a kind of multilayer neural network, convolutional Neural net
It is underlying parameter that network model, which has two sets of parameters, one kind, another kind of such as the weight and biasing of convolutional layer or full articulamentum, is super ginseng
Number, such as learning rate when network training, weight attenuation coefficient, Dropout ratios, need to set before model training.
The training process of convolutional neural networks model is exactly the process according to loss adjust automatically underlying parameter, if it is desired to
Suitable underlying parameter is quickly recalled, obtains higher performance accuracy rate, it is necessary to select suitable hyper parameter.The choosing of hyper parameter
Basic skills there are two types of selecting：It is artificial that ginseng and automation is adjusted to adjust ginseng.Automation adjusts ginseng adaptively to be adjusted to hyper parameter, reduces
Manual intervention to convolutional neural networks model training process, to reduce convolutional neural networks model training difficulty.
Automation tune ginseng algorithm common at present includes grid search and random search.The wherein algorithm time of grid search
Complexity can exponentially rise as hyper parameter increases, and may be only available for small-scale neural network and the less situation of hyper parameter；
The algorithm of random search needs successive ignition sampling to be possible to determine optimal hyper parameter, can be missed most when sampling number deficiency
Excellent hyper parameter combination, therefore, the speed of decision of optimal hyper parameter need to be improved in automatic parameter adjustment method common at present.
Invention content
In order to overcome drawbacks described above, the technical problem to be solved in the present invention is to provide a kind of parameter adjustment method, device and storages
Medium, to improve the speed of decision of the optimal value of hyper parameter in convolutional neural networks model.
In order to solve the above technical problems, a kind of parameter adjustment method in the embodiment of the present invention, including：
According to preset repetitive exercise frequency n, the repetitive exercise of convolutional neural networks model is carried out to the value of hyper parameter；
Determine hyper parameter value of the value of the hyper parameter in each repetitive exercise；
The value corresponding to maximum hyper parameter value in the hyper parameter value that the n times repetitive exercise is obtained is as institute
State the optimal value of hyper parameter.
In order to solve the above technical problems, a kind of tune in the embodiment of the present invention joins device, including memory and processor, institute
It states memory and is stored with computer program, the processor executes the computer program, to realize the step of method as described above
Suddenly.
In order to solve the above technical problems, a kind of computer readable storage medium in the embodiment of the present invention, is stored with calculating
Machine program, when the computer program is executed by least one processor, the step of to realize method as described above.
The present invention has the beneficial effect that：
The each embodiment of the present invention carries out convolutional neural networks according to preset repetitive exercise frequency n, to hyper parameter value
The repetitive exercise of model, and be worth by hyper parameter, determine hyper parameter valence of the hyper parameter value in each repetitive exercise
Value, and then maximum hyper parameter in the n times repetitive exercise is worth corresponding hyper parameter value as the optimal of the hyper parameter
Value, to effectively improve the speed of decision of optimal hyper parameter in convolutional neural networks model.
Description of the drawings
Fig. 1 is a kind of flow chart of parameter adjustment method of the embodiment of the present invention；
Fig. 2 is a kind of flow chart of optionally parameter adjustment method of the embodiment of the present invention；
Fig. 3 is the flow chart that Fig. 2 is described in detail；
Fig. 4 is a kind of structural schematic diagram for adjusting ginseng device in the embodiment of the present invention.
Specific implementation mode
In order to solve problems in the prior art, it the present invention provides a kind of parameter adjustment method, device and storage medium, ties below
Attached drawing and embodiment are closed, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only
To explain the present invention, the present invention is not limited.
In subsequent description, using for indicating that the suffix of such as " module ", " component " or " unit " of element is only
The explanation for being conducive to the present invention, itself does not have a specific meaning.Therefore, " module ", " component " or " unit " can mix
Ground uses.
Using for distinguishing element " first ", the prefixes such as " second " only for being conducive to the explanation of the present invention,
Itself is without specific meaning.
Embodiment one
The embodiment of the present invention provides a kind of parameter adjustment method, as shown in Figure 1, the method includes：
S101 carries out the value of hyper parameter the iteration of convolutional neural networks model according to preset repetitive exercise frequency n
Training；
S102 determines hyper parameter value of the value of the hyper parameter in each repetitive exercise；
S103, the maximum hyper parameter during the hyper parameter that the n times repetitive exercise is obtained is worth are worth corresponding value
Optimal value as the hyper parameter.
In detail, hyper parameter is very more in convolutional Neural neural network, but convolutional neural networks are unwise to part hyper parameter
Sense, whether these hyper parameters are that the influence that optimal value shows model performance is little, therefore the hyper parameter in the embodiment of the present invention
The hyper parameter being generally affected to model performance；Such as learning rate, momentum, L2 regularization coefficients, batch processing size, convolution
The initialization type etc. of weight filter in layer and full articulamentum.
The embodiment of the present invention carries out convolutional neural networks model according to preset repetitive exercise frequency n, to hyper parameter value
Repetitive exercise, and be worth by hyper parameter, determine hyper parameter value of the hyper parameter value in each repetitive exercise, into
And maximum hyper parameter in the n times repetitive exercise is worth corresponding hyper parameter value as the optimal value of the hyper parameter,
To effectively improve the speed of decision of optimal hyper parameter in convolutional neural networks model.
In embodiments of the present invention, optionally, described according to preset repetitive exercise frequency n, hyper parameter value is carried out
The repetitive exercise of convolutional neural networks model, including：Ith iteration is trained：
When the i is equal to 1, using the sampled value of first time stochastical sampling as the value of the hyper parameter, described in progress
The first time repetitive exercise of convolutional neural networks model；
When the i is more than 1 and is less than the n, by the sampled value of ith stochastical sampling or predetermined optimal experience
Value of the sampled value as the hyper parameter carries out the ith iteration training of the convolutional neural networks model.
In alternative embodiment of the present invention, optimal experience sampled value is introduced into the sampling process that ginseng is adjusted in hyper parameter automation, into
And the more efficient speed of decision for improving optimal hyper parameter in convolutional neural networks model.
Wherein, optionally, using predetermined optimal experience sampled value as the hyper parameter value, the convolution is carried out
Before the ith iteration training of neural network model, including：
The value corresponding to maximum hyper parameter value during the hyper parameter that preceding i-1 repetitive exercise obtains is worth is determined as
The optimal experience sampled value.
Alternative embodiment of the present invention effectively solves grid search by the determining optimal experience adopted value of hyper parameter value
Algorithms T-cbmplexity can increase the problem of exponentially rising with hyper parameter, and the algorithm needs for effectively solving random search are more
The problem of secondary iteration sampling is possible to determine optimal hyper parameter, optimal hyper parameter combination can be missed when sampling number deficiency.
In embodiments of the present invention, optionally, super ginseng of the determination hyper parameter value in each repetitive exercise
Number value, including：
For an arbitrary repetitive exercise, at the end of an arbitrary repetitive exercise, changed according to currently completed
The corresponding each hyper parameter value effect value of generation training determines that the value of the hyper parameter is instructed in the arbitrary an iteration
Corresponding hyper parameter value in white silk；If that is, an arbitrary repetitive exercise is pth time repetitive exercise, in pth
At the end of secondary repetitive exercise, according to the corresponding hyper parameter value effect value of each repetitive exercise in preceding p repetitive exercise, institute is determined
State the value of hyper parameter hyper parameter value corresponding in the pth time repetitive exercise；The p is less than or equal to n.
Hyper parameter value effect value is for evaluating hyper parameter difference value to model training effect in the embodiment of the present invention
Influence degree；Statistics of the hyper parameter value for measuring hyper parameter value effect value in multiple training is special in the embodiment of the present invention
Property, each optimal experience sampled value of hyper parameter is worth after determining repeatedly training using maximization hyper parameter.
Alternative embodiment of the present invention further increases the speed of decision of optimal hyper parameter in convolutional neural networks model, and
Hyper parameter is arranged by hyper parameter value effect value to be worth, can effectively improve the performance of model.
Wherein, optionally, according to the currently corresponding each hyper parameter value effect value of completed repetitive exercise, really
The hyper parameter that the value of the fixed hyper parameter is corresponding in an arbitrary repetitive exercise is worth, including：
The mean value of each hyper parameter value effect value is determined as the value of the hyper parameter described arbitrary primary
Corresponding hyper parameter value in repetitive exercise.
Alternative embodiment of the present invention indicates hyper parameter value by the mean value of hyper parameter value effect value, so as to obtain
The statistical nature of hyper parameter value effect value effectively avoids preserving the memory overhead that all effect values are brought.Hyper parameter value is used
In evaluation hyper parameter difference value to the influence degree of model training effect, hyper parameter value is higher, then shows the hyper parameter
Value is more conducive to model performance.
In embodiments of the present invention, optionally, currently completed repetitive exercise is corresponding each super for the basis
Parameter value effect value determines the value of hyper parameter hyper parameter value corresponding in an arbitrary repetitive exercise
Before, including：
After each repetitive exercise in the current completed repetitive exercise, institute in each repetitive exercise is determined
State the penalty values of convolutional neural networks model；
According to the penalty values, the hyper parameter value effect value of each repetitive exercise is determined.
Wherein, optionally, according to the penalty values, the corresponding hyper parameter value effect of each repetitive exercise is determined
Value, including：
Judge whether the penalty values of convolutional neural networks model described in each repetitive exercise are normal；
If normal, the corresponding trained accuracy rate of each repetitive exercise is rounded up, obtains each iteration instruction
Practice the corresponding hyper parameter value effect value；
The corresponding hyper parameter value effect value of each repetitive exercise is assigned a value of preset value if abnormal,.
The embodiment of the present invention can determine the optimal value of 1 hyper parameter, can also determine at least two hyper parameter simultaneously
Optimal value, below with j-th of hyper parameter θjFor, the embodiment of the present invention is illustrated, wherein j is the optimal value of determination
Hyper parameter in any one.For example, being defined in the training of ith network model, used hyper parameter value effect value
For：
Wherein, θjIndicate j-th of hyper parameter, yijThe value for being j-th of hyper parameter when ith is trained (or be known as adopting
Sample value), x is that current iteration training corresponding trained accuracy rate in model evaluation is x%, and ceil () expressions take x upwards
Integer.Above formula indicates, if training normal termination (i.e. the loss of repetitive exercise is normal), is used in this training process every
A hyper parameter value effect value is ceil (x), is remained unchanged if there is penalty values, is more than 80 or positive and negative infinite feelings occur
Condition is then assigned a value of preset value, such as -50.Succinct in order to describe, hyper parameter value effect value can also claim in the embodiment of the present invention
Be effect value；Repetitive exercise is referred to as training in the embodiment of the present invention.
Defining hyper parameter value is：The hyper parameter value effect value that some value of hyper parameter is obtained at i times in training
The mean value hyper parameter value corresponding in ith iteration training as the value of hyper parameter, such as following formula：
Wherein, E [] indicates to average to the hyper parameter value effect value obtained in i training, Qij(y) ith instruction is indicated
The hyper parameter that the value of j-th of hyper parameter is y during white silk is worth, vij(yij) it is j-th of hyper parameter in ith training process
It is (y in valueij) when the effect value that obtains.In specific calculating process, i belongs to variable in above formula, such as i=m, m tables
Show the current frequency of training completed, wherein repetitive exercise can all generate a hyper parameter value effect value each time.That is,
It is v completing the hyper parameter value effect value corresponding to the 1st training j-th of hyper parameter1j(y1j), complete the 2nd training
Hyper parameter value effect value corresponding to j-th of hyper parameter is v2j(y2j), and so on, complete the m times training jth
Hyper parameter value effect value corresponding to a hyper parameter is vmj(ymj), above formula Q at this timeij(yij)=(v1j(y1j)+v2j(y2j)
+......+vmj(ymj))/m, as j-th of hyper parameter hyper parameter value corresponding in being trained in ith iteration.
In embodiments of the present invention, optionally, described by the sampled value of ith stochastical sampling or predetermined optimal warp
Sampled value is tested as the hyper parameter value, before the ith iteration training for carrying out the convolutional neural networks model, including：
Probability is chosen according to preset first, using the sampled value of the ith stochastical sampling as the hyper parameter value；
Probability is chosen according to preset second, using the optimal experience sampled value as the hyper parameter value.
For example, the first selection probability can be probability ρ, the second selection probability can be probability (1- ρ).
Stochastical sampling and optimal experience sampled value are used in the embodiment of the present invention simultaneously, for each hyper parameter, with general
Rate ρ chooses stochastical sampling for exploring new hyper parameter value, chooses optimal experience sampled value with probability (1- ρ) and is used for from history
Optimal empirical value is extracted in priori, reduces the iterative process of hyper parameter sampling-model training-model evaluation, to quickly
Determine the optimum combination of convolutional neural networks model hyper parameter.
For example, for adjusting multiple hyper parameters simultaneously, in stochastical sampling and optimal experience sampling process, the first round is removed
Repetitive exercise using outside stochastical sampling, train while carrying out hyper parameter using stochastical sampling and the sampling of optimal empirical value by successive iterations
Sampling, for each hyper parameter, with probability ρ in hyper parameter valued space stochastical sampling, if not carrying out stochastical sampling,
Optimal experience sampling is then carried out, i.e., then chooses current hyper parameter value Qij(yij) highest hyper parameter value y is as current iteration
Training sampled value.
During model training and model evaluation, hyper parameter sampling is carried out from hyper parameter valued space, uses the group
Hyper parameter combination carries out convolutional neural networks model training and model evaluation, and is updated after model training and model evaluation
Hyper parameter is worth Qij(yij).If exceptional value occurs in loss during model training, i.e. penalty values remain unchanged, are more than 80
Or there is positive and negative infinite situation, then it is assumed that failure to train, direct deconditioning, update hyper parameter are worth Qij(yij), and start
Hyper parameter sampling next time and training.
During determining optimal hyper parameter, when the sampling number of all hyper parameters all reaches default sampling number, knot
Shu Xunhuan searches for all hyper parameter values sampled, selects Q in each hyper parameterij(yij) highest yijValue is as optimal super
Parameter carries out model training using optimal hyper parameter, and model verifies performance evaluation result of the accuracy rate as the model.
Embodiment two
The embodiment of the present invention provides a kind of optionally parameter adjustment method, as shown in Fig. 2, method includes real in the embodiment of the present invention
Example one is applied, realizes that the fast search of optimal hyper parameter in convolutional neural networks model, main flow include Initialize installation, update
Hyper parameter value, hyper parameter sampling, model training and model evaluation, the determination of optimal hyper parameter etc.；In detail, as shown in figure 3, originally
Method includes in inventive embodiments：
S301, Initialize installation.Configuration carries out the value range that the hyper parameter of ginseng, each hyper parameter are adjusted in automation,
Hyper parameter valued space is constituted, the sampling number of each hyper parameter is set.
For example, setting carries out the hyper parameter and its codomain, each hyper parameter sampling number that ginseng is adjusted in automation.Convolution god
It is very more through hyper parameter in neural network, but convolutional neural networks are insensitive to part hyper parameter, and whether these hyper parameters are most
The influence that the figure of merit shows model performance is little, therefore automation provided in an embodiment of the present invention adjusts ginseng scheme that configuration is supported to need
Carry out the hyper parameter that ginseng is adjusted in automation.Recommend carry out automation adjust ginseng hyper parameter have learning rate, momentum, L2 regularization coefficients,
The initialization type etc. of weight filter in batch processing size, convolutional layer and full articulamentum.
The value of hyper parameter can be to be set as successive value or centrifugal pump.Part hyper parameter may be only configured to centrifugal pump, such as criticize
Handle size, the initialization type etc. of weight filter.The hyper parameters such as learning rate, momentum, L2 regularization coefficients can be both arranged
For discrete enumerated value, continuous value range can also be provided.
Need the sampling number (i.e. repetitive exercise number) that each hyper parameter is set as automation in the embodiment of the present invention
Adjust the end condition of ginseng.The importance of different hyper parameters is, the hyper parameter being affected to convolutional neural networks can be with
Larger sampling number is set.Once the sampling number of all hyper parameters is finished, then automates tune ginseng and terminate.
Hyper parameter is arranged to be worth：
It is defined in the training of ith network model, all hyper parameter value effect values used are：
Wherein, θjIndicate j-th of hyper parameter, yijFor value of j-th of hyper parameter when ith is trained, x trains for this
Accuracy rate (penalty values) in model evaluation is x%, and ceil () indicates to round up number to x.Above formula indicates, if training
Normal termination, the then each hyper parameter value effect value used in this training process are ceil (x), are protected if there is penalty values
Hold it is constant, be more than 80 or positive and negative infinite situation occur, then be assigned a value of -50.
Define the mean value for the effect value that hyper parameter value is obtained by some value of hyper parameter in multiple model training, example
Such as：
Wherein, Qij(yij) indicate that the value of j-th of hyper parameter in ith training process is yijHyper parameter value, vij
(yij) it is that j-th of hyper parameter in value is y in ith training processijWhen the effect value that obtains.
Hyper parameter is worth the influence degree for evaluating hyper parameter difference value to model training effect, and hyper parameter value is got over
Height then shows that the hyper parameter value is more conducive to model performance；Indicate that hyper parameter value can obtain with effect value mean value simultaneously
The statistical nature of effect value, and avoid preserving the memory overhead that all effect values are brought.
S302, stochastical sampling and the sampling of optimal experience.
In addition to first round iteration is using stochastical sampling, successive iterations are carried out using random search and the sampling of optimal experience simultaneously
Hyper parameter sample, for each hyper parameter, with probability ρ in hyper parameter valued space stochastical sampling, if do not carry out with
Machine samples, then carries out optimal experience sampling, i.e., then chooses current hyper parameter value Qij(yij) highest value yijIt is adopted as this
Sample value.
In embodiments of the present invention, optionally, the sampled value of the hyper parameter described in the i+1 time repetitive exercise belongs to random
When sampling, after i+1 time repetitive exercise, by iterations plus 1；
When the sampled value of the hyper parameter described in the i+1 time repetitive exercise is the optimal empirical value, in i+1 time iteration
After training, the iterations are constant.
For example, if some hyper parameter has carried out stochastical sampling, the sampling number of corresponding hyper parameter adds 1, if carried out
Optimal experience sampling, then sampling number is constant.Further, if hyper parameter θjSampling number have reached it is preset sampling time
It counts, and does not reach default sampling number there are hyper parameter, then θjIt is no longer sampled, it is highest directly to choose hyper parameter value
Value is used for subsequent iteration.
S303, model training.
For example, carrying out hyper parameter sampling from hyper parameter valued space and optimal empirical value, combined using this group of hyper parameter
Carry out convolutional neural networks model training.
S304, model evaluation judge whether the loss of this time training is abnormal；If abnormal, S306 is executed, if just
Often, S305 is executed.
Hyper parameter is updated after model training and model evaluation is worth Qij(yij).If during model training
There is exceptional value in loss, i.e. penalty values remain unchanged, are more than 80 or positive and negative infinite situation occur, then it is assumed that failure to train, directly
Deconditioning is connect, update hyper parameter is worth Qij(yij), and start hyper parameter sampling next time and training.
S306, update hyper parameter are worth Qij(yij) value.
S307, judges whether repetitive exercise terminates.If it is not, executing S308, if terminating, S309 is executed.
S308, optimal empirical value sampling or stochastical sampling, execute S303.
S309 determines optimal hyper parameter.
When the sampling number of all hyper parameters all reaches default sampling number, end loop, search is all to be sampled
Hyper parameter value selects Q in each hyper parameterij(yij) highest yijAs optimal hyper parameter, (i.e. the optimal of hyper parameter takes value
Value), model training is carried out using optimal hyper parameter, model verifies performance evaluation result of the accuracy rate as the model.
Method passes through Initialize installation, the update of hyper parameter value, hyper parameter sampling, model training in this embodiment of the present invention
It carries out optimal experience on the basis of random search with contents such as model evaluation, the determinations of optimal hyper parameter using priori and adopts
Sample quickly determines that convolutional neural networks model is super to reduce the iterative process of hyper parameter sampling-model training-model evaluation
The optimum combination of parameter.
Embodiment three
The embodiment of the present invention provides a kind of tune ginseng device of convolutional neural networks model, as shown in figure 4, described device includes
Memory 10 and processor 12, the memory 10 are stored with computer program, and the processor 12 executes the computer journey
Sequence, to realize such as the step of embodiment one to any one of embodiment two the method.
For example, the processor 12 executes the computer program, to realize following steps：
According to preset repetitive exercise frequency n, the repetitive exercise of convolutional neural networks model is carried out to the value of hyper parameter；
Determine hyper parameter value of the value of the hyper parameter in each repetitive exercise；
The value corresponding to maximum hyper parameter value in the hyper parameter value that the n times repetitive exercise is obtained is as institute
State the optimal value of hyper parameter.
In embodiments of the present invention, optionally, described according to preset repetitive exercise frequency n, hyper parameter value is carried out
The repetitive exercise of convolutional neural networks model, including：
Ith iteration is trained, when the i is equal to 1, using the sampled value of first time stochastical sampling as the super ginseng
Number value, carries out the first time repetitive exercise of the convolutional neural networks model；
When the i is more than 1 and is less than the n, by the sampled value of ith stochastical sampling or predetermined optimal experience
Sampled value carries out the ith iteration training of the convolutional neural networks model as the hyper parameter value.
In embodiments of the present invention, optionally, wherein using predetermined optimal experience sampled value as the hyper parameter
Value, before the ith iteration training for carrying out the convolutional neural networks model, including：
The value corresponding to maximum hyper parameter value during the hyper parameter that preceding i-1 repetitive exercise obtains is worth is determined as
The optimal experience sampled value.
In embodiments of the present invention, optionally, described by the sampled value of ith stochastical sampling or predetermined optimal warp
Sampled value is tested as the hyper parameter value, before the ith iteration training for carrying out the convolutional neural networks model, including：
Probability is chosen according to preset first, using the sampled value of the ith stochastical sampling as the hyper parameter value；
Probability is chosen according to preset second, using the optimal experience sampled value as the hyper parameter value.
In embodiments of the present invention, optionally, super ginseng of the determination hyper parameter value in each repetitive exercise
Number value, including：
For an arbitrary repetitive exercise, at the end of an arbitrary repetitive exercise, changed according to currently completed
The corresponding each hyper parameter value effect value of generation training determines that the value of the hyper parameter is instructed in the arbitrary an iteration
Corresponding hyper parameter value in white silk.
In embodiments of the present invention, optionally, currently completed repetitive exercise is corresponding each super for the basis
Parameter value effect value determines the value of hyper parameter hyper parameter valence corresponding in an arbitrary repetitive exercise
Value, including：
The mean value of each hyper parameter value effect value is determined as the value of the hyper parameter described arbitrary primary
Corresponding hyper parameter value in repetitive exercise.
In embodiments of the present invention, optionally, currently completed repetitive exercise is corresponding each super for the basis
Parameter value effect value determines the value of hyper parameter hyper parameter value corresponding in an arbitrary repetitive exercise
Before, including：
After each repetitive exercise in the current completed repetitive exercise, institute in each repetitive exercise is determined
State the penalty values of convolutional neural networks model；
According to the penalty values, the corresponding hyper parameter value effect value of each repetitive exercise is determined.
In embodiments of the present invention, optionally, according to the penalty values, the corresponding super ginseng of each repetitive exercise is determined
Number value effect value, including：
Judge whether the penalty values of convolutional neural networks model described in each repetitive exercise are normal；
If normal, the corresponding trained accuracy rate of each repetitive exercise is rounded up, obtains each iteration instruction
Practice corresponding hyper parameter value effect value；
The corresponding hyper parameter value effect value of each repetitive exercise is assigned a value of preset value if abnormal,.
The embodiment of the present invention can have corresponding technique effect in specific implementation refering to above-mentioned each embodiment.
Example IV
The embodiment of the present invention provides a kind of computer readable storage medium, and the storage medium is stored with computer program,
When the computer program is executed by least one processor, to realize as described in any one of embodiment one to embodiment two
The step of method.
The embodiment of the present invention can have corresponding technique effect in specific implementation refering to above-mentioned each embodiment.
Computer of embodiment of the present invention readable storage medium storing program for executing can be RAM memory, flash memory, ROM memory, EPROM
Memory, eeprom memory, register, hard disk, mobile hard disk, CD-ROM or any other form known in the art
Storage medium.A kind of storage medium lotus root can be connected to processor, to enable a processor to from the read information,
And information can be written to the storage medium；Or the storage medium can be the component part of processor.Processor and storage are situated between
Matter can be located in application-specific integrated circuit.
Above-described specific implementation mode has carried out further the purpose of the present invention, technical solution and advantageous effect
It is described in detail, it should be understood that the foregoing is merely the specific implementation mode of the present invention, is not intended to limit the present invention
Protection domain, all within the spirits and principles of the present invention, any modification, equivalent substitution, improvement and etc. done should all include
Within protection scope of the present invention.
Claims (10)
1. a kind of parameter adjustment method, which is characterized in that the method includes：
According to preset repetitive exercise frequency n, the repetitive exercise of convolutional neural networks model is carried out to the value of hyper parameter；
Determine hyper parameter value of the value of the hyper parameter in each repetitive exercise；
The value corresponding to maximum hyper parameter value in the hyper parameter value that the n times repetitive exercise is obtained is as described super
The optimal value of parameter.
2. the method as described in claim 1, which is characterized in that it is described according to preset repetitive exercise frequency n, hyper parameter is taken
Value carries out the repetitive exercise of convolutional neural networks model, including：
Ith iteration is trained, when the i is equal to 1, using the sampled value of first time stochastical sampling as the hyper parameter
Value carries out the first time repetitive exercise of the convolutional neural networks model；
When the i is more than 1 and is less than the n, the sampled value of ith stochastical sampling or predetermined optimal experience are sampled
It is worth the value as the hyper parameter, carries out the ith iteration training of the convolutional neural networks model.
3. method as claimed in claim 2, which is characterized in that wherein by the sampled value of ith stochastical sampling or predefine
Value of the optimal experience sampled value as the hyper parameter, carry out the ith iteration training of the convolutional neural networks model
Before, including：
The value corresponding to maximum hyper parameter value during the hyper parameter that preceding i-1 repetitive exercise obtains is worth is determined as described
Optimal experience sampled value.
4. method as claimed in claim 2, which is characterized in that described by the sampled value of ith stochastical sampling or predetermined
Optimal experience sampled value as the hyper parameter value, the ith iteration for carrying out the convolutional neural networks model trains it
Before, including：
Probability is chosen according to preset first, using the sampled value of the ith stochastical sampling as the hyper parameter value；
Probability is chosen according to preset second, using the optimal experience sampled value as the hyper parameter value.
5. the method as described in any one of claim 1-4, which is characterized in that the value of the determination hyper parameter exists
Hyper parameter value in each repetitive exercise, including：
For an arbitrary repetitive exercise, at the end of an arbitrary repetitive exercise, instructed according to current completed iteration
Practice corresponding each hyper parameter value effect value, determines the value of the hyper parameter in an arbitrary repetitive exercise
Corresponding hyper parameter value.
6. method as claimed in claim 5, which is characterized in that currently completed repetitive exercise is corresponding for the basis
Each hyper parameter value effect value determines the value of the hyper parameter super ginseng corresponding in an arbitrary repetitive exercise
Number value, including：
The mean value of each hyper parameter value effect value is determined as the value of the hyper parameter in the arbitrary an iteration
Corresponding hyper parameter value in training.
7. method as claimed in claim 5, which is characterized in that currently completed repetitive exercise is corresponding for the basis
Each hyper parameter value effect value determines the value of the hyper parameter super ginseng corresponding in an arbitrary repetitive exercise
Before number value, including：
After each repetitive exercise in the current completed repetitive exercise, determines and rolled up described in each repetitive exercise
The penalty values of product neural network model；
According to the penalty values, the corresponding hyper parameter value effect value of each repetitive exercise is determined.
8. the method for claim 7, which is characterized in that according to the penalty values, determine each repetitive exercise pair
The hyper parameter value effect value answered, including：
Judge whether the penalty values of convolutional neural networks model described in each repetitive exercise are normal；
If normal, the corresponding trained accuracy rate of each repetitive exercise is rounded up, obtains each repetitive exercise pair
The hyper parameter value effect value answered；
The corresponding hyper parameter value effect value of each repetitive exercise is assigned a value of preset value if abnormal,.
9. a kind of tune joins device, which is characterized in that described device includes memory and processor, and the memory is stored with calculating
Machine program, the processor execute the computer program, to realize the step such as any one of claim 1-8 the method
Suddenly.
10. a kind of computer readable storage medium, which is characterized in that the storage medium is stored with computer program, the meter
When calculation machine program is executed by least one processor, to realize such as the step of any one of claim 1-8 the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810473912.9A CN108764455A (en) | 2018-05-17 | 2018-05-17 | Parameter adjustment method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810473912.9A CN108764455A (en) | 2018-05-17 | 2018-05-17 | Parameter adjustment method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108764455A true CN108764455A (en) | 2018-11-06 |
Family
ID=64008505
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810473912.9A Pending CN108764455A (en) | 2018-05-17 | 2018-05-17 | Parameter adjustment method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764455A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711548A (en) * | 2018-12-26 | 2019-05-03 | 歌尔股份有限公司 | Choosing method, application method, device and the electronic equipment of hyper parameter |
| CN110414578A (en) * | 2019-07-16 | 2019-11-05 | 上海电机学院 | A kind of transfer learning method based on the multiple batches of training of dynamic and colour gamut conversion |
| CN110443126A (en) * | 2019-06-27 | 2019-11-12 | 平安科技（深圳）有限公司 | Model hyper parameter adjusts control method, device, computer equipment and storage medium |
| CN110766090A (en) * | 2019-10-30 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Model training method, device, equipment, system and storage medium |
| CN111709519A (en) * | 2020-06-17 | 2020-09-25 | 湖南大学 | Deep learning parallel computing architecture method and hyper-parameter automatic configuration optimization thereof |
| WO2020259502A1 (en) * | 2019-06-27 | 2020-12-30 | 腾讯科技（深圳）有限公司 | Method and device for generating neural network model, and computer-readable storage medium |
| CN114067183A (en) * | 2021-11-24 | 2022-02-18 | 北京百度网讯科技有限公司 | Neural network model training method, image processing method, device and equipment |
-
2018
- 2018-05-17 CN CN201810473912.9A patent/CN108764455A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711548A (en) * | 2018-12-26 | 2019-05-03 | 歌尔股份有限公司 | Choosing method, application method, device and the electronic equipment of hyper parameter |
| CN110443126A (en) * | 2019-06-27 | 2019-11-12 | 平安科技（深圳）有限公司 | Model hyper parameter adjusts control method, device, computer equipment and storage medium |
| WO2020259502A1 (en) * | 2019-06-27 | 2020-12-30 | 腾讯科技（深圳）有限公司 | Method and device for generating neural network model, and computer-readable storage medium |
| CN110414578A (en) * | 2019-07-16 | 2019-11-05 | 上海电机学院 | A kind of transfer learning method based on the multiple batches of training of dynamic and colour gamut conversion |
| CN110766090A (en) * | 2019-10-30 | 2020-02-07 | 腾讯科技（深圳）有限公司 | Model training method, device, equipment, system and storage medium |
| CN111709519A (en) * | 2020-06-17 | 2020-09-25 | 湖南大学 | Deep learning parallel computing architecture method and hyper-parameter automatic configuration optimization thereof |
| CN111709519B (en) * | 2020-06-17 | 2024-02-06 | 湖南大学 | Deep learning parallel computing architecture method and super-parameter automatic configuration optimization thereof |
| CN114067183A (en) * | 2021-11-24 | 2022-02-18 | 北京百度网讯科技有限公司 | Neural network model training method, image processing method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764455A (en) | Parameter adjustment method, device and storage medium | |
| CN103426042B (en) | The group technology of social networks and system | |
| CN110691422A (en) | Multi-channel intelligent access method based on deep reinforcement learning | |
| CN101873638B (en) | Heterogeneous wireless network access selection method based on fuzzy neural network | |
| CN108900333A (en) | A kind of appraisal procedure and assessment device of quality of wireless network | |
| CN109961098A (en) | A kind of training data selection method of machine learning | |
| CN109429247A (en) | A kind of method and device of the improvement subzone network based on problem scenes | |
| CN112232508A (en) | Model training method, system, device and medium | |
| CN109934330A (en) | The method of prediction model is constructed based on the drosophila optimization algorithm of diversified population | |
| CN113422695B (en) | Optimization method for improving robustness of topological structure of Internet of things | |
| CN117852616B (en) | Big language model alignment fine tuning method and system based on enhanced reject sampling training | |
| CN108271176A (en) | Determine base station cell matter difference root because method and system | |
| CN110519816A (en) | A kind of radio roaming control method, device, storage medium and terminal device | |
| CN104640222A (en) | Pilot frequency scheduling method for multi-input multi-output system and synergetic equipment | |
| CN109067427A (en) | A kind of frequency hop sequences prediction technique based on Optimization-type wavelet neural network | |
| Huang et al. | Zwei: A self-play reinforcement learning framework for video transmission services | |
| CN106951615B (en) | Electric power transportation network Topology Structure Design method based on close female algorithm | |
| US8787510B2 (en) | Channel selection exploiting multiple observations of the channel delay spread | |
| Zhu et al. | Adaptive deep reinforcement learning for non-stationary environments | |
| CN106022475A (en) | Web service composition method based on dynamic self-adaptive chaos ant colony algorithm | |
| CN105451268A (en) | High-energy-efficiency heterogeneous community access method | |
| CN109672498A (en) | Interpretation method and device | |
| CN111917529A (en) | Underwater sound OFDM resource allocation method based on improved EXP3 algorithm | |
| CN114401192B (en) | Multi-SDN controller cooperative training method | |
| CN115146702A (en) | Transformer fault diagnosis method, medium and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20190529 Address after: 518057 Nanshan District high tech Industrial Park, Shenzhen, Guangdong, Ministry of justice, Zhongxing Road, South China road. Applicant after: ZTE Corporation Address before: 210000 No. 68, Bauhinia Road, Ningnan street, Yuhuatai District, Nanjing, Jiangsu Applicant before: Nanjing Zhongxing Software Co., Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
| RJ01 | Rejection of invention patent application after publication |