CN103559542A - Extension neural network pattern recognition method based on priori knowledge - Google Patents

Abstract

The invention discloses an extension neural network pattern recognition method based on priori knowledge. The method includes the following steps that (1) a training sample set and a knowledge base are prepared; (2) an initial weight value of an extension neural network is determined according to training samples and the priori knowledge; (3) the extension neural network can be trained by the utilization of the training samples, if a training process is converged or the total error rate reaches a preset value, training is stopped, and a weight value vector, after the training, of the extension neural network is kept, and otherwise the training is continued; (4) the trained extension neural network is used for performing pattern recognition until recognition of all objects to be recognized is completed. According to the extension neural network pattern recognition method, under the common driving of the priori knowledge and the training samples, learning of the extension neural network is guided, training and learning of the extension neural network are completed, the learning burden of the extension neural network is relieved, the performance of the extension neural network is effectively improved, training time is shortened, and recognition accuracy is improved.

Description

Extension neural network mode identification method based on priori

Technical field

The present invention relates to network mode identification field, relate in particular to the extension neural network mode identification method under the common driving of a kind of combination priori and training sample.

Background technology

Extension neural network (Extension Neural Network, ENN) is to open up product theoretical and that nerual network technique organically combines.Extension neural network is another the new network type after fuzzy neural network, genetic neural network, Evolutionary Neural Network, its appearance and development can not only be expanded the further application that can open up theory self, also will promote further developing of nerual network technique and intelligent computation.At present, extension neural network < < Extension neural network and its applications > > (the Neural Networks that a kind of two power that M.H.Wang proposes connects, 2003, the 16th the 5th phase of volume) be most widely used extension neural network model, it is to separate in the problems such as the classification of block eigenvector eigenwert in interval range, identification, cluster effect remarkable.Than traditional neural network, extension neural network generalization ability is strong, plasticity good, the training time is short, can adapt to quickly new information, and in actual mechanical process, having certain rule can follow, therefore, the extension neural network sorter based on two power has had certain application at aspects such as pattern-recognition, detection, fault diagnosises.

Weighing the most important index of neural network classification recognition performance is generalization ability.Affecting one of most important factor of generalization ability is training sample, and high-quality training sample can bring high-quality network performance.Yet obtaining of high quality training sample is not a pipe course, although reason is observation data (sample data) and has no data from same distribution that common observation data limited amount can not be portrayed well raw data and distribute; In addition, although quantity is enough sometimes for sample data, there are the little data of bulk information amount, or have the similar redundant data of bulk information amount; Meanwhile, in sample data production process,, there are various noises in the training sample especially obtaining under rugged surroundings possibly.Therefore, facing training sample quality poor in the situation that, the performance of extension neural network sharply declines, so we need to solve and face under the environment of difference sample, how to improve this realistic problem of performance of extension neural network.

In actual application, sometimes be difficult to obtain on the one hand high-quality sample data, but then, although some system is very complicated, cannot understand all structures of its inside, but conventionally can there is certain understanding to some process mechanism, some priori information of knowing these processes, therefore, hope can make full use of these information, set up observation data and have no contacting of data, in conjunction with the method for empirical modeling, carrying out the foundation of model.Adding priori may be that machine learning realizes the only resource of Generalization Ability (generalization ability) under finite data.Much research shows, utilizes priori, before neural metwork training, they are applied to certain constraint, is conducive to improve the performance of network.Therefore, must, in conjunction with the priori of concrete problem concerning study, could learn out to be applicable to " optimum " sorter of specific problem concerning study.

Summary of the invention

The object of this invention is to provide a kind of extension neural network mode identification method based on priori, can effectively improve the performance of extension neural network; Even if it is poor to face training sample quality, be applied in Complex System Environment, the extension neural network based on priori still has outstanding Classification and Identification performance.

The present invention adopts following technical proposals:

An extension neural network mode identification method for priori, comprises the following steps: (1), preparation training sample set and knowledge base, training sample set is the observation data of having obtained, and supposes that training sample set is n wherein _pbe the total number of sample of sample set, i schedule of samples is shown

wherein n is the total number of feature that sampling feature vectors comprises, and i sample class label is p; Knowledge base is the information of storage about having known in the face of concrete object; The knowledge feature embodying for extension neural network weights, the classical territory extreme value of each eigenwert of alternative proper vector,

l _kjrepresent that k kind pattern is about the quantitative scope of j characteristic attribute;

(2), according to training sample and priori, determine the initial weight of extension neural network;

(3), utilize training sample to train extension neural network; If training process convergence or total error rate arrive preset value, deconditioning, preserves the weight vector of the extension neural network after training; Otherwise continue training;

(4), utilize the extension neural network training to carry out pattern-recognition, until the whole identification of all identifying objects is complete.

Described step (2) is specific as follows: the neuron number n of input layer and the neuron number n of output layer that first determine extension neural network _c, the neuron number n of input layer equals the number of features training eigenwert, the neuron number n of output layer _cequal the number of mode state; The mode that adopts two power to connect between input layer and output layer, one of them weights represents the lower limit in the classical territory of a certain feature, another one weights are representing the higher limit in the classical territory of individual features, and two weights that connect j node of input layer and k node of output layer are used respectively with

represent, the initial weight of extension neural network can obtain according to formula (2)～(3):

Described step (3) specifically comprises the following steps:

Wherein the initial center point of each pattern is as described below:

Z _k={z _k1,z _k2,...,z _kn} (4)

$z_{\mathrm{kj}}=(w_{\mathrm{kj}}^L+w_{\mathrm{kj}}^U)/2,k=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n_c;j=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n---\left(5\right)$

(31), read in i sample and safe state mode class label p corresponding to this sample, i=1,2 ..., N _p;

(32), utilize formula (6) to calculate sample

opened up distance with k class safe state mode:

$\left\{\begin{array}{c}{\mathrm{ED}}_{\mathrm{ik}}=\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}[\frac{|x_{\mathrm{ij}}^p-z_{\mathrm{kj}}|-(w_{\mathrm{kj}}^U-w_{\mathrm{kj}}^L)/2}{|(w_{\mathrm{kj}}^U-w_{\mathrm{kj}}^L)/2|}+1],k=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n_c\\ z_{\mathrm{kj}}=(w_{\mathrm{kj}}^U+w_{\mathrm{kj}}^L)/2\end{array}\right.---\left(6\right)$

(33), ask for k ^*, make

if k ^*=p, enters step (35); Otherwise enter step (34), carry out the adjustment of network weight;

(34), according to formula (7)～(10), adjust p class and k ^*the weights that quasi-mode is corresponding: (a), the adjustment at p quasi-mode class center is suc as formula shown in (7); K ^*the adjustment at quasi-mode class center is suc as formula shown in (8):

$z_{\mathrm{pj}}^{\mathrm{new}}=z_{\mathrm{pj}}^{\mathrm{old}}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})---\left(7\right)$

$z_{k^{*}j}^{\mathrm{new}}=z_{k^{*}j}^{\mathrm{old}}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})---\left(8\right)$

(b), the adjustment of p quasi-mode weights is suc as formula shown in (9); K ^*the adjustment of quasi-mode weights is suc as formula shown in (10):

$\left\{\begin{array}{c}{w}_{\mathrm{pj}}^{L\left(\mathrm{new}\right)}=w_{\mathrm{pj}}^{L\left(\mathrm{old}\right)}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})\\ w_{\mathrm{pj}}^{U\left(\mathrm{new}\right)}=w_{\mathrm{pj}}^{U\left(\mathrm{old}\right)}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})\end{array}\right.---\left(9\right)$

$\left\{\begin{array}{c}{w}_{k^{*}j}^{L\left(\mathrm{new}\right)}=w_{k^{*}j}^{L\left(\mathrm{old}\right)}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})\\ w_{k^{*}j}^{U\left(\mathrm{new}\right)}=w_{k^{*}j}^{U\left(\mathrm{old}\right)}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})\end{array}\right.---\left(10\right)$

represent respectively k ^*quasi-mode is adjusted Hou Lei center and is adjusted Qian Lei center,

represent respectively p quasi-mode adjustment Hou Lei center and adjust Qian Lei center;

represent the weights after p quasi-mode is adjusted, represent the weights before p quasi-mode is adjusted;

represent k ^*weights after quasi-mode is adjusted,

represent k ^*weights before quasi-mode is adjusted;

η represents learning rate;

(35): circulation execution step (31) to (34); If all samples are all classified, a Learning Step finishes, and enters step (36);

(36) if training process convergence, or total error rate arrives preset value, deconditioning, otherwise return to execution step (31).

Total error rate E in described step (3) _t=N _m/ N _p, N _mthe number of misclassification, N _pit is number of samples.

The present invention proposes a kind of extension neural network mode identification method based on priori, by extension neural network being added to effective priori, priori is embedded in the middle of neural network weight effectively, under priori and the common driving of training sample, instruct the study of extension neural network, and then complete the training of extension neural network.Extension neural network based on priori has alleviated the learning burden of network effectively, and the performance (learning performance, generalization ability, fault-tolerant ability etc.) of extension neural network is significantly improved.

The present invention utilizes priori, before extension neural network training, the weights of network are applied to certain constraint, can debug sample, imperfect sample adverse effect that study is brought, even face in Practical Project, obtain that training sample quality is not high, under Complex System Environment, also can make extension neural network still there is outstanding Classification and Identification performance, be conducive to the raising of extension neural network performance.Therefore, the present invention has good value for applications, and can be applied to the aspects such as industry spot fault diagnosis, the detection of working environment safe condition and image processing.

Accompanying drawing explanation

Fig. 1 is extension neural network structural drawing;

Fig. 2 is based on data and the common extension neural network illustraton of model driving of priori;

Fig. 3 is opening up apart from schematic diagram between a point and an interval;

Fig. 4 is method flow diagram of the present invention.

Embodiment

The invention discloses a kind of extension neural network mode identification method based on priori, as shown in Figure 4, comprise the following steps:

(1), prepare training sample set and knowledge base, training sample set is the observation data of having obtained, and supposes that training sample set is

n wherein _pbe the total number of sample of sample set, i schedule of samples is shown

wherein n is the total number of feature that sampling feature vectors comprises, and i sample class label is p; Knowledge base is the priori information of storage about having known towards concrete object; The knowledge feature embodying for extension neural network weights, the classical territory extreme value of each eigenwert of alternative proper vector,

l _kjrepresent that k kind pattern is about the quantitative scope of j characteristic attribute;

In actual process, how much we have gained some understanding to some characteristic interval scope of the proper vector of identifying object sometimes, such as, in coal mine environment safe condition recognition system, carbon monoxide content is an important indicator, when carbon monoxide content is in 0.0006%～0.0012%, can think that mine is in a safe condition, and when carbon monoxide content is in 0.0018%～0.0024% time, need to give a warning, illustrate that now the residing state of coal mine environment is more dangerous.Therefore the accumulation of the information of this class and to obtain be to be relatively easy to.

(2), determine the structure of extension neural network; According to training sample and priori, determine the initial weight of extension neural network;

Specific as follows: first to determine the neuron number n of extension neural network input layer and the neuron number n of output layer _c, the neuron number n of input layer equals the number of proper vector eigenwert, the neuron number n of output layer _cequal the number of schema category; The mode that adopts two power to connect between input layer and output layer, one of them weights represents the lower limit of the classical territory of a certain feature scope, another one weights are representing the higher limit of the classical territory of individual features scope; Be illustrated in figure 1 extension neural network structural representation, Figure 2 shows that the extension neural network illustraton of model based under training sample and the common driving of priori.

According to training sample and priori, set up the matter-element model of every kind of pattern, as the formula (1).

$R_k=\left[\begin{array}{ccc}{N}_k& c_1,& V_{k1}\\ & C_2,& V_{K2}\\ & \&CenterDot;\&CenterDot;\&CenterDot;& \&CenterDot;\&CenterDot;\&CenterDot;\\ & c_n,& V_{\mathrm{Kn}}\end{array}\right],k=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n_c---\left(1\right)$

Can open up in theory c _jrepresent k quasi-mode N _kj feature,

represent that k quasi-mode is about the classical territory scope of j characteristic index, classical territory scope is determined jointly by training sample and priori.Two weights that connect j node of input layer and k node of output layer are used respectively

with

represent, the initial weight of extension neural network can obtain according to formula (2)～(3):

(3), utilize training sample to train extension neural network;

Wherein the initial center point of each pattern is as described below:

Z _k={z _k1,z _k2,...,z _kn} (4)

$z_{\mathrm{kj}}=(w_{\mathrm{kj}}^L+w_{\mathrm{kj}}^U)/2,k=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n_c;j=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n---\left(5\right)$

Specifically comprise the following steps:

(31), read in i sample and safe state mode class label p corresponding to this sample, i=1,2 ..., N _p;

(32), utilize formula (6) to calculate sample

opened up distance with k class safe state mode:

$\left\{\begin{array}{c}{\mathrm{ED}}_{\mathrm{ik}}=\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}[\frac{|x_{\mathrm{ij}}^p-z_{\mathrm{kj}}|-(w_{\mathrm{kj}}^U-w_{\mathrm{kj}}^L)/2}{|(w_{\mathrm{kj}}^U-w_{\mathrm{kj}}^L)/2|}+1],k=\mathrm{1,2},\&CenterDot;\&CenterDot;\&CenterDot;,n_c\\ z_{\mathrm{kj}}=(w_{\mathrm{kj}}^U+w_{\mathrm{kj}}^L)/2\end{array}\right.---\left(6\right)$

As shown in Figure 3, be a some x and an interval

between open up apart from schematic diagram, can open up distance and be used for judging the similarity degree at tested point Yu Lei center,

work as ED=0, represent object under test and class center superposition, completely similar, can open up apart from larger, the similarity of expression is less.

From formula (6), even wrong sample input, also can not affect training and the study of extension neural network, its main cause is the information that initial weight has embedded our known systems, this category information can effectively be got rid of the data that depart from classical territory scope, makes this class sample can not carry out wrong adjustment to initial weight, and the study of extension neural network is played to directive function, not only alleviate the training burden of extension neural network, and reduced training time and computed losses.

(33), ask for k ^*, make

if k ^*=p, enters step (35); Otherwise enter step (34), carry out the adjustment of network weight.

(34), according to formula (7)～(10), adjust p class and k ^*the weights that quasi-mode is corresponding: (a), the adjustment at p quasi-mode class center is suc as formula shown in (7); K ^*the adjustment at quasi-mode class center is suc as formula shown in (8):

$z_{\mathrm{pj}}^{\mathrm{new}}=z_{\mathrm{pj}}^{\mathrm{old}}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})---\left(7\right)$

$z_{k^{*}j}^{\mathrm{new}}=z_{k^{*}j}^{\mathrm{old}}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})---\left(8\right)$

(b), the adjustment of p quasi-mode weights is suc as formula shown in (9); K ^*the adjustment of quasi-mode weights is suc as formula shown in (10):

$\left\{\begin{array}{c}{w}_{\mathrm{pj}}^{L\left(\mathrm{new}\right)}=w_{\mathrm{pj}}^{L\left(\mathrm{old}\right)}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})\\ w_{\mathrm{pj}}^{U\left(\mathrm{new}\right)}=w_{\mathrm{pj}}^{U\left(\mathrm{old}\right)}+\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{\mathrm{pj}}^{\mathrm{old}})\end{array}\right.---\left(9\right)$

$\left\{\begin{array}{c}{w}_{k^{*}j}^{L\left(\mathrm{new}\right)}=w_{k^{*}j}^{L\left(\mathrm{old}\right)}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})\\ w_{k^{*}j}^{U\left(\mathrm{new}\right)}=w_{k^{*}j}^{U\left(\mathrm{old}\right)}-\mathrm{\&eta;}(x_{\mathrm{ij}}^p-z_{k^{*}j}^{\mathrm{old}})\end{array}\right.---\left(10\right)$

represent respectively k ^*quasi-mode is adjusted Hou Lei center and is adjusted Qian Lei center,

represent respectively p quasi-mode adjustment Hou Lei center and adjust Qian Lei center;

represent the weights after p quasi-mode is adjusted, represent the weights before p quasi-mode is adjusted;

represent k ^*weights after quasi-mode is adjusted,

represent k ^*weights before quasi-mode is adjusted;

η represents learning rate;

(35): circulation execution step (31) to (34).If all samples are all classified, a Learning Step finishes, and enters step (36);

(36) if training process convergence, or total error rate E _tarrive preset value, deconditioning.Here E _t=N _m/ N _p, N _mthe number of misclassification, N _mcan in training process, obtain; N _pbe number of samples, otherwise return to execution step (31).

(4), utilize the extension neural network training to carry out pattern-recognition to identifying object.Utilization can be opened up the opened up distance of distance metric target to be measured and each safe mode, tries to achieve k ^*make

output

in order to indicate the safe mode of target to be measured, be k ^*, until the whole identification of all identifying objects is complete.

Claims (3)

1. the extension neural network mode identification method based on priori, is characterized in that: comprise the following steps: (1), preparation training sample set and knowledge base, training sample set is the observation data of having obtained, and supposes that training sample set is

n wherein _pbe the total number of sample of sample set, i schedule of samples is shown

wherein n is the total number of feature that sampling feature vectors comprises, and i sample class label is p; Knowledge base is the information of storage about having known in the face of concrete object; The knowledge feature embodying for extension neural network weights, the classical territory extreme value of each eigenwert of alternative proper vector,

l _kjrepresent that k kind pattern is about the quantitative scope of j characteristic attribute;

(2), according to training sample and priori, determine the initial weight of extension neural network;

(3), utilize training sample to train extension neural network; If training process convergence or total error rate arrive preset value, deconditioning, preserves the weight vector of the extension neural network after training; Otherwise continue training;

(4), utilize the extension neural network training to carry out pattern-recognition, until the whole identification of all identifying objects is complete.

2. the extension neural network mode identification method based on priori according to claim 1, is characterized in that: described step (2) is specific as follows: the neuron number n of input layer and the neuron number n of output layer that first determine extension neural network _c, the neuron number n of input layer equals the number of features training eigenwert, the neuron number n of output layer _cequal the number of mode state; The mode that adopts two power to connect between input layer and output layer, one of them weights represents the lower limit in the classical territory of a certain feature, another one weights are representing the higher limit in the classical territory of individual features, and two weights that connect j node of input layer and k node of output layer are used respectively

with

represent, the initial weight of extension neural network can obtain according to formula (2)～(3):

。

3. the extension neural network mode identification method based on priori according to claim 1, is characterized in that: described step (3) specifically comprises the following steps:

Wherein the initial center point of each pattern is as described below:

Z _k={z _k1,z _k2,...,z _kn} (4)

(31), read in i sample and safe state mode class label p corresponding to this sample, i=1,2 ..., N _p;

(32), utilize formula (6) to calculate sample

opened up distance with k class safe state mode:

(33), ask for k ^*, make

if k ^*=p, enters step (35); Otherwise enter step (34), carry out the adjustment of network weight;

(34), according to formula (7)～(10), adjust p class and k ^*the weights that quasi-mode is corresponding: (a), the adjustment at p quasi-mode class center is suc as formula shown in (7); K ^*the adjustment at quasi-mode class center is suc as formula shown in (8):

(b), the adjustment of p quasi-mode weights is suc as formula shown in (9); K ^*the adjustment of quasi-mode weights is suc as formula shown in (10):

represent respectively k ^*quasi-mode is adjusted Hou Lei center and is adjusted Qian Lei center, represent respectively p quasi-mode adjustment Hou Lei center and adjust Qian Lei center;

represent the weights after p quasi-mode is adjusted,

represent the weights before p quasi-mode is adjusted;

represent k ^*weights after quasi-mode is adjusted,

represent k ^*weights before quasi-mode is adjusted;

η represents learning rate;

(35): circulation execution step (31) to (34); If all samples are all classified, a Learning Step finishes, and enters step (36);

(36) if training process convergence, or total error rate arrives preset value, deconditioning, otherwise return to execution step (31).

Extension neural network mode identification method based on priori according to claim 3, is characterized in that: the total error rate E in described step (3) _t=N _m/ N _p, N _mthe number of misclassification, N _pit is number of samples.

Images