Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides a post-SMT anomaly detection method which integrates a CMA-ES algorithm and a sequential extreme learning machine, so that the post-SMT anomaly detection can be completed by a reinforcement learning method, the detection result is more accurate, and the detection efficiency is higher.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention discloses a post-chip anomaly detection method integrating a CMA-ES algorithm and a sequential extreme learning machine, which is characterized by comprising the following steps of:
step 1, obtaining a gray level histogram of N normally welded PCBs from a normally welded PCB database and forming a sample training set;
step 2, carrying out image enhancement processing on the training set to obtain an enhanced training set;
2.1, converting the N gray level histograms in the training set into a uniform distribution map;
step 2.2, modifying the gray value of each pixel in the uniform distribution map by utilizing a gray stretching algorithm to obtain an enhanced training set;
step 3, utilizing Haar transformation to extract features of the enhanced training set to obtain the difference between pixel sums of white areas and black areas of the images in the training set, and using the difference as a feature vector of the training set;
step 4, constructing a single hidden layer feedforward neural network and initializing a sequential extreme learning machine model;
step 4.1, taking a Sigmoid function as a hidden layer activation function;
step 4.2, randomly generating input weight { W } i L =1,2, \8230;, L } and hidden layer bias { b [ ] i L } to determine the input to output relationship; wherein, W i Representing the ith input weight, b i Represents the ith hidden layer bias;
step 5, performing initialization training on the sequential extreme learning machine model;
according to the feature vectors of the training set, an initial neuron matrix H and an output vector T are constructed, the least square solution of the initial neuron matrix H and the output vector T is solved, and a model parameter { beta is obtained i |i=1,2,…,L},β i Representing the ith model parameter, and L representing the total number of the model parameters, thereby completing the initial training of the model;
step 6, performing iterative optimization on the initialized sequential extreme learning machine model by utilizing a CMA-ES algorithm so as to obtain the optimal parameters of the sequential extreme learning machine model;
step 6.1, setting the minimum iteration number as eta, defining the current iteration number as g, and initializing g =1;
utilizing the input weight { W } of the initialized sequential extreme learning machine model obtained in the step 5 i I =1,2, \ 8230;, L } and hidden layer bias { b [ ] i L } forms the g-th generation population;
with the model parameter { beta
i L =1,2, \ 8230;, L } is used as the model parameter corresponding to the g-th generation population
Wherein,
representing model parameters corresponding to ith individuals of the g-th generation population, and taking the initial neuron matrix H as the g-th generation as a neuron matrix H
(g) ;
Step 6.2, calculating the fitness of the ith individual in the population of the g generation by using the formula (1)
Thus obtaining the fitness of each individual of the g-th generation population, and ranking each individual according to the fitness from high to low:
in the formula (1), the reaction mixture is,
is the average output vector;
is a model parameter corresponding to the ith individual in the g generation population
Calculating a predicted feature vector;
step 6.3, iteration and mutation are carried out on the g generation population by using the formula (2), so that the ith individual of the g +1 generation population is obtained
Thereby forming a g +1 generation population to update a g generation neuron matrix H
(g) And obtaining a g +1 th generation neuron matrix H
(g+1) :
In the formula (2), m (g) Is the mean vector of the mu individuals with the individual fitness ranking in the g-th generation population, N i (0,C (g) ) Is a Gaussian distribution function obeyed by the ith individual in the g-th generation population; epsilon (g) For the step size of the population evolution of the g generation, when g =1, the epsilon is initialized (g) =1;C (g) Is a covariance matrix of the g-th generation population;
step 6.4, calculating the mean vector m of the g +1 th generation population
(g+1) Step size epsilon of population evolution
(g+1) Model parameters
Step 6.4.1, obtaining the mean vector m of the g +1 th generation population by using the formula (3) (g+1) :
In the formula (3), ω
n Is the optimized weight of the nth individual in the mu individuals before fitness ranking, an
Is the nth individual of the mu individuals in the population of the g generation;
step 6.4.2, obtaining the step size epsilon of the population evolution of the g +1 generation by using the formula (4) (g+1) :
In the formula (4), c (g) Is a step size epsilon (g) Updating parameters of (1);
step 6.4.3, according to the g +1 th generation neuron matrix H
(g+1) Obtaining model parameters corresponding to the g +1 generation population according to the process of the step 5
Step 6.5, obtaining a convergence criterion S by using the formula (5), if the S is less than theta for continuous lambda times, stopping iteration, obtaining a sequential extreme learning machine model of the optimal parameter, and simultaneously outputting the optimal parameter, wherein the step comprises the following steps: optimal input weights
Optimal hidden layer biasing
And corresponding optimal model parameters
Wherein,
represents the ith optimal input weight,
indicating the ith optimal hidden layer bias,
and (3) representing the ith optimal model parameter, otherwise, returning to the step 6.2, wherein theta is a convergence boundary:
in the formula (5), the reaction mixture is,
is the fitness of the ith individual in the g-1 generation population, and is initialized when g =1
Step 7, anomaly detection after mounting:
step 7.1, obtaining the average value sigma of the feature vectors of the training set by using the formula (6):
in the formula (6), c is the total number of the feature vectors in the training set, X j Is the j-th feature vector of the input;
step 7.2, defining the current time as t, and initializing t =0;
optimizing model parameters
As model parameters at the current time t
Is shown asThe ith model parameter at the previous t moment;
obtaining a predicted feature vector of a model output at a time t +1 using equation (7)
In the formula (7), the reaction mixture is,
representing a hidden layer activation function;
step 7.3, according to the prediction characteristic vector
The prediction variance at time t +1 is obtained by equation (8)
In the formula (8), the reaction mixture is,
is the predicted variance at time t; when t =0, let
And 7.4, obtaining a detection threshold value omega by using the formula (8):
in the formula (9), T t+1 Representing the actual feature vector at time t + 1;
step 7.5, calculating t +1 by using the formula (10)Predicted distance difference phi of scales t+1 :
If phi is t+1 If the value is larger than omega, the paster is abnormal at the time of t +1, otherwise, the paster is normal at the time of t + 1;
step 8, acquiring the patch data of the welded PCB in real time and processing the data according to the steps 1 to 3 to obtain the patch characteristic vector of the welded PCB; sequentially inputting the patch characteristic vectors of the welded PCB into the sequential extreme learning machine model of the optimal parameters for on-line training so as to update the optimal parameters, thereby obtaining the sequential extreme learning machine model with strong adaptability;
step 9, scanning the area needing patch detection by adopting a multi-scale sliding window, and processing the test image in the area where each sliding window is located according to the steps 1-3 to obtain a feature vector to be detected;
step 10, inputting the feature vector to be detected into a strong-adaptability sequential extreme learning machine model for anomaly detection, thereby obtaining the areas of all windows with the patch anomalies;
and 11, screening all window areas with the patch abnormity, so as to obtain the positions of the patch abnormity in the test image.
Compared with the prior art, the invention has the beneficial effects that:
1. the method is based on the sequential extreme learning machine, the sequential extreme learning machine is a high-efficiency classifier with high classification speed and few learning parameters, further optimization of a model can be realized by continuously adding new samples, active learning is realized, feedback is accurately obtained, the method has strong self-learning and large-scale parallel processing capacity, compared with the traditional neural network structure, the method has the characteristics of being updatable and optimizable, meanwhile, on the basis of the extreme learning machine, the concept of time is introduced into network training, and the SMT post-paster anomaly detection method with high training speed, strong generalization capability and high precision is established, so that the method is suitable for detection and repair of post-paster anomalies;
2. according to the invention, the sequential extreme learning machine model is improved based on the CMA-ES algorithm, the CMA-ES algorithm has the characteristics of good overall performance, high optimization efficiency and high convergence rate, and the parameters of the sequential extreme learning machine model are improved by the CMA-ES algorithm, so that the sequential extreme learning machine model has better generalization, avoids falling into local optimization, better promotes the accuracy and reliability of abnormal detection after SMT (surface mount technology) chip mounting, and improves the production efficiency;
3. the method is based on the single hidden layer feedforward neural network, the output weight connecting the hidden layer and the output layer is determined by an analytic method, the parameters are simple and convenient to select, iteration is not needed, the learning speed is high, and the detection and repair of the abnormity after the surface mounting are realized;
4. in the detection stage, the whole image is not scanned any more when the multi-scale sliding window is adopted, and only the marked salient region of the test image needs to be scanned, so that the generation of a plurality of candidate regions is reduced, and the detection speed is further improved.
5. The invention has wide application range: additional auxiliary information is not required to be added, the method can be widely applied to abnormality detection after SMT patch, and has wide application prospect.
Detailed Description
In this embodiment, as shown in fig. 1, a post-patch anomaly detection method that combines a CMA-ES (covariance matrix adaptive evolution policy) algorithm and a sequential extreme learning machine is performed according to the following steps:
step 1, obtaining a gray level histogram of N normally welded PCBs from a normally welded PCB database to form a sample training set;
step 2, carrying out image enhancement on the training set to obtain an enhanced training set;
step 2.1, converting N gray level histograms in the training set into an averageA uniform distribution pattern, i.e. the number of pixels of the image above each grey level is the same, thereby achieving the effect of increasing the dynamic value range with the abscissa being the frequency with which the image grey level ordinate is the frequency with which the grey level pixel appears in the image, the grey level range being [ l ] 1 ,l 2 ]The gray histogram function is:
h(r k )=n k ,k∈[0,L-1] (1)
in the formula (1), r k Is the kth gray level of the image, and n k Is a gray level r in the image k The number of pixels of (a);
step 2.2, modifying the gray value of each pixel in the uniform distribution map by using a gray stretching algorithm to obtain an enhanced training set;
let the original image in the training set have a gray scale range of l 1 ,l 2 ]. To obtain a compound having [ l 3 ,l 4 ]The transfer function required for the grayscale range image is:
in the formula (2), r is the gray level of the image;
after the linear stretching function is adopted, the gray value of the pixel points of all the digital images is subjected to gray level transformation, so that the contrast of the images can be improved, the images are enhanced, and the corresponding characteristic vectors are obtained, thereby facilitating model training by using a sequential extreme learning machine;
step 3, utilizing Haar transformation to extract features of the enhanced training set to obtain the difference between pixel sums of white areas and black areas of images in the training set, and using the difference as a feature vector of the training set; the difference between the pixel sums of the white area and the black area is calculated in a mode of constructing an integral graph, the integral graph is constructed in a mode of scanning an image line by line, the accumulated sum of the row direction of each pixel (i, j) is calculated in a recursion mode, and the formula is as follows:
s(i,j)=s(i,j-1)+f(i,j) (3)
ii(i,j)=ii(i-1,j)+s(i,j) (4)
in equations (3) and (4), s (i, j) represents the accumulated pixel in the row direction, and ii (i, j) represents the value of the integral image.
Three types of features of the image can be extracted through Haar transform: edge features, linear features, center features, and diagonal features;
step 4, constructing a single hidden layer feedforward neural network, as shown in fig. 2, initializing a sequential extreme learning machine (OS-ELM) model
Step 4.1, taking a Sigmoid function as a hidden layer activation function:
in the formula (5), W i =[w i,1 ,w i,2 ,…,w i,n ] T To input the weight, X i =[x i,1 ,x i,2 ,…,x i,n ] T As input feature vectors, b i The offset value of the ith hidden layer unit which is randomly output is a random number in the range of (0, 1). W is a group of i ·X j Represents W i And X j The inner product of (d).
Step 4.2, randomly generating input weight { W } i I =1,2, \ 8230;, L } and hidden layer bias { b [ ] i I =1,2, \8230 |, L }, thereby determining the input to output relationship; wherein, the ith input weight is represented, and the ith hidden layer bias is represented; the input-output relationship can be expressed as:
in formula (6), g (W) i ·X j +b i ) For the activation function, n is the number of input nodes, and n =4 is taken; l is the number of hidden layer nodes, and L =30 is taken for obtaining better effect; t is j And the network output is the feature vector of the (j + n) th image in the training set.
The minimum loss function during model initialization is:
in the formula (7), N is the total number of neuron nodes.
Step 5, performing initialization training on the sequential extreme learning machine model; constructing an initial neuron matrix H and an output vector T, solving the least square solution of the initial neuron matrix H and the output vector T to obtain a model parameter { beta [ [ beta ] ] i |i=1,2,…,L},β i Representing the ith model parameter, and L representing the total number of the model parameters, thereby completing the initial training of the model; the method comprises the following specific steps:
writing equation (6) in matrix form:
Hβ=T (8)
in formula (8), H is a neuron matrix:
solving the least square solution of the formula (9) can obtain the output weight:
β=PH T T (10)
in formula (10):
P=(H T H) -1 (11)
due to the model parameter beta i The solution of (2) adopts an analytic method, and the learning speed is high, so the method is an extreme learning machine.
Step 6, carrying out iterative optimization on the initialized sequential extreme learning machine model by utilizing a CMA-ES algorithm so as to obtain the optimal parameters of the sequential extreme learning machine model;
step 6.1, setting the minimum iteration number as eta, defining the current iteration number as g, and initializing g =1;
utilizing the input weight { W of the initialized sequential extreme learning machine model obtained in the step 5 i L =1,2, \8230;, L } and hidden layer bias { b [ ] i L =1,2, \8230 |, L } constitutes the g-th generation population;
by model parameter { beta
i L =1,2, \ 8230;, L } is used as the model parameter corresponding to the g-th generation population
Wherein,
representing model parameters corresponding to ith individuals of the g generation population, and taking the initial neuron matrix H as the g generation as the neuron matrix H
(g) ;
Step 6.2, calculating the fitness of the ith individual in the population of the g generation by using the formula (12)
Thus obtaining the fitness of each individual of the g-th generation population, and ranking each individual according to the fitness from high to low:
in the formula (12), the reaction mixture is,
is the average output vector;
is a model parameter corresponding to the ith individual in the g generation population
Calculating a predicted feature vector;
step 6.3, iteration and mutation are carried out on the g generation population by using the formula (13), so that the ith individual of the g +1 generation population is obtained
Thereby forming a g +1 generation population to update a g generation neuron matrix H
(g) And obtaining a g +1 th generation neuron matrix H
(g+1) :
In formula (13), m (g) Is the mean vector of the mu individuals with the individual fitness ranking in the g-th generation population, N i (0,C (g) ) Is a gaussian distribution function obeyed by the ith individual in the population of the g generation; epsilon (g) For the step size of the population evolution of the g generation, when g =1, the epsilon is initialized (g) =1;C (g) Is a covariance matrix of the g-th generation population;
step 6.4, calculating the mean vector m of the g +1 th generation population
(g+1) Step size epsilon of population evolution
(g+1) Model parameters
Step 6.4.1, obtaining the mean vector m of the g +1 th generation population by using the formula (14) (g+1) :
In formula (14), ω
n Is the optimized weight of the nth individual in the mu individuals before the fitness ranking, and
is the nth individual of the mu individuals in the population of the g generation;
step 6.4.2, obtaining the step size epsilon of the population evolution of the g +1 generation by using the formula (15) (g+1) :
In the formula (15), c (g) Is a step size epsilon (g) Updating parameters of (2);
step 6.4.3, according to the g +1 th generation neuron matrix H
(g+1) Obtaining model parameters corresponding to the g +1 generation population according to the process of the step 5
Step 6.5, obtaining a convergence criterion S by using the formula (16), if S is less than theta for continuous lambda times and g is more than eta, stopping iteration and obtaining a sequential extreme learning machine model of the optimal parameters, and simultaneously outputting the optimal parameters, wherein the step comprises the following steps of: optimal input weights
Optimal hidden layer biasing
And corresponding optimal model parameters
Wherein,
represents the ith optimal input weight and,
indicating the ith optimal hidden layer bias,
and (4) representing the ith optimal model parameter, otherwise, returning to the step 6.2, wherein theta is a convergence boundary:
in the formula (16), the compound represented by the formula,
is the fitness of the ith individual in the g-1 generation population, and is initialized when g =1
In summary, the flow of the CMA-ES algorithm is shown in FIG. 3.
Step 7, anomaly detection after mounting:
step 7.1, obtaining the average value sigma of the feature vectors of the training set by using the formula (17):
in the formula (17), c is the total number of the feature vectors in the training set, X j Is the j-th feature vector of the input;
step 7.2, defining the current time as t, and initializing t =0;
optimizing model parameters
As model parameters at the current time t
Representing the ith model parameter at the current time t;
obtaining a predicted feature vector of a model output at time t +1 using equation (18)
In the formula (18), the reaction mixture,
representing a hidden layer activation function;
step 7.3, predicting the feature vector according to the prediction
The prediction variance at time t +1 is obtained by equation (19)
In the formula (19), the reaction mixture is,
is the predicted variance at time t; when t =0, let
And 7.4, obtaining a detection threshold value omega by using the formula (20):
in the formula (20), T t+1 Representing the actual feature vector at time t + 1;
step 7.5, calculating the prediction distance difference phi at the t +1 moment by using the formula (21) t+1 :
If phi is t+1 If omega is larger than omega, the paster is abnormal at the time of t +1, otherwise, the paster is normal at the time of t + 1;
because the prediction model is trained by adopting a large number of PCBs with normal patches, whether the PCBs are abnormal or not can be reversely deduced.
Step 8, acquiring the patch data of the welded PCB in real time and processing the data according to the steps 1 to 3 to obtain the patch characteristic vector of the welded PCB; sequentially inputting the patch characteristic vectors of the welded PCB into a sequential extreme learning machine model with optimal parameters for on-line training so as to update the optimal parameters, thereby obtaining the sequential extreme learning machine model with strong adaptability; the new training sample is input into the model in a sequential mode, and the model parameter beta is updated in a recursion mode, and the method comprises the following specific steps:
model parameter β at time t +1 t+1 Obtained according to the following formula:
in the formulae (21) and (22),
h t+1 =[g(W i ,b i ,X j ) g(W i ,b i ,X j ) ... g(W i ,b i ,X j )] (24)
step 9, scanning the area needing to be subjected to patch detection by adopting multi-scale sliding windows, and processing the test image in the area where each sliding window is located according to the steps 1-3 to obtain a feature vector to be detected; every time sliding is carried out, the area where the window is located is scaled into an image with the resolution of 160X 90;
step 10, inputting the feature vector to be detected into a strong-adaptability sequential extreme learning machine model for anomaly detection, thereby obtaining the areas of all windows with the patch anomalies;
in the method, according to the size of a region to be detected in a picture, 10 different windows are set to scan the test picture, and the sliding step length is set to be 10 pixels;
and 11, screening all window areas with the patch abnormity, so as to obtain the positions of the patch abnormity in the test image.
When a multi-scale sliding window is adopted to detect the abnormal area of the patch, most of the adopted detection windows are larger than a target to be detected, window combination is needed, and the combination principle is as follows: and if the ratio of the intersection area of the two mutually overlapped detection windows to the smaller area of the two windows is greater than 0.5, selecting the window with higher output score.