CN107145905B

CN107145905B - Image recognition detection method for looseness of elevator fastening nut

Info

Publication number: CN107145905B
Application number: CN201710301044.1A
Authority: CN
Inventors: 唐朝伟; 胡佩; 陈冠豪; 金卓义; 章景昆; 王丹; 陈世玉; 吕艳; 尹建峰; 杨科; 马国鹏; 李显斌; 李伟全; 李忠; 谭量; 何自立; 刘向民
Original assignee: Chongqing University; Winone Elevator Co Ltd
Current assignee: Chongqing University; Winone Elevator Co Ltd
Priority date: 2017-05-02
Filing date: 2017-05-02
Publication date: 2020-04-21
Anticipated expiration: 2037-05-02
Also published as: CN107145905A

Abstract

The invention provides an image recognition and detection method for looseness of an elevator fastening nut, which comprises the following steps: s1, obtaining an optimal distance allowable threshold value and an optimal calibration matching candidate shape among nuts by training a positive sample image of a sufficiently tightened bumper bolt; s2, selecting a plurality of candidate frames through a BING algorithm, and filtering out redundant candidate frames according to color values; s3, performing edge detection on the reserved candidate frame to obtain an edge closed shape of the bolt color marking part; and S4, judging whether the bolt is loosened according to the edge geometry and the distance.

Description

Image recognition detection method for looseness of elevator fastening nut

Technical Field

The invention relates to the field of computer control, in particular to an image recognition and detection method for looseness of an elevator fastening nut.

Background

Elevators are motor-powered elevator devices that are widely used in human society for the transportation of people or goods between multi-story buildings today in modernization and high-speed. In recent years, frequent electric accidents cause social concerns, normal operation of an elevator is guaranteed, and personal safety of passengers is concerned, so that the safety and reliability operation of equipment is guaranteed by detecting and researching looseness of a bumper bolt in operation of the elevator.

The traditional bolt detection comprises two main types of manual detection and automatic detection. The manual detection means that a special worker is equipped to regularly observe whether the bolt is loosened or not by naked eyes. The method is simple and easy to implement, and does not need to depend on complex equipment. But the method has the disadvantages of low efficiency, high labor intensity and easy fatigue of detection personnel; secondly, the detection quality is difficult to guarantee due to the fact that the detection method greatly depends on professional quality and working attitude of workers.

The second broad category of methods is to automatically detect the loosening of the bolt using a device or built-in algorithm. The bolt looseness detection method for the detector is characterized in that a signal sent by the sensor is read through the detector, the driving wheel can be driven to rotate when the nut is loosened, and when the area where the metal coating on the surface of the driven gear is located rotates to the position above the sensor mounting groove, the signal sent by the sensor to the detector can be blocked, so that the looseness of a fastener can be detected.

A more common built-in algorithm uses bolt rotation angle to account for bumper bolt looseness. Generally, a continuous, vertical and clear mark is drawn at the bolt connecting part, when the bolts are loosened, the angles among the bolts are inevitably changed, and at the moment, the pictures are extracted and the angles among the marks are detected to detect whether the bolts are loosened, but the method has poor fault removal effect when the bolts just rotate an integer of turns.

Disclosure of Invention

The invention aims to at least solve the technical problems in the prior art, and particularly innovatively provides a novel

In order to achieve the above object, the present invention provides an image recognition and detection method for loosening of an elevator fastening nut, comprising:

s1, obtaining an optimal distance allowable threshold value and an optimal calibration matching candidate shape among nuts by training a positive sample image of a sufficiently tightened bumper bolt;

s2, selecting a plurality of candidate frames through a BING algorithm, and filtering out redundant candidate frames according to color values;

s3, performing edge detection on the reserved candidate frame to obtain an edge closed shape of the bolt color marking part;

and S4, judging whether the bolt is loosened according to the edge geometry and the distance.

In the image recognition and detection method for elevator fastening nut loosening, preferably, the S1 includes:

and S1-1, obtaining the tiny spacing between nuts through the positive sample image of the sufficiently tightened bumper bolt, average allowable errors of horizontal alignment and vertical alignment of the nut calibration shape, geometric and color information of the calibration pairing candidate shape, and training to obtain an optimal distance allowable threshold value and an optimal calibration pairing candidate shape.

In the image recognition and detection method for elevator fastening nut looseness, preferably, the step S2 of selecting a plurality of candidate boxes through a BING algorithm includes:

s2-1, firstly, selecting a sample image of a labeling target position in a BING algorithm to generate positive and negative samples with different scales, and scaling the size of each sample to be fixed 8 multiplied by 8; under the size, the detail characteristics such as bolt texture, local shape and the like are lost, and only the boundary gradient contour characteristics of the object are reserved; the gradient value of the object outline is higher, and the object outline is strongly contrasted with the surrounding background area;

s2-2, training a positive sample set and a negative sample set by using a linear support vector machine to obtain a target similarity detection template; a pixel size candidate box of the same size as the target object is more likely to express the target object, so the template is also guaranteed to be 8 × 8 in 64 dimensions; then, calculating and sequencing the filtering scores under each scale by using a scoring system mechanism, and eliminating local redundancy by using a Non-maximum suppression algorithm (NMS);

and S2-3, finally finding the size of the candidate frame corresponding to the scoring point in the original image and storing the size.

Preferably, the image recognition and detection method for detecting loosening of elevator fastening nut, in which S2-3 includes:

a: BING calculates gradient feature for each size sample, and obtains a group of candidate frames with filtering score according to gradient feature matching, and the filtering score s_lThe definition is as follows:

s_(i,x,y)＝＜w,G_(i,x,y)＞，

where w is the model parameter obtained from training, G_(i,x,y)Is an NG (normalized gradients) feature, symbol, at a small scale scaled to the image frame of position coordinates (x, y) of scale i "<，>"denotes inner product operation; w and G are both matrixes;

and for the gradient amplitude calculation of each pixel, obtaining by adopting a Prewitt operator mode:

g_(i,x,y)＝min{|g_x|+|g_yi, 255, where g (i, x, y) represents the bits of scale iSet coordinate (x, y)

Magnitude of gradient per pixel, g_xPixel gradient amplitude, g, in X coordinate_yPixel gradient magnitude for the Y coordinate;

since the filtering scores may be biased by different sizes, when reordering the final candidate scores, calibration is required to obtain candidate window group scores with different final sizes, and the objective score o is defined_lThe following were used:

o_(i,x,y)＝v_is_(i,x,y)+t_i，

wherein v is_iIs the independent learning coefficient, t, at the size i_iAn offset representing a dimension i;

the obtained candidate box not only has the highest score in the global scope, but also has the highest score in the neighborhood, so as to avoid the phenomenon that the face is missed to be detected because the score of the highest window is too concentrated in a certain area; therefore, the non-maximum constraint algorithm is used for selecting the edge points, so that the final candidate window has a global higher score and a neighborhood with the highest score.

In the image recognition and detection method for elevator fastening nut loosening, preferably, the S2 includes:

in order to improve the calculation speed and accelerate the feature extraction and test process, the specific steps of the template and feature binarization approximation process are as follows:

characteristic binarization: because the gradient amplitude value ranges [0, 255], binary bit stream replacement is carried out by using the following formula mode;

wherein, b_kE {0,1} represents the k-th bit binary value, N_gThe number of the selected binary digits is represented, and the influence of the latter four digits on the gradient is not great;

jointly representing the k-th bit corresponding to each 8 x 8 point as a binary stream b_k∈{0,1}⁶⁴(ii) a Thus, corresponding to the 8 × 8 matrix area:

template binarization: for the model parameters obtained by training, w is regarded as the combination of a plurality of basis vectors and is approximately expressed as

Wherein, β_jCoefficient representing the jth basis vector, a_jDenotes the jth basis vector, a_j∈{-1,1}⁶⁴，N_wIndicates the number of basis vectors, a_jConversion to [0,1]The range is as follows:

therefore, w is represented as:

formula s_(i,x,y)＝＜w,G_(i,x,y)The binarization equation for > is expressed as follows:

wherein

Is referred to as a_jNegative values in the vector are set to zero, and positive values are unchanged.

In the image recognition and detection method for the looseness of the elevator fastening nut, preferably, the process of the template w includes:

inputting: the basis vectors of the initialization are,

and (3) outputting: the number of the templates w is,

6:for j＝1to N_wdo

7:a_j＝sign(ε)

8:

9:ε＝ε-β_ja_j

10:w＝w+β_ja_j

the image recognition and detection method for the looseness of the elevator fastening nut preferably further comprises the following steps:

in order to eliminate redundant selection frames, the method is specifically implemented as follows:

(1) sorting the scores of all the frames in descending order, selecting the highest score and the frame corresponding to the highest score,

(2) the remaining boxes are traversed and, if the area of overlap (IOU) with the current highest sub-box is greater than a certain threshold, we delete the box,

(3) and continuing to select one with the highest score from the unprocessed boxes, and repeating the process.

Preferably, the image recognition and detection method for detecting loosening of an elevator fastening nut, in which S3 further includes:

s3-1, performing edge detection on the candidate region by using Canny, performing convolution by using a 2D Gaussian filtering template to eliminate image noise, calculating the gradient magnitude and direction of each pixel point in the filtered image, finding an adjacent pixel in the gradient direction of the pixel through the gradient direction, finally obtaining a bolt edge closed shape curve through non-maximum suppression, thresholding and edge linking, and calculating the area of the edge closed shape;

s3-2, the algorithm process is as follows:

the gaussian kernel with size 5 used in convolutional noise reduction using a 2D gaussian filter template is as follows:

calculating the amplitude and direction of the pixel points, wherein the steps of the Sobel filter are introduced:

a. using a pair of convolution arrays, acting on the x, y direction, respectively

b. The gradient values and directions are calculated using the following formulas:

the gradient direction is approximated to one of four possible angles;

detecting a maximum value point, namely an edge point, of the modulus value along the amplitude direction, traversing pixel points, comparing the partial derivative value of each pixel with the modulus value of an adjacent pixel, taking the maximum value as the edge point, and setting the gray value of the pixel to be 0;

s3-3, detecting and connecting edges by using a double-purpose double-threshold algorithm:

a. two thresholds th for non-maximum suppressed images₁And th₂The relationship th between them₁＝0.4th₂(ii) a We set the gradient value less than th₁The gray value of the pixel of (1) is set to 0 to obtain an image 1; then the gradient value is less than th₂The gray value of the pixel of (2) is set to 0, and an image 2 is obtained; most of the noise is removed due to the higher threshold of image 2, but at the same time the useful edge information is also lost; the threshold of the image 1 is lower, more information is reserved, and the edge of the image can be connected by taking the image 1 as a supplement on the basis of the image 2;

b. the specific steps of linking the edges are as follows:

scanning the image 2, and tracking a contour line taking p (x, y) as a starting point until an end point q (x, y) of the contour line when encountering a pixel p (x, y) with non-zero gray; examining 8 neighboring regions of a point s (x, y) in the image 1 corresponding to the position of the q (x, y) point in the image 2; if a non-zero pixel s (x, y) exists in an 8-neighborhood of the s (x, y) point, it is included in the image 2 as an r (x, y) point; starting from r (x, y), the first step is repeated until we cannot continue in both image 1 and image 2;

after completing the concatenation of the contour line containing p (x, y), marking this contour line as visited; returning to the first step, and searching a next contour line; repeating the first step, the second step and the third step until no new contour line can be found in the image 2;

and finishing the edge detection of the canny operator to obtain the closed shapes of the two edges of the red marked part of the bolt.

In the image recognition and detection method for elevator fastening nut loosening, preferably, the S4 includes:

s4-1, calculating the edge closed shapes of the two red marked parts of the bolt, comparing the edge closed shapes with the closed shapes in the positive sample, and when the area of the closed shapes is more different from that of the positive sample or only one closed shape is provided, considering that the bolt is loosened;

s4-2, on the premise of meeting the geometric shape, judging whether the distance between the two closed shapes is within the allowed threshold range, if so, determining that the bolt is not loosened, and if not, determining that the bolt is loosened.

In summary, due to the adoption of the technical scheme, the invention has the beneficial effects that:

whether the bolt loosens or not is judged from the two aspects of the angle and the geometric shape, and the detection accuracy is further improved.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a flow chart of the method for detecting bolt looseness of a bumper during the operation of an elevator according to the present invention;

FIG. 2 is a schematic view showing the bolt tightening and loosening of the detection method of the bolt loosening of the lever for the elevator in operation according to the present invention;

FIG. 3 is a diagram of the steps for performing edge detection using the canny algorithm;

FIG. 4A is a schematic diagram of an edge direction, and FIG. 4B is a schematic diagram of an 8-neighborhood argument direction;

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, it is to be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.

In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.

The invention provides a detection method for bolt looseness of a safety lever in elevator running, which innovatively solves the problem of inaccurate bolt looseness identification from the perspective of distance and geometric shape.

In order to achieve the above object, the present invention provides a method for detecting bolt looseness of a bumper during elevator operation, comprising:

step 1: obtaining an optimal distance allowable threshold value and an optimal calibration matching candidate shape among all nuts by training a positive sample image of a sufficiently tightened bumper bolt;

step 2: and selecting a plurality of candidate frames through a BING algorithm, and filtering redundant candidate frames according to the color values.

And step 3: performing edge detection on the reserved candidate frame to obtain an edge closed shape of the color marking part of the bolt;

and 4, step 4: and judging whether the bolt is loosened or not according to the edge geometric shape and the distance.

The beneficial effects of the above technical scheme are: the bolt detection method provided by the invention detects the bolt looseness comprehensively from two standards of geometric shape and distance, has higher accuracy, can automatically detect, saves time and labor, improves the pertinence of bolt looseness detection, and ensures the safe and reliable operation of equipment; the invention can detect whether the bolt is loosened in real time only by regularly shooting the bolt picture and transmitting the picture to the processing end without adding any equipment, and can give an alarm in time.

Preferably, the method for detecting loosening of the safety lever bolt in the operation of the elevator comprises the following steps of 1:

step 1-1, obtaining a tiny space between nuts through a positive sample image of a sufficiently tightened bumper bolt, obtaining average allowable errors of horizontal alignment and vertical alignment of a nut calibration shape, and obtaining optimal distance allowable threshold and optimal calibration matching candidate shapes through training according to geometric and color information of calibration matching candidate shapes;

the technical method has the beneficial effects that: the optimal allowable threshold value and the optimal calibration matching candidate shape are obtained by training the positive sample, so that more accurate judgment and marking are provided for bolt loosening, and the precision ratio and the recall ratio of bolt loosening are improved.

Preferably, the method for detecting loosening of the safety lever bolt in the operation of the elevator comprises the following steps of:

step 2-1, detecting a bolt candidate frame area by using a BING algorithm,

(1) in the BING algorithm, firstly, a sample image of a labeling target position is selected to generate positive and negative samples with different scales, and the size of each sample is scaled to be fixed 8 multiplied by 8. At this size, detailed features such as bolt texture, local shape, etc. will be lost, and only the boundary gradient contour feature of the object is retained. The object profile gradient is higher in value and strongly contrasts with its surrounding background area.

(2) And secondly, training a positive sample set and a negative sample set by using a linear support vector machine to obtain a target similarity detection template. A pixel size candidate box of the same size as the target object is more likely to represent the target object, so the template is also guaranteed to be 8 x 8 in size of 64 dimensions. And then, calculating and sequencing the filtering scores at each scale by using a scoring system mechanism, and eliminating local redundancy by using a Non-maximum suppression (NMS) algorithm.

(3) And finally, finding the size of the candidate frame corresponding to the scoring point in the original image and storing the size.

s_(i,x,y)＝＜w,G_(i,x,y)＞

wherein w is a model parameter (hereinafter referred to as a template w) obtained by training, G_(i,x,y)Scaling an image frame with dimension i and position coordinates (x, y) to an NG feature at small size, here 8 x 8 size. And symbol "<，>"denotes inner product operation.

For the gradient magnitude calculation of each pixel, we use the Prewitt operator approach described in the second chapter to obtain:

g_(i,x,y)＝min{|g_x|+|g_y|,255}

since the filtering scores may be biased by size, for example, the probability of a face being present in a 10 × 80 window is much smaller than in a 30 × 30 window. Therefore, when reordering the final candidate scores, calibration is needed to obtain the final candidate window group scores with different sizes, and the objective score o is defined_lThe following were used:

o_(i,x,y)＝v_is_(i,x,y)+t_i，

wherein v is_iIs the independent learning coefficient and t at the size i_iIndicating the offset of dimension i.

The obtained candidate box should not only have the highest score in the global scope, but also have the highest score in the neighborhood, so as to avoid the phenomenon that the face is missed because the score of the highest window is too concentrated in a certain area. Therefore, the non-maximum constraint algorithm is used for selecting the edge points, so that the final candidate window has a global higher score and a neighborhood with the highest score.

1. characteristic binarization: since the gradient magnitude value ranges from 0, 255, the binary bit stream replacement can be performed using the following formula.

Wherein, b_kE {0,1} represents the k-th bit binary value, N_gRepresenting the number of selected binary bits, where N is a further acceleration since the latter four bits do not have much influence on the gradient_gOnly the upper four digits are retained for 4. Taking the value 122 as an example, 01111010 is obtained after binarization, and 0111 is reserved as the upper four-digit value.

Extending to the 8 x 8 region, there are 64 gradient amplitudes that coexist, and a binary bit stream is available for each point. The k-th bit corresponding to each 8 x 8 dot is now jointly represented as a binary stream b_k∈{0,1}⁶⁴. Thus, corresponding to the 8 × 8 matrix area:

2. template binarization: for the template w, it can be seen as a combination of a plurality of basis vectors, approximately expressed as

Wherein, β_jCoefficient representing the jth basis vector, a_jDenotes the jth basis vector, a_j∈{-1,1}⁶⁴，N_wIndicates the number of basis vectors, which is 2. For convenient treatment, a_jConversion to [0,1]The range is as follows:

therefore w can be expressed as:

the specific calculation flow of the template w is as follows:

in summary, the formula s_(i,x,y)＝＜w,G_(i,x,y)The binarization equation for > is expressed as follows:

b: the application of non-maximum suppression is very wide, and in order to eliminate redundant selection boxes, the following specific implementation is implemented:

The technical method has the beneficial effects that: all regions that are close to the positive sample gradient value can be marked using the BING algorithm and a portion of the regions that do not meet the requirements can be eliminated in conjunction with NMS.

Step 2-2: filtering a part of candidate frames according to the color value range of the marks in the bolts to obtain the best candidate frame containing the bolts;

in an actual scene, a clear and vertical red strip-shaped frame with the width of about 1 cm needs to be drawn at the bolt connecting part, the red strip-shaped frame can be kept fadeless for a long time, the marking operation is simple, the application cost is low, and a great auxiliary effect can be realized on the following automatic identification. In this experiment, the RGB tristimulus values of the red-labeled region are 119.37, 43.45 and 53.73, respectively, so we can define that the candidate frame is considered to contain the bolt region only when the object RGB in the candidate frame satisfies the tristimulus range.

The RGB three-color range of the red marked area on the bolt is as follows:

according to the RGB three-color range value, the candidate frame which does not contain the color information can be filtered out, and the best candidate frame is obtained.

The technical method has the beneficial effects that: the candidate frame obtained in step 2 is only the area with similar gradient value to the positive sample, and contains the area of the bolt part, so the method of sampling the color mark can further filter out the area with non-conforming color information.

and 3-1, performing edge detection on the candidate region by using Canny, performing convolution by using a 2D Gaussian filtering template to eliminate image noise, calculating the gradient magnitude and direction of each pixel point in the filtered image, finding an adjacent pixel in the gradient direction of the pixel through the gradient direction, finally obtaining a bolt edge closed shape curve through non-maximum inhibition, thresholding and edge linking, and calculating the area of the edge closed shape.

The algorithm process is as follows:

(1) the gaussian kernel with size 5 used in convolutional noise reduction using a 2D gaussian filter template is as follows:

(2) calculating the amplitude and direction of the pixel points, wherein the steps of the Sobel filter are introduced:

a. using a pair of convolution arrays (acting in x, y directions respectively)

the gradient direction approximates to one of four possible angles (typically 0, 45, 90, 135 degrees)

(3) Obtaining only global gradients is not sufficient to determine edges, so to determine edges, the point at which the local gradient is maximal must be preserved. The essence of the Non-maximum suppression algorithm (NMS) is to search for local maxima and suppress Non-maxima elements.

as shown in fig. 4, the maximum value point, i.e., the edge point, of the modulus value is detected along the argument direction, 8 direction pixel points are traversed, the partial derivative value of each pixel is compared with the modulus value of the adjacent pixel, the maximum value is taken as the edge point, and the gray value of the pixel is set to be 0.

(4) Detecting and connecting edges with a dual-purpose dual-threshold algorithm:

a. two thresholds th for non-maximum suppressed images₁And th₂The relationship th between them₁＝0.4th₂. We set the gradient value less than th₁The gradation value of the pixel of (2) is set to 0, and an image 1 is obtained. Then the gradient value is less than th₂The gradation value of the pixel of (2) is set to 0, and an image 2 is obtained. Since the threshold of image 2 is high, most of the noise is removed, but at the same time useful edge information is also lost. While the threshold for image 1 is lower, and more information is retained, we can connect the edges of the images based on image 2 and supplemented with image 1.

b. The specific steps of linking the edges are as follows:

the image 2 is scanned and when a non-zero gray pixel p (x, y) is encountered, the contour line starting at p (x, y) is traced until the end point q (x, y) of the contour line. Consider the 8 neighborhood of point s (x, y) in image 1 corresponding to the position of the q (x, y) point in image 2. If a non-zero pixel s (x, y) exists in an 8-neighborhood of the s (x, y) point, it is included in the image 2 as the r (x, y) point. Starting from r (x, y), the first step is repeated until we cannot continue in both image 1 and image 2.

When the concatenation of the contour line containing p (x, y) is completed, this contour line is marked as visited. And returning to the first step, and searching the next contour line. And repeating the first step, the second step and the third step until no new contour line can be found in the image 2.

The technical method has the beneficial effects that: when the color information in the picture is similar to the color information marked on the bolt, the sample containing the bolt area cannot be judged well. Therefore, the edge geometry of the object included in the candidate frame is calculated by combining with the edge detection and compared with the geometry included in the positive sample, so as to further filter out the candidate frame which does not satisfy the condition, and finally find the area where the bolt is located.

and 4-1, calculating the edge closed shapes of the two red marked parts of the bolt, comparing the edge closed shapes with the closed shapes in the positive sample, and when the area of the closed shapes is more different from that of the positive sample or only one closed shape is provided, determining that the bolt is loosened.

And 4-2, judging whether the intermediate distance between the two closed shapes is within an allowed threshold range on the premise of meeting the geometric shape, if so, determining that the bolt is not loosened, and if not, determining that the bolt is loosened.

The technical method has the beneficial effects that: whether the bolt is loosened or not is judged in the aspects of geometric angle and distance, the probability of false detection and missed detection can be reduced to a certain extent, and therefore the engineering application value of the whole algorithm is improved.

(1) a detection program is built in, so that the bolt loosening condition can be automatically detected, the low efficiency of manual detection is avoided, an alarm can be given at the first time when the bolt loosening is detected, and the potential safety hazard caused by the bolt loosening is avoided;

(2) according to the scheme, other auxiliary equipment does not need to be purchased, and only the program is required to be built in the single chip microcomputer, so that the trouble and labor are saved, and the scheme is economic.

(3) The bolt looseness detection algorithm can detect the looseness of the bolt from two aspects of geometric shape and distance, can detect the looseness with high probability, and has low omission factor and false detection.

The invention discloses a method for detecting looseness of a bolt of a bumper in elevator running, which comprises the following steps:

As shown in fig. 1, when the photographing mode is adopted, the bolt looseness measuring method based on the geometric shape and the center distance comprises the following steps:

and S1, calculating the geometric shape of the bolt area of the positive sample picture and the center distance of the two marked parts, and determining the geometric shape and the distance threshold.

And S2, erecting a fixed camera on the elevator shaft wall or the pipe shaft, and continuously shooting pictures of the area where the bolt is located.

Usually, the position of the bolt in the elevator is fixed and unchangeable, and the picture that the camera was shot has comparatively fixed background, and this has the auxiliary effect to following automated inspection. The invention comprehensively considers the factors of floor height, elevator moving speed, camera view angle width, camera frame rate, terminal storage capacity and the like, and selects proper time intervals for shooting.

And S3, performing correlation processing on the shot picture by using a BING algorithm, and extracting a candidate box meeting the correlation requirement.

In order to better detect the loosening of the bolt, firstly, a positive sample picture without loosening of the bolt is input, relevant information of the area is obtained, secondly, the picture to be detected is sent to a processor, and a candidate frame similar to the positive sample information is selected by utilizing a BING algorithm. After the detection is finished, a candidate frame with lower confidence coefficient is filtered out by combining a non-maximum suppression algorithm, so that the subsequent detection is ensured to be carried out smoothly.

S4: the candidate frames obtained in the previous step are usually more, so that a part of the candidate frames are further filtered out by using the color information marked by the bolts.

Firstly, acquiring an RGB average value of a red mark part of the bolt in a positive sample, and setting different TGB value ranges according to the value; and then calculating the RGB color information of the object contained in each candidate frame in the picture to be detected, if the RGB value of the object contained in the candidate frame is in the set range, reserving the candidate frame, and if not, rejecting the candidate frame.

S5, the edge shape of the mark area in the candidate box is retained, and the geometric information is calculated.

The candidate frames retained in the last two steps are consistent with the positive sample in terms of gradient and color information, but when an object with a color similar to that of the marked part appears in the background picture, the subsequent identification work is interfered. And detecting the to-be-detected picture marked with the candidate frames, carrying out edge detection on the object in the candidate frames by using a canny algorithm to obtain the edge closed shape of the object, and calculating the geometric information of the object.

S6: and judging whether the calculated geometric information accords with the geometric information of the positive sample or not, if not, indicating that the bolt is loosened, if so, further judging whether the middle distance between the two closed areas is smaller than a set related threshold value, if so, also considering that the bolt is loosened, and alarming at the first time. If both of the above conditions are satisfied, we consider the bolt to be operating normally.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An image recognition detection method for looseness of an elevator fastening nut is characterized by comprising the following steps:

s3, performing edge detection on the reserved candidate frame to obtain the edge geometry of the color marking part of the bolt;

2. The image recognition detecting method of elevator fastening nut loosening according to claim 1, wherein said S1 includes:

and S1-1, obtaining the tiny spacing between nuts through the positive sample image of the sufficiently tightened bumper bolt, average allowable errors of horizontal alignment and vertical alignment of the nut calibration shape, and geometric shape and color information of the calibration pairing candidate, and training to obtain an optimal distance allowable threshold value and an optimal calibration pairing candidate shape.

3. The image recognition detection method of elevator cage nut loosening as claimed in claim 1, wherein said S2 selecting candidate boxes by the BING algorithm comprises:

s2-1, firstly, selecting a sample image of a labeling target position in a BING algorithm to generate positive and negative samples with different scales, and scaling the size of each sample to be fixed 8 multiplied by 8; under the size, the bolt texture and the local shape detail features are lost, and only the boundary gradient contour features of the object are reserved; the gradient value of the object outline is higher, and the object outline is strongly contrasted with the surrounding background area;

4. The image recognition detection method of elevator cage nut loosening according to claim 3, wherein said S2-3 includes:

a: BING calculates gradient feature for each size sample, matches to obtain a group of candidate frames with filtering score according to the gradient feature, and the filtering score S_(i,x,y)The definition is as follows:

S_(i,x,y)＝<W,G_(i,x,y)>，

wherein W is a model parameter obtained by training, G_(i,x,y)Is an NG (normalized gradients) feature, symbol, at a small scale scaled to the image frame of position coordinates (x, y) of scale i "<，>"denotes inner product operation; w and G are both matrixes;

g_(i,x,y)＝min{|g_x|+|g_yl, 255}, where g_(i,x,y)Each pixel gradient magnitude, g, is represented by a position coordinate (x, y) of scale i_xPixel gradient amplitude, g, in X coordinate_yPixel gradient magnitude for the Y coordinate;

since the filtering scores may be biased by different sizes, when reordering the final candidate scores, calibration is required to obtain candidate window group scores with different final sizes, and the objective score o is defined_(i,x,y)The following were used:

o_(i,x,y)＝V_iS_(i,x,y)+t_i,

wherein, V_iIs the independent learning coefficient, t, at the size i_iIndicating the offset of dimension i.

5. The image recognition detecting method of elevator fastening nut loosening according to claim 1, wherein said S2 includes:

template binarization: for the model parameters obtained by training W, the model parameters are regarded as the combination of a plurality of basis vectors and are approximately expressed as

therefore, W is represented as:

formula S_(i,x,y)＝<W,G_(i,x,y)>The binarization formula of (a) is expressed as follows:

wherein the content of the first and second substances,

6. The image recognition detecting method for loosening of elevator fastening nut as claimed in claim 5, wherein the construction process of the template W comprises:

inputting: the basis vectors of the initialization are,

and (3) outputting: the number of the die plates W is,

1:for j＝1to N_Wdo

2:a_j＝sign(ε)

3:

4：ε＝ε-β_ja_j

5：W＝W+β_ja_j。

7. the method of image recognition and detection of elevator fastening nut loosening of claim 5, further comprising:

(2) traversing the rest of the boxes, if the overlapping area (IOU) with the current highest sub-box is larger than a certain threshold, deleting the boxes,

8. The image recognition detecting method of elevator fastening nut loosening according to claim 1, wherein said S3 further comprises:

s3-1, performing edge detection on the candidate region by using Canny, performing convolution by using a 2D Gaussian filtering template to eliminate image noise, calculating the gradient size and direction of each pixel point in the filtered image, finding an adjacent pixel in the gradient direction of the pixel through the gradient direction, finally obtaining a bolt edge closed shape curve through non-maximum inhibition, thresholding and edge linking, and calculating the area of the edge closed shape;

s3-2, the algorithm process is as follows:

the gradient direction is approximated to one of four possible angles;

s3-3, detecting and connecting edges by using a double-threshold algorithm:

b. the specific steps for connecting the edges are as follows:

9. The image recognition detecting method of elevator fastening nut loosening according to claim 1, wherein said S4 includes: