CN112365461A - Fastener loosening identification method, system, terminal and storage medium - Google Patents
Fastener loosening identification method, system, terminal and storage medium Download PDFInfo
- Publication number
- CN112365461A CN112365461A CN202011227065.1A CN202011227065A CN112365461A CN 112365461 A CN112365461 A CN 112365461A CN 202011227065 A CN202011227065 A CN 202011227065A CN 112365461 A CN112365461 A CN 112365461A
- Authority
- CN
- China
- Prior art keywords
- fastener
- point cloud
- cloud data
- mask
- detection frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000003860 storage Methods 0.000 title claims abstract description 16
- 238000001514 detection method Methods 0.000 claims abstract description 32
- 238000004590 computer program Methods 0.000 claims description 22
- 238000000513 principal component analysis Methods 0.000 claims description 12
- 238000013135 deep learning Methods 0.000 claims description 8
- 238000003709 image segmentation Methods 0.000 claims description 6
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 229910052742 iron Inorganic materials 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides a fastener loosening identification method, a fastener loosening identification system, a terminal and a storage medium, and relates to a quality detection technology. The fastener loosening identification method comprises the following steps: expanding and dividing the obtained fastener detection frame to obtain a mask of the mark lines of the fastener; extracting corresponding depth values from the depth map according to the mask of the marking lines of the fastener and obtaining corresponding point cloud data according to the internal parameters of the camera; determining that the fastener is loose when there is a discontinuity in the point cloud data and the primary directions are different. In the embodiment of the application, the spatial information of the fastener is more perfect, the accuracy is higher compared with the judgment result based on the two-dimensional image, the manual inspection and identification can be replaced, the efficiency is higher, and the data can be rechecked.
Description
Technical Field
The present disclosure relates to quality detection technologies, and in particular, to a method, a system, a terminal, and a storage medium for identifying loosening of a fastener.
Background
In a rail vehicle such as a high-speed rail or a subway, a plurality of members are generally connected by a fastener such as a bolt. In order to prevent safety accidents caused by loosening of the fastening pieces, a loosening prevention piece wound with 8-shaped iron wires or the like is usually inserted between the two fastening pieces. Factors such as vibration that produce among the rail vehicle operation process lead to locking parts such as iron wire very easily to split, and then lead to fasteners such as bolt still to have the risk of becoming flexible.
In the related art, it is common to manually check to determine whether the fastener is loose. Specifically, a marking line is drawn on a fastened bolt, an inspector visually observes whether the marking line is misaligned, and when the marking line is observed to be misaligned, it is determined that the fastener has loosened. However, with manual observation, workers are easily fatigued and have missed their inspections.
Disclosure of Invention
In order to solve one of the technical defects, embodiments of the present application provide a method, a system, a terminal and a storage medium for identifying fastener loosening.
The embodiment of the first aspect of the application provides a fastener loosening identification method, which comprises the following steps:
expanding and dividing the obtained fastener detection frame to obtain a mask of the mark lines of the fastener;
extracting corresponding depth values from the depth map according to the mask of the marking lines of the fastener and obtaining corresponding point cloud data according to the internal parameters of the camera;
determining that the fastener is loose when there is a discontinuity in the point cloud data and the primary directions are different.
An embodiment of a second aspect of the present application provides a fastener loosening identification system, including:
the first processing module is used for expanding and dividing the obtained fastener detection frame to obtain a mask of the mark line of the fastener;
the second processing module is used for extracting corresponding depth values from the depth map according to the mask of the marking lines of the fastener and obtaining corresponding point cloud data according to the internal reference of the camera;
and the third processing module is used for determining that the fastening piece is loosened when the point cloud data is disconnected and the main directions are different.
An embodiment of a third aspect of the present application provides a terminal, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement a method as claimed in any preceding claim.
A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement a method as claimed in any preceding claim.
The embodiment of the application provides a method, a system, a terminal and a storage medium for identifying loosening of fasteners. In the embodiment of the application, the spatial information of the fastener is more perfect, the accuracy is higher compared with the judgment result based on the two-dimensional image, the manual inspection and identification can be replaced, the efficiency is higher, and the data can be rechecked.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a schematic flow chart of a method provided in an exemplary embodiment;
FIG. 2 is a schematic flow chart of a method provided in an exemplary embodiment;
FIG. 3 is a schematic block diagram of a system provided in an exemplary embodiment;
FIG. 4a is a schematic illustration of an exemplary embodiment provided with a bolt not loosened;
fig. 4b is a schematic diagram of an exemplary embodiment providing a loosened bolt.
Detailed Description
In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following further detailed description of the exemplary embodiments of the present application with reference to the accompanying drawings makes it clear that the described embodiments are only a part of the embodiments of the present application, and are not exhaustive of all embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
In the related art, it is common to manually check to determine whether the fastener is loose. Specifically, a marking line is drawn on a fastened bolt, an inspector visually observes whether the marking line is misaligned, and when the marking line is observed to be misaligned, it is determined that the fastener has loosened. However, with manual observation, workers are easily fatigued and have missed their inspections.
In addition, there are also some ways to determine whether the marker line is in a straight line based on the two-dimensional image. However, due to the reasons that the direction of the mark line in the two-dimensional image is not fixed or the photographing angle is not good, it is impossible to accurately determine whether the mark line is misaligned in the two-dimensional image.
In order to overcome at least one of the above problems, embodiments of the present application provide a method, a system, a terminal, and a storage medium for identifying loosening of a fastener, where images acquired based on a grayscale camera and a depth camera automatically determine whether the fastener is loosened in a three-dimensional space, so that spatial information of the fastener is more complete, accuracy is higher compared with a determination result based on a two-dimensional image, inspection and identification can be completed instead of manual work, efficiency is higher, and data can be reviewed.
The following describes functions and implementation processes of a method, a system, a terminal and a storage medium for identifying fastener loosening provided by the embodiments of the present application with reference to the accompanying drawings.
As shown in fig. 1, the method for identifying a loose fastener provided by this embodiment includes:
s101, expanding and dividing the obtained fastener detection frame to obtain a mask of the mark line of the fastener;
s102, extracting corresponding depth values from the depth map according to a mask of a marking line of the fastener and obtaining corresponding point cloud data according to camera internal parameters;
s103, determining that the fastening piece is loosened when the point cloud data is disconnected and the main directions are different.
Before step S101, as shown in fig. 2, a detection frame of the fastener needs to be acquired. Specifically, the method comprises the following steps:
acquiring a gray scale image and a depth image which are respectively acquired by an aligned gray scale camera and a depth camera;
and detecting and positioning the fastener according to the gray-scale image and the depth image to obtain a fastener detection frame.
The method comprises the following steps of shooting by using a camera with gray scale and depth alignment, and completing segmentation and positioning of fastener marking lines based on a gray scale image; the depth camera can acquire three-dimensional point cloud data, and the judgment is more accurate compared with two-dimensional images.
And finishing the detection and positioning of the position of the fastener based on the gray-scale image to obtain a fastener detection frame. Specifically, a convolutional neural network based on deep learning, such as a fast r-cnn target detection network, can be adopted to output position information [ x1, y1, w, h ] of a plurality of fasteners in a graph, wherein x1 and y1 are coordinates of the upper left corner of the fastener, and w and h are width and height of the fastener; and obtaining a detection frame of each fastener according to the position information of the plurality of fasteners.
In step S101, the method specifically includes:
expanding the obtained fastener detection frame, and cutting according to a preset area;
and inputting the cut region into an image segmentation network based on deep learning to obtain a mask of the marking lines of the fastener.
The fastener marking lines are generally within a certain range of the fastener detection frame, so that the fastener detection frame can be subjected to certain range of edge expansion, preset regions such as regions with the sizes of [ x1-w/2, y1-h/2,2w,2h ] are cut, and the cut regions are sent to an image segmentation network based on deep learning, such as mask r-cnn, so that a mask of the fastener marking lines is obtained.
In step S102, a corresponding depth value is extracted from the depth map according to the mask result, and corresponding three-dimensional point cloud data is obtained according to the camera internal parameters.
In step S103, connectivity analysis and PCA are performed on the point cloud data, and whether the fastener is loose is determined according to the analysis result.
When all points in the point cloud data are communicated, the marking lines are not dislocated, and the fastening piece is not loosened.
When the point cloud data is determined to have disconnected points, PCA is carried out on the disconnected points; when the resulting primary direction is not in a straight line, fastener loosening is determined, as shown in FIG. 4b, with curved segments L2 and L3 in FIG. 4b being used to illustrate two sections of bolt identification lines that are not in a straight line.
When it is determined that there are disconnected points in the point cloud data and the obtained main directions are on the same straight line, it is indicated that the fastener marking line is on the straight line and the fastener is not loosened, as shown in fig. 4a, a curved line segment L1 in fig. 4a is used for illustrating the bolt identification line.
In the concrete implementation, after the disconnected points are determined, the main directions of x, y and z are solved by the disconnected points through a PCA algorithm; if the main directions of the points which are not communicated are the same, the marking lines of the fastening pieces are on the same straight line, and the fastening pieces are not loosened; if the main directions of the disconnected points are not on a straight line, the fastener loosens.
Respectively acquiring nearest neighbor Euclidean distances between each point and the rest points in the point cloud data; and when the nearest neighbor Euclidean distance is larger than the threshold value, determining that a disconnected point exists.
The following description will not be made by taking the fastener as a bolt as an example.
As shown in fig. 2, a grayscale map and a depth map are collected. And shooting by using a camera with gray scale and depth alignment to obtain an image about the bolt.
And (3) a convolutional neural network bolt detection model based on a gray scale map. Based on the acquired gray-scale map, the detection and positioning of the bolt position can be completed, and a convolutional neural network based on deep learning, such as a fast r-cnn target detection network, can be adopted to output position information [ x1, y1, w, h ] of a plurality of bolts in a map, wherein x1 and y1 are coordinates of the upper left corner of the bolt, and w and h are width and height of the bolt.
And expanding the bolt segmentation model based on the detection result. The bolt marking line is generally within a certain range of the bolt detection frame, so that the bolt detection frame can be subjected to certain range of edge expansion, the region with the size of [ x1-w/2, y1-h/2,2w,2h ] is cut, and the cut region is sent to an image segmentation network based on deep learning, such as mask r-cnn, so as to obtain the mask of the bolt marking line.
And extracting point clouds of the marking lines from the depth map based on the segmentation result. And extracting corresponding depth values from the depth map according to the mask result of the last step and obtaining corresponding three-dimensional point cloud data according to the camera internal parameters.
And judging whether the bolt mark lines are communicated or not based on the point cloud. Performing connectivity judgment on the point cloud, and if the connectivity shows that the marking lines are not dislocated, the bolt is not loosened, as shown in fig. 4 a; if not, respectively solving the x, y and z main directions of the point clouds which are not communicated through a PCA algorithm, and if the main directions are the same, indicating that the bolt marking lines are on a straight line, the bolt is not loosened as shown in FIG. 4 a; if the primary direction is not in line, the bolt identification line is not in line, as shown in FIG. 4b, the bolt is loose.
According to the fastener loosening identification method provided by the embodiment of the application, based on the acquired images of the gray camera and the depth camera, the depth value corresponding to the mask of the mark line of the fastener is extracted, corresponding point cloud data is obtained according to internal parameters of the camera, connectivity analysis and PCA are carried out on the point cloud data, and the loosened fastener is automatically identified according to the analysis result. According to the method, the spatial information of the fastener is more perfect, the accuracy is higher compared with the judgment result based on the two-dimensional image, the manual inspection and identification can be replaced, the efficiency is higher, and the data can be verified again.
The fastener loosening identification system provided by the embodiment is a product embodiment corresponding to the method embodiment, and the function and implementation process of the fastener loosening identification system are the same as or similar to those of the previous embodiment, and the description of the embodiment is omitted here.
As shown in fig. 3, the present embodiment provides a fastener loosening recognition system, including:
the first processing module 11 is configured to expand and divide the acquired fastener detection frame to obtain a mask of a mark line of the fastener;
the second processing module 12 is configured to extract a corresponding depth value from the depth map according to a mask of a mark line of the fastener and obtain corresponding point cloud data according to camera internal parameters;
and the third processing module 13 is used for determining that the fastening piece is loosened when the point cloud data has no connection and the main directions are different.
In one possible implementation manner, the first processing module 11 is specifically configured to:
expanding the obtained fastener detection frame, and cutting according to a preset area;
and inputting the cut region into an image segmentation network based on deep learning to obtain a mask of the marking lines of the fastener.
In one possible implementation manner, the third processing module 13 is specifically configured to:
when it is determined that there are disconnected points in the point cloud data, performing Principal Component Analysis (PCA) on the disconnected points;
determining that the fastener is loose when the resulting primary directions are not in line.
In one possible implementation manner, the third processing module 13 is specifically configured to:
respectively obtaining nearest neighbor Euclidean distances between each point and the rest points in the point cloud data;
and when the nearest neighbor Euclidean distance is larger than the threshold value, determining that a disconnected point exists.
In one possible implementation manner, the first processing module 11 is further configured to:
acquiring a gray scale image and a depth image which are respectively acquired by an aligned gray scale camera and a depth camera;
and detecting and positioning the fastener according to the gray-scale image and the depth image to obtain a fastener detection frame.
The fastener loosening identification system provided by the embodiment of the application extracts the depth value corresponding to the mask of the marking line of the fastener based on the collected images of the gray camera and the depth camera, obtains corresponding point cloud data according to the internal reference of the camera, performs connectivity analysis and PCA on the point cloud data, and automatically identifies the loosened fastener according to the analysis result. In the system that this application provided, the spatial information of fastener is more perfect, and is higher for the judged result accuracy based on two-dimensional image, and can replace the manual work to accomplish inspection discernment, and efficiency is higher, and data can be reviewed.
The present embodiment provides a terminal, including:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the respective method.
The memory is used for storing a computer program, and the processor executes the computer program after receiving the execution instruction, and the method executed by the apparatus defined by the flow process disclosed in the foregoing corresponding embodiments can be applied to or implemented by the processor.
The Memory may comprise a Random Access Memory (RAM) and may also include a non-volatile Memory, such as at least one disk Memory. The memory can implement communication connection between the system network element and at least one other network element through at least one communication interface (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the method disclosed in the first embodiment may be implemented by hardware integrated logic circuits in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The corresponding methods, steps, and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software elements in the decoding processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, among other storage media that are well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.
The present embodiment provides a computer-readable storage medium having stored thereon a computer program; the computer program is executed by a processor in a corresponding method. For specific implementation, reference may be made to the method embodiments, which are not described herein again.
It should be noted that: unless specifically stated otherwise, the relative steps, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of the present invention. In all examples shown and described herein, unless otherwise specified, any particular value should be construed as merely illustrative, and not restrictive, and thus other examples of example embodiments may have different values.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a unit, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (12)
1. A fastener loosening identification method, comprising:
expanding and dividing the obtained fastener detection frame to obtain a mask of the mark lines of the fastener;
extracting corresponding depth values from the depth map according to the mask of the marking lines of the fastener and obtaining corresponding point cloud data according to the internal parameters of the camera;
determining that the fastener is loose when there is a discontinuity in the point cloud data and the primary directions are different.
2. The method of claim 1, wherein the expanding and dividing the captured fastener detection frame to obtain a mask of fastener marking lines comprises:
expanding the obtained fastener detection frame, and cutting according to a preset area;
and inputting the cut region into an image segmentation network based on deep learning to obtain a mask of the marking lines of the fastener.
3. The method of claim 1, wherein determining that the fastener is loose when there is a discontinuity in point cloud data and a primary direction is different comprises:
when the point cloud data is determined to have disconnected points, carrying out Principal Component Analysis (PCA) on the disconnected points;
determining that the fastener is loose when the resulting primary directions are not in a straight line.
4. The method of claim 3, wherein determining that there are disconnected points in the point cloud data comprises:
respectively acquiring nearest neighbor Euclidean distances between each point in the point cloud data and the rest points;
and when the nearest neighbor Euclidean distance is larger than a threshold value, determining that a disconnected point exists.
5. The method of claim 1, prior to the expanding and segmenting the captured fastener detection frame, comprising:
acquiring a gray scale image and a depth image which are respectively acquired by an aligned gray scale camera and a depth camera;
and detecting and positioning the fastener according to the gray-scale image and the depth image to obtain a fastener detection frame.
6. A fastener loosening identification system, comprising:
the first processing module is used for expanding and dividing the obtained fastener detection frame to obtain a mask of the mark line of the fastener;
the second processing module is used for extracting corresponding depth values from the depth map according to the mask of the marking lines of the fastener and obtaining corresponding point cloud data according to the internal reference of the camera;
and the third processing module is used for determining that the fastening piece is loosened when the point cloud data is disconnected and the main directions are different.
7. The system of claim 6, wherein the first processing module is specifically configured to:
expanding the obtained fastener detection frame, and cutting according to a preset area;
and inputting the cut region into an image segmentation network based on deep learning to obtain a mask of the marking lines of the fastener.
8. The system of claim 6, wherein the third processing module is specifically configured to:
when the point cloud data is determined to have disconnected points, carrying out Principal Component Analysis (PCA) on the disconnected points;
determining that the fastener is loose when the resulting primary directions are not in a straight line.
9. The system of claim 8, wherein the third processing module is specifically configured to:
respectively acquiring nearest neighbor Euclidean distances between each point in the point cloud data and the rest points;
and when the nearest neighbor Euclidean distance is larger than a threshold value, determining that a disconnected point exists.
10. The system of claim 6, wherein the first processing module is further configured to:
acquiring a gray scale image and a depth image which are respectively acquired by an aligned gray scale camera and a depth camera;
and detecting and positioning the fastener according to the gray-scale image and the depth image to obtain a fastener detection frame.
11. A terminal, comprising:
a memory;
a processor; and
a computer program;
wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1-5.
12. A computer-readable storage medium, having stored thereon a computer program; the computer program is executed by a processor to implement the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011227065.1A CN112365461A (en) | 2020-11-06 | 2020-11-06 | Fastener loosening identification method, system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011227065.1A CN112365461A (en) | 2020-11-06 | 2020-11-06 | Fastener loosening identification method, system, terminal and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112365461A true CN112365461A (en) | 2021-02-12 |
Family
ID=74509601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011227065.1A Pending CN112365461A (en) | 2020-11-06 | 2020-11-06 | Fastener loosening identification method, system, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112365461A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112967257A (en) * | 2021-03-11 | 2021-06-15 | 杭州申昊科技股份有限公司 | Subway nut looseness detection method based on visual angle conversion |
| CN113469940A (en) * | 2021-05-26 | 2021-10-01 | 成都唐源电气股份有限公司 | Fastener looseness detection method based on three-dimensional point cloud and two-dimensional image processing technology |
| CN113808096A (en) * | 2021-09-14 | 2021-12-17 | 成都主导软件技术有限公司 | Non-contact bolt looseness detection method and system |
| CN113984315A (en) * | 2021-11-30 | 2022-01-28 | 东风商用车有限公司 | Method and system for detecting bolt in vibration simulation test |
| CN114445435A (en) * | 2021-12-23 | 2022-05-06 | 中数智科(杭州)科技有限公司 | Vehicle body bolt looseness judging method based on mark line displacement deviation |
| CN115222731A (en) * | 2022-09-07 | 2022-10-21 | 西南交通大学 | Train fastener abnormity detection method based on two-dimensional image-point cloud mapping |
| CN116071353A (en) * | 2023-03-06 | 2023-05-05 | 成都盛锴科技有限公司 | Bolt assembly detection method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107830813A (en) * | 2017-09-15 | 2018-03-23 | 浙江理工大学 | The longaxones parts image mosaic and flexural deformation detection method of laser wire tag |
| CN110009602A (en) * | 2019-03-08 | 2019-07-12 | 唐山百川智能机器股份有限公司 | Detection method, system storage medium and the computer equipment of fastener locking mark |
| WO2019183475A1 (en) * | 2018-03-23 | 2019-09-26 | University Of Kansas | Vision-based fastener loosening detection |
| CN110634123A (en) * | 2018-06-05 | 2019-12-31 | 成都精工华耀科技有限公司 | Track fastener loosening detection method adopting depth image |
-
2020
- 2020-11-06 CN CN202011227065.1A patent/CN112365461A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107830813A (en) * | 2017-09-15 | 2018-03-23 | 浙江理工大学 | The longaxones parts image mosaic and flexural deformation detection method of laser wire tag |
| WO2019183475A1 (en) * | 2018-03-23 | 2019-09-26 | University Of Kansas | Vision-based fastener loosening detection |
| CN110634123A (en) * | 2018-06-05 | 2019-12-31 | 成都精工华耀科技有限公司 | Track fastener loosening detection method adopting depth image |
| CN110009602A (en) * | 2019-03-08 | 2019-07-12 | 唐山百川智能机器股份有限公司 | Detection method, system storage medium and the computer equipment of fastener locking mark |
Non-Patent Citations (1)
| Title |
|---|
| 胡广胜;王菁;孙福庆;: "基于图像识别的轨道交通车辆装配过程检测***", 城市轨道交通研究, no. 04, 10 April 2020 (2020-04-10) * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112967257A (en) * | 2021-03-11 | 2021-06-15 | 杭州申昊科技股份有限公司 | Subway nut looseness detection method based on visual angle conversion |
| CN113469940A (en) * | 2021-05-26 | 2021-10-01 | 成都唐源电气股份有限公司 | Fastener looseness detection method based on three-dimensional point cloud and two-dimensional image processing technology |
| CN113469940B (en) * | 2021-05-26 | 2023-08-04 | 成都唐源电气股份有限公司 | Fastener loosening detection method based on three-dimensional point cloud and two-dimensional image processing technology |
| CN113808096A (en) * | 2021-09-14 | 2021-12-17 | 成都主导软件技术有限公司 | Non-contact bolt looseness detection method and system |
| CN113808096B (en) * | 2021-09-14 | 2024-01-30 | 成都主导软件技术有限公司 | Non-contact bolt loosening detection method and system |
| CN113984315A (en) * | 2021-11-30 | 2022-01-28 | 东风商用车有限公司 | Method and system for detecting bolt in vibration simulation test |
| CN114445435A (en) * | 2021-12-23 | 2022-05-06 | 中数智科(杭州)科技有限公司 | Vehicle body bolt looseness judging method based on mark line displacement deviation |
| CN115222731A (en) * | 2022-09-07 | 2022-10-21 | 西南交通大学 | Train fastener abnormity detection method based on two-dimensional image-point cloud mapping |
| CN115222731B (en) * | 2022-09-07 | 2022-12-02 | 西南交通大学 | Train fastener abnormity detection method based on two-dimensional image-point cloud mapping |
| CN116071353A (en) * | 2023-03-06 | 2023-05-05 | 成都盛锴科技有限公司 | Bolt assembly detection method and system |
| CN116071353B (en) * | 2023-03-06 | 2023-09-05 | 成都盛锴科技有限公司 | Bolt assembly detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112365461A (en) | Fastener loosening identification method, system, terminal and storage medium | |
| JP6904614B2 (en) | Object detection device, prediction model creation device, object detection method and program | |
| CN114387268A (en) | Bolt looseness detection method and device | |
| JP2010117772A (en) | Feature value extracting device, object identification device, and feature value extracting method | |
| KR20200025480A (en) | Boundary based structure crack detection apparatus and method | |
| CN112906534A (en) | Lock catch loss fault detection method based on improved Faster R-CNN network | |
| US9508000B2 (en) | Object recognition apparatus | |
| JP6177541B2 (en) | Character recognition device, character recognition method and program | |
| JP2020038132A (en) | Crack on concrete surface specification method, crack specification device, and crack specification system, and program | |
| CN117094975A (en) | Method and device for detecting surface defects of steel and electronic equipment | |
| CN112712495A (en) | Anti-loose piece fracture identification method, system, terminal and storage medium | |
| JP2011007728A (en) | Method, apparatus and program for defect detection | |
| CN109767426B (en) | Shield tunnel water leakage detection method based on image feature recognition | |
| CN117011290B (en) | Defect diagnosis method, system and computer equipment | |
| CN114549389A (en) | Wheel set tread detection method, device, system, terminal and storage medium | |
| CN112784633B (en) | Lane boundary processing method and device, electronic equipment and storage medium | |
| CN116071335A (en) | Wall surface acceptance method, device, equipment and storage medium | |
| KR101944604B1 (en) | Apparatus and method for inspection automation to count object quantity | |
| JP7186821B2 (en) | Bar arrangement inspection device, bar arrangement inspection method and program | |
| JP4470513B2 (en) | Inspection method and inspection apparatus | |
| KR101371875B1 (en) | A vehicle detection and distance measurement system and method between the vehicles using stereo vision | |
| JP5439069B2 (en) | Character recognition device and character recognition method | |
| KR101714131B1 (en) | Device and method for recognizing parking stall | |
| KR102156488B1 (en) | Method of detecting car number plate | |
| CN109215068B (en) | Image magnification measuring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |