CN117218123B - Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud - Google Patents
Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud Download PDFInfo
- Publication number
- CN117218123B CN117218123B CN202311482390.6A CN202311482390A CN117218123B CN 117218123 B CN117218123 B CN 117218123B CN 202311482390 A CN202311482390 A CN 202311482390A CN 117218123 B CN117218123 B CN 117218123B
- Authority
- CN
- China
- Prior art keywords
- data
- point cloud
- detection
- point
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 125
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 26
- 239000010959 steel Substances 0.000 title claims abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 27
- 238000001914 filtration Methods 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 14
- 239000011159 matrix material Substances 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 25
- 238000006243 chemical reaction Methods 0.000 claims description 19
- 238000004458 analytical method Methods 0.000 claims description 13
- 238000012544 monitoring process Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 5
- 238000007781 pre-processing Methods 0.000 abstract 1
- 238000003860 storage Methods 0.000 description 8
- 238000013519 translation Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a fault detection method and system for cold-rolled strip steel wire flying equipment based on point cloud, and belongs to the technical field of fault detection of production equipment. The embodiment of the invention discloses a fault detection method and system for cold-rolled strip steel wire flying equipment based on point cloud. Wherein the method comprises the following steps: acquiring point cloud data through a data acquisition device; preprocessing point cloud data, removing irrelevant interference data, filtering noise/outliers, and performing data downsampling; fusing the processed point cloud data with a visible light image, establishing a 3D point cloud visible light fusion scene, and setting a virtual detection frame in the reconstructed three-dimensional area; and performing fault detection by taking the number of the detected points in the virtual detection frame as a detection value. The real-time and omnibearing display of the detection data and the running state of the equipment is realized, and the accurate and rapid fault detection of the equipment is realized.
Description
Technical Field
The invention belongs to the technical field of production equipment fault detection, and particularly relates to a fault detection method and system for cold-rolled strip steel wire flight equipment based on point cloud.
Background
The cold-rolled strip steel has high dimensional accuracy, good surface quality, good mechanical property and good technological property, so the cold-rolled strip steel is widely applied to manufacturing departments and industrial departments, but the steel plate collides with conveying equipment in the movement process due to flying wires or warping of the edge of the steel plate, so that the equipment is blocked and the production line is stopped, and personal casualties can be generated under severe conditions. Therefore, fault detection is required to be carried out on cold-rolled strip steel wire flying equipment in the production process, and the current detection method is to shoot through a camera, and then identify the warping and blocking faults of strip steel through an image analysis algorithm. The method for identifying the image still has the following to be improved:
(1) The image acquisition requires the use of a sufficiently stable light source on site, but the stability is difficult to ensure due to the uncertainty of actual conditions in a factory, so that the identification effect is affected.
(2) The existing image recognition algorithm is based on the graph characteristics of the problems, false recognition or false recognition can be caused due to the fact that the shape of the warping or flying wire is greatly different, the image recognition is 2D, and the warping or blocking problems are difficult to accurately and quantitatively describe by using a 2D image.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a fault detection method and a fault detection system for cold-rolled strip steel wire flying equipment based on point cloud,
the aim of the invention can be achieved by the following technical scheme:
s1, performing S1; scanning detection equipment through a data acquisition device to obtain scanning light, wherein the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light is reflected to a 3D point cloud scanning sensor through a steel belt, and point cloud data are acquired through the 3D point cloud scanning sensor;
s2: performing direct filtering processing on the point cloud data to eliminate a background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
s3: fusing the modeling data with a visible light image to establish a 3D point cloud visible light fusion scene, setting a monitoring area in the 3D point cloud visible light fusion scene, and solving a conversion matrix by adopting a matching calibration mode for data acquired by a laser radar and an RGB camera, wherein the calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
s4: converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein a conversion formula is as follows:
，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
Specifically, the step S2 specifically includes the following steps:
setting a value range for each dimension of the point cloud data, judging whether the value of each point in the point cloud data in the current dimension is in the corresponding value range or not by traversing the point cloud data, if yes, reserving the point, otherwise, deleting the point, and forming the reserved point into the non-interference data;
setting a neighborhood radius and a neighbor point threshold, traversing each point in the non-interference data, judging whether the number of neighbor points of each point of the non-interference data in the neighborhood radius is larger than the neighbor point threshold, if yes, reserving the point, if not, deleting the point, and forming the reserved point into communication data;
and establishing a 3D voxel grid through the connected data, and substituting the grid centroid of each point falling in the 3D voxel grid with the point to obtain modeling data.
A fault detection system of cold-rolled strip steel wire-flying equipment based on point cloud comprises a scanning module, a scanning data processing module, a monitoring analysis module and a fault detection module;
the scanning module is used for scanning the detection equipment through the data acquisition device to obtain scanning light rays, the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light rays are reflected to the 3D point cloud scanning sensor through the steel belt, and the 3D point cloud scanning sensor is used for acquiring point cloud data;
the scanning data processing module is used for performing direct filtering processing on the point cloud data to eliminate the background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
the monitoring analysis module is used for fusing the modeling data with the visible light image to establish a 3D point cloud visible light fusion scene, a monitoring area is set in the 3D point cloud visible light fusion scene, a conversion matrix is solved by adopting a matching calibration mode for data collected by the laser radar and the RGB camera, and a calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
the fault detection module is used for converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein the conversion formula is as follows:
，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
The beneficial effects of the invention are as follows:
(1) Scanning is carried out on detection equipment through an RGB camera and a laser radar to obtain scanning light, the scanning light is reflected to a 3D point cloud scanning sensor through a steel belt, point cloud data are acquired through the 3D point cloud scanning sensor, the ambient brightness during detection is improved, measurement characteristics are directly determined, the imaging effect of the detection equipment is improved, the data processing process is simplified, and the stability and reliability of data transmission are guaranteed.
(2) The 3D point cloud visible light fusion scene is established by fusing the point cloud data with the visible light image, a three-dimensional detection frame is arranged in the 3D fusion scene, and the problems of equipment surface warping, blocking and the like are quantitatively represented by calculating the 3D point cloud data in the detection frame, so that the complexity of an identification system is reduced, and the positioning, measuring and identifying precision of the system is improved.
Drawings
The present invention is further described below with reference to the accompanying drawings for the convenience of understanding by those skilled in the art.
FIG. 1 is a schematic flow chart of a fault detection method of cold-rolled strip steel wire-flying equipment based on point cloud.
Detailed Description
In order to further describe the technical means and effects adopted by the invention for achieving the preset aim, the following detailed description is given below of the specific implementation, structure, characteristics and effects according to the invention with reference to the attached drawings and the preferred embodiment.
Referring to fig. 1, a fault detection method for a cold-rolled strip steel wire flying device based on point cloud specifically includes the following steps:
s1, performing S1; scanning detection equipment through a data acquisition device to obtain scanning light, wherein the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light is reflected to a 3D point cloud scanning sensor through a steel belt, and point cloud data are acquired through the 3D point cloud scanning sensor;
s2: performing direct filtering processing on the point cloud data to eliminate a background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
s3: fusing the modeling data with a visible light image to establish a 3D point cloud visible light fusion scene, setting a monitoring area in the 3D point cloud visible light fusion scene, and solving a conversion matrix by adopting a matching calibration mode for data acquired by a laser radar and an RGB camera, wherein the calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
s4: converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein a conversion formula is as follows:
，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
In this embodiment, in the present embodiment, the internal parameters of the RGB camera need to be acquired, the conversion from the laser radar coordinate system to the RGB camera coordinate system is implemented by means of the rotation matrix and the translation vector, the internal parameters of the RGB camera, the rotation matrix and the translation vector are combined into three rows and four columns of M matrices, and more than 6 groups of points corresponding to the points [ u, v ] on the visible light RGB image from the object points [ UV, W ] scanned by the laser radar are used to solve the M matrices. Setting basic parameters of a detection frame, namely length, width and height, changing the size of the detection frame by modifying the parameters, setting a position difference value of coordinates of a central point of the detection frame as a translation vector, setting each angle of rotation of the detection frame by the central point as a rotation matrix, reversely pushing out the rotation matrix and the translation vector by the M matrix and an internal reference matrix, converting the rotation matrix and the translation vector obtained by the mode when converting the point cloud data into a coordinate system of the detection frame, and counting the number of point clouds in the detection frame.
Specifically, the step S2 specifically includes the following steps:
setting a value range for each dimension of the point cloud data, judging whether the value of each point in the point cloud data in the current dimension is in the corresponding value range or not by traversing the point cloud data, if yes, reserving the point, otherwise, deleting the point, and forming the reserved point into the non-interference data;
setting a neighborhood radius and a neighbor point threshold, traversing each point in the non-interference data, judging whether the number of neighbor points of each point of the non-interference data in the neighborhood radius is larger than the neighbor point threshold, if yes, reserving the point, if not, deleting the point, and forming the reserved point into communication data;
and establishing a 3D voxel grid through the connected data, and substituting the grid centroid of each point falling in the 3D voxel grid with the point to obtain modeling data.
In this embodiment, PCL open source library is used to process point cloud data, points within a range of values no longer preset in a specified dimension are filtered, a filter object pass is created through PCL:: passchrough < PCL::: pointXYZ > pass, filtering is performed in various axis directions using pass.setFilterFieldName ("pos") settings, a filtering range is set through pass.setFilterLimits (min, max), PCL:: radio outlieRemoval < PCL::: pointXYZ > filters are used for filtering results to achieve a radius filtering function, based on the above analysis, a 3D voxel grid is created on the input point cloud using PCL::: voxel < PCL:: pointXYZ > filters, points within each grid are approximately represented by the centroid of the grid, and the PCL PointPointXud 2 class is used to preserve the point cloud.
A fault detection system of cold-rolled strip steel wire-flying equipment based on point cloud comprises a scanning module, a scanning data processing module, a monitoring analysis module and a fault detection module;
the scanning module is used for scanning the detection equipment through the data acquisition device to obtain scanning light rays, the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light rays are reflected to the 3D point cloud scanning sensor through the steel belt, and the 3D point cloud scanning sensor is used for acquiring point cloud data;
the scanning data processing module is used for performing direct filtering processing on the point cloud data to eliminate the background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
the monitoring analysis module is used for fusing the modeling data with the visible light image to establish a 3D point cloud visible light fusion scene, a monitoring area is set in the 3D point cloud visible light fusion scene, a conversion matrix is solved by adopting a matching calibration mode for data collected by the laser radar and the RGB camera, and a calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
the fault detection module is used for converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein the conversion formula is as follows:
，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
In the embodiment, a monitoring device is arranged between a welding machine and a steel structure, a scanning module adopts an RGB camera and a laser radar to collect data simultaneously, two pieces of equipment are placed in the same direction, the collected data are uploaded to an industrial personal computer through a sensor, the industrial personal computer processes the scanning data, the industrial personal computer is provided with a ubuntu environment, and a program is written to link a PCL3D point cloud library to preprocess the scanning data; reconstructing a three-dimensional scene by using an SfM algorithm, counting and judging the quantity of the point cloud data transmitted in real time in a detection frame, returning an early warning instruction, and analyzing the early warning instruction by an industrial personal computer to send a safety signal; the industrial personal computer transmits data to the operation terminal and the server of the PLC room through a network, the server can manage a plurality of sets of detection devices, and the operation terminal can check real-time data and abnormal early warning historical data of any set of detection devices at any time.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The present invention is not limited to the above embodiments, but is capable of modification and variation in detail, and other modifications and variations can be made by those skilled in the art without departing from the scope of the present invention.
Claims (3)
1. The fault detection method of the cold-rolled strip steel wire-flying equipment based on the point cloud is characterized by comprising the following steps of:
s1: scanning detection equipment through a data acquisition device to obtain scanning light, wherein the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light is reflected to a 3D point cloud scanning sensor through a steel belt, and point cloud data are acquired through the 3D point cloud scanning sensor;
s2: performing direct filtering processing on the point cloud data to eliminate a background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
s3: fusing the modeling data with a visible light image to establish a 3D point cloud visible light fusion scene, setting a monitoring area in the 3D point cloud visible light fusion scene, and solving a conversion matrix by adopting a matching calibration mode for data acquired by a laser radar and an RGB camera, wherein the calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
s4: converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein a conversion formula is as follows:，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
2. The method according to claim 1, wherein said step S2 comprises the steps of:
setting a value range for each dimension of the point cloud data, judging whether the value of each point in the point cloud data in the current dimension is in the corresponding value range or not by traversing the point cloud data, if yes, reserving the point, otherwise, deleting the point, and forming the reserved point into the non-interference data;
setting a neighborhood radius and a neighbor point threshold, traversing each point in the non-interference data, judging whether the number of neighbor points of each point of the non-interference data in the neighborhood radius is larger than the neighbor point threshold, if yes, reserving the point, if not, deleting the point, and forming the reserved point into communication data;
and establishing a 3D voxel grid through the connected data, and substituting the grid centroid of each point falling in the 3D voxel grid with the point to obtain modeling data.
3. A cold-rolled strip steel wire-flying equipment fault detection system based on point cloud, which is characterized by being operated by the method of any one of claims 1-2 and comprising a scanning module, a scanning data processing module, a monitoring analysis module and a fault detection module;
the scanning module is used for scanning the detection equipment through the data acquisition device to obtain scanning light rays, the data acquisition device is an RGB camera and a laser radar, data are acquired in the same direction, the scanning light rays are reflected to the 3D point cloud scanning sensor through the steel belt, and the 3D point cloud scanning sensor is used for acquiring point cloud data;
the scanning data processing module is used for performing direct filtering processing on the point cloud data to eliminate the background and obtain interference-free data, performing radius filtering analysis on the interference-free data to obtain connected data, and performing downsampling processing on the connected data to obtain modeling data;
the monitoring analysis module is used for fusing the modeling data with the visible light image to establish a 3D point cloud visible light fusion scene, a monitoring area is set in the 3D point cloud visible light fusion scene, a conversion matrix is solved by adopting a matching calibration mode for data collected by the laser radar and the RGB camera, and a calculation formula is as follows:
，
wherein,uis the abscissa of the pixel points of the RGB camera,vis the ordinate of the pixel points of the RGB camera,Min order to transform the matrix,Uis the radius of the laser radar coordinate system,Vis the azimuth angle of the laser radar coordinate system,Wis the polar angle of a laser radar coordinate system;
converting the point cloud data into RGB image coordinates through the conversion matrix and obtaining RGB color information;
establishing a 3D point cloud visible light fusion scene by the RGB image coordinates, the RGB color information and the modeling data;
a cuboid detection frame with a fixed size is established at the space origin of the 3D point cloud visible light fusion scene;
the fault detection module is used for converting the point cloud data into RGB image coordinates to obtain space coordinates, and converting the space coordinates into a detection frame coordinate system to obtain detection coordinates, wherein the conversion formula is as follows:
，
wherein,Qin order to detect the coordinates of the object,Rin order to rotate the matrix is rotated,Tin order to translate the vector of the vector,Pis a space coordinate;
when the detection coordinates are in the detection frame, counting the number of detection points in the detection frame as detection values;
and counting the number of the point cloud data in the space virtual detection frame to serve as a detection value, presetting a safety threshold and an alarm threshold, judging the detection value, the safety threshold and the alarm threshold, returning to a normal operation signal if the detection value is smaller than the safety threshold, returning to process an early warning information signal if the detection value is larger than the safety threshold and smaller than the alarm threshold, and returning to emergency processing alarm signal if the detection value is larger than the alarm threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311482390.6A CN117218123B (en) | 2023-11-09 | 2023-11-09 | Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311482390.6A CN117218123B (en) | 2023-11-09 | 2023-11-09 | Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117218123A CN117218123A (en) | 2023-12-12 |
| CN117218123B true CN117218123B (en) | 2024-02-02 |
Family
ID=89037492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311482390.6A Active CN117218123B (en) | 2023-11-09 | 2023-11-09 | Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218123B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709934A (en) * | 2020-06-17 | 2020-09-25 | 浙江大学 | Injection molding impeller warping defect detection method based on point cloud characteristic comparison |
| CN113111887A (en) * | 2021-04-26 | 2021-07-13 | 河海大学常州校区 | Semantic segmentation method and system based on information fusion of camera and laser radar |
| CN113204029A (en) * | 2021-04-19 | 2021-08-03 | 北京科技大学 | Hot-rolled plate blank warpage detection device and method |
| KR20220095468A (en) * | 2020-12-30 | 2022-07-07 | 주식회사 딥인스펙션 | Steel quality inspection system based on xai and large scale deep learning hpc system |
| CN114758333A (en) * | 2020-12-29 | 2022-07-15 | 北京瓦特曼科技有限公司 | Method and system for identifying off-hook of ladle lifted by crane of ladle crane |
| US11403860B1 (en) * | 2022-04-06 | 2022-08-02 | Ecotron Corporation | Multi-sensor object detection fusion system and method using point cloud projection |
| CN115330734A (en) * | 2022-08-18 | 2022-11-11 | 南京邮电大学 | Automatic robot repair welding system based on three-dimensional target detection and point cloud defect completion |
| CN115731545A (en) * | 2022-12-06 | 2023-03-03 | 国网江苏省电力有限公司 | Cable tunnel inspection method and device based on fusion perception |
| CN115877400A (en) * | 2022-11-23 | 2023-03-31 | 太原理工大学 | Tunnel roof support steel belt drilling positioning method based on radar and vision fusion |
| CN116466355A (en) * | 2023-03-23 | 2023-07-21 | 浙江吉利控股集团有限公司 | Point cloud target detection method and device and computer readable storage medium |
| CN116520302A (en) * | 2023-01-31 | 2023-08-01 | 新石器慧通（北京）科技有限公司 | Positioning method applied to automatic driving system and method for constructing three-dimensional map |
| CN116573017A (en) * | 2023-04-19 | 2023-08-11 | 长安大学 | Urban rail train running clearance foreign matter sensing method, system, device and medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111352112B (en) * | 2020-05-08 | 2022-11-29 | 泉州装备制造研究所 | Target detection method based on vision, laser radar and millimeter wave radar |
| GB2596867A (en) * | 2020-07-10 | 2022-01-12 | Scout Drone Inspection As | Inspection device for inspecting a building or structure |
| JP2023122951A (en) * | 2022-02-24 | 2023-09-05 | 株式会社神戸製鋼所 | Workpiece measurement method, workpiece measurement system, and program |
-
2023
- 2023-11-09 CN CN202311482390.6A patent/CN117218123B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111709934A (en) * | 2020-06-17 | 2020-09-25 | 浙江大学 | Injection molding impeller warping defect detection method based on point cloud characteristic comparison |
| CN114758333A (en) * | 2020-12-29 | 2022-07-15 | 北京瓦特曼科技有限公司 | Method and system for identifying off-hook of ladle lifted by crane of ladle crane |
| KR20220095468A (en) * | 2020-12-30 | 2022-07-07 | 주식회사 딥인스펙션 | Steel quality inspection system based on xai and large scale deep learning hpc system |
| CN113204029A (en) * | 2021-04-19 | 2021-08-03 | 北京科技大学 | Hot-rolled plate blank warpage detection device and method |
| CN113111887A (en) * | 2021-04-26 | 2021-07-13 | 河海大学常州校区 | Semantic segmentation method and system based on information fusion of camera and laser radar |
| US11403860B1 (en) * | 2022-04-06 | 2022-08-02 | Ecotron Corporation | Multi-sensor object detection fusion system and method using point cloud projection |
| CN115330734A (en) * | 2022-08-18 | 2022-11-11 | 南京邮电大学 | Automatic robot repair welding system based on three-dimensional target detection and point cloud defect completion |
| CN115877400A (en) * | 2022-11-23 | 2023-03-31 | 太原理工大学 | Tunnel roof support steel belt drilling positioning method based on radar and vision fusion |
| CN115731545A (en) * | 2022-12-06 | 2023-03-03 | 国网江苏省电力有限公司 | Cable tunnel inspection method and device based on fusion perception |
| CN116520302A (en) * | 2023-01-31 | 2023-08-01 | 新石器慧通（北京）科技有限公司 | Positioning method applied to automatic driving system and method for constructing three-dimensional map |
| CN116466355A (en) * | 2023-03-23 | 2023-07-21 | 浙江吉利控股集团有限公司 | Point cloud target detection method and device and computer readable storage medium |
| CN116573017A (en) * | 2023-04-19 | 2023-08-11 | 长安大学 | Urban rail train running clearance foreign matter sensing method, system, device and medium |
Non-Patent Citations (7)
| Title |
|---|
| Development and application of substation intelligent inspection robot supporting deep learning accelerating;Lei Su et al.;《Journal of Physics: Conference Series》;第1754卷;第1-8页 * |
| Steel Plate Surface Defect Recognition Method Based on Depth Information;Chang Zhao et al.;《2019 IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS)》;第322-327页 * |
| 一种车载激光扫描点云中路面坑槽自动提取方法;杨雷;刘如飞;卢秀山;马新江;;《测绘工程》;第29卷(第1期);第66-71页 * |
| 基于图像和激光点云的桥梁混凝土表面缺陷检测方法研究;邢维聪;《万方数据》;第1-82页 * |
| 基于激光雷达的地铁隧道侵限检测方法;刘璐宾等;《机械工程与自动化》(第3期);第16-18页 * |
| 无人机载多载荷输电线路巡检方法研究;陈科羽等;《电力大数据》;第23卷(第2期);第80-86页 * |
| 钢坯表面缺陷标注机器人控制***设计与实现;刘智锋;《万方数据》;第1-62页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117218123A (en) | 2023-12-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108009515B (en) | Power transmission line positioning and identifying method of unmanned aerial vehicle aerial image based on FCN | |
| CN113345019B (en) | Method, equipment and medium for measuring potential hazards of transmission line channel target | |
| CN111798407B (en) | Electrified equipment fault diagnosis method based on neural network model | |
| CN112686938B (en) | Power transmission line clear distance calculation and safety alarm method based on binocular image ranging | |
| CN111412842B (en) | Method, device and system for measuring cross-sectional dimension of wall surface | |
| CN115294145B (en) | Method and system for measuring sag of power transmission line | |
| CN115731165B (en) | Detection system and method for on-line size spot inspection of lithium battery | |
| CN114255405A (en) | Hidden danger target identification method and device | |
| CN113023293A (en) | Inspection method, device, equipment and system for belt conveyor | |
| CN112926483A (en) | Standard cabinet state indicator lamp identification monitoring method, device and system | |
| CN116147842A (en) | Gas leakage detection method and device, equipment and storage medium | |
| CN115995058A (en) | Power transmission channel safety on-line monitoring method based on artificial intelligence | |
| CN117218123B (en) | Cold-rolled strip steel wire flying equipment fault detection method and system based on point cloud | |
| CN113567451A (en) | Cable defect detection and diameter measurement method | |
| CN117314986A (en) | Unmanned aerial vehicle cross-mode power distribution equipment inspection image registration method based on semantic segmentation | |
| CN115979972B (en) | Real-time monitoring method and system for hyperspectral of crude oil film on sea surface | |
| CN115861443A (en) | Multi-camera internal reference calibration method and device, electronic equipment and storage medium | |
| CN114112972B (en) | Closed space SF 6 Gas leakage infrared telemetry device and imaging positioning method thereof | |
| CN112686162B (en) | Method, device, equipment and storage medium for detecting clean state of warehouse environment | |
| CN111798476B (en) | Extraction method for conductive arm axis of high-voltage isolating switch | |
| CN114092857A (en) | Gateway-based collection card image acquisition method, system, equipment and storage medium | |
| CN111583336A (en) | Robot and inspection method and device thereof | |
| CN113516120A (en) | Raise dust detection method, image processing method, device, equipment and system | |
| Carratù et al. | Vision-Based System for Measuring the Diameter of Wood Logs | |
| Bao et al. | Defect detection system for smartphone front camera based on improved template matching algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |