WO2022252698A1 - Defect detection method and apparatus based on structured light field video stream - Google Patents

Defect detection method and apparatus based on structured light field video stream Download PDF

Info

Publication number
WO2022252698A1
WO2022252698A1 PCT/CN2022/076878 CN2022076878W WO2022252698A1 WO 2022252698 A1 WO2022252698 A1 WO 2022252698A1 CN 2022076878 W CN2022076878 W CN 2022076878W WO 2022252698 A1 WO2022252698 A1 WO 2022252698A1
Authority
WO
WIPO (PCT)
Prior art keywords
microlens
light field
time
video stream
real
Prior art date
Application number
PCT/CN2022/076878
Other languages
French (fr)
Chinese (zh)
Inventor
金欣
康今世
Original Assignee
清华大学深圳国际研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学深圳国际研究生院 filed Critical 清华大学深圳国际研究生院
Publication of WO2022252698A1 publication Critical patent/WO2022252698A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01—Arrangements or apparatus for facilitating the optical investigation
    • G01N2021/0106—General arrangement of respective parts
    • G01N2021/0112—Apparatus in one mechanical, optical or electronic block

Definitions

  • the invention relates to the fields of computer vision and digital video processing, in particular to a defect detection method and device based on structured light field video streams.
  • Machine vision defect detection is a technology that takes the image captured by the optical sensor as input, extracts the surface image features of the object through computer vision methods, and then identifies its physical defects. Compared with traditional manual identification, this technology has high detection accuracy and fast detection speed, and has broad application prospects in the field of industrial quality inspection.
  • traditional image recognition and stereo vision technologies mostly rely on image feature points and texture information, and are often powerless in the face of low-texture or no-texture objects to be detected and working scenes; detection technology based on structured light can reconstruct the three-dimensional information of the object surface , but this type of system is expensive, and the detection accuracy is significantly affected by the material and processing technology of the object to be detected.
  • the above methods based on traditional visual sensors have poor robustness when detecting small defects, and there are many false detections and missed detections. When dealing with objects with complex surface structures, multiple multi-angle imaging is required, which is inefficient.
  • Light field imaging technology simultaneously captures four-dimensional light information including space dimension and angle dimension through a special optical path structure design, and realizes single-exposure multi-angle stereoscopic imaging.
  • the related research on the application of field imaging technology to industrial inspection applications is still in its infancy.
  • the present invention proposes a defect detection method and device based on structured light field video stream.
  • the present invention adopts the following technical solutions:
  • a defect detection method based on structured light field video stream comprising the steps of:
  • A1 Construct a detection device based on an active coded structured light source and a light field vision sensor
  • A2 use the white image to calibrate the light field visual sensor, decode the structured light field video stream obtained by the light field visual sensor according to the calibration result, and obtain a real-time microlens image video stream;
  • A3 Perform motion correction, grayscale correction, and region-of-interest correction on the real-time microlens image video stream to calculate the similarity between frames, and establish a local defect detection result for the microlens;
  • A4 Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
  • the detection device includes an active coding structured light source system capable of emitting light and displaying specific coding patterns and a light field visual sensor system capable of collecting light field video streams, wherein the light emitted by the active coding structured light source system passes through the object to be detected After being reflected, it is captured by the light field visual sensor system; wherein, the active coding structured light source system is controlled by a computer program, and can display a specified coding pattern at a specified time according to the requirements of the program.
  • the coding pattern includes One or more of two-dimensional square wave stripes, two-dimensional sine wave stripes, and two-dimensional checkerboard grids;
  • the light field visual sensor system includes one or more visual sensor devices with single-exposure stereoscopic imaging functions, preferably, using Light field camera or camera array light field camera based on multiplexing technology.
  • the initial estimated value of the microlens center is input into the optimization model corresponding to the camera microlens distribution, after nonlinear optimization Obtain the grid vector corresponding to the microlens geometric distribution of the light field vision sensor, and then obtain the precise pixel coordinates of the microlens center; preferably, the specific operations are as follows:
  • ⁇ the microlens angle distribution parameter matched with the microlens array
  • H refined new transformation matrix generated according to r refined ;
  • x k,refined ,y k,refined The precise pixel coordinate value of the center of the kth microlens.
  • the light field video stream refers to a light field image sequence continuously collected by a light field visual sensor; for each frame of light field original image in the sequence, traverse all microlens centers in the calibration result, and use its precise pixel coordinates
  • the values x i,refined and y i,refined are the center, and the size of the neighborhood close to the size of the microlens is selected for segmentation to obtain a real-time microlens image video stream.
  • Described step A3 comprises:
  • A31 Grayscale correction of real-time microlens image video stream
  • A32 Perform region of interest (ROI) correction on the real-time microlens image video stream;
  • A33 Motion correction of real-time microlens image video streams
  • A34 Calculate the inter-frame similarity in real time based on the corrected frame difference video stream, and establish the local defect detection result of the microlens.
  • step A2 For the real-time microlens image video stream obtained in step A2, two adjacent frames of microlens images are intercepted, the gray average value of the two frames of images is calculated, and a frame with a smaller grayscale is linearly transformed to obtain a consistent grayscale change.
  • Video stream preferably, the specific operations are as follows:
  • I(t) real-time microlens image at time t
  • ⁇ t the frame sampling interval of the light field vision sensor
  • I'(t) real-time microlens image after grayscale correction at time t.
  • Bin() binarization operation
  • I′′(t) real-time microlens image after ROI correction at time t.
  • step A32 For the real-time microlens image video stream with consistent grayscale and ROI obtained in step A32, intercept two adjacent frames of microlens images, move the position of the previous frame and calculate the MSE between frames corresponding to different displacements, and the displacement with the minimum MSE value It is the best motion estimation between two frames, and the motion error of the previous frame can be corrected according to the estimated value; preferably, the specific operation is as follows:
  • I′′ t a real-time microlens image with consistent gray scale and ROI at time t;
  • i, j the pixel coordinates of the microlens image used for traversal
  • MSE t time t corresponds to all MSE values used for traversal inter-frame displacement estimation
  • I′′' t real-time microlens image with consistent gray scale, consistent ROI, and no motion error at time t.
  • the specific operations are as follows:
  • Result(t) local defect detection result of the microlens at time t.
  • Described step A4 comprises:
  • A41 Use the spatial correlation of light field images to count the distribution of defect features on adjacent microlenses, and establish the detection results of defect positions in single frame difference light field images;
  • A42 Use the time-domain correlation of light field video to calculate the change rule of defect position between adjacent frames, and establish the final defect detection result of structured light field video.
  • step A41 Preferably, in the step A41:
  • step A34 Use the microlens real-time local defect detection results obtained in step A34 to generate a series of defective microlens center spatial position distributions, and calculate the defect microlens density corresponding to each part of the current light field image frame.
  • the greater the density, the presence of physical defects The greater the possibility; more preferably, the specific operations are as follows:
  • Dist() distance measurement function, including but not limited to Euclidean distance, Gaussian distance, etc.
  • FrameResult(k) the detection result of the defect position at the center of the kth microlens in the current frame.
  • step A42 Preferably, in the step A42:
  • step A41 Using the single-frame differential light field image defect position detection results obtained in step A41, for areas that may have physical defects, if defects are detected in the area between adjacent frames, and the area position moves at a speed between frames If it is close to the moving speed of the object set by the detection device, it is more likely to judge that there is a defect in the position; the defect position with time domain correlation is marked and output in real time on the light field video stream.
  • a defect detection device based on a structured light field video stream comprising a processor and a detection device based on an active coded structured light source and a light field visual sensor, when the processor executes a computer program on a computer-readable storage medium, the described Steps A2-A4 of the defect detection method based on structured light field video stream.
  • the present invention proposes a defect detection method and device based on structured light field video streams, combined with coded structured light technology and light field imaging technology, using an active coded structured light source to convert physical defects on the surface of an object to be detected into a reflective coded pattern on the surface of the object
  • the geometric distortion of using light field imaging to capture this geometric distortion in a higher dimension with fewer exposures, transforming the large-scale inconspicuous distortion in the field of view of traditional cameras into light field microlens image level with spatial correlation Significant distortion in a small range.
  • the method of the present invention can effectively improve the accuracy and stability of structured light defect detection, solve the problem of "starting from scratch” in the detection of complex workpiece light field visual defects, and significantly improve the quality of light field video
  • the real-time performance of the detection method is of great significance for industries such as precision machining and the popularization and application of light field imaging technology.
  • FIG. 1 is a flow chart of a defect detection method based on structured light field video streams according to an embodiment of the present invention.
  • the present invention proposes a method and device for detecting defects based on structured light field video streams.
  • the main idea is: combining coded structured light technology and light field imaging technology, using active coded structured light sources to convert physical defects of objects to be detected into coded patterns Geometric distortion, using light field imaging to capture this geometric distortion in a higher dimension with fewer exposures, transforming the large-scale inconspicuous distortion in the field of view of traditional cameras into spatial correlation at the level of light field microlens images
  • Significant distortion in a small range enables accurate, efficient and stable detection of physical defects in objects of various materials.
  • the defect detection method includes the following steps:
  • A1 Construct a detection device based on an active coded structured light source and a light field vision sensor
  • A2 Use the white image to calibrate the light field vision sensor, and decode the structured light field video stream obtained by the sensor according to the calibration result to obtain a real-time microlens image video stream;
  • A3 Calculate the inter-frame similarity after performing gray scale correction, region of interest correction and motion correction on the real-time microlens image video stream, and establish the local defect detection results of the microlens;
  • A4 Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
  • A1 Construct a detection device based on an active coded structured light source and a light field vision sensor.
  • the device includes an active coded structured light source system capable of emitting light and displaying specific coded patterns, and a light field visual sensor system capable of collecting light field video streams.
  • the active coded structured light source system is controlled by a computer program, which can display a designated coded pattern at a designated time according to the requirements of the program.
  • Coding patterns include but are not limited to two-dimensional square wave stripes, two-dimensional sine wave stripes, two-dimensional checkerboard and other patterns containing spatial modulation information. Controlled display devices such as electronic display screens.
  • the light field visual sensor system consists of one or more visual sensor devices capable of single-exposure stereoscopic imaging, and its hardware carriers include but are not limited to light field cameras based on multiplexing technology, camera array light field cameras, and other Stereo imaging device for obtaining light field images.
  • the light emitted by the light source system in the system is captured by the light field vision sensor system after being reflected by the object to be detected.
  • A2 Use the white image to calibrate the light field vision sensor, and decode the structured light field video stream acquired by the sensor according to the calibration result to obtain a real-time microlens image video stream.
  • the initial estimated value of the microlens center is input into the corresponding microlens distribution of the camera.
  • the model is optimized. After nonlinear optimization, the grid vector corresponding to the geometric distribution of the microlens of the light field vision sensor is obtained, and then the precise pixel coordinates of the center of the microlens are obtained.
  • the specific operation is as follows:
  • ⁇ the microlens angle distribution parameter matched with the microlens array
  • H refined new transformation matrix generated according to r refined ;
  • x i,refined ,y i,refined The precise pixel coordinate value of the i-th microlens center.
  • a light field video stream refers to a sequence of light field images continuously captured using a light field vision sensor. For each frame of the light field original image in the sequence, traverse all the centers of the microlenses in the above calibration results, center on their precise pixel coordinate values x i,refined ,y i,refined , and select a neighborhood with a size similar to the size of the microlens
  • the real-time microlens image video stream can be obtained by dividing the size.
  • A3 Perform gray scale correction, region of interest correction, and motion correction on the real-time microlens image video stream to calculate the similarity between frames, and establish the local defect detection results of the microlens.
  • A31 Grayscale correction of real-time microlens image video stream
  • step A2 For the real-time microlens image video stream obtained in step A2, two adjacent frames of microlens images are intercepted, the gray average value of the two frames of images is calculated, and a frame with a smaller grayscale is linearly transformed to obtain the grayscale Variation consistent video stream.
  • the specific operation is as follows (assuming that a frame with a smaller gray scale is the previous frame):
  • I(t) real-time microlens image at time t
  • ⁇ t the frame sampling interval of the light field vision sensor
  • I'(t) real-time microlens image after grayscale correction at time t.
  • A32 Perform region of interest (ROI) correction on the real-time microlens image video stream;
  • step A31 For the real-time microlens image video stream with consistent gray levels obtained in step A31, two adjacent frames of microlens images are intercepted, and the intersection of the ROIs of the two frames of images is calculated as a new ROI, and invalid information other than the ROI is removed.
  • the specific operation is as follows:
  • Bin() binarization operation
  • I′′(t) real-time microlens image after ROI correction at time t.
  • A33 Motion correction of real-time microlens image video streams
  • step A32 intercept two adjacent frames of microlens images, move the position of the previous frame and calculate the inter-frame MSE corresponding to different displacements, with the minimum MSE
  • the displacement of the value is the best motion estimate between two frames, and the motion error of the previous frame can be corrected according to the estimated value.
  • I′′ t a real-time microlens image with consistent gray scale and ROI at time t;
  • MSE t time t corresponds to all MSE values used for traversal inter-frame displacement estimation
  • I′′' t real-time microlens image with consistent gray scale, consistent ROI, and no motion error at time t.
  • A34 Calculate the inter-frame similarity in real time based on the corrected frame difference video stream, and establish the local defect detection result of the microlens.
  • a similarity measurement function suitable for specific tasks including but not limited to Absolute error, mean square error, two-dimensional correlation coefficient, spectral correlation coefficient and other similarity measures
  • a threshold judgment for the similarity between frames set a threshold judgment for the similarity between frames, and output the local defect detection results of the microlens.
  • Result(t) local defect detection result of the microlens at time t.
  • the method can output real-time local defect detection results for each microlens.
  • A4 Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
  • A41 Use the spatial correlation of light field images to count the distribution of defect features on adjacent microlenses, and establish the detection results of defect positions in single frame difference light field images;
  • step A34 use the microlens real-time local defect detection results obtained in step A34 to generate a series of defective microlens center spatial position distributions, and calculate the defect microlens density corresponding to each part of the current light field image frame.
  • Dist() distance measurement function, including but not limited to Euclidean distance, Gaussian distance, etc.
  • FrameResult(k) the detection result of the defect position at the center of the kth microlens in the current frame.
  • A42 Use the time-domain correlation of light field video to calculate the change rule of defect position between adjacent frames, and establish the final defect detection result of structured light field video.
  • step A41 using the single-frame difference light field image defect position detection results obtained in step A41, for areas that may have physical defects, if defects are detected in the area between adjacent frames, and the area position is within each frame If the moving speed between objects is close to the moving speed set by the detection device, then there is a greater possibility of defects at this position.
  • Real-time marking and outputting the defect position with time domain correlation on the light field video stream can obtain the final real-time structural light field video defect detection result.
  • the present invention is based on the defect detection method and device of structured light field video stream, combined with coding structured light technology and light field imaging technology, captures the geometric distortion of coding patterns caused by surface defects in light field video stream in a higher dimension, and improves the Accuracy and stability of machine vision defect detection.
  • the Background of the Invention section may contain background information about the problem or circumstances of the invention without necessarily describing prior art. Accordingly, inclusion in the Background section is not an admission by the applicant of prior art.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

A defect detection method and apparatus based on a structured light field video stream, the method comprising: A1: constructing a detection apparatus comprising an active encoding structured light source and a light field vision sensor; A2: using a white image to calibrate the light field vision sensor and, on the basis of the calibration result, decoding the structured light field video stream acquired by the light field vision sensor to obtain a real-time microlens image video stream; A3: performing motion correction, greyscale correction, and region-of-interest correction on the real-time microlens image video stream to calculate inter-frame similarity, and establishing a local defect detection result for the microlens; and A4: using the spatial domain and time domain correlation of the light field video, counting the distribution of defect features on an adjacent micro lens to obtain a structured light field video defect detection result; structured light encoding technology and light field imaging technology are combined to capture in a higher dimension the geometric distortion of an encoding pattern caused by surface defects in the light field video stream, thereby improving the accuracy and stability of machine vision defect detection.

Description

一种基于结构光场视频流的缺陷检测方法和装置A defect detection method and device based on structured light field video stream 技术领域technical field
本发明涉及计算机视觉与数字视频处理领域，特别是涉及一种基于结构光场视频流的缺陷检测方法和装置。The invention relates to the fields of computer vision and digital video processing, in particular to a defect detection method and device based on structured light field video streams.
背景技术Background technique
机器视觉缺陷检测是以光学传感器捕获的图像作为输入，通过计算机视觉方法提取物体表面图像特征，进而对其物理缺陷进行识别的技术。与传统人工识别相比，该技术检测精度高，检测速度快，在工业质检领域具有广阔的应用前景。然而，传统图像识别和立体视觉技术大多依赖于图像特征点与纹理信息，面对低纹理或无纹理的待检测物体以及工作场景时往往无能为力；基于结构光的检测技术能够重建物体表面的三维信息，但这类***造价高昂，检测精度受待检测物体材质、加工工艺影响明显。此外，以上基于传统视觉传感器的方法在检测微小缺陷时鲁棒性差，虚检、漏检较多，处理表面结构复杂的物体时需多次多角度成像，效率低下。Machine vision defect detection is a technology that takes the image captured by the optical sensor as input, extracts the surface image features of the object through computer vision methods, and then identifies its physical defects. Compared with traditional manual identification, this technology has high detection accuracy and fast detection speed, and has broad application prospects in the field of industrial quality inspection. However, traditional image recognition and stereo vision technologies mostly rely on image feature points and texture information, and are often powerless in the face of low-texture or no-texture objects to be detected and working scenes; detection technology based on structured light can reconstruct the three-dimensional information of the object surface , but this type of system is expensive, and the detection accuracy is significantly affected by the material and processing technology of the object to be detected. In addition, the above methods based on traditional visual sensors have poor robustness when detecting small defects, and there are many false detections and missed detections. When dealing with objects with complex surface structures, multiple multi-angle imaging is required, which is inefficient.
光场成像技术通过特殊的光路结构设计同时捕获包含空间维和角度维的四维光线信息，实现了单曝光多角度立体成像，但光场数据维度高，处理复杂，很难实现实时检测处理，将光场成像技术应用于工业检测应用的相关研究尚处于起步阶段。Light field imaging technology simultaneously captures four-dimensional light information including space dimension and angle dimension through a special optical path structure design, and realizes single-exposure multi-angle stereoscopic imaging. The related research on the application of field imaging technology to industrial inspection applications is still in its infancy.
需要说明的是，在上述背景技术部分公开的信息仅用于对本申请的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。It should be noted that the information disclosed in the above background technology section is only for understanding the background of the application, and therefore may include information that does not constitute prior art known to those of ordinary skill in the art.
发明内容Contents of the invention
针对现有机器视觉缺陷检测技术准确性、稳定性、鲁棒性以及实时性较差的问题，本发明提出了一种基于结构光场视频流的缺陷检测方法和装置。Aiming at the problems of poor accuracy, stability, robustness and real-time performance of the existing machine vision defect detection technology, the present invention proposes a defect detection method and device based on structured light field video stream.
为实现上述目的，本发明采用以下技术方案：To achieve the above object, the present invention adopts the following technical solutions:
一种基于结构光场视频流的缺陷检测方法，包括如下步骤：A defect detection method based on structured light field video stream, comprising the steps of:
A1：构建包含基于主动式编码结构光源和光场视觉传感器的检测装置；A1: Construct a detection device based on an active coded structured light source and a light field vision sensor;
A2：使用白图像对所述光场视觉传感器进行标定，根据标定结果对所述 光场视觉传感器获取的结构光场视频流进行解码，得到实时微透镜图像视频流；A2: use the white image to calibrate the light field visual sensor, decode the structured light field video stream obtained by the light field visual sensor according to the calibration result, and obtain a real-time microlens image video stream;
A3：对所述实时微透镜图像视频流进行运动校正、灰度校正以及感兴趣区域校正后计算帧间相似度，建立对微透镜的局部缺陷检测结果；A3: Perform motion correction, grayscale correction, and region-of-interest correction on the real-time microlens image video stream to calculate the similarity between frames, and establish a local defect detection result for the microlens;
A4：利用光场视频的空域、时域相关性，统计缺陷特征在相邻微透镜上的分布情况，得到最终的结构光场视频缺陷检测结果。A4: Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
进一步地：further:
所述检测装置包含能够发光并显示特定编码图案的主动式编码结构光源***和具备光场视频流采集能力的光场视觉传感器***，其中所述主动式编码结构光源***发出的光经待检测物体反射后被所述光场视觉传感器***所捕获；其中，所述主动式编码结构光源***由计算机程序控制，能够根据程序的要求在指定时间显示指定的编码图案，优选地，所述编码图案包括二维方波条纹、二维正弦波条纹、二维棋盘格中的一种或多种；所述光场视觉传感器***包括一至多台具备单曝光立体成像功能的视觉传感器设备，优选地，采用基于多路复用技术的光场相机或相机阵列式光场相机。The detection device includes an active coding structured light source system capable of emitting light and displaying specific coding patterns and a light field visual sensor system capable of collecting light field video streams, wherein the light emitted by the active coding structured light source system passes through the object to be detected After being reflected, it is captured by the light field visual sensor system; wherein, the active coding structured light source system is controlled by a computer program, and can display a specified coding pattern at a specified time according to the requirements of the program. Preferably, the coding pattern includes One or more of two-dimensional square wave stripes, two-dimensional sine wave stripes, and two-dimensional checkerboard grids; the light field visual sensor system includes one or more visual sensor devices with single-exposure stereoscopic imaging functions, preferably, using Light field camera or camera array light field camera based on multiplexing technology.
所述步骤A2中：In said step A2:
对于光场视觉传感器捕获的白图像，使用边缘检测算法对白图像中对应的微透镜中心进行初始估计后，将微透镜中心初始估计值输入与相机微透镜分布对应的优化模型，经过非线性优化后得到与光场视觉传感器微透镜几何分布对应的网格矢量，进而得到微透镜中心的精确像素坐标；优选地，具体操作如下：For the white image captured by the light field visual sensor, after using the edge detection algorithm to initially estimate the corresponding microlens center in the white image, the initial estimated value of the microlens center is input into the optimization model corresponding to the camera microlens distribution, after nonlinear optimization Obtain the grid vector corresponding to the microlens geometric distribution of the light field vision sensor, and then obtain the precise pixel coordinates of the microlens center; preferably, the specific operations are as follows:
Figure PCTCN2022076878-appb-000001
Figure PCTCN2022076878-appb-000001
Figure PCTCN2022076878-appb-000002
Figure PCTCN2022076878-appb-000002
Figure PCTCN2022076878-appb-000003
Figure PCTCN2022076878-appb-000003
Figure PCTCN2022076878-appb-000004
Figure PCTCN2022076878-appb-000004
上式中：In the above formula:
r—微透镜几何分布网格矢量；r—microlens geometric distribution grid vector;
θ—与微透镜阵列匹配的微透镜角度分布参数；θ—the microlens angle distribution parameter matched with the microlens array;
H—从像素坐标系到微透镜坐标系的变换矩阵；H—transformation matrix from pixel coordinate system to microlens coordinate system;
x k,y k—第k个微透镜中心的像素坐标值的初始估计； x k , y k —the initial estimate of the pixel coordinate value of the kth microlens center;
r refined—优化后的微透镜几何分布网格矢量； r refined — the optimized microlens geometric distribution grid vector;
H refined—根据r refined生成的新变换矩阵； H refined — new transformation matrix generated according to r refined ;
x k,refined,y k,refined—第k个微透镜中心的精确像素坐标值。 x k,refined ,y k,refined —The precise pixel coordinate value of the center of the kth microlens.
优选地，光场视频流指使用光场视觉传感器连续采集的光场图像序列；对序列中的每一帧光场原始图像，遍历所述标定结果中的所有微透镜中心，以其精确像素坐标值x i,refined,y i,refined为中心，选择与微透镜尺寸相近的邻域大小进行分割，得到实时微透镜图像视频流。 Preferably, the light field video stream refers to a light field image sequence continuously collected by a light field visual sensor; for each frame of light field original image in the sequence, traverse all microlens centers in the calibration result, and use its precise pixel coordinates The values x i,refined and y i,refined are the center, and the size of the neighborhood close to the size of the microlens is selected for segmentation to obtain a real-time microlens image video stream.
所述步骤A3包括：Described step A3 comprises:
A31：对实时微透镜图像视频流进行灰度校正；A31: Grayscale correction of real-time microlens image video stream;
A32：对实时微透镜图像视频流进行感兴趣区域(ROI)校正；A32: Perform region of interest (ROI) correction on the real-time microlens image video stream;
A33：对实时微透镜图像视频流进行运动校正；A33: Motion correction of real-time microlens image video streams;
A34：根据校正后的帧差视频流实时计算帧间相似度，建立对微透镜的局部缺陷检测结果。A34: Calculate the inter-frame similarity in real time based on the corrected frame difference video stream, and establish the local defect detection result of the microlens.
所述步骤A31中：In the step A31:
对于步骤A2得到的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像灰度均值，并对灰度较小的一帧进行线性变换以得到灰度变化一致的视频流；优选地，具体操作如下：For the real-time microlens image video stream obtained in step A2, two adjacent frames of microlens images are intercepted, the gray average value of the two frames of images is calculated, and a frame with a smaller grayscale is linearly transformed to obtain a consistent grayscale change. Video stream; preferably, the specific operations are as follows:
Figure PCTCN2022076878-appb-000005
Figure PCTCN2022076878-appb-000005
上式中：In the above formula:
I(t)—t时刻的实时微透镜图像；I(t)—real-time microlens image at time t;
Δt—光场视觉传感器的帧采样间隔；Δt—the frame sampling interval of the light field vision sensor;
I(t+Δt)—t时刻后的第一帧微透镜图像；I(t+Δt)—the first frame of microlens image after time t;
I′(t)—t时刻灰度校正后的实时微透镜图像。I'(t)—real-time microlens image after grayscale correction at time t.
所述步骤A32中：In the step A32:
对于步骤A31得到的灰度一致的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像ROI的交集部分作为新的ROI，去除ROI以外的无效信息；优选地，具体操作如下：For the real-time microlens image video stream with the same gray scale obtained in step A31, two adjacent frames of microlens images are intercepted, and the intersection of the two frames of image ROI is calculated as a new ROI, and invalid information other than ROI is removed; preferably, The specific operation is as follows:
I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t)     (6)I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t) (6)
上式中：In the above formula:
Bin()—二值化运算；Bin()—binarization operation;
I″(t)—t时刻ROI校正后的实时微透镜图像。I″(t)—real-time microlens image after ROI correction at time t.
所述步骤A33中：In the step A33:
对于步骤A32得到的灰度、ROI一致的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，移动前帧的位置并计算不同位移对应的帧间MSE，具有最小MSE值的位移即为两帧间的最佳运动估计，根据该估计值可对前帧运动误差进行修正；优选地，具体操作如下：For the real-time microlens image video stream with consistent grayscale and ROI obtained in step A32, intercept two adjacent frames of microlens images, move the position of the previous frame and calculate the MSE between frames corresponding to different displacements, and the displacement with the minimum MSE value It is the best motion estimation between two frames, and the motion error of the previous frame can be corrected according to the estimated value; preferably, the specific operation is as follows:
Figure PCTCN2022076878-appb-000006
Figure PCTCN2022076878-appb-000006
Figure PCTCN2022076878-appb-000007
Figure PCTCN2022076878-appb-000007
I t″′(i,j)＝I t″(i+u t,j+v t)(9) I t "'(i,j)=I t "(i+u t ,j+v t )(9)
上式中：In the above formula:
I″ t—t时刻灰度、ROI一致的实时微透镜图像； I″ t —a real-time microlens image with consistent gray scale and ROI at time t;
M,N—微透镜图像尺寸；M, N—microlens image size;
i,j—用于遍历的微透镜图像像素坐标；i, j—the pixel coordinates of the microlens image used for traversal;
u,v—用于遍历的帧间位移估计；u,v—interframe displacement estimation for traversal;
MSE t—t时刻对应所有用于遍历的帧间位移估计的MSE值； MSE t — time t corresponds to all MSE values used for traversal inter-frame displacement estimation;
u t,v t—t时刻的最佳帧间运动估计； u t , v t —the best inter-frame motion estimation at time t;
I″′ t—t时刻灰度、ROI一致、无运动误差的实时微透镜图像。 I″' t —real-time microlens image with consistent gray scale, consistent ROI, and no motion error at time t.
所述步骤A34中：In the step A34:
对于步骤A33得到的灰度、ROI一致且无运动误差的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，选择适合具体任务的相似性测度函数，对帧间相似度设置阈值判断，输出对微透镜的局部缺陷检测结果；优选地，具体操作如下：For the real-time microlens image video stream with consistent grayscale and ROI and no motion error obtained in step A33, intercept two adjacent frames of microlens images, select a similarity measurement function suitable for specific tasks, and set a threshold for the similarity between frames Judging and outputting the local defect detection result of the microlens; preferably, the specific operations are as follows:
Result(t)＝Bin[Similarity(I″′ t,I″′ t+Δt)]      (10) Result(t)=Bin[Similarity(I″′ t , I″′ t+Δt )] (10)
上式中：In the above formula:
Similarity()—相似性测度函数；Similarity()—similarity measure function;
Result(t)—t时刻该微透镜的局部缺陷检测结果。Result(t)—local defect detection result of the microlens at time t.
所述步骤A4包括：Described step A4 comprises:
A41：利用光场图像的空域相关性，统计缺陷特征在相邻微透镜上的分布情况，建立单帧差光场图像中的缺陷位置检测结果；A41: Use the spatial correlation of light field images to count the distribution of defect features on adjacent microlenses, and establish the detection results of defect positions in single frame difference light field images;
A42：利用光场视频的时域相关性，统计缺陷位置在相邻帧间的变化规律，建立最终的结构光场视频缺陷检测结果。A42: Use the time-domain correlation of light field video to calculate the change rule of defect position between adjacent frames, and establish the final defect detection result of structured light field video.
优选地，所述步骤A41中：Preferably, in the step A41:
使用步骤A34所得到的微透镜实时局部缺陷检测结果，生成一系列具有缺陷的微透镜中心空间位置分布，计算当前光场图像帧各处对应的缺陷微透镜密度，密度越大则存在物理缺陷的可能性越大；更优选地，具体操作如下：Use the microlens real-time local defect detection results obtained in step A34 to generate a series of defective microlens center spatial position distributions, and calculate the defect microlens density corresponding to each part of the current light field image frame. The greater the density, the presence of physical defects The greater the possibility; more preferably, the specific operations are as follows:
Figure PCTCN2022076878-appb-000008
Figure PCTCN2022076878-appb-000008
FrameResult(k)＝Bin[D(k)]      (12)上式中：FrameResult(k)＝Bin[D(k)] (12) In the above formula:
Dist()—距离测度函数，包括但不限于欧氏距离、高斯距离等；Dist()—distance measurement function, including but not limited to Euclidean distance, Gaussian distance, etc.;
S—当前帧存在缺陷的微透镜总数；S—the total number of defective microlenses in the current frame;
D(k)—第k个微透镜中心处的缺陷密度；D(k)—defect density at the center of the kth microlens;
FrameResult(k)—当前帧第k个微透镜中心处的缺陷位置检测结果。FrameResult(k)—the detection result of the defect position at the center of the kth microlens in the current frame.
优选地，所述步骤A42中：Preferably, in the step A42:
使用步骤A41所得到的单帧差光场图像缺陷位置检测结果，对于可能存在物理缺陷的区域，如果该区域在相邻的数帧之间均检出缺陷，且区域位置在各帧间移动速度与检测装置设定的物体移动速度接近，则判断该位置存在缺陷的可能性较大；在光场视频流上对这种存在时域相关性的缺陷位置进行实时标记和输出。Using the single-frame differential light field image defect position detection results obtained in step A41, for areas that may have physical defects, if defects are detected in the area between adjacent frames, and the area position moves at a speed between frames If it is close to the moving speed of the object set by the detection device, it is more likely to judge that there is a defect in the position; the defect position with time domain correlation is marked and output in real time on the light field video stream.
一种基于结构光场视频流的缺陷检测装置，包括处理器和基于主动式编码结构光源和光场视觉传感器的检测装置，所述处理器执行计算机可读存储介质上的计算机程序时，实现所述的基于结构光场视频流的缺陷检测方法的步骤A2-A4。A defect detection device based on a structured light field video stream, comprising a processor and a detection device based on an active coded structured light source and a light field visual sensor, when the processor executes a computer program on a computer-readable storage medium, the described Steps A2-A4 of the defect detection method based on structured light field video stream.
本发明具有如下有益效果：The present invention has following beneficial effects:
本发明提出了一种基于结构光场视频流的缺陷检测方法和装置，结合编码结构光技术与光场成像技术，利用主动式编码结构光源将待检测物体表面物理缺陷转化为物体表面反射编码图案的几何畸变，利用光场成像在 更高的维度上通过更少的曝光次数捕捉这种几何畸变，将传统相机视域内的大范围不明显畸变转化为光场微透镜图像水平上具有空域相关性的小范围显著畸变。与现有方法相比，本发明的方法能够有效提升结构光缺陷检测的准确度与稳定性，解决了复杂工件光场视觉缺陷检测“从无到有”的问题，同时显著提升了光场视频检测方法的实时性，对于精密机械加工等行业以及光场成像技术的推广应用具有重要意义。The present invention proposes a defect detection method and device based on structured light field video streams, combined with coded structured light technology and light field imaging technology, using an active coded structured light source to convert physical defects on the surface of an object to be detected into a reflective coded pattern on the surface of the object The geometric distortion of , using light field imaging to capture this geometric distortion in a higher dimension with fewer exposures, transforming the large-scale inconspicuous distortion in the field of view of traditional cameras into light field microlens image level with spatial correlation Significant distortion in a small range. Compared with the existing methods, the method of the present invention can effectively improve the accuracy and stability of structured light defect detection, solve the problem of "starting from scratch" in the detection of complex workpiece light field visual defects, and significantly improve the quality of light field video The real-time performance of the detection method is of great significance for industries such as precision machining and the popularization and application of light field imaging technology.
附图说明Description of drawings
图1为本发明实施例的基于结构光场视频流的缺陷检测方法的流程图。FIG. 1 is a flow chart of a defect detection method based on structured light field video streams according to an embodiment of the present invention.
具体实施方式Detailed ways
以下对本发明的实施方式做详细说明。应该强调的是，下述说明仅仅是示例性的，而不是为了限制本发明的范围及其应用。Embodiments of the present invention will be described in detail below. It should be emphasized that the following description is only exemplary and not intended to limit the scope of the invention and its application.
本发明提出一种基于结构光场视频流的缺陷检测方法和装置，主要思路为：结合编码结构光技术与光场成像技术，利用主动式编码结构光源将待检测物体物理缺陷转化为编码图案的几何畸变，利用光场成像在更高的维度上通过更少的曝光次数捕捉这种几何畸变，将传统相机视域内的大范围不明显畸变转化为光场微透镜图像水平上具有空域相关性的小范围显著畸变，实现了对于多种材质物体物理缺陷的准确、高效且稳定的检测。在一些实施例中，如图1所示，所述缺陷检测方法包括如下步骤：The present invention proposes a method and device for detecting defects based on structured light field video streams. The main idea is: combining coded structured light technology and light field imaging technology, using active coded structured light sources to convert physical defects of objects to be detected into coded patterns Geometric distortion, using light field imaging to capture this geometric distortion in a higher dimension with fewer exposures, transforming the large-scale inconspicuous distortion in the field of view of traditional cameras into spatial correlation at the level of light field microlens images Significant distortion in a small range enables accurate, efficient and stable detection of physical defects in objects of various materials. In some embodiments, as shown in Figure 1, the defect detection method includes the following steps:
A1：构建包含基于主动式编码结构光源、光场视觉传感器的检测装置；A1: Construct a detection device based on an active coded structured light source and a light field vision sensor;
A2：使用白图像对光场视觉传感器进行标定，根据标定结果对该传感器获取的结构光场视频流进行解码，得到实时微透镜图像视频流；A2: Use the white image to calibrate the light field vision sensor, and decode the structured light field video stream obtained by the sensor according to the calibration result to obtain a real-time microlens image video stream;
A3：对实时微透镜图像视频流进行灰度校正、感兴趣区域校正以及运动校正后计算帧间相似度，建立对微透镜的局部缺陷检测结果；A3: Calculate the inter-frame similarity after performing gray scale correction, region of interest correction and motion correction on the real-time microlens image video stream, and establish the local defect detection results of the microlens;
A4：利用光场视频的空域、时域相关性，统计缺陷特征在相邻微透镜上的分布情况，得到最终的结构光场视频缺陷检测结果。A4: Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
在具体实施例中执行以上步骤时，可以按照以下方式操作。需注意的是，在实施过程中所采用的具体方法都仅为举例说明，本发明所涵盖的范围包括但不局限于所列举的以下具体方法。When performing the above steps in a specific embodiment, the operation may be performed in the following manner. It should be noted that the specific methods used in the implementation process are only examples, and the scope of the present invention includes but is not limited to the following specific methods listed.
A1：构建包含基于主动式编码结构光源、光场视觉传感器的检测装置。A1: Construct a detection device based on an active coded structured light source and a light field vision sensor.
具体地，装置包含能够发光并显示特定编码图案的主动式编码结构光源***和具备光场视频流采集能力的光场视觉传感器***。主动式编码结构光源***由计算机程序控制，能够根据程序的要求在指定时间显示指定的 编码图案。编码图案包括但不限于二维方波条纹、二维正弦波条纹、二维棋盘格等包含空间调制信息的图案，光源***硬件载体包括但不限于编码图案覆盖的摄影灯、灯箱以及可由计算机程序控制的电子显示屏等显示设备。光场视觉传感器***由一至多台具备单曝光立体成像功能的视觉传感器设备构成，其硬件载体包括但不限于基于多路复用技术的光场相机、相机阵列式光场相机以及其他能够通过解码得到光场图像的立体成像设备。***中光源***发出的光经待检测物体反射后被光场视觉传感器***所捕获。Specifically, the device includes an active coded structured light source system capable of emitting light and displaying specific coded patterns, and a light field visual sensor system capable of collecting light field video streams. The active coded structured light source system is controlled by a computer program, which can display a designated coded pattern at a designated time according to the requirements of the program. Coding patterns include but are not limited to two-dimensional square wave stripes, two-dimensional sine wave stripes, two-dimensional checkerboard and other patterns containing spatial modulation information. Controlled display devices such as electronic display screens. The light field visual sensor system consists of one or more visual sensor devices capable of single-exposure stereoscopic imaging, and its hardware carriers include but are not limited to light field cameras based on multiplexing technology, camera array light field cameras, and other Stereo imaging device for obtaining light field images. The light emitted by the light source system in the system is captured by the light field vision sensor system after being reflected by the object to be detected.
A2：使用白图像对光场视觉传感器进行标定，根据标定结果对该传感器获取的结构光场视频流进行解码，得到实时微透镜图像视频流。A2: Use the white image to calibrate the light field vision sensor, and decode the structured light field video stream acquired by the sensor according to the calibration result to obtain a real-time microlens image video stream.
具体地，对于光场视觉传感器捕获的白图像，使用Hough圆变换等传统边缘检测算法对白图像中对应的微透镜中心进行初始估计后，将微透镜中心初始估计值输入与相机微透镜分布对应的优化模型，经过非线性优化后得到与光场视觉传感器微透镜几何分布对应的网格矢量，进而得到微透镜中心的精确像素坐标。具体操作如下：Specifically, for the white image captured by the light field vision sensor, after initial estimation of the corresponding microlens center in the white image using traditional edge detection algorithms such as Hough circle transform, the initial estimated value of the microlens center is input into the corresponding microlens distribution of the camera. The model is optimized. After nonlinear optimization, the grid vector corresponding to the geometric distribution of the microlens of the light field vision sensor is obtained, and then the precise pixel coordinates of the center of the microlens are obtained. The specific operation is as follows:
Figure PCTCN2022076878-appb-000009
Figure PCTCN2022076878-appb-000009
Figure PCTCN2022076878-appb-000010
Figure PCTCN2022076878-appb-000010
Figure PCTCN2022076878-appb-000011
Figure PCTCN2022076878-appb-000011
Figure PCTCN2022076878-appb-000012
Figure PCTCN2022076878-appb-000012
上式中：In the above formula:
r—微透镜几何分布网格矢量；r—microlens geometric distribution grid vector;
θ—与微透镜阵列匹配的微透镜角度分布参数；θ—the microlens angle distribution parameter matched with the microlens array;
H—从像素坐标系到微透镜坐标系的变换矩阵；H—transformation matrix from pixel coordinate system to microlens coordinate system;
x i,y i—第i个微透镜中心的像素坐标值的初始估计； x i , y i —the initial estimation of the pixel coordinate value of the i-th microlens center;
r refined—优化后的微透镜几何分布网格矢量； r refined — the optimized microlens geometric distribution grid vector;
H refined—根据r refined生成的新变换矩阵； H refined — new transformation matrix generated according to r refined ;
x i,refined,y i,refined—第i个微透镜中心的精确像素坐标值。 x i,refined ,y i,refined —The precise pixel coordinate value of the i-th microlens center.
光场视频流指使用光场视觉传感器连续采集的光场图像序列。对序列中的每一帧光场原始图像，遍历上述标定结果中的所有微透镜中心，以其精确像素坐标值x i,refined,y i,refined为中心，选择与微透镜尺寸相近的邻域大小进行分割，即可得到实时微透镜图像视频流。 A light field video stream refers to a sequence of light field images continuously captured using a light field vision sensor. For each frame of the light field original image in the sequence, traverse all the centers of the microlenses in the above calibration results, center on their precise pixel coordinate values x i,refined ,y i,refined , and select a neighborhood with a size similar to the size of the microlens The real-time microlens image video stream can be obtained by dividing the size.
A3：对实时微透镜图像视频流进行灰度校正、感兴趣区域校正以及运动校正后计算帧间相似度，建立对微透镜的局部缺陷检测结果。A3: Perform gray scale correction, region of interest correction, and motion correction on the real-time microlens image video stream to calculate the similarity between frames, and establish the local defect detection results of the microlens.
A31：对实时微透镜图像视频流进行灰度校正；A31: Grayscale correction of real-time microlens image video stream;
具体地，对于步骤A2得到的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像灰度均值，并对灰度较小的一帧进行线性变换以得到灰度变化一致的视频流。具体操作如下(假设灰度较小的一帧为前帧)：Specifically, for the real-time microlens image video stream obtained in step A2, two adjacent frames of microlens images are intercepted, the gray average value of the two frames of images is calculated, and a frame with a smaller grayscale is linearly transformed to obtain the grayscale Variation consistent video stream. The specific operation is as follows (assuming that a frame with a smaller gray scale is the previous frame):
Figure PCTCN2022076878-appb-000013
Figure PCTCN2022076878-appb-000013
上式中：In the above formula:
I(t)—t时刻的实时微透镜图像；I(t)—real-time microlens image at time t;
Δt—光场视觉传感器的帧采样间隔；Δt—the frame sampling interval of the light field vision sensor;
I(t+Δt)—t时刻后的第一帧微透镜图像；I(t+Δt)—the first frame of microlens image after time t;
I′(t)—t时刻灰度校正后的实时微透镜图像。I'(t)—real-time microlens image after grayscale correction at time t.
A32：对实时微透镜图像视频流进行感兴趣区域(ROI)校正；A32: Perform region of interest (ROI) correction on the real-time microlens image video stream;
具体地，对于步骤A31得到的灰度一致的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像ROI的交集部分作为新的ROI，去除ROI以外的无效信息。具体操作如下：Specifically, for the real-time microlens image video stream with consistent gray levels obtained in step A31, two adjacent frames of microlens images are intercepted, and the intersection of the ROIs of the two frames of images is calculated as a new ROI, and invalid information other than the ROI is removed. The specific operation is as follows:
I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t)     (6)I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t) (6)
上式中：In the above formula:
Bin()—二值化运算；Bin()—binarization operation;
I″(t)—t时刻ROI校正后的实时微透镜图像。I″(t)—real-time microlens image after ROI correction at time t.
A33：对实时微透镜图像视频流进行运动校正；A33: Motion correction of real-time microlens image video streams;
具体地，对于步骤A32得到的灰度、ROI一致的实时微透镜图像视频 流，截取其中相邻的两帧微透镜图像，移动前帧的位置并计算不同位移对应的帧间MSE，具有最小MSE值的位移即为两帧间的最佳运动估计，根据该估计值即可对前帧运动误差进行修正。具体操作如下：Specifically, for the grayscale and ROI-consistent real-time microlens image video stream obtained in step A32, intercept two adjacent frames of microlens images, move the position of the previous frame and calculate the inter-frame MSE corresponding to different displacements, with the minimum MSE The displacement of the value is the best motion estimate between two frames, and the motion error of the previous frame can be corrected according to the estimated value. The specific operation is as follows:
Figure PCTCN2022076878-appb-000014
Figure PCTCN2022076878-appb-000014
Figure PCTCN2022076878-appb-000015
Figure PCTCN2022076878-appb-000015
I″′ t(i,j)＝I″ t(i+u t,j+v t)      (9) I″′ t (i,j)=I″ t (i+u t ,j+v t ) (9)
上式中：In the above formula:
I″ t—t时刻灰度、ROI一致的实时微透镜图像； I″ t —a real-time microlens image with consistent gray scale and ROI at time t;
M,N—微透镜图像尺寸；M, N—microlens image size;
u,v—用于遍历的帧间位移估计；u,v—interframe displacement estimation for traversal;
MSE t—t时刻对应所有用于遍历的帧间位移估计的MSE值； MSE t — time t corresponds to all MSE values used for traversal inter-frame displacement estimation;
u t,v t—t时刻的最佳帧间运动估计； u t , v t —the best inter-frame motion estimation at time t;
I″′ t—t时刻灰度、ROI一致、无运动误差的实时微透镜图像。 I″' t —real-time microlens image with consistent gray scale, consistent ROI, and no motion error at time t.
A34：根据校正后的帧差视频流实时计算帧间相似度，建立对微透镜的局部缺陷检测结果。A34: Calculate the inter-frame similarity in real time based on the corrected frame difference video stream, and establish the local defect detection result of the microlens.
具体地，对于步骤A33得到的灰度、ROI一致且无运动误差的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，选择适合具体任务的相似性测度函数(包括但不限于绝对误差、均方误差、二维相关系数、频谱相关系数等相似性测度)，对帧间相似度设置阈值判断，输出对微透镜的局部缺陷检测结果。具体操作如下：Specifically, for the real-time microlens image video stream with consistent grayscale and ROI and no motion error obtained in step A33, two adjacent frames of microlens images are intercepted, and a similarity measurement function suitable for specific tasks (including but not limited to Absolute error, mean square error, two-dimensional correlation coefficient, spectral correlation coefficient and other similarity measures), set a threshold judgment for the similarity between frames, and output the local defect detection results of the microlens. The specific operation is as follows:
Result(t)＝Bin[Similarity(I″′ t,I″′ t+Δt)]     (10) Result(t)=Bin[Similarity(I″′ t , I″′ t+Δt )] (10)
上式中：In the above formula:
Similarity()—相似性测度函数；Similarity()—similarity measure function;
Result(t)—t时刻该微透镜的局部缺陷检测结果。Result(t)—local defect detection result of the microlens at time t.
该方法可输出对于每一个微透镜的实时局部缺陷检测结果。The method can output real-time local defect detection results for each microlens.
A4：利用光场视频的空域、时域相关性，统计缺陷特征在相邻微透镜上的分布情况，得到最终的结构光场视频缺陷检测结果。A4: Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
A41：利用光场图像的空域相关性，统计缺陷特征在相邻微透镜上的分 布情况，建立单帧差光场图像中的缺陷位置检测结果；A41: Use the spatial correlation of light field images to count the distribution of defect features on adjacent microlenses, and establish the detection results of defect positions in single frame difference light field images;
具体地，使用步骤A34所得到的微透镜实时局部缺陷检测结果，生成一系列具有缺陷的微透镜中心空间位置分布，计算当前光场图像帧各处对应的缺陷微透镜密度，密度越大则存在物理缺陷的可能性越大。具体操作如下：Specifically, use the microlens real-time local defect detection results obtained in step A34 to generate a series of defective microlens center spatial position distributions, and calculate the defect microlens density corresponding to each part of the current light field image frame. The greater the likelihood of physical defects. The specific operation is as follows:
Figure PCTCN2022076878-appb-000016
Figure PCTCN2022076878-appb-000016
FrameResult(k)＝Bin[D(k)]     (12)FrameResult(k)＝Bin[D(k)] (12)
上式中：In the above formula:
Dist()—距离测度函数，包括但不限于欧氏距离、高斯距离等；Dist()—distance measurement function, including but not limited to Euclidean distance, Gaussian distance, etc.;
S—当前帧存在缺陷的微透镜总数；S—the total number of defective microlenses in the current frame;
D(k)—第k个微透镜中心处的缺陷密度；D(k)—defect density at the center of the kth microlens;
FrameResult(k)—当前帧第k个微透镜中心处的缺陷位置检测结果。FrameResult(k)—the detection result of the defect position at the center of the kth microlens in the current frame.
A42：利用光场视频的时域相关性，统计缺陷位置在相邻帧间的变化规律，建立最终的结构光场视频缺陷检测结果。A42: Use the time-domain correlation of light field video to calculate the change rule of defect position between adjacent frames, and establish the final defect detection result of structured light field video.
具体地，使用步骤A41所得到的单帧差光场图像缺陷位置检测结果，对于可能存在物理缺陷的区域，如果该区域在相邻的数帧之间均检出缺陷，且区域位置在各帧间移动速度与检测装置设定的物体移动速度接近，则该位置存在缺陷的可能性较大。在光场视频流上对这种存在时域相关性的缺陷位置进行实时标记和输出，即可得到最终的实时结构光场视频缺陷检测结果。Specifically, using the single-frame difference light field image defect position detection results obtained in step A41, for areas that may have physical defects, if defects are detected in the area between adjacent frames, and the area position is within each frame If the moving speed between objects is close to the moving speed set by the detection device, then there is a greater possibility of defects at this position. Real-time marking and outputting the defect position with time domain correlation on the light field video stream can obtain the final real-time structural light field video defect detection result.
本发明基于结构光场视频流的缺陷检测方法和装置，结合编码结构光技术与光场成像技术，在更高的维度上捕捉表面缺陷在光场视频流中导致的编码图案几何畸变，提高了机器视觉缺陷检测的准确度与稳定性。The present invention is based on the defect detection method and device of structured light field video stream, combined with coding structured light technology and light field imaging technology, captures the geometric distortion of coding patterns caused by surface defects in light field video stream in a higher dimension, and improves the Accuracy and stability of machine vision defect detection.
本发明的背景部分可以包含关于本发明的问题或环境的背景信息，而不一定是描述现有技术。因此，在背景技术部分中包含的内容并不是申请人对现有技术的承认。The Background of the Invention section may contain background information about the problem or circumstances of the invention without necessarily describing prior art. Accordingly, inclusion in the Background section is not an admission by the applicant of prior art.
以上内容是结合具体/优选的实施方式对本发明所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，其还可以对这 些已描述的实施方式做出若干替代或变型，而这些替代或变型方式都应当视为属于本发明的保护范围。在本说明书的描述中，参考术语“一种实施例”、“一些实施例”、“优选实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。尽管已经详细描述了本发明的实施例及其优点，但应当理解，在不脱离专利申请的保护范围的情况下，可以在本文中进行各种改变、替换和变更。The above content is a further detailed description of the present invention in conjunction with specific/preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, they can also make some substitutions or modifications to the described embodiments, and these substitutions or modifications should be regarded as Belong to the protection scope of the present invention. In the description of this specification, references to the terms "one embodiment," "some embodiments," "preferred embodiments," "examples," "specific examples," or "some examples" are intended to mean A specific feature, structure, material, or characteristic described by an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. Those skilled in the art may combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that the various changes, substitutions and alterations could be made herein without departing from the protection scope of the patent application.

Claims (10)

  1. 一种基于结构光场视频流的缺陷检测方法，其特征在于，包括如下步骤：A defect detection method based on a structured light field video stream, characterized in that it comprises the following steps:
    A1：构建包含基于主动式编码结构光源和光场视觉传感器的检测装置；A1: Construct a detection device based on an active coded structured light source and a light field vision sensor;
    A2：使用白图像对所述光场视觉传感器进行标定，根据标定结果对所述光场视觉传感器获取的结构光场视频流进行解码，得到实时微透镜图像视频流；A2: Use the white image to calibrate the light field visual sensor, and decode the structured light field video stream obtained by the light field visual sensor according to the calibration result to obtain a real-time microlens image video stream;
    A3：对所述实时微透镜图像视频流进行运动校正、灰度校正以及感兴趣区域校正后计算帧间相似度，建立对微透镜的局部缺陷检测结果；A3: Perform motion correction, grayscale correction, and region-of-interest correction on the real-time microlens image video stream to calculate the similarity between frames, and establish a local defect detection result for the microlens;
    A4：利用光场视频的空域、时域相关性，统计缺陷特征在相邻微透镜上的分布情况，得到最终的结构光场视频缺陷检测结果。A4: Using the spatial domain and time domain correlation of light field video, the distribution of defect features on adjacent microlenses is counted, and the final defect detection result of structured light field video is obtained.
  2. 如权利要求1所述的检测装置，其特征在于，所述检测装置包含能够发光并显示特定编码图案的主动式编码结构光源***和具备光场视频流采集能力的光场视觉传感器***，其中所述主动式编码结构光源***发出的光经待检测物体反射后被所述光场视觉传感器***所捕获；其中，所述主动式编码结构光源***由计算机程序控制，能够根据程序的要求在指定时间显示指定的编码图案，优选地，所述编码图案包括二维方波条纹、二维正弦波条纹、二维棋盘格中的一种或多种；所述光场视觉传感器***包括一至多台具备单曝光立体成像功能的视觉传感器设备，优选地，采用基于多路复用技术的光场相机或相机阵列式光场相机。The detection device according to claim 1, wherein the detection device comprises an active coding structure light source system capable of emitting light and displaying a specific coding pattern and a light field visual sensor system capable of collecting light field video streams, wherein the The light emitted by the active coded structured light source system is captured by the light field vision sensor system after being reflected by the object to be detected; wherein, the active coded structured light source system is controlled by a computer program and can be detected at a specified time according to the requirements of the program. Display a specified coding pattern, preferably, the coding pattern includes one or more of two-dimensional square wave stripes, two-dimensional sine wave stripes, and two-dimensional checkerboard; the light field vision sensor system includes one or more The visual sensor device with single-exposure stereoscopic imaging function preferably adopts a light field camera or camera array light field camera based on multiplexing technology.
  3. 如权利要求1或2所述的方法，其特征在于，所述步骤A2中：The method according to claim 1 or 2, characterized in that, in the step A2:
    对于光场视觉传感器捕获的白图像，使用边缘检测算法对白图像中对应的微透镜中心进行初始估计后，将微透镜中心初始估计值输入与相机微透镜分布对应的优化模型，经过非线性优化后得到与光场视觉传感器微透镜几何分布对应的网格矢量，进而得到微透镜中心的精确像素坐标；优选地，具体操作如下：For the white image captured by the light field visual sensor, after using the edge detection algorithm to initially estimate the corresponding microlens center in the white image, the initial estimated value of the microlens center is input into the optimization model corresponding to the camera microlens distribution, after nonlinear optimization Obtain the grid vector corresponding to the microlens geometric distribution of the light field vision sensor, and then obtain the precise pixel coordinates of the microlens center; preferably, the specific operations are as follows:
    Figure PCTCN2022076878-appb-100001
    Figure PCTCN2022076878-appb-100001
    Figure PCTCN2022076878-appb-100002
    Figure PCTCN2022076878-appb-100002
    Figure PCTCN2022076878-appb-100003
    Figure PCTCN2022076878-appb-100003
    Figure PCTCN2022076878-appb-100004
    Figure PCTCN2022076878-appb-100004
    上式中：In the above formula:
    r—微透镜几何分布网格矢量；r—microlens geometric distribution grid vector;
    θ—与微透镜阵列匹配的微透镜角度分布参数；θ—the microlens angle distribution parameter matched with the microlens array;
    H—从像素坐标系到微透镜坐标系的变换矩阵；H—transformation matrix from pixel coordinate system to microlens coordinate system;
    x k,y k—第k个微透镜中心的像素坐标值的初始估计； x k , y k —the initial estimate of the pixel coordinate value of the kth microlens center;
    r refined—优化后的微透镜几何分布网格矢量； r refined — the optimized microlens geometric distribution grid vector;
    H refined—根据r refined生成的新变换矩阵； H refined — new transformation matrix generated according to r refined ;
    x k,refined,y k,refined—第k个微透镜中心的精确像素坐标值； x k,refined ,y k,refined — the precise pixel coordinate value of the center of the kth microlens;
    优选地，光场视频流指使用光场视觉传感器连续采集的光场图像序列；对序列中的每一帧光场原始图像，遍历所述标定结果中的所有微透镜中心，以其精确像素坐标值x i,refined,y i,refined为中心，选择与微透镜尺寸相近的邻域大小进行分割，得到实时微透镜图像视频流。 Preferably, the light field video stream refers to a light field image sequence continuously collected by a light field visual sensor; for each frame of light field original image in the sequence, traverse all microlens centers in the calibration result, and use its precise pixel coordinates The values x i,refined and y i,refined are the center, and the size of the neighborhood close to the size of the microlens is selected for segmentation to obtain a real-time microlens image video stream.
  4. 如权利要求1至3任一项所述的方法，其特征在于，所述步骤A3包括：The method according to any one of claims 1 to 3, wherein said step A3 comprises:
    A31：对实时微透镜图像视频流进行灰度校正；A31: Grayscale correction of real-time microlens image video stream;
    A32：对实时微透镜图像视频流进行感兴趣区域ROI校正；A32: Perform ROI correction on the real-time microlens image video stream;
    A33：对实时微透镜图像视频流进行运动校正；A33: Motion correction of real-time microlens image video streams;
    A34：根据校正后的帧差视频流实时计算帧间相似度，建立对微透镜的局部缺陷检测结果。A34: Calculate the inter-frame similarity in real time based on the corrected frame difference video stream, and establish the local defect detection results of the microlens.
  5. 如权利要求4所述的方法，其特征在于，所述步骤A31中：The method according to claim 4, characterized in that, in the step A31:
    对于步骤A2得到的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像灰度均值，并对灰度较小的一帧进行线性变换以得到灰度变化一致的视频流；优选地，具体操作如下：For the real-time microlens image video stream obtained in step A2, two adjacent frames of microlens images are intercepted, the gray average value of the two frames of images is calculated, and a frame with a smaller grayscale is linearly transformed to obtain a consistent grayscale change. Video stream; preferably, the specific operations are as follows:
    Figure PCTCN2022076878-appb-100005
    Figure PCTCN2022076878-appb-100005
    上式中：In the above formula:
    I(t)—t时刻的实时微透镜图像；I(t)—real-time microlens image at time t;
    Δt—光场视觉传感器的帧采样间隔；Δt—the frame sampling interval of the light field vision sensor;
    I(t+Δt)—t时刻后的第一帧微透镜图像；I(t+Δt)—the first frame of microlens image after time t;
    I′(t)—t时刻灰度校正后的实时微透镜图像。I'(t)—real-time microlens image after grayscale correction at time t.
  6. 如权利要求4或5所述的方法，其特征在于，所述步骤A32中：The method according to claim 4 or 5, characterized in that, in the step A32:
    对于步骤A31得到的灰度一致的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，计算两帧图像ROI的交集部分作为新的ROI，去除ROI以外的无效信息；优选地，具体操作如下：For the real-time microlens image video stream with the same gray scale obtained in step A31, two adjacent frames of microlens images are intercepted, and the intersection of the two frames of image ROI is calculated as a new ROI, and invalid information other than ROI is removed; preferably, The specific operation is as follows:
    I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t)    (6)I″(t)＝∩{Bin[I′(t)],Bin[I′(t+Δt)]}·I′(t) (6)
    上式中：In the above formula:
    Bin()—二值化运算；Bin()—binarization operation;
    I″(t)—t时刻ROI校正后的实时微透镜图像。I″(t)—real-time microlens image after ROI correction at time t.
  7. 如权利要求4至6任一项所述的方法，其特征在于，所述步骤A33中：The method according to any one of claims 4 to 6, characterized in that, in the step A33:
    对于步骤A32得到的灰度、ROI一致的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，移动前帧的位置并计算不同位移对应的帧间MSE，具有最小MSE值的位移即为两帧间的最佳运动估计，根据该估计值可对前帧运动误差进行修正；优选地，具体操作如下：For the real-time microlens image video stream with consistent grayscale and ROI obtained in step A32, intercept two adjacent frames of microlens images, move the position of the previous frame and calculate the MSE between frames corresponding to different displacements, and the displacement with the minimum MSE value It is the best motion estimation between two frames, and the motion error of the previous frame can be corrected according to the estimated value; preferably, the specific operation is as follows:
    Figure PCTCN2022076878-appb-100006
    Figure PCTCN2022076878-appb-100006
    Figure PCTCN2022076878-appb-100007
    Figure PCTCN2022076878-appb-100007
    I″′ t(i,j)＝I″ t(i+u t,j+v t)     (9) I″′ t (i,j)=I″ t (i+u t ,j+v t ) (9)
    上式中：In the above formula:
    I″ t—t时刻灰度、ROI一致的实时微透镜图像； I″ t —a real-time microlens image with consistent gray scale and ROI at time t;
    M,N—微透镜图像尺寸；M, N—microlens image size;
    i,j—用于遍历的微透镜图像像素坐标；i, j—the pixel coordinates of the microlens image used for traversal;
    u,v—用于遍历的帧间位移估计；u,v—interframe displacement estimation for traversal;
    MSE t—t时刻对应所有用于遍历的帧间位移估计的MSE值； MSE t — time t corresponds to all MSE values used for traversal inter-frame displacement estimation;
    u t,v t—t时刻的最佳帧间运动估计； u t , v t —the best inter-frame motion estimation at time t;
    I″′ t—t时刻灰度、ROI一致、无运动误差的实时微透镜图像。 I″' t —real-time microlens image with consistent gray scale, consistent ROI, and no motion error at time t.
  8. 如权利要求4至7任一项所述的方法，其特征在于，所述步骤A34中：The method according to any one of claims 4 to 7, characterized in that, in the step A34:
    对于步骤A33得到的灰度、ROI一致且无运动误差的实时微透镜图像视频流，截取其中相邻的两帧微透镜图像，选择适合具体任务的相似性测度函数，对帧间相似度设置阈值判断，输出对微透镜的局部缺陷检测结果；优选地，具体操作如下：For the real-time microlens image video stream with consistent grayscale and ROI and no motion error obtained in step A33, intercept two adjacent frames of microlens images, select a similarity measurement function suitable for specific tasks, and set a threshold for the similarity between frames Judging and outputting the local defect detection result of the microlens; preferably, the specific operations are as follows:
    Result(t)＝Bin[Similarity(I″′ t,I″′ t+Δt)]   (10) Result(t)=Bin[Similarity(I″′ t , I″′ t+Δt )] (10)
    上式中：In the above formula:
    Similarity()—相似性测度函数；Similarity()—similarity measure function;
    Result(t)—t时刻该微透镜的局部缺陷检测结果。Result(t)—local defect detection result of the microlens at time t.
  9. 如权利要求1至8任一项所述的方法，其特征在于，所述步骤A4包括：The method according to any one of claims 1 to 8, wherein said step A4 comprises:
    A41：利用光场图像的空域相关性，统计缺陷特征在相邻微透镜上的分布情况，建立单帧差光场图像中的缺陷位置检测结果；A41: Use the spatial correlation of light field images to count the distribution of defect features on adjacent microlenses, and establish the detection results of defect positions in single frame difference light field images;
    A42：利用光场视频的时域相关性，统计缺陷位置在相邻帧间的变化规律，建立最终的结构光场视频缺陷检测结果；A42: Utilize the time domain correlation of light field video, count the change rule of defect position between adjacent frames, and establish the final defect detection result of structured light field video;
    优选地，所述步骤A41中：Preferably, in the step A41:
    使用步骤A34所得到的微透镜实时局部缺陷检测结果，生成一系列具有缺陷的微透镜中心空间位置分布，计算当前光场图像帧各处对应的缺陷微透镜密度，密度越大则存在物理缺陷的可能性越大；更优选地，具体操作如下：Use the microlens real-time local defect detection results obtained in step A34 to generate a series of defective microlens center spatial position distributions, and calculate the defect microlens density corresponding to each part of the current light field image frame. The greater the density, the presence of physical defects The greater the possibility; more preferably, the specific operations are as follows:
    Figure PCTCN2022076878-appb-100008
    Figure PCTCN2022076878-appb-100008
    FrameResult(k)＝Bin[D(k)]    (12)FrameResult(k)＝Bin[D(k)] (12)
    上式中：In the above formula:
    Dist()—距离测度函数，包括但不限于欧氏距离、高斯距离等；Dist()—distance measurement function, including but not limited to Euclidean distance, Gaussian distance, etc.;
    S—当前帧存在缺陷的微透镜总数；S—the total number of defective microlenses in the current frame;
    D(k)—第k个微透镜中心处的缺陷密度；D(k)—defect density at the center of the kth microlens;
    FrameResult(k)—当前帧第k个微透镜中心处的缺陷位置检测结果；FrameResult(k)—the defect position detection result at the center of the kth microlens in the current frame;
    优选地，所述步骤A42中：Preferably, in the step A42:
    使用步骤A41所得到的单帧差光场图像缺陷位置检测结果，对于可能存在物理缺陷的区域，如果该区域在相邻的数帧之间均检出缺陷，且区域位置在各帧间移动速度与检测装置设定的物体移动速度接近，则判断该位置存在缺陷的可能性较大；在光场视频流上对这种存在时域相关性的缺陷位置进行实时标记和输出。Using the single-frame differential light field image defect position detection results obtained in step A41, for areas that may have physical defects, if defects are detected in the area between adjacent frames, and the area position moves at a speed between frames If it is close to the moving speed of the object set by the detection device, it is more likely to judge that there is a defect in the position; the defect position with time domain correlation is marked and output in real time on the light field video stream.
  10. 一种基于结构光场视频流的缺陷检测装置，其特征在于，包括处理器和基于主动式编码结构光源和光场视觉传感器的检测装置，所述处理器执行计算机可读存储介质上的计算机程序时，实现如权利要求1至9任一项所述的基于结构光场视频流的缺陷检测方法的步骤A2-A4。A defect detection device based on a structured light field video stream, characterized in that it includes a processor and a detection device based on an active coded structured light source and a light field visual sensor, and when the processor executes a computer program on a computer-readable storage medium , realizing the steps A2-A4 of the defect detection method based on the structured light field video stream according to any one of claims 1 to 9.
PCT/CN2022/076878 2021-05-31 2022-02-18 Defect detection method and apparatus based on structured light field video stream WO2022252698A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110598368.2 2021-05-31
CN202110598368.2A CN113406111B (en) 2021-05-31 2021-05-31 Defect detection method and device based on structural light field video stream

Publications (1)

Publication Number Publication Date
WO2022252698A1 true WO2022252698A1 (en) 2022-12-08

Family

ID=77675291

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/076878 WO2022252698A1 (en) 2021-05-31 2022-02-18 Defect detection method and apparatus based on structured light field video stream

Country Status (2)

Country Link
CN (1) CN113406111B (en)
WO (1) WO2022252698A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116402819A (en) * 2023-06-08 2023-07-07 季华实验室 Mirror-like defect detection method, device, electronic equipment and storage medium
CN118072230A (en) * 2024-04-19 2024-05-24 深圳市猿人创新科技有限公司 Image recognition method and system based on artificial intelligence

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113406111B (en) * 2021-05-31 2022-09-23 清华大学深圳国际研究生院 Defect detection method and device based on structural light field video stream

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08285557A (en) * 1995-04-17 1996-11-01 Nissan Motor Co Ltd Surface defect inspection instrument
US20160131885A1 (en) * 2013-07-17 2016-05-12 Nikon Corporation Structured illumination device and structured illumination microscope device
US20170132772A1 (en) * 2015-11-05 2017-05-11 Nuflare Technology, Inc. Inspection method and inspection apparatus
US10041833B1 (en) * 2016-04-05 2018-08-07 The United States of America as Represented by the Adminstrator of the NASA System and method for active multispectral imaging and optical communications
CN110146507A (en) * 2019-04-30 2019-08-20 杭州晶耐科光电技术有限公司 Automobile finish surface appearance defects detection system and method
CN110352346A (en) * 2017-03-15 2019-10-18 通用电气公司 Method and apparatus for checking assets
CN112824881A (en) * 2020-04-28 2021-05-21 奕目(上海)科技有限公司 System and method for detecting defects of transparent or semitransparent medium based on light field camera
CN113406111A (en) * 2021-05-31 2021-09-17 清华大学深圳国际研究生院 Defect detection method and device based on structural light field video stream

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109506589B (en) * 2018-12-25 2020-07-28 东南大学苏州医疗器械研究院 Three-dimensional profile measuring method based on structural light field imaging
CN111583323B (en) * 2020-04-30 2023-04-25 深圳大学 Single-frame structure light field three-dimensional imaging method and system
CN112630469B (en) * 2020-12-07 2023-04-25 清华大学深圳国际研究生院 Three-dimensional detection method based on structured light and multiple light field cameras

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08285557A (en) * 1995-04-17 1996-11-01 Nissan Motor Co Ltd Surface defect inspection instrument
US20160131885A1 (en) * 2013-07-17 2016-05-12 Nikon Corporation Structured illumination device and structured illumination microscope device
US20170132772A1 (en) * 2015-11-05 2017-05-11 Nuflare Technology, Inc. Inspection method and inspection apparatus
US10041833B1 (en) * 2016-04-05 2018-08-07 The United States of America as Represented by the Adminstrator of the NASA System and method for active multispectral imaging and optical communications
CN110352346A (en) * 2017-03-15 2019-10-18 通用电气公司 Method and apparatus for checking assets
CN110146507A (en) * 2019-04-30 2019-08-20 杭州晶耐科光电技术有限公司 Automobile finish surface appearance defects detection system and method
CN112824881A (en) * 2020-04-28 2021-05-21 奕目(上海)科技有限公司 System and method for detecting defects of transparent or semitransparent medium based on light field camera
CN113406111A (en) * 2021-05-31 2021-09-17 清华大学深圳国际研究生院 Defect detection method and device based on structural light field video stream

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KONG XIANGXIONG; LI JIAN: "Automated fatigue crack identification through motion tracking in a video stream", PROCEEDINGS OF SPIE, vol. 10598, 27 March 2018 (2018-03-27), US , pages 105980V - 105980V-8, XP060105893, ISBN: 978-1-5106-1533-5, DOI: 10.1117/12.2296602 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116402819A (en) * 2023-06-08 2023-07-07 季华实验室 Mirror-like defect detection method, device, electronic equipment and storage medium
CN116402819B (en) * 2023-06-08 2023-09-12 季华实验室 Mirror-like defect detection method, device, electronic equipment and storage medium
CN118072230A (en) * 2024-04-19 2024-05-24 深圳市猿人创新科技有限公司 Image recognition method and system based on artificial intelligence

Also Published As

Publication number Publication date
CN113406111A (en) 2021-09-17
CN113406111B (en) 2022-09-23

Similar Documents

Publication Publication Date Title
WO2022252698A1 (en) Defect detection method and apparatus based on structured light field video stream
CN103325112B (en) Moving target method for quick in dynamic scene
CN110807355B (en) Pointer instrument detection and reading identification method based on mobile robot
CN112116576B (en) Polarization structure light imaging and improved defect detection method
Song et al. DOE-based structured-light method for accurate 3D sensing
CN102201058B (en) Cat eye effect object recognition algorithm of active and passive imaging system sharing same aperture
CN107862713B (en) Camera deflection real-time detection early warning method and module for polling meeting place
CN110189375A (en) A kind of images steganalysis method based on monocular vision measurement
Wang et al. Error analysis and improved calibration algorithm for LED chip localization system based on visual feedback
WO2023236886A1 (en) Cloud occlusion prediction method based on dense optical flow method
CN107633507A (en) LCD defect inspection methods based on contour detecting and characteristic matching
CN108629790A (en) A kind of optical strip image threshold segmentation method based on depth residual error network
Liang et al. Research on concrete cracks recognition based on dual convolutional neural network
CN109064536B (en) Page three-dimensional reconstruction method based on binocular structured light
CN108510544A (en) A kind of striation localization method of feature based cluster
Shibo et al. A new approach to calibrate range image and color image from Kinect
CN114494169A (en) Industrial flexible object detection method based on machine vision
CN115880643B (en) Social distance monitoring method and device based on target detection algorithm
CN115184362B (en) Rapid defect detection method based on structured light projection
CN108010076B (en) End face appearance modeling method for intensive industrial bar image detection
Liu et al. Underwater polarization image restoration based on logarithmic transformation and dark channel
Li et al. Structured light based high precision 3D measurement and workpiece pose estimation
CN113409334A (en) Centroid-based structured light angle point detection method
CN113393395A (en) High-dynamic infrared image segmentation threshold self-adaptive calculation method
Yin et al. Motion detection and tracking using the 3D-camera

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22814747

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22814747

Country of ref document: EP

Kind code of ref document: A1