CN115728309A - Ink-jet printing line defect identification method and process regulation and control method - Google Patents
Ink-jet printing line defect identification method and process regulation and control method Download PDFInfo
- Publication number
- CN115728309A CN115728309A CN202211445797.7A CN202211445797A CN115728309A CN 115728309 A CN115728309 A CN 115728309A CN 202211445797 A CN202211445797 A CN 202211445797A CN 115728309 A CN115728309 A CN 115728309A
- Authority
- CN
- China
- Prior art keywords
- ink
- jet printing
- dimensional
- image
- view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses an ink-jet printing line defect identification method and a process regulation and control method, wherein an ink-jet printing device adopting a transparent substrate and a multi-view vision acquisition device are firstly set up, and a certain number of multi-view two-dimensional vision images of ink-jet printing lines with known defects are acquired through the multi-view vision acquisition device; performing three-dimensional reconstruction on the surface of the ink-jet printing circuit through a multi-view two-dimensional visual image to obtain a three-dimensional point cloud model; identifying the defect type of the ink-jet printing circuit through artificial intelligence learning; the printing parameters of the ink-jet printing device are adjusted according to the identified defect type, so that a plurality of limitations of the existing equipment are broken through, raw materials are saved, and the performance of devices and the yield of products are improved. And the given quality standard is used as a reference, the printing parameters are optimized in real time, and the aims of improving the ink-jet printing quality, improving the product yield, reducing the consumable waste and the like are finally achieved.
Description
Technical Field
The invention belongs to the field of electronic manufacturing, relates to a flexible electronic manufacturing process regulation and control technology, and particularly relates to an ink-jet printing line defect identification method and a process regulation and control method.
Background
Ink jet printing technology has emerged since the middle of the last century and has been developed to date as a manufacturing technology that can be used to process electronic devices. The basic operating principle of ink-jet printing technology for processing electronic devices is as follows: first, an electronic device wiring image stored in an electronic computer is input to an inkjet printing apparatus; and calculating the ink amount of the corresponding channel by the electronic computer, and controlling the transducer to spray atomized ink drops to the surface of the substrate to form an electronic device circuit. However, the quality of the electronic device circuit formed on the substrate by the ink is easily affected by various factors such as ink droplet form, printing distance, printing temperature, ink droplet carrier liquid volatilization efficiency and the like, and the device performance can only be quantified by testing after processing is completed, so that the product yield cannot be effectively improved, and raw material waste is caused. Therefore, a need exists for a process control method for inkjet printing to fabricate electronic devices.
Disclosure of Invention
The invention provides a method for identifying the defects of an ink-jet printing circuit and a process regulation and control method based on multi-vision, which aim at the quality control problem of a circuit of a device formed on a substrate by ink when an existing ink-jet printing device processes an electronic device, breaks through multiple limitations of existing equipment, saves raw materials, and improves the performance of the device and the yield of products.
In order to solve the technical problems, the invention adopts the following technical scheme:
the invention provides an ink-jet printing line defect identification method based on multi-view vision, which comprises the following steps:
step 5, constructing a CNN convolutional neural network, and training the CNN convolutional neural network by using the image sample data set to obtain a trained CNN convolutional neural network model;
and 6, acquiring a multi-view two-dimensional visual image of the ink-jet printing line to be detected through multi-view vision, processing the multi-view two-dimensional visual image of the ink-jet printing line to be detected through the same method as the steps 2 and 3 to obtain a surface three-dimensional point cloud model of the ink-jet printing line to be detected, and inputting the surface three-dimensional point cloud model into the trained CNN convolutional neural network model in the step 5 to obtain the defect type of the ink-jet printing line to be detected.
The invention also provides an ink-jet printing process regulation and control method based on multi-view vision, which comprises the following steps:
s1, identifying the defect information of the ink-jet printing line to be detected on line by adopting any one of the ink-jet printing line defect identification methods;
s2, adjusting the ink-jet printing parameters in real time based on the acquired defect information of the ink-jet printing line to be detected; the specific method is that parameters such as ink jet voltage, printing distance and the like of the ink jet printing equipment are adjusted in real time according to the actual precision of ink jet printing processing and the actual performance requirement of a target device, so that the defects of ink jet printing lines are eliminated.
The invention also provides an ink-jet printing system based on multi-view vision, which comprises
The ink-jet printing device is used for performing ink-jet printing on the transparent substrate to form a required circuit;
the multi-view vision acquisition device comprises a plurality of cameras with different visual angles and is used for shooting ink printing lines printed by the ink-jet printing device at different angles to obtain two-dimensional vision images;
the defect identification module is used for executing the ink-jet printing line defect identification method and identifying the defects of the printed current ink printing line;
and the process adjusting module is used for adjusting the printing parameters of the ink-jet printing device according to the defect type identified by the defect identifying module.
When ink drops are ejected from the printing head and drop on the substrate to form a device circuit, the existing device cannot monitor the circuit forming process in real time and cannot ensure the processing quality of the device. The invention provides that a plurality of groups of cameras are arranged in an ink-jet printing device to acquire visual information of a plurality of visual angles when ink forms a device circuit in the ink-jet printing processing process and reconstruct the three-dimensional appearance of the surface of the device circuit, thereby realizing real-time monitoring of the quality of the ink printing circuit through a convolutional neural network and adjusting printing parameters according to a detection result.
Compared with the prior art, the invention has the following beneficial effects:
compared with the traditional ink-jet printing processing process, the detection blind area is reduced and the detection precision of the surface defects of the printing device line is improved by acquiring the multi-view real-time image of the ink printing line in the ink-jet printing process; and the given quality standard is used as a reference, the printing parameters are optimized in real time, and the aims of improving the ink-jet printing quality, improving the product yield, reducing the consumable waste and the like are finally achieved.
Drawings
Fig. 1 is a schematic diagram of an inkjet printing device and a multi-view vision acquisition device of an inkjet printing system based on multi-view vision in an embodiment of the invention.
FIG. 2 is a flowchart of an algorithm of a method for regulating and controlling an inkjet printing process based on multi-vision in an embodiment of the present invention.
Fig. 3 is a schematic diagram of a CNN convolutional neural network framework in the embodiment of the present invention.
1-a transparent printing table; 2-a transparent substrate; 3-ink jet printing circuit; 4-top right lens; 5-flying ink droplets; 6-ink jet print head; 7-top medium lens; 8-top left lens; 9-left lens; 10-current print area; 11-bottom left lens; 12-bottom center lens; 13-bottom right lens; 14-right lens; 15-top plane light source; 16-bottom plane light source.
Detailed Description
The embodiments of the present invention will be described in further detail with reference to the drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
As shown in fig. 1, the present invention provides an inkjet printing circuit defect identification system based on multi-view vision, which includes an inkjet printing device, a multi-view vision acquisition device, and a controller with a built-in program for executing the inkjet printing process regulation and control method based on multi-view vision;
the ink-jet printing device comprises a transparent printing table 1, a transparent substrate 2 and an ink-jet printing head 6, wherein the transparent substrate 2 is positioned on the upper surface of the transparent printing table 1, the ink-jet printing head 6 is positioned above the transparent substrate 2, and an ink-jet printing line 3 is formed on the transparent substrate 2 by printing;
in the embodiment of the invention, the transparent printing table 1 is made of transparent toughened glass and is used for bearing a printing substrate; can move spatially along the three-axis directions of XYZ, and ensures the flatness required just during the working process of the ink-jet printing.
The transparent substrate 2 is made of transparent or semitransparent materials such as PDMS, PET, PI and the like, is attached to the transparent printing table, and ink is deposited on the substrate to form a circuit of an electronic device.
The inkjet print head 6 has a piezoelectric transducer integrated therein, which is capable of ejecting ink droplets on demand onto a transparent substrate in response to a signal from a control device.
Flying ink droplets 5, ejected by an inkjet print head 6, will deposit on the transparent substrate 2 and form the electronics circuit.
And the ink-jet printing circuit 3 is formed by depositing ink drops on the surface of the transparent substrate 2 so as to realize the required functions of the target electronic device.
The multi-view vision acquisition device comprises 8 cameras and two planar light sources, wherein the two planar light sources are respectively positioned above the transparent substrate and below the transparent printing table, and are used for respectively supplementing light to the upper surface, the lower surface and the side surface of the ink-jet printing circuit, namely a bottom planar light source 16 and a top planar light source 15;8 cameras are distributed above, below and around the ink-jet printing circuit, are respectively a top right lens 4, a top middle lens 7, a top left lens 8, a left lens 9, a bottom left lens 11, a bottom middle lens 12, a bottom right lens 13 and a right lens 14, and respectively shoot current printing areas of the ink-jet printing circuit at different angles to obtain multi-view two-dimensional visual images.
And the right lens 4 is arranged on the right side of the top of the transparent substrate 2, inclines towards the center of the transparent printing table 1, focuses on an ink-jet printing processing area, and captures visual information in the direction of the right side of the top when ink forms a device circuit.
And the top left lens 8 is arranged on the left side of the top of the transparent substrate 2, inclines towards the center of the transparent printing table 1, focuses on the ink-jet printing processing area, and captures visual information of the left side of the top when ink forms a device circuit.
And the top middle lens 7 is arranged in the center of the top of the transparent substrate 2, focuses on an ink-jet printing processing area and captures visual information in the top middle direction when ink forms a device circuit.
And the top plane light source 15 is arranged in the center of the bottom of the transparent substrate 2, and provides uniform and sufficient illumination for the current printing area from the top, so that the interference of top ambient light on the visual information of the line surface is reduced.
And the left lens 9 is arranged on the left side of the ink-jet printing device, focuses on the ink-jet printing processing area and captures the visual information of the left side when the ink forms the circuit of the device.
And the right lens 14 is arranged on the right side of the ink-jet printing device, focuses on the ink-jet printing processing area, and captures the visual information of the right side when the ink forms the circuit of the device.
And the bottom left lens 11 is arranged at the left side of the bottom of the transparent printing table 1, focuses on the ink-jet printing processing area, and captures visual information in the direction of the left side of the bottom when ink forms a circuit of a device.
And the bottom-in-center lens 12 is arranged in the center of the bottom of the transparent printing table 1, focuses on an ink-jet printing processing area, and captures visual information in the bottom-in-center direction when ink forms a device circuit.
And the bottom right lens 13 is arranged on the right side of the bottom of the transparent printing table 1, focuses on an ink-jet printing processing area, and captures visual information in the direction of the right side of the bottom when ink forms a device circuit.
And the bottom plane light source 16 is arranged in the center of the bottom of the transparent printing table 1, provides uniform and sufficient illumination for the current printing area from the bottom, and reduces the interference of bottom ambient light on the visual information of the surface of the line.
The current print zone 10, the inkjet printing process zone.
When ink drops are ejected from the printing head and drop on the substrate to form a device circuit, the existing device cannot monitor the circuit forming process in real time and cannot ensure the processing quality of the device. The invention provides that a plurality of groups of cameras are arranged in an ink-jet printing device to acquire visual information of a plurality of visual angles when ink forms a device circuit in the ink-jet printing processing process, and reconstruct the three-dimensional appearance of the surface of the device circuit, thereby realizing real-time monitoring of the quality of the ink printing circuit through a convolutional neural network, and adjusting printing parameters according to a detection result.
It should be noted that, for the number of the cameras, the above 8 are only examples, and specifically, the number of the cameras is selected according to the complexity of the inkjet printing circuit, and may be reduced or increased on the basis of 8, so as to be able to clearly photograph possible defects of the inkjet printing circuit.
The invention provides an ink-jet printing line defect identification method based on multi-view vision, which comprises the following steps:
2.1, performing feature extraction on the multi-view two-dimensional visual image of the ink-jet printing line by adopting an SIFT algorithm to obtain a key feature point feature descriptor of the two-dimensional image of each ink-jet printing line; the method comprises the following specific steps:
step 2.1.1, performing Gaussian blur operation on the acquired multi-view two-dimensional visual image, and convolving a two-dimensional Gaussian kernel function G (x, y, sigma) with the variation scale of sigma with the multi-view two-dimensional image I (x, y) to generate a scale space L (x, y, sigma) of the ink-jet printing line two-dimensional image so as to simulate the multi-scale characteristics of the ink-jet printing line image data:
L(x,y,σ)＝G(x,y,σ)*I(x,y)
wherein, (x, y) is a plane coordinate of a certain pixel point in the ink-jet printing circuit, the variation scale sigma is selected according to the actual ink-jet printing processing precision requirement, the high sigma value corresponds to a fine scale, and the low sigma value corresponds to a coarse scale;
step 2.1.2, constructing a Gaussian difference scale space of the two-dimensional image of the ink-jet printing line;
constructing a Gaussian difference scale space corresponding to the scale space of the two-dimensional image of the ink-jet printing line constructed in the step 2.1.1 on the basis of the scale space; and (3) performing difference on the two-dimensional images of the ink-jet printing lines with different scales after Gaussian blur, for example, subtracting the jth layer of the ith group from the jth layer of the ith group of the Gaussian pyramid to obtain a Gaussian blur response value image D (x, y, sigma) corresponding to the jth layer of the ith group of the DoG pyramid:
D(x,y,σ)＝[G(x,y,kσ)-G(x,y,σ)*(x,y)＝L(x,y,kσ)-L(x,y,σ)
where k is a constant of the adjacent scale space multiples.
Carrying out non-maximum suppression on the D (x, y, sigma) to obtain local extreme points of the two-dimensional image of the ink-jet printing line, namely the key feature points;
step 2.1.3, allocating reference directions for key characteristic points of the two-dimensional image of the ink-jet printing line, which comprises the following specific steps:
in order to make the two-dimensional image of the ink-jet printing line have rotation invariance, for the key feature points of the two-dimensional image of the ink-jet printing line detected in the step 2.1.2, the gradient and direction distribution features of pixels are collected in a neighborhood window with 3 multiplied by 1.5 sigma as a radius in the Gaussian blur response value image D (x, y, sigma); and by utilizing the gradient direction distribution characteristics of the pixels in the neighborhood window, assigning a direction parameter to each key feature point in the two-dimensional image of the ink-jet printing line:
where m (x, y) is the magnitude of the gradient of the key feature point and θ (x, y) is the direction of the gradient of the key feature point.
Counting the directions of image pixel points in the neighborhood of the extreme point by adopting a gradient histogram statistical method, dividing the neighborhood into BpxBp sub-regions, wherein the size of each sub-region is m sigma pixel elements, and selecting parameters m =3 and Bp =4 as a preferred embodiment of the invention, wherein sigma is the scale value of the characteristic point determined in the step 2.1.1; the gradient histogram divides the direction range of 0-360 degrees into 36 columns, wherein each column is 10 degrees; taking the peak direction of the histogram as the main direction of the key feature point, and keeping the direction in which the peak value is greater than 80% of the peak value of the main direction as the auxiliary direction of the key feature point;
step 2.1.4, establishing a description vector based on the key feature points in the two-dimensional image of the ink-jet printing line, and specifically comprising the following steps:
rotating the image in the neighborhood of the feature point by a certain angle:
wherein, x ', y' are coordinates of pixels after the image in the neighborhood is rotated, and theta is an included angle between the feature point and the original direction x axis of the neighborhood image thereof and the main direction x axis distributed for the feature point in the step 2.1.3. For the sub-regions divided in step 2.1.3, as a preferred embodiment of the present invention, the gradient histogram divides the direction range of 0-360 degrees into 8 columns, wherein each column has 45 degrees, so as to obtain the gradient strength information of the corresponding direction of the column; up to this point, a Bp × 8=128 dimensional feature description vector corresponding to the feature points can be constructed. And then, gaussian weighting processing is carried out on the constructed feature description vector by adopting a standard Gaussian function with the variance of m sigma Bp/2, so as to prevent mismatching.
Step 2.2, according to the key feature point feature descriptors obtained in the step 2.1, the Euclidean distance between feature points is used as similarity judgment measurement of the feature points of the adjacent view images in a group of multi-view ink-jet printing line two-dimensional images, and feature point matching operation is carried out on the adjacent view images of the ink-jet printing lines; performing the characteristic point matching operation on the two-dimensional images at all visual angles of the ink-jet printing circuit to obtain characteristic matching point pairs of the two-dimensional images at multiple visual angles of the ink-jet printing circuit; the specific steps of feature point matching are as follows:
searching by adopting a K-dimensional binary balance search tree data structure, firstly establishing a Kd binary balance search tree according to Euclidean distances among all feature points, and then searching original image feature points most adjacent to the feature points of a target image and next-adjacent original image feature points by taking key points of the target image as a reference; and performing the characteristic point matching operation on the two-dimensional images of all the visual angles of the ink-jet printing circuit to obtain characteristic matching point pairs of the multi-visual angle two-dimensional images of the ink-jet printing circuit.
Step 2.3, verifying the validity of the multi-view two-dimensional image feature matching point pair of the ink-jet printing circuit by adopting an extreme line constraint method, and correcting the coordinates of the feature matching point pair; the method comprises the following specific steps:
step 2.3.1, calculating a corresponding epipolar line l of p in the I2 and a distance d1 from p 'to the epipolar line l for the matching characteristic point pairs p and p' in the two-dimensional images I1 and I2 in the collected multi-view two-dimensional images of the ink-jet printing circuit;
step 2.3.2, when d1 exceeds a threshold value t, judging that p and p' are in wrong matching, and removing t from the characteristic point pair set;
and 2.3.3, when d1 is smaller than the threshold t, judging that p and p 'are correctly matched, updating the coordinate of p' into the projection from p to the epipolar line l, and realizing the coordinate correction of the feature matching point pair.
The selection mode of the judgment threshold t is determined according to the precision required by actual ink-jet printing processing, the value range of the judgment threshold t is 1-2 pixels, a larger threshold t corresponds to coarser reconstruction quality, and a smaller threshold t corresponds to finer reconstruction quality.
And 2.4, matching the characteristic point pairs based on the multi-view two-dimensional image of the ink-jet printing circuit, and performing three-dimensional reconstruction on the surface of the ink-jet printing circuit to obtain a surface three-dimensional point cloud model of the ink-jet printing circuit.
step 3.1, measuring to obtain the minimum distance between two points in the surface three-dimensional point cloud model, and taking the minimum distance between the two points as a reference for parameter selection when the surface three-dimensional point cloud model is further optimized;
3.2, performing radius filtering on the surface three-dimensional point cloud model of the ink-jet printing circuit to remove noise points in the surface three-dimensional point cloud model of the ink-jet printing circuit;
the screening radius of the radius filtering method is determined according to the actual point density of the surface three-dimensional point cloud model; for the surface defect detection of a common ink-jet printing circuit, the value of the screening radius is selected as integral multiple of the minimum distance (reference) between two points in the point cloud.
Step 5, constructing a CNN convolutional neural network, training the CNN convolutional neural network by utilizing an image sample data set to obtain a trained CNN convolutional neural network model, wherein in the training process, 70% of image sample data are concentrated to be used as a sample training set for training the CNN convolutional neural network, and the rest 30% of image sample data are used as a sample testing machine for testing the trained CNN convolutional neural network;
as shown in fig. 3, the CNN convolutional neural network includes a feature extraction module and a feature combination module, where the feature extraction module includes a convolutional layer, a nonlinear layer, a maximum pooling layer, a convolutional layer, a nonlinear layer, and a maximum pooling layer, which are connected in sequence; the characteristic extraction module comprises two full connection layers, and two nonlinear layers are connected behind the two full connection layers respectively.
The classification layer of the CNN convolutional neural network adopts a plurality of classifiers, particularly a Softmax function, and the output value of the classification layer can be regarded as the probability that the sample three-dimensional model belongs to a certain defect type.
And 6, acquiring a multi-view two-dimensional visual image of the ink-jet printing line to be detected through multi-view vision, processing the multi-view two-dimensional visual image of the ink-jet printing line to be detected through the same method as the steps 2 and 3 to obtain a surface three-dimensional point cloud model of the ink-jet printing line to be detected, and inputting the surface three-dimensional point cloud model into the trained CNN convolutional neural network model in the step 5 to obtain the defect type of the ink-jet printing line to be detected.
The invention also provides an ink-jet printing process regulating and controlling method based on multi-view vision, which comprises the following steps:
s1, identifying the defect information of the ink-jet printing line to be detected on line by adopting the ink-jet printing line defect identification method;
s2, adjusting the ink-jet printing parameters in real time based on the acquired defect information of the ink-jet printing line to be detected; the method comprises the steps of adjusting parameters such as ink-jet voltage, printing distance and the like of the ink-jet printing equipment in real time according to actual precision of ink-jet printing processing and actual performance requirements of a target device so as to eliminate ink-jet printing circuit defects, and achieve the effects of improving ink-jet printing quality, improving product yield, reducing consumable waste and the like.
According to the invention, through a technical means of collecting multi-view two-dimensional images of a product to be detected and fusing two-dimensional and three-dimensional visual information, the problems that the defect identification is easily affected by false defects such as oxidation and oil stain and the like only according to the characteristics of the two-dimensional images and the defect identification capability is poor due to low contrast are solved; and the defects such as cracks can not be detected only through three-dimensional point cloud registration. Meanwhile, the method and the device adopt a means of fusing two-dimensional and three-dimensional visual information to make up the defects of the means in the prior art, reduce the negative influence on detection caused by lower image acquisition quality, less visual information quantity and smaller detection area range, improve the detection efficiency and the detection rate and reduce the omission factor and the false detection rate.
The above embodiments are merely illustrative of the present invention and are not to be construed as limiting the invention. Although the present invention has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that various combinations, modifications or equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention, and the technical solution of the present invention is covered by the claims of the present invention.
Claims (10)
1. An ink-jet printing line defect identification method based on multi-view vision is characterized by comprising the following steps:
step 1, a certain number of multi-view two-dimensional visual images of ink-jet printing lines with known defects are acquired through a multi-view visual acquisition device;
step 2, three-dimensional reconstruction, namely performing three-dimensional reconstruction on the surface of the ink-jet printing circuit through the multi-view two-dimensional visual image to obtain a surface three-dimensional point cloud model of the ink-jet printing circuit;
step 3, point cloud data preprocessing, namely preprocessing the surface three-dimensional point cloud model obtained in the step 1 to remove noise;
step 4, constructing an image sample data set by taking the denoised surface three-dimensional point cloud model as a sample and the known defect information of the ink-jet printing line as a label;
step 5, constructing a CNN convolutional neural network, and training the CNN convolutional neural network by using the image sample data set to obtain a trained CNN convolutional neural network model;
and 6, acquiring a multi-view two-dimensional visual image of the ink-jet printing line to be detected through multi-view vision, processing the multi-view two-dimensional visual image of the ink-jet printing line to be detected through the same method as the steps 2 and 3 to obtain a surface three-dimensional point cloud model of the ink-jet printing line to be detected, and inputting the surface three-dimensional point cloud model into the CNN convolutional neural network model trained in the step 5 to obtain the defect type of the ink-jet printing line to be detected.
2. The inkjet-printed line defect identifying method as defined in claim 1, wherein: in step 1, the viewing angles of the multi-view two-dimensional visual image at least include 8 viewing angles, namely, upper, lower, left, right, upper left, lower left, upper right and lower right.
3. The inkjet-printed line defect identifying method as defined in claim 1, wherein: in step 2, the three-dimensional reconstruction method is as follows:
2.1, performing feature extraction on the multi-view two-dimensional visual image of the ink-jet printing line by adopting an SIFT algorithm to obtain a key feature point feature descriptor of the two-dimensional image of each ink-jet printing line;
step 2.2, according to the key feature point feature descriptors obtained in the step 2.1, the Euclidean distance between feature points is used as similarity judgment measurement of the feature points of the adjacent view images in a group of multi-view ink-jet printing line two-dimensional images, and feature point matching operation is carried out on the adjacent view images of the ink-jet printing lines; performing the characteristic point matching operation on the two-dimensional images at all visual angles of the ink-jet printing circuit to obtain characteristic matching point pairs of the two-dimensional images at multiple visual angles of the ink-jet printing circuit;
step 2.3, verifying the validity of the multi-view two-dimensional image feature matching point pair of the ink-jet printing circuit by adopting an epipolar constraint method, and correcting the coordinates of the feature matching point pair;
and 2.4, matching the characteristic point pairs based on the multi-view two-dimensional image of the ink-jet printing circuit, and performing three-dimensional reconstruction on the surface of the ink-jet printing circuit to obtain a surface three-dimensional point cloud model of the ink-jet printing circuit.
4. The inkjet-printed line defect identifying method as defined in claim 3, wherein: the specific steps of step 2.1 are as follows:
step 2.1.1, performing Gaussian blur operation on the acquired multi-view two-dimensional visual image, and convolving a two-dimensional Gaussian kernel function G (x, y, sigma) with the variation scale of sigma with the multi-view two-dimensional image I (x, y) to generate a scale space L (x, y, sigma) of the ink-jet printing line two-dimensional image so as to simulate the multi-scale characteristics of the ink-jet printing line image data:
L(x,y,σ)＝G(x,y,σ)*I(x,y)
wherein, (x, y) is a plane coordinate of a certain pixel point in the ink-jet printing circuit, the variation scale sigma is selected according to the actual ink-jet printing processing precision requirement, the high sigma value corresponds to a fine scale, and the low sigma value corresponds to a coarse scale;
step 2.1.2, constructing a Gaussian difference scale space of the two-dimensional image of the ink-jet printing circuit;
constructing a corresponding Gaussian difference scale space based on the scale space of the two-dimensional image of the ink-jet printing line constructed in the step 2.1.1; and (3) performing difference on the two-dimensional images of the ink-jet printing lines with different scales after Gaussian blur, for example, subtracting the jth layer of the ith group from the jth layer of the ith group of the Gaussian pyramid to obtain a Gaussian blur response value image D (x, y, sigma) corresponding to the jth layer of the ith group of the DoG pyramid:
D(x,y,σ)＝[G(x,y,kσ)-G(x,y,σ)]*I(x,y)＝L(x,y,kσ)-L(x,y,σ)
wherein k is a constant of the adjacent scale space multiple;
carrying out non-maximum suppression on the D (x, y, sigma) to obtain local extreme points of the two-dimensional image of the ink-jet printing line, namely the key feature points;
step 2.1.3, allocating reference directions for key characteristic points of the two-dimensional image of the ink-jet printing line, and specifically comprising the following steps:
in order to make the two-dimensional image of the ink-jet printing line have rotation invariance, for the key feature points of the two-dimensional image of the ink-jet printing line detected in the step 2.1.2, the gradient and direction distribution features of pixels are collected in a neighborhood window with 3 multiplied by 1.5 sigma as a radius in the Gaussian blur response value image D (x, y, sigma); and by utilizing the gradient direction distribution characteristics of the pixels in the neighborhood window, assigning a direction parameter for each key feature point in the two-dimensional image of the ink-jet printing line:
wherein m (x, y) is the amplitude of the gradient of the key feature point, and theta (x, y) is the direction of the gradient of the key feature point;
counting the directions of image pixel points in the extreme point neighborhood by adopting a gradient histogram statistical method, dividing the neighborhood into BpxBp sub-regions, wherein the size of each sub-region is m sigma pixels, and sigma is the scale value of the characteristic point determined in the step 2.1.1; the gradient histogram divides the direction range of 0-360 degrees into 36 columns, wherein each column is 10 degrees; taking the peak direction of the histogram as the main direction of the key feature point, and keeping the direction in which the peak value is greater than 80% of the peak value of the main direction as the auxiliary direction of the key feature point;
step 2.1.4, establishing a description vector based on key feature points in the two-dimensional image of the ink-jet printing line, and specifically comprising the following steps:
rotating the image in the neighborhood of the feature point by a certain angle:
wherein x ', y' are coordinates of pixels after the image in the neighborhood is rotated, and theta is an included angle between the feature point and the original direction x axis of the neighborhood image and the main direction x axis allocated to the feature point in the step 2.1.3.
5. The inkjet-printed line defect identifying method as defined in claim 4, wherein: in step 2.2, the specific steps of feature point matching are as follows:
searching by adopting a K-dimensional binary balance search tree data structure, firstly establishing a Kd binary balance search tree according to Euclidean distances among all feature points, and then searching original image feature points most adjacent to the feature points of a target image and next-adjacent original image feature points by taking key points of the target image as a reference; and performing the characteristic point matching operation on the two-dimensional images of all the visual angles of the ink-jet printing circuit to obtain characteristic matching point pairs of the multi-visual angle two-dimensional images of the ink-jet printing circuit.
6. The inkjet printing line defect identifying method according to claim 4, wherein: in step 2.3, the specific steps are as follows:
step 2.3.1, calculating a epipolar line l corresponding to p in the I2 and a distance d1 from p 'to the epipolar line l for the two-dimensional image I1 in the collected multi-view two-dimensional image of the ink-jet printing line and the matching characteristic point pairs p and p' in the two-dimensional image I2;
step 2.3.2, when d1 exceeds a threshold value t, judging that p and p' are in wrong matching, and removing t from the characteristic point pair set;
and 2.3.3, when d1 is smaller than a threshold t, judging that p and p 'are correctly matched, updating the coordinate of p' into the projection from p to the epipolar line l, and realizing the coordinate correction of the feature matching point pair.
7. The inkjet-printed line defect identifying method as defined in claim 4, wherein: in step 3, the specific method for removing noise points is as follows:
step 3.1, measuring to obtain the minimum distance between two points in the surface three-dimensional point cloud model, and taking the minimum distance between the two points as a reference for parameter selection when the surface three-dimensional point cloud model is further optimized;
3.2, performing radius filtering on the surface three-dimensional point cloud model of the ink-jet printing circuit to remove noise points in the surface three-dimensional point cloud model of the ink-jet printing circuit;
the screening radius of the radius filtering method is determined according to the actual point density of the surface three-dimensional point cloud model; for the surface defect detection of a common ink-jet printing circuit, the value of the screening radius is selected from integral multiples of the minimum distance between two points in the point cloud.
8. The inkjet printing line defect identifying method according to claim 4, wherein: in step 5, the CNN convolutional neural network comprises a feature extraction module and a feature combination module, wherein the feature extraction module comprises a convolutional layer, a nonlinear layer, a maximum pooling layer, a convolutional layer, a nonlinear layer and a maximum pooling layer which are sequentially connected; the characteristic extraction module comprises two full connection layers, and two nonlinear layers are connected behind the two full connection layers respectively.
9. A multi-view vision-based ink-jet printing process regulation and control method is characterized by comprising the following steps:
s1, identifying the defect information of the ink-jet printing line to be detected on line by adopting the ink-jet printing line defect identification method of any one of claims 1 to 8;
s2, adjusting the ink-jet printing parameters in real time based on the acquired defect information of the ink-jet printing line to be detected; the method comprises the steps of adjusting parameters such as ink-jet voltage, printing distance and the like of the ink-jet printing equipment in real time according to actual precision of ink-jet printing processing and actual performance requirements of a target device so as to eliminate defects of ink-jet printing lines.
10. An ink jet printing system based on multi-vision comprises
The ink-jet printing device is used for carrying out ink-jet printing on the transparent substrate to form a required circuit;
the multi-view vision acquisition device comprises a plurality of cameras with different visual angles and is used for shooting ink printing lines printed by the ink jet printing device at different angles to obtain two-dimensional vision images;
the defect identification module is used for executing the ink-jet printing line defect identification method and identifying the defects of the printed current ink printing line;
and the process adjusting module is used for adjusting the printing parameters of the ink-jet printing device according to the defect type identified by the defect identifying module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211445797.7A CN115728309A (en) | 2022-11-18 | 2022-11-18 | Ink-jet printing line defect identification method and process regulation and control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211445797.7A CN115728309A (en) | 2022-11-18 | 2022-11-18 | Ink-jet printing line defect identification method and process regulation and control method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115728309A true CN115728309A (en) | 2023-03-03 |
Family
ID=85296507
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211445797.7A Pending CN115728309A (en) | 2022-11-18 | 2022-11-18 | Ink-jet printing line defect identification method and process regulation and control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115728309A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116402809A (en) * | 2023-05-31 | 2023-07-07 | 华中科技大学 | Defect identification method and device in three-dimensional sand mould printing and sanding process |
| CN117853482A (en) * | 2024-03-05 | 2024-04-09 | 武汉软件工程职业学院(武汉开放大学) | Multi-scale-based composite defect detection method and equipment |
-
2022
- 2022-11-18 CN CN202211445797.7A patent/CN115728309A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116402809A (en) * | 2023-05-31 | 2023-07-07 | 华中科技大学 | Defect identification method and device in three-dimensional sand mould printing and sanding process |
| CN116402809B (en) * | 2023-05-31 | 2023-08-11 | 华中科技大学 | Defect identification method and device in three-dimensional sand mould printing and sanding process |
| CN117853482A (en) * | 2024-03-05 | 2024-04-09 | 武汉软件工程职业学院(武汉开放大学) | Multi-scale-based composite defect detection method and equipment |
| CN117853482B (en) * | 2024-03-05 | 2024-05-07 | 武汉软件工程职业学院(武汉开放大学) | Multi-scale-based composite defect detection method and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108548820B (en) | Cosmetic paper label defect detection method | |
| CN115728309A (en) | Ink-jet printing line defect identification method and process regulation and control method | |
| CN106650721B (en) | A kind of industrial character identifying method based on convolutional neural networks | |
| CN111007073B (en) | Method and system for online detection of part defects in additive manufacturing process | |
| CN108986086A (en) | The detection of typographical display panel inkjet printing picture element flaw and classification method and its device | |
| CN104992449B (en) | Information identification and surface defect online test method based on machine vision | |
| CN112686812B (en) | Bank card inclination correction detection method and device, readable storage medium and terminal | |
| CN110163177B (en) | Unmanned aerial vehicle automatic sensing and identifying method for wind turbine generator blades | |
| CN108334881B (en) | License plate recognition method based on deep learning | |
| CN103927534A (en) | Sprayed character online visual detection method based on convolutional neural network | |
| CN108734189A (en) | Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather | |
| CN109886925A (en) | A kind of aluminium material surface defect inspection method that Active Learning is combined with deep learning | |
| CN113109348B (en) | Paddle image transfer printing defect identification method based on machine vision | |
| CN110991439B (en) | Handwriting character extraction method based on pixel-level multi-feature joint classification | |
| CN114119526A (en) | Steel plate surface defect detection and identification system and method based on machine vision | |
| CN113706566B (en) | Edge detection-based perfuming and spraying performance detection method | |
| CN104050684A (en) | Video moving object classification method and system based on on-line training | |
| CN111260609B (en) | Cigarette appearance defect detection method based on deep learning | |
| CN112354874A (en) | Coal and gangue identification method and gangue automatic separation system | |
| CN116482104B (en) | Thermal transfer film detection method | |
| Lall et al. | Deep Learning for Prediction of Print Parameters and Realized Electrical Performance and Geometry on Inkjet Platform | |
| CN111950556A (en) | License plate printing quality detection method based on deep learning | |
| CN115565141A (en) | Truck axle type detection method based on visual infrared fusion | |
| CN111161215B (en) | Detection apparatus capable of identifying color of printer | |
| CN114148103A (en) | Solder paste printing mark identification method and system based on neural network learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |