CN117115228A

CN117115228A - SOP chip pin coplanarity detection method and device

Info

Publication number: CN117115228A
Application number: CN202311370037.9A
Authority: CN
Inventors: 邓耀华; 陈冠浩; 李泽杭
Original assignee: Guangdong University of Technology
Current assignee: Guangdong University of Technology
Priority date: 2023-10-23
Filing date: 2023-10-23
Publication date: 2023-11-24

Abstract

The invention discloses a method and a device for detecting coplanarity of pins of an SOP chip, which are used for solving the technical problems of low detection precision and poor imaging effect caused by easiness in external environment influence in the existing SOP chip detection method. The method comprises the following steps: collecting chip point clouds of the SOP chip to be tested, and carrying out clustering segmentation on the chip point clouds to obtain pin point clouds to be tested, wherein the pin point clouds to be tested correspond to a plurality of pins to be tested; performing space correction processing on the pin point cloud to be detected by adopting a preset reference pin point cloud to obtain a space correction point cloud; european clustering is carried out on the space correction point cloud, and the height from each pin to be detected to the XOY plane is calculated; and calculating the coplanarity of the pins based on the heights of the pins, wherein the coplanarity of the pins is used for judging the coplanarity defect of the pins of the SOP chip to be tested.

Description

SOP chip pin coplanarity detection method and device

Technical Field

The present invention relates to the field of chip detection technologies, and in particular, to a method for detecting pin coplanarity of an SOP chip, a device for detecting pin coplanarity of an SOP chip, an electronic device, and a storage medium.

Background

In integrated circuits, SOP (Small Out-Line Package) is a very common form of component packaging. The SOP chip is widely applied to the electronic fields of various smart phones, tablet computers, digital products and the like due to the characteristics of small appearance, strong flexibility and the like, but in the SOP chip production and manufacturing process, the defects of broken pins, missing pins, coplanarity, tilting degree and the like of pins cannot be completely avoided due to the influence of factors such as manufacturing process, materials, production equipment and the like, and the chip with the defects often causes the problems of functional failure and even scrapping of subsequent mounted finished products. Therefore, SOP chip pin detection is increasingly taking up the weight of the whole chip appearance detection.

Currently, most enterprises generally use traditional manual visual inspection and industrial 2D (Two-Dimensional) visual inspection schemes for inspecting SOP chip pins, but due to lack of depth information in industrial 2D inspection, if 3D (Three-Dimensional) indexes such as coplanarity, warpage and the like of chip pins are inspected, the following problems are often easy to exist:

(1) When detection is carried out, three-dimensional reconstruction is needed to be carried out on the SOP chip, the process is complex, and the reconstruction precision can influence the actual detection precision;

(2) When SOP chips with different sizes are used, the layout of the optical path needs to be redesigned, so that the detection flexibility is low;

(3) Because the SOP chip is imaged on a plurality of sides at the same time, the SOP chip has high requirements on illumination conditions, is easily influenced by external environment and has poor imaging effect.

Disclosure of Invention

The invention provides an SOP chip pin coplanarity detection method, an SOP chip pin coplanarity detection device, electronic equipment and a storage medium, which are used for solving or partially solving the technical problems of low detection precision and poor imaging effect caused by easy influence of external environment in the existing SOP chip detection method.

The invention provides a method for detecting coplanarity of pins of an SOP chip, which comprises the following steps:

collecting chip point clouds of an SOP chip to be tested, and carrying out clustering segmentation on the chip point clouds to obtain pin point clouds to be tested, wherein the pin point clouds to be tested correspond to a plurality of pins to be tested;

performing space correction processing on the pin point cloud to be detected by adopting a preset reference pin point cloud to obtain a space correction point cloud;

european clustering is carried out on the space correction point cloud, and the pin heights from the pins to be detected to the XOY plane are calculated;

and calculating the coplanarity of the pins based on the heights of the pins, wherein the coplanarity of the pins is used for judging the coplanarity defect of the pins of the SOP chip to be tested.

Optionally, the performing cluster segmentation on the chip point cloud to obtain a pin point cloud to be tested includes:

carrying out statistical filtering on the chip point cloud to obtain a denoising chip point cloud;

voxel downsampling is conducted on the denoising chip point cloud to obtain a downsampled chip point cloud, and Z-axis straight-through filtering is conducted on the downsampled chip point cloud to obtain a straight-through filtering point cloud;

and carrying out Euclidean clustering on the through filtering point cloud to obtain clustered point cloud, and dividing the clustered point cloud to obtain pin point cloud to be detected.

Optionally, the performing euclidean clustering on the through filtering point cloud to obtain a clustered point cloud, and dividing the clustered point cloud to obtain a pin point cloud to be tested, including:

step S01: establishing a K-D Tree search structure corresponding to the through filtering point cloud, and randomly selecting a first clustering point;

step S02: determining k points closest to the first clustering point, and a distance threshold value between each point in the k points and the first clustering point, and scribing the points with the distance threshold value smaller than a preset distance threshold value into a first clustering point set;

step S03: randomly selecting points except the first clustering point set as second clustering points in the first clustering point set, and repeatedly executing the step S02 until all the points meeting the conditions are marked into the first clustering point set, and completing clustering of the first clustering point set when new points are not marked into the first clustering point set any more;

step S04: selecting a point which is closest to the first clustering point and does not belong to the first clustering point set as a third clustering point, repeatedly executing steps S02 to S03 to finish clustering of the third clustering point, repeatedly executing step S04 to finish all clusters corresponding to the through filtering point cloud, and outputting a clustering point cloud which comprises a plurality of clustered clustering point clouds formed after clustering;

Step S05: and calculating the number of points of each clustering point cloud set, and deleting the clustering point cloud set with the largest number of points to obtain the segmented pin point cloud to be detected.

Optionally, the performing spatial correction processing on the pin point cloud to be detected by using a preset reference pin point cloud to obtain a spatial correction point cloud includes:

taking the preset reference pin point cloud as a target point cloud and taking the pin point cloud to be detected as a source point cloud;

performing centroid transformation calculation by adopting the target point cloud and the source point cloud to obtain a transformation matrix, and performing center alignment on the source point cloud according to the transformation matrix to obtain a coarse registration point cloud;

and carrying out regional clustering on the rough alignment point cloud to obtain a centroid of a pin region to be detected, and carrying out ICP registration with the target point cloud by taking the centroid of the pin region to be detected as a matching characteristic point to obtain a space correction point cloud.

Optionally, the performing area clustering on the rough alignment point cloud to obtain a centroid of a pin area to be detected, performing ICP registration with the target point cloud by using the centroid of the pin area to be detected as a matching feature point, and obtaining a spatial correction point cloud, including:

step S11: determining an initial corresponding point set, performing European clustering on the rough alignment point cloud, and extracting a clustering center in the rough alignment point cloud as a matching characteristic point set, wherein the matching characteristic point set comprises a plurality of matching characteristic points;

Step S12: searching a point closest to the matching characteristic point in the target point cloud as a matching corresponding point, and integrating the searched matching corresponding points into a matching corresponding point set;

step S13: combining the matched characteristic point set and the matched corresponding point set, and carrying out iterative computation by adopting a least square method to obtain optimal coordinate transformation, wherein the optimal coordinate transformation corresponds to a rotation matrix and a translation vector;

step S14: updating the rough alignment point cloud by adopting the rotation matrix and the translation vector to obtain a rotation point cloud, and calculating the average square distance between the rotation point cloud and the target point cloud;

step S15: if the average distance square is greater than or equal to a preset distance square threshold, repeating the steps S11 to S14 to perform iterative updating until the calculated average distance square is less than the preset distance square threshold;

step S16: and if the calculated average distance square is smaller than the preset distance square threshold, stopping iteration, completing space correction, and outputting space correction point cloud.

Optionally, the performing the european style clustering on the spatial correction point cloud, calculating a pin height from each pin to be measured to an XOY plane, including:

European clustering is carried out on the space correction point cloud so as to divide each pin to be detected of the SOP chip to be detected into a cluster;

and calculating the average distance from all points in each cluster to the XOY plane, wherein the average distance is the height from the pin to be detected to the XOY plane corresponding to the cluster.

Optionally, the calculating the pin coplanarity based on the pin heights includes:

screening the pin height with the largest pin height value from the pin heights as the largest pin height, and screening the pin height with the smallest pin height value from the pin heights as the smallest pin height;

and calculating a difference value according to the maximum pin height and the minimum pin height to obtain the coplanarity of the pins.

The invention also provides an SOP chip pin coplanarity detection device, which comprises:

the pin clustering segmentation module is used for collecting chip point clouds of the SOP chip to be detected, carrying out clustering segmentation on the chip point clouds to obtain pin point clouds to be detected, wherein the pin point clouds to be detected correspond to a plurality of pins to be detected;

the space correction processing module is used for carrying out space correction processing on the pin point cloud to be detected by adopting a preset reference pin point cloud to obtain a space correction point cloud;

The pin height calculating module is used for carrying out European clustering on the space correction point cloud and calculating the pin height from each pin to be detected to the XOY plane;

and the pin coplanarity calculating module is used for calculating pin coplanarity based on the pin heights, and the pin coplanarity is used for judging the chip pin coplanarity defect of the SOP chip to be tested.

The invention also provides an electronic device comprising a processor and a memory:

the memory is used for storing program codes and transmitting the program codes to the processor;

the processor is configured to execute the SOP chip pin coplanarity detection method according to any one of the above instructions in the program code.

The present invention also provides a computer-readable storage medium for storing program code for executing the SOP chip pin coplanarity detection method as set forth in any one of the above.

From the above technical scheme, the invention has the following advantages: aiming at the problems existing in the current SOP chip pin detection method, the SOP chip pin coplanarity detection method based on space correction is provided, and the pin area for detecting the point cloud can be independently segmented through a Euclidean-based pin segmentation algorithm, so that the calculation amount of detecting the point cloud of the pin is reduced; the pin correction method using the center alignment and ICP registration is provided, standard pins are used as templates, the center alignment is carried out on the pin point cloud, the ICP registration is carried out on the centroid of the extraction area, the spatial correction of the pin point cloud is realized, the influence caused by different positions and postures of the point cloud can be eliminated, the point cloud is regular in form, and the subsequent processing and analysis are convenient; clustering the pins by a clustering algorithm, then calculating the distance from each clustered pin cluster to an XOY plane (reference plane) to obtain the height of the pins, and then taking the maximum height difference between the pins as a coplanarity index to realize coplanarity detection on the pins of the SOP chip.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from these drawings without inventive faculty for a person skilled in the art.

FIG. 1 is a flow chart of steps of a method for detecting coplanarity of pins of an SOP chip according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a data acquisition device according to an embodiment of the present invention;

fig. 3 is a schematic structural diagram of a SOP chip with different sizes according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of a pin dividing step of an SOP chip according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a K-D Tree search structure when partitioning a cluster point cloud according to an embodiment of the present invention;

FIG. 6 is a schematic flow chart of a pin point cloud space correction step according to an embodiment of the present invention;

FIG. 7 is a schematic flow chart of a pin coplanarity detecting step according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a coplanarity defect of an SOP chip according to an embodiment of the present invention;

FIG. 9 is a schematic overall flow chart of a method for detecting coplanarity of pins of an SOP chip according to an embodiment of the present invention;

fig. 10 is a block diagram of an apparatus for detecting coplanarity of pins of an SOP chip according to an embodiment of the present invention.

Detailed Description

The embodiment of the invention provides an SOP chip pin coplanarity detection method, an SOP chip pin coplanarity detection device, electronic equipment and a storage medium, which are used for solving or partially solving the technical problems of low detection precision and poor imaging effect caused by easy influence of external environment in the existing SOP chip detection method.

In order to make the objects, features and advantages of the present invention more comprehensible, the technical solutions in the embodiments of the present invention are described in detail below with reference to the accompanying drawings, and it is apparent that the embodiments described below are only some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

As an example, for SOP chip pin detection, currently, most enterprises use traditional manual visual inspection and industrial 2D visual inspection schemes, and due to lack of depth information in industrial 2D inspection, if 3D indexes such as coplanarity, tilting degree and the like of chip pins are detected, the following problems tend to exist easily:

Therefore, one of the core inventions of the embodiments of the present invention is: aiming at the problems existing in the current SOP chip pin detection method, the SOP chip pin coplanarity detection method based on space correction is provided, firstly, the SOP chips with different sizes can be subjected to coplanarity detection by providing a data acquisition device based on a line laser sensor and a one-dimensional motion platform; secondly, through a Euclidean-based pin segmentation algorithm, a pin area for detecting the point cloud can be segmented independently, so that the calculation amount of detecting the point cloud of the pin is reduced; meanwhile, a pin correction method using center alignment and ICP (Iterative Closest Point ) registration is provided, standard pins are used as templates, center alignment is carried out on pin point clouds, ICP registration is carried out on the centroids of the extraction areas, spatial correction on the pin point clouds is achieved, influences caused by different point cloud pose can be eliminated, and the point cloud is regular in form and convenient to process and analyze in the follow-up process; finally, clustering each pin by a clustering algorithm, calculating the distance from each clustered pin cluster to an XOY plane (reference plane) to obtain the height of the pins, and taking the maximum height difference between the pins as a coplanarity index to realize coplanarity detection on the pins of the SOP chip.

Referring to fig. 1, a step flowchart of a method for detecting coplanarity of pins of an SOP chip provided by an embodiment of the present invention may specifically include the following steps:

step 101, collecting chip point clouds of an SOP chip to be tested, and carrying out clustering segmentation on the chip point clouds to obtain pin point clouds to be tested, wherein the pin point clouds to be tested correspond to a plurality of pins to be tested;

for data acquisition of the SOP chip, referring to fig. 2, a schematic structural diagram of a data acquisition device provided by an embodiment of the present invention is shown.

As can be seen from fig. 2, the data acquisition device mainly comprises a high-precision line laser sensor 201 and a one-dimensional motion platform 202, and can acquire point cloud data of a plurality of SOP chips 203 with different sizes (such as the SOP chips with different sizes shown in fig. 3), and the acquired chip point cloud data can be transmitted to an industrial personal computer 204 for subsequent data processing.

Next, a process of performing cluster division on the chip point cloud will be described with reference to a general flowchart of the SOP chip pin division step shown in fig. 4.

The left side of fig. 4 shows a general flow of clustering and splitting the chip point cloud, and the right side shows a point cloud processing schematic diagram corresponding to each processing step for more intuitively explaining the chip point cloud.

Then specifically, the clustering and segmentation are performed on the chip point cloud to obtain a pin point cloud to be tested, which may be:

step S401: carrying out statistical filtering on the chip point cloud to obtain a denoising chip point cloud;

first, for the obtained chip point cloudStatistical filtering is carried out to remove existing outliers and noise points, and chip point clouds after denoising are obtained>。

Wherein the statistical filtering is based on the statistical characteristics of Gaussian distribution, and is applied to chip point cloudsFirstly, searching K neighbors of the point by establishing a K-D Tree data structure, calculating the average value and standard deviation of coordinate values of the point and the neighbor point, and judging the average value of the distance from the point to the neighbor point +.>Whether the confidence coefficient is larger than the specified confidence coefficient of Gaussian distribution or not, the statistical filtering is realized, and the specific realization steps are as follows:

(1) And establishing a K-D Tree data structure to realize efficient searching of K points in each point field.

A K-D Tree (K-Dimensional Tree) is understood to be a binary Tree with K-Dimensional points for each node, wherein all non-leaf nodes can be seen as a hyperplane to partition a Space into two Half-spaces (Half-spaces), i.e. a Space partition data structure for partitioning points in a high-latitude Space.

(2) The distance of each point to its neighbors is then calculated according to the following formula：

；

Wherein,representing denoising chip point cloud->Midpoint (at the middle point)>To its neighbor point->Distance of (2), i.e，/>Representing denoising chip point cloud->There are a total of n points and,indicating that each point has k neighbors.

(3) Then according to Gaussian distributionModeling distance parameters, calculating the mean value of all the points and the neighbor distances +.>Standard deviation of distance->：

；

(4) Calculating the distance average value from each point to the neighbor point：

；

(5) Traversing all points if the distance mean satisfiesThis point is deleted and statistical filtering is achieved.

Step S402: voxel downsampling is conducted on the denoising chip point cloud to obtain a downsampling chip point cloud, and Z-axis straight-through filtering is conducted on the downsampling chip point cloud to obtain a straight-through filtering point cloud;

when data processing is performed, for large-scale point cloud processing, the three-dimensional structure information can be well reserved by directly extracting the characteristics of the point cloud, but the direct processing mode requires higher calculation cost when searching the neighborhood due to the disorder of the point cloud.

Aiming at the problems, dimension reduction can be realized by carrying out downsampling processing on the point clouds, and the operation on all the point clouds is converted to the key points obtained by downsampling, so that the geometric structure of the original point clouds is well reserved while the calculation amount is reduced. Therefore, the denoising chip point cloud obtained by statistical filtering can be obtained Performing voxel downsampling operation to obtain downsampled point cloud +.>。

In a specific implementation of the present invention,regarding the processing principle of voxel downsampling, firstly, denoising chip point clouds can be performedThe 3D space is subdivided to form a voxel grid, then the average value of all data in the voxel grid, namely the coordinates of centroid points, is calculated, then the centroid points are adopted to replace other points in the voxel grid, and the specific implementation steps of voxel downsampling are as follows:

(1) Calculating the point cloud to be sampled (i.e. denoising chip point cloud) Maximum and minimum of each dimension:

；

wherein,representing the point cloud to be sampled->Maximum in the dimension x-coordinate direction, +.>Representing the point cloud to be sampled->Minimum in the dimension x-coordinate direction, +.>Representing the point cloud to be sampled->Maximum in the y-coordinate direction of the dimension, +.>Representing the point cloud to be sampled->Minimum in the dimension y-coordinate direction, +.>Representing the point cloud to be sampled->At the maximum in the z-coordinate direction of the dimension,representing the point cloud to be sampled->Minimum in the z-coordinate direction of the dimension.

(2) The voxel grid size is then determined as:

；

wherein,representing the length of the voxel grid corresponding to dimension x,/->Representing the length of the voxel grid corresponding to dimension y,/- >Representing the length size of the voxel grid corresponding to the dimension z.

(3) The dimensions of the voxel grid are calculated by the following formula:

；

wherein,corresponding dimension x->Corresponding dimension y->Corresponding to dimension z.

(4) Then calculate the index of the voxel where each point is located:

；

wherein,index representing the corresponding dimension x of the voxel where the point (x, y, z) is located, +.>Index representing the corresponding dimension y of the voxel where the point (x, y, z) is located, +.>Representing the index of the corresponding dimension z of the voxel in which the point (x, y, z) is located.

(5) Then calculating the center coordinates of all the points with the same index to obtain the center of mass coordinates of each voxel grid, deleting all the points except the center of mass, realizing the downsampling of the point cloud voxels, and obtaining the downsampled chip point cloud after the downsampling process。

When aiming at denoising chip point cloudAfter the voxel downsampling process is completed, the obtained downsampled chip point cloud can be subjected toPerforming Z-axis direct filtering to remove point cloud in the Z-axis direction, and specifically setting a Z-axis direction threshold ++>Will beThe points in the threshold value are reserved, all the points outside the threshold value range are deleted, the point cloud direct-pass filtering is realized, and after the direct-pass filtering is obtainedIs>。

Specifically, the point cloud data can be filtered by using a pass-through filter and a statistical filter in the PCL (Point Cloud Library ), then points out of range in the Z-axis direction are filtered by the pass-through filter, and outliers and noise points are removed by using the statistical filter.

Step S403: and carrying out Euclidean clustering on the through filtering point cloud to obtain clustered point cloud, and dividing the clustered point cloud to obtain pin point cloud to be detected.

The euclidean clustering algorithm (Euclidean Clustering Algorithm, ECA) is a distance metric based clustering algorithm, and is also an unsupervised learning algorithm.

The Euclidean clustering algorithm is based on the principle that data points in a sample space are divided according to the distance, so that the distance between the data points in the same group is as small as possible, the distance between the data points in different groups is as large as possible, and the Euclidean clustering algorithm is suitable for the conditions of scattered data distribution and obvious clustering.

For better explanation, referring to fig. 5, a schematic diagram of a K-D Tree search structure when partitioning a cluster point cloud is shown in the embodiment of the present invention.

In a specific implementation, euclidean clustering is performed on the straight-through filtering point cloud to obtain clustered point cloud, and the clustered point cloud is segmented to obtain pin point cloud to be detectedMay include the steps of:

step S01: establishing a K-D Tree search structure corresponding to the straight-through filtering point cloud, and randomly selecting a first clustering point K ₁₂ ；

First, a pin point cloud to be tested as shown in (a) to (c) in fig. 5 is establishedK-D Tree search Structure of (E)And randomly select the first cluster point K ₁₂ 。

Wherein (a) in fig. 5 can be represented as a basic principle of a K-D Tree search structure, and (b) in fig. 5 can be represented as a pin point cloud to be measuredIn FIG. 5 (c) can be expressed as +.A for the pin point cloud to be tested>And constructing a K-D Tree search structure.

Step S02: determining a distance from a first cluster point K ₁₂ Nearest K points, and each of the K points to the first cluster point K ₁₂ And a point with the distance threshold smaller than a preset distance threshold r is marked into a first clustering point set;

the distance K can then be found using the K-D Tree ₁₂ Nearest K points and judging the K points to K ₁₂ Dividing the points with the distance threshold smaller than the set threshold r into a first cluster point set.

Step S03: randomly selecting and dividing the first clustering point K in the first clustering point set ₁₂ The other points are used as second clustering points, the step S02 is repeatedly executed until all the points meeting the conditions are marked into a first clustering point set, and clustering of the first clustering point set is completed when new points are not marked into the first clustering point set any more;

step S04: selecting a distance from the first cluster point K ₁₂ The nearest points which do not belong to the first clustering point set are used as third clustering points, the steps S02 to S03 are repeatedly executed to finish clustering of the third clustering points, then the step S04 is repeatedly executed to finish all clusters corresponding to the through filtering point cloud, the clustering point cloud is output, and the clustering point cloud comprises a plurality of clustered clustering point clouds;

step S05: calculating the number of points of each cluster point cloud set, deleting the cluster point cloud set with the largest number of points, and obtaining the segmented pin point cloud to be detected。

In step 101, in the embodiment of the present invention, from the standpoint that the detection efficiency is low due to the large amount of the point cloud data of the chip, a comprehensive application of the point cloud downsampling and the segmentation algorithm based on euclidean is provided, and the point cloud of the pin area is individually segmented by setting the preprocessing and segmentation judgment conditions of the point cloud of the chip, so that the amount of the point cloud data to be detected can be effectively reduced, and the subsequent pin detection efficiency is improved.

102, performing spatial correction processing on the pin point cloud to be detected by adopting a preset reference pin point cloud to obtain a spatial correction point cloud;

in the actual imaging process, the chip point cloud may have problems of inclination, rotation and the like due to different placement postures, so that the point cloud is irregular in form, which may bring a certain difficulty to subsequent processing and analysis. Therefore, through space correction, the influence caused by different positions and postures of the point cloud can be eliminated, so that the point cloud is regular in form and convenient to process and analyze.

Referring to fig. 6, the general idea of the spatial correction processing flow provided by the embodiment of the invention is: firstly, taking a standard pin point cloud (namely a preset reference pin point cloud) as a target point cloudThe pin point cloud to be tested is->As a source point cloud; then based on the target point cloud->Pin point cloud to be tested>Calculating a rotation matrix M according to the mass center, and applying the rotation matrix M to the pin point cloud to be testedTo realize +.>Center alignment is performedCoarse registration is realized, and a coarse registration point cloud is obtained; then carrying out regional clustering on the rough alignment point cloud to obtain clustering centroids of different pins and obtaining the centroids of the region centroids of the pins to be detected +.>Then, taking the pin centroid as a matching characteristic point to be cloud-added with the target point>ICP registration is carried out, and pin space correction is achieved.

The ICP algorithm is mainly used for accurately splicing depth images in computer vision, and accurate splicing is achieved by continuously iterating and minimizing corresponding points of source data and target data.

In a specific implementation, the step of performing spatial correction processing on the pin point cloud to be tested by adopting the preset reference pin point cloud to obtain the spatial correction point cloud may include:

step S601: taking the preset reference pin point cloud as a target point cloud The pin point cloud to be tested is->As a source point cloud;

step S602: using a target point cloudSource Point cloud->Performing centroid transformation calculation to obtain a transformation matrix, and aligning the centers of the source point clouds according to the transformation matrix to obtain a coarse registration point cloud;

further, the implementation procedure in step S602 may specifically be:

(1) Calculating the pin point cloud to be measured through the following formulaBarycentric coordinates->Target point cloud->Barycentric coordinates->：

；

Wherein,for the point cloud of the pin to be measured->Number of (I) and (II)>Is the target point cloud->Number of the pieces.

(2) By calculating the translation vector between two point clouds, a transformation matrix M can be obtained, expressed as:

；

(3) Cloud the pin points to be testedCenter of mass->Expressed in homogeneous form and transposed into column vectors:

；

wherein,representing +.>The centroid representation obtained after transformation.

(4) Then apply the transformation matrix, apply the transformation matrix M to the centroidIn (a):

；

wherein,representing the use of the transform matrix M pair->Transformed homogeneous form of coordinates +.>Representing the use of the transform matrix M pair->X coordinate value obtained after transformation, +.>Representing the use of the transform matrix M pair->Y coordinate value obtained after transformation, +.>Representing the use of the transform matrix M pair- >The z coordinate value obtained after the transformation is performed.

(5) Then the homogeneous coordinatesConverting into a three-dimensional coordinate form:

；

(6) Cloud the pin points to be testedAnd (3) executing the steps (4) to (5) to obtain pin point clouds after center alignment, namely coarse registration point clouds.

Step S603: and carrying out regional clustering on the rough alignment point cloud to obtain the centroid of the pin area to be detected, and carrying out ICP registration with the target point cloud by taking the centroid of the pin area to be detected as a matching characteristic point to obtain the space correction point cloud.

Further, performing region clustering on the rough alignment point cloud to obtain a centroid of a pin region to be detected, performing ICP registration with the target point cloud by taking the centroid of the pin region to be detected as a matching feature point, and obtaining a spatial correction point cloud, which may include:

step S11: determining an initial corresponding point set, performing European clustering on the rough alignment point cloud, extracting a clustering center in the rough alignment point cloud as a matching characteristic point set, wherein the matching characteristic point set comprises a plurality of matching characteristic points;

the Euclidean clustering algorithm is based on greedy thought, and is used for dividing all points in the data set into k categories through continuous iteration, and each point belongs to the category closest to the point.

In a specific implementation, an initial corresponding point set can be determined, european clustering is then performed on the rough registration point cloud, and a clustering center in the rough registration point cloud is extracted as a matching characteristic point of the rough registration point cloud。

the target point cloud can then be foundMiddle distance Point->Nearest dot->As a dot->In the embodiment of the present invention, a 28 pairs point set, which is a matching feature point set, can be found altogether>Matching corresponding point set +.>。

Step S13: combining the matched characteristic point set and the matched corresponding point set, and adopting a least square method to perform iterative computation to obtain an optimal coordinate transformation, wherein the optimal coordinate transformation corresponds to a rotation matrix and a translation vector;

the solution of the ICP coordinate transformation can then be performed, in particular in combination with a set of matching feature pointsMatching corresponding point set +.>(the number of the corresponding point pairs is 28), and the optimal coordinate transformation is calculated through the least square method iteration.

The rotation matrix R and the translation vector t can be calculated by the following formula such that the objective function F (R, t) reaches a minimum:

；

Wherein,argminrepresenting the objective functionF（R,t) The variable value when the minimum value is taken (i.e. the optimal coordinate transformation),w _i represent the firstiWeights of matching points, in the embodiment of the invention, the centroid is extracted as the matching characteristic point, and ICP registration is performed, sow _i The value of (2) is 1.

Step S14: updating the coarse registration point cloud by adopting a rotation matrix and a translation vector to obtain a rotation point cloud, and calculating the average square distance between the rotation point cloud and the target point cloud;

then the coarse registration point cloud can be updated by adopting the rotation matrix R and the translation vector t, and whether the registration accuracy requirement is met or not is judged based on the updating result, specifically, the calculated rotation matrix R and the translation vector can be usedtThe method is applied to the rough alignment point cloud to obtain a rotated rotation point cloudThen calculate the rotation point cloud +.>Cloud of target points->Is the average distance squared of (c).

Step S15: if the average distance square is greater than or equal to the preset distance square threshold, repeating steps S11 to S14 to perform iterative updating until the calculated average distance square is less than the preset distance square threshold;

and (3) taking the average distance square as an iteration judgment value, if the average distance square is larger than a preset distance square threshold value set by iteration, repeatedly executing the steps S11 to S14, and judging whether iteration needs to be continued or not based on the recalculated average distance square until the calculated average distance square is smaller than the set threshold value.

And when the calculated average distance square is smaller than a preset distance square threshold, stopping iteration, completing space correction, and outputting space correction point cloud.

In step 102, aiming at the problem of pose error in detection, the embodiment of the invention provides a method for carrying out spatial correction processing on pin point clouds by adopting a center alignment and ICP registration combination algorithm, so that the pose error of pins is eliminated, the pose problem of the pins after segmentation is solved, the robustness of subsequent coplanarity detection is effectively improved, and the pin coplanarity detection can be more accurately realized after the pins are spatially corrected.

Step 103, performing European clustering on the space correction point cloud, and calculating the pin height from each pin to be detected to an XOY plane;

further, performing European clustering on the space correction point cloud, and calculating the pin height from each pin to be detected to the XOY plane may include: firstly, european clustering is carried out on the space correction point cloud so as to divide each pin to be detected of the SOP chip to be detected into a cluster; the average distance from all points in each cluster to the XOY plane (i.e., the reference plane) is then calculated by the following formula:

；

Wherein s is the number of points contained in each cluster,irepresent the firstiA point of the light-emitting diode is located,jrepresent the firstjThe number of clusters of the clusters is,represent the firstjThe average distance between all points in the clusters and the XOY plane can be understood as the height of the pins to be detected in the clusters from the XOY plane.

And 104, calculating the coplanarity of the pins based on the heights of the pins, wherein the coplanarity of the pins is used for judging the coplanarity defect of the pins of the SOP chip to be tested.

Further, calculating the pin coplanarity based on the respective pin heights may be: firstly, screening the pin height with the largest pin height value from all pin heights as the largest pin height, and screening the pin height with the smallest pin height value from all pin heights as the smallest pin height; and then, calculating a difference value according to the maximum pin height and the minimum pin height by the following formula to obtain the pin coplanarity Dis:

；

wherein,representing maximum pin height, +.>Representing the minimum pin height.

And then comparing the pin coplanarity Dis with a coplanarity threshold value of 0.1mm to judge whether the pin has the coplanarity defect, when the pin coplanarity Dis is larger than or equal to the coplanarity threshold value of 0.1mm, the existence of the coplanarity defect of the SOP chip to be tested is indicated, and when the pin coplanarity Dis is smaller than the coplanarity threshold value of 0.1mm, the existence of the coplanarity defect of the SOP chip to be tested is indicated, or the defect is negligible.

For better explanation, referring to fig. 7, a general flow chart of a pin coplanarity detection step provided by an embodiment of the present invention is shown.

Firstly, inputting pin point cloud data subjected to space correction;

then clustering the pins by adopting an Euclidean distance algorithm (namely European clustering) so as to divide each pin to be detected of the SOP chip to be detected into a cluster;

calculating the average distance from all points in each pin (i.e. cluster) to the XOY planeThen calculating the coplanarity Dis of the pins, and comparing the coplanarity Dis of the pins with a coplanarity threshold value of 0.1 mm;

when the pin coplanarity Dis is greater than or equal to the coplanarity threshold value of 0.1mm, the existence of the pin coplanarity defect of the SOP chip to be tested is indicated, at the moment, the pin serial number causing the coplanarity defect can be output, and the detection flow is ended;

and when the pin coplanarity Dis is smaller than the coplanarity threshold value of 0.1mm, the SOP chip to be tested is free of coplanarity defects or the defects can be ignored (namely, the pin coplanarity is normal), and the detection flow can be directly ended at the moment.

Fig. 8 is a schematic diagram of an SOP chip coplanarity defect provided in an embodiment of the present invention. It can be seen from the figure that the coplanarity Dis of the pins is in fact a difference between the maximum pin height and the minimum pin height.

In steps 103 to 104, in combination with the foregoing steps, aiming at the problem of pose error in detection, the embodiment of the invention firstly proposes that a center alignment and ICP registration combination algorithm is adopted to perform space pose correction processing on the segmented pin point clouds so as to place the pin point clouds on the same horizontal plane, thereby eliminating the pin pose error, solving the pose problem of the segmented pins, effectively improving the robustness of subsequent coplanarity detection, and simultaneously, after the space pose correction is performed on the pin point clouds, only the distance from the point clouds in each pin to the XOY plane needs to be calculated in the subsequent calculation process, and an average value is taken as a pin height value, so that the calculated pin height is more representative of the height of the whole pin compared with the calculated centroid, thereby realizing the pin coplanarity detection more accurately.

In the embodiment of the invention, aiming at the problems existing in the current SOP chip pin detection method, the SOP chip pin coplanarity detection method based on space correction is provided, firstly, the coplanarity detection of SOP chips with various sizes is realized by providing a data acquisition device based on a line laser sensor and a one-dimensional motion platform; secondly, through a Euclidean-based pin segmentation algorithm, a pin area for detecting the point cloud can be segmented independently, so that the calculation amount of detecting the point cloud of the pin is reduced; meanwhile, a pin correction method using center alignment and ICP registration is provided, standard pins are used as templates, center alignment is carried out on pin point clouds, ICP registration is carried out on the centroid of an extraction area, spatial correction on the pin point clouds is achieved, influences caused by different positions and postures of the point clouds can be eliminated, and the point clouds are regular in form and convenient to process and analyze in follow-up mode; finally, clustering each pin by a clustering algorithm, calculating the distance from each clustered pin cluster to an XOY plane (reference plane) to obtain the height of the pins, and taking the maximum height difference between the pins as a coplanarity index to realize coplanarity detection on the pins of the SOP chip.

For better explanation, referring to fig. 9, an overall flow chart of an SOP chip pin coplanarity detection method provided by the embodiment of the present invention is shown, and it should be noted that, in this example, only the general flow of the SOP chip pin coplanarity detection method is briefly described, and detailed descriptions related to each step may refer to relevant matters in the foregoing embodiment, which is not repeated herein, but it is to be understood that the invention is not limited thereto.

As can be seen from fig. 9, the implementation system for detecting pin coplanarity of an SOP chip may further include a detection module for detecting pin coplanarity, in addition to a data acquisition device for data acquisition, where the detection module may be divided according to a processing flow, and mainly divided into a pin dividing module 901 for dividing a pin point cloud, a space correction module 902 for performing a space correction process on the pin point cloud, and a coplanarity detection module 903 for performing pin coplanarity detection.

After the chip point cloud is obtained through the line laser sensor of the data acquisition device and the one-dimensional motion platform, the chip point cloud is transmitted to the pin segmentation module 901. In the pin segmentation module 901, the pin point cloud to be detected can be segmented independently by performing data preprocessing (statistical filtering, voxel downsampling and Z-axis straight-through filtering processing) on the chip point cloud, and then based on euclidean clustering segmentation, so that the part of the point cloud only needs to be detected in the subsequent processing flow.

For the pin segmentation module 901, a segmentation algorithm based on region growth can also be used for clustering segmentation of chip pins. In combination with an actual application scene, the Euclidean clustering algorithm is higher in calculation efficiency compared with the area growth algorithm, so that the Euclidean clustering algorithm is selected to segment the pin point cloud, and the difference between the pin shape and the surface shape of the actually acquired SOP chip point cloud data is considered to be larger.

Then in the space correction module 902, the pin point cloud to be detected and the standard pin point cloud are aligned in the center, then the pin point cloud to be detected is used as the source point cloud, the standard pin point cloud is used as the target point cloud, the center alignment is performed first, the ICP registration is performed, the detection error caused by different pin pose is eliminated, and the space correction of the pin to be detected is realized.

For the pin space correction module 902, besides the mode of combining center alignment and an ICP algorithm proposed by the embodiment of the invention, other registration algorithms may be used, for example, an ICP registration algorithm based on feature point matching and registration and based on feature descriptors may be involved in a problem of local optimal solution, so that a good correction effect is not achieved, and meanwhile, compared with the algorithm proposed by the invention, the two algorithms are more complex in calculation and may take longer time.

Then, in the coplanarity detection module 903, the heights of the pins to be detected to the XOY plane are calculated by performing European clustering on the pin point cloudAnd calculates pin coplanarity +/based on the vertical distance between the highest pin and the lowest pin>Chip pin coplanarity defect determination is performed in conjunction with fig. 8.

For the coplanarity detection module 903, an algorithm that the height of the centroid represents the height of the entire pin exists in the related art, but this calculation mode does not consider the distribution of pins in the entire plane, and in the embodiment of the present invention, by calculating the average distance from each pin to the XOY plane as the pin height value, this comprehensive consideration can represent the height feature of the entire pin.

Based on the functional description of each module, the general flow of the SOP chip pin coplanarity detection method can be obtained as follows:

1) The system is started, a line laser sensor and a one-dimensional motion platform are initialized, and SOP chip point cloud data are obtained in a shooting mode;

2) Inputting the acquired chip point cloud data into a pin segmentation module 901, performing statistical filtering, voxel downsampling and Z-axis straight-through filtering processing, and segmenting a pin area based on Euclidean segmentation algorithm to obtain independent pin point clouds to be detected;

3) Inputting the segmented pin point cloud to be detected into a space correction module 902, and performing center alignment and ICP registration on the pin point cloud to be detected by taking standard pin point cloud data as a target to obtain a space correction point cloud, so as to realize pin space correction;

4) And then inputting the space correction point cloud to a pin coplanarity detection module 903, performing European clustering on pins, calculating the average distance from each pin to an XOY plane, further calculating the pin coplanarity, judging whether the SOP chip has the coplanarity defect based on the pin coplanarity, and outputting a detection result.

5) If the pins are normal after detection, marking the chips as normal chips, if the pins have coplanarity defects, marking the chips as defective chips, classifying the normal chips and the chips with different defects according to the marks, and conveying the chips with the defects to a scrapping frame or a next detection process.

In the embodiment of the invention, firstly, the chip point cloud is obtained through a high-precision line laser sensor and a one-dimensional motion platform, and meanwhile, a preprocessing algorithm and a Euclidean-based clustering segmentation algorithm are used for carrying out pin segmentation on the chip point cloud, so that the number of the point cloud is reduced from 45 ten thousand points to about 2800 points, and the calculated amount of point cloud detection is reduced by 160 times; secondly, a method of combining center alignment and ICP registration is provided for carrying out space correction on the segmented pins, so that detection errors caused by chip pose are eliminated, and the accuracy and the robustness of subsequent coplanarity index detection are enhanced; meanwhile, the pin coplanarity detection algorithm provided by the embodiment of the invention can effectively detect coplanarity indexes of the pins of the SOP chip.

Referring to fig. 10, a block diagram of an apparatus for detecting coplanarity of pins of an SOP chip according to an embodiment of the present invention may specifically include:

the pin clustering and splitting module 1001 is configured to collect chip point clouds of an SOP chip to be tested, and cluster and split the chip point clouds to obtain pin point clouds to be tested, where the pin point clouds to be tested correspond to a plurality of pins to be tested;

the space correction processing module 1002 is configured to perform space correction processing on the pin point cloud to be detected by using a preset reference pin point cloud to obtain a space correction point cloud;

a pin height calculating module 1003, configured to perform european style clustering on the spatial correction point cloud, and calculate a pin height from each to-be-detected pin to an XOY plane;

and the pin coplanarity calculating module 1004 is configured to calculate pin coplanarity based on each pin height, where the pin coplanarity is used for determining chip pin coplanarity defects of the SOP chip to be tested.

In an alternative embodiment, the pin cluster segmentation module 1001 includes:

the statistical filtering module is used for carrying out statistical filtering on the chip point cloud to obtain a denoising chip point cloud;

the direct filtering point cloud processing module is used for carrying out voxel downsampling on the denoising chip point cloud to obtain a downsampled chip point cloud, and carrying out Z-axis direct filtering on the downsampled chip point cloud to obtain a direct filtering point cloud;

And the clustering point cloud segmentation module is used for carrying out Euclidean clustering on the through filtering point cloud to obtain clustering point cloud, and segmenting the clustering point cloud to obtain pin point cloud to be detected.

In an alternative embodiment, the cluster point cloud segmentation module includes:

the first cluster point selection module is configured to execute step S01: establishing a K-D Tree search structure corresponding to the through filtering point cloud, and randomly selecting a first clustering point;

the first cluster point set dividing module is configured to execute step S02: determining k points closest to the first clustering point, and a distance threshold value between each point in the k points and the first clustering point, and scribing the points with the distance threshold value smaller than a preset distance threshold value into a first clustering point set;

the first clustering point set repeated clustering module is configured to execute step S03: randomly selecting points except the first clustering point set as second clustering points in the first clustering point set, and repeatedly executing the step S02 until all the points meeting the conditions are marked into the first clustering point set, and completing clustering of the first clustering point set when new points are not marked into the first clustering point set any more;

the clustering point cloud set repeated clustering module is used for executing step S04: selecting a point which is closest to the first clustering point and does not belong to the first clustering point set as a third clustering point, repeatedly executing steps S02 to S03 to finish clustering of the third clustering point, repeatedly executing step S04 to finish all clusters corresponding to the through filtering point cloud, and outputting a clustering point cloud which comprises a plurality of clustered clustering point clouds formed after clustering;

The pin point cloud generating module to be tested is used for executing step S05: and calculating the number of points of each clustering point cloud set, and deleting the clustering point cloud set with the largest number of points to obtain the segmented pin point cloud to be detected.

In an alternative embodiment, the spatial correction processing module 1002 includes:

the pin point cloud setting module is used for taking the preset reference pin point cloud as a target point cloud and taking the pin point cloud to be detected as a source point cloud;

the center alignment module is used for carrying out centroid transformation calculation by adopting the target point cloud and the source point cloud to obtain a transformation matrix, and carrying out center alignment on the source point cloud according to the transformation matrix to obtain a coarse registration point cloud;

and the point cloud space correction sub-module is used for carrying out region clustering on the rough alignment point cloud to obtain a to-be-detected pin region centroid, and carrying out ICP registration with the target point cloud by taking the to-be-detected pin region centroid as a matching characteristic point to obtain a space correction point cloud.

In an alternative embodiment, the point cloud space correction submodule includes:

the matching feature point set extracting module is configured to execute step S11: determining an initial corresponding point set, performing European clustering on the rough alignment point cloud, and extracting a clustering center in the rough alignment point cloud as a matching characteristic point set, wherein the matching characteristic point set comprises a plurality of matching characteristic points;

The matching corresponding point set generating module is configured to execute step S12: searching a point closest to the matching characteristic point in the target point cloud as a matching corresponding point, and integrating the searched matching corresponding points into a matching corresponding point set;

the optimal coordinate transformation calculation module is configured to execute step S13: combining the matched characteristic point set and the matched corresponding point set, and carrying out iterative computation by adopting a least square method to obtain optimal coordinate transformation, wherein the optimal coordinate transformation corresponds to a rotation matrix and a translation vector;

the average distance square calculation module is configured to execute step S14: updating the rough alignment point cloud by adopting the rotation matrix and the translation vector to obtain a rotation point cloud, and calculating the average square distance between the rotation point cloud and the target point cloud;

the iterative update repeated execution module is configured to execute step S15: if the average distance square is greater than or equal to a preset distance square threshold, repeating the steps S11 to S14 to perform iterative updating until the calculated average distance square is less than the preset distance square threshold;

the spatial correction point cloud output module is configured to execute step S16: and if the calculated average distance square is smaller than the preset distance square threshold, stopping iteration, completing space correction, and outputting space correction point cloud.

In an alternative embodiment, the pin height calculation module 1003 includes:

the cluster division module is used for performing European clustering on the space correction point cloud so as to divide each pin to be detected of the SOP chip to be detected into a cluster;

and the pin height calculating sub-module is used for calculating the average distance from all points in each cluster to the XOY plane, wherein the average distance is the pin height from the pin to be detected to the XOY plane corresponding to the cluster.

In an alternative embodiment, the pin coplanarity calculating module 1004 is specifically configured to:

For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the foregoing method embodiments for relevant points.

The embodiment of the invention also provides electronic equipment, which comprises a processor and a memory:

The memory is used for storing the program codes and transmitting the program codes to the processor;

the processor is used for executing the SOP chip pin coplanarity detection method according to any embodiment of the application according to the instructions in the program code.

The embodiment of the application also provides a computer readable storage medium, which is used for storing program codes, and the program codes are used for executing the SOP chip pin coplanarity detection method of any embodiment of the application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The SOP chip pin coplanarity detection method is characterized by comprising the following steps:

2. The SOP chip pin coplanarity detection method according to claim 1, wherein the clustering segmentation is performed on the chip point cloud to obtain a pin point cloud to be detected, and the method comprises:

3. The SOP chip pin coplanarity detection method according to claim 2, wherein the performing euclidean clustering on the through filtering point cloud to obtain a clustered point cloud, and dividing the clustered point cloud to obtain a pin point cloud to be detected includes:

4. The SOP chip pin coplanarity detection method according to any one of claims 1 to 3, wherein the performing spatial correction processing on the pin point cloud to be detected by using a preset reference pin point cloud to obtain a spatial correction point cloud includes:

5. The SOP chip pin coplanarity detection method according to claim 4, wherein the performing region clustering on the coarse alignment point cloud to obtain a pin region centroid to be detected, performing ICP registration with the target point cloud by using the pin region centroid to be detected as a matching feature point, and obtaining a spatial correction point cloud includes:

6. The SOP chip pin coplanarity detection method according to any one of claims 1 to 3, wherein performing euclidean clustering on the spatial correction point cloud, calculating a pin height from each pin to be detected to an XOY plane, includes:

7. The SOP chip pin coplanarity detection method of claim 6, wherein the calculating pin coplanarity based on each of the pin heights comprises:

8. The utility model provides a SOP chip pin coplanarity detection device which characterized in that includes:

9. An electronic device, the device comprising a processor and a memory:

the processor is configured to execute the SOP chip pin coplanarity detection method according to any one of claims 1-7 according to instructions in the program code.

10. A computer-readable storage medium storing program code for performing the SOP chip pin coplanarity detection method of any of claims 1-7.