CN116563357B - Image matching method, device, computer equipment and computer readable storage medium - Google Patents
Image matching method, device, computer equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN116563357B CN116563357B CN202310838392.8A CN202310838392A CN116563357B CN 116563357 B CN116563357 B CN 116563357B CN 202310838392 A CN202310838392 A CN 202310838392A CN 116563357 B CN116563357 B CN 116563357B
- Authority
- CN
- China
- Prior art keywords
- image
- fourier
- transformation
- production image
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 82
- 238000003860 storage Methods 0.000 title claims abstract description 18
- 238000004519 manufacturing process Methods 0.000 claims abstract description 161
- 238000013461 design Methods 0.000 claims abstract description 135
- 230000009466 transformation Effects 0.000 claims abstract description 84
- 238000010586 diagram Methods 0.000 claims abstract description 77
- 230000008859 change Effects 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 21
- 230000004044 response Effects 0.000 claims description 19
- 230000007547 defect Effects 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 238000012937 correction Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000013519 translation Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000007689 inspection Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- IKGXIBQEEMLURG-NVPNHPEKSA-N rutin Chemical compound O[C@@H]1[C@H](O)[C@@H](O)[C@H](C)O[C@H]1OC[C@@H]1[C@@H](O)[C@H](O)[C@@H](O)[C@H](OC=2C(C3=C(O)C=C(O)C=C3OC=2C=2C=C(O)C(O)=CC=2)=O)O1 IKGXIBQEEMLURG-NVPNHPEKSA-N 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010363 phase shift Effects 0.000 description 1
- 150000003071 polychlorinated biphenyls Chemical class 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application discloses an image matching method, an image matching device, computer equipment and a computer readable storage medium. The method comprises the following steps: respectively carrying out Fourier Merlin transformation on the production image and the design image of the PCB to obtain a production image after the Fourier Merlin transformation and a design image after the Fourier Merlin transformation; correcting the production image after Fourier-Mellin transformation by using the design image after Fourier-Mellin transformation based on a phase correlation method to obtain a corrected production image; dividing the design image after Fourier Merlin transformation and the corrected production image into a plurality of block diagrams with the same number, and obtaining the block diagram of the design image and the block diagram of the production image; and matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result. By adopting the application, the matching between the design image and the production image of the PCB can be realized rapidly and accurately.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to an image matching method, an image matching device, a computer device, and a computer readable storage medium.
Background
In modern electronics, printed circuit boards (PCBs, printed Circuit Board) are very widely used. In the defect inspection process of the PCB, it is required to precisely align the design drawing and the production drawing of the PCB. However, due to the complexity of the PCB scene and the huge amount of image data (for example, the pixels of the image generally reach 30000×30000), the general image matching method is difficult to meet the actual requirement, and the efficiency and accuracy of the subsequent PCB defect detection are seriously affected.
In the conventional process, the design drawing and the production drawing of the PCB are matched, a plurality of characteristic points are selected on the PCB, and a transformation relation is calculated according to the characteristic points, so that the alignment of the design drawing and the production drawing is realized. The matching method based on the characteristic points has low matching precision and low speed when the image data quantity of the PCB is large.
Disclosure of Invention
In view of this, embodiments of the present application provide an image matching method, apparatus, computer device, and computer readable storage medium, which can quickly and accurately match a design image and a production image of a PCB.
In a first aspect, the present application provides an image matching method, including:
respectively carrying out Fourier Merlin transformation on the production image and the design image of the PCB to obtain a production image after the Fourier Merlin transformation and a design image after the Fourier Merlin transformation;
correcting the production image after Fourier-Mellin transformation by using the design image after Fourier-Mellin transformation based on a phase correlation method to obtain a corrected production image;
dividing the design image after Fourier Merlin transformation and the corrected production image into a plurality of block diagrams with the same number, and obtaining the block diagram of the design image and the block diagram of the production image;
and matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
In a second aspect, the present application provides an image matching apparatus comprising:
the transformation unit is used for respectively carrying out Fourier-Merlin transformation on the production image and the design image of the PCB to obtain a production image after the Fourier-Merlin transformation and a design image after the Fourier-Merlin transformation;
the correction unit is used for correcting the production image after Fourier-Merlin transformation by utilizing the design image after Fourier-Merlin transformation based on a phase correlation method to obtain a corrected production image;
the segmentation unit is used for segmenting the design image after Fourier-Merlin transformation and the corrected production image into the same number of block diagrams to obtain the block diagrams of the design image and the block diagrams of the production image;
and the matching unit is used for matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
In a third aspect, the application provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the method described above when the processor executes the computer program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method described above.
In a fifth aspect, the application provides a computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method described above.
The image matching method, the device, the computer equipment, the computer readable storage medium and the computer program product adopt a mode of firstly roughly aligning and then finely aligning, wherein the roughly aligning is used for improving the speed of overall matching; fine alignment is mainly to improve the accuracy of matching. The rough alignment specifically comprises the steps of respectively carrying out Fourier-Merlin transformation on a production image and a design image of the PCB to obtain a production image after the Fourier-Merlin transformation and a design image after the Fourier-Merlin transformation; and correcting the Fourier-Merlin transformed production image by using the Fourier-Merlin transformed design image based on a phase correlation method to obtain a corrected production image, namely obtaining the change values of the scale and the angle between the production image and the design image and the offset values of the x direction and the y direction, and obtaining the corrected production image according to the change values of the scale and the angle and the offset values of the x direction and the y direction. And performing fine alignment, namely blocking the design image after Fourier Merlin transformation and the corrected production image, and calculating the pixel matching result between each block diagram of the design image and each block diagram of the corresponding production image in parallel. Therefore, when the images are matched, coarse alignment is performed before fine alignment, and matching between the design image and the production image of the PCB can be rapidly and accurately realized.
Drawings
Fig. 1 is an application scene diagram of an image matching method provided by an embodiment of the present application;
fig. 2 is a flowchart of an image matching method according to an embodiment of the present application;
FIG. 3 is a flowchart of another image matching method according to an embodiment of the present application;
FIG. 4 is a schematic illustration of a fine alignment provided by an embodiment of the present application;
fig. 5 is a schematic diagram of an image matching apparatus according to an embodiment of the present application;
FIG. 6 is a diagram illustrating an internal architecture of a computer device according to an embodiment of the present application;
FIG. 7 is an internal block diagram of another computer device according to an embodiment of the present application;
fig. 8 is an internal structural diagram of a computer-readable storage medium according to an embodiment of the present application.
Detailed Description
In order to make the purposes, technical schemes and advantages of the application more clear, application scenes of the technical scheme provided by the embodiment of the application are introduced.
The image matching method provided by the embodiment of the application is applied to a matching process before defect detection is carried out on the PCB, and mainly comprises the steps of matching a design drawing (cam drawing) and a production drawing (test drawing, also known as a physical drawing) of the PCB, and carrying out the defect detection on the PCB after the matching. However, because the image data of the PCB is huge, in order to achieve the matching speed and accuracy, the application adopts a coarse-to-fine matching mode, firstly performs coarse alignment on the whole image, then performs fine alignment on the blocks based on the result after the coarse alignment, namely, divides the design image and the production image after the coarse alignment into equal number of block images, aligns each block image in parallel, and performs one-to-one matching on the block image of the design image and the corresponding block image of the production image to obtain a matching result.
The image matching method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a communication network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.
In order to facilitate understanding of the technical solution of the embodiments of the present application, technical terms related to the embodiments of the present application will be described first.
Fourier-Mellin Transform: the method is a frequency domain transformation method, in the image matching process, the problems of translation, rotation, scale transformation, shielding, deformation and the like are often required to be solved, and the Fourier Merlin transformation can be used for well coping with translation, in-plane rotation, scaling and shielding, so that the method is a method with stronger robustness.
Log polar transformation (Log-polar): similar to the polar transformation, the fourier-mellin transformed image is transformed into the logarithmic polar domain. According to the logarithmic polar transformation, if there is only rotation and scaling of the two pictures. Rotation and scaling are converted to translational relationships in log polar coordinates, and then rotation and scaling amounts can be obtained using phase correlation techniques.
Phase correlation: the phase correlation method is used for detecting the direct relative displacement of two images with the same content, and is based on the displacement theorem of Fourier transformation, and the Fourier transformation of the function after the displacement is simply the product of the Fourier transformation of the function without translation and an exponential factor with linear phase, namely, the translation of the space domain can cause the phase shift of the frequency spectrum in the frequency domain.
In order that the above-recited objects, features and advantages of the present application will become more readily apparent, a more particular description of embodiments of the application will be rendered by reference to the appended drawings and appended drawings.
Referring to fig. 2, the flowchart of an image matching method according to an embodiment of the present application is shown.
The image matching method provided by the embodiment of the application comprises the following steps:
s201: and carrying out Fourier-Merlin transformation on the production image and the design image of the PCB respectively to obtain a production image after the Fourier-Merlin transformation and a design image after the Fourier-Merlin transformation.
The fourier melin transform functions to reject the effects of the translation factors of the production image and the design image of the PCB in the frequency domain, i.e., the x-direction and y-direction offsets.
S202: and correcting the production image after Fourier-Mellin transformation by using the design image after Fourier-Mellin transformation based on a phase correlation method, and obtaining a corrected production image.
The scale change value, the angle change value and the xy shift value can be obtained by using a phase correlation method.
The embodiment of the application utilizes the high efficiency of Fourier Merlin transformation in frequency domain calculation, for example, the requirement of completing coarse matching in about 80ms can be realized. For an oversized graph with the pixel size of 3w or more, the matching precision of the technical scheme of coarse alignment can reach within 32 pixels.
S203: and cutting the design image after Fourier Merlin transformation and the corrected production image into the same number of block diagrams to obtain the block diagrams of the design image and the block diagrams of the production image.
The design image and the corrected production image are segmented into the same number of tiles according to the size of the design image, e.g. for tiles at the same location, the tiles of the design image are smaller than the tiles of the production image.
It should be understood that the number of the block diagrams is not particularly limited, and may be set according to actual needs, for example, the larger the size of the reference design image, the larger the number of the block diagrams may be set, or may be set according to the matching speed and accuracy, for example, the higher the speed, the smaller the number of the block diagrams may be. The higher the accuracy, the greater the number of tiles may be.
S204: and matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
It should be understood that, in order to increase the matching speed, the matching result between the block diagram of each design image and the block diagram of the corresponding production image may be calculated in parallel, so as to implement the matching at the pixel level, specifically, the matching may be performed by using the template, and the higher the similarity, the more successful the matching is indicated.
For example, the precise alignment provided by the embodiment of the application can realize precise matching, and the final matching precision can reach within 3 pixels.
The image matching method provided by the embodiment of the application adopts a mode of coarse alignment and then fine alignment, and the coarse alignment is used for improving the overall matching speed; fine alignment is mainly to improve the accuracy of matching. The rough alignment specifically comprises the steps of respectively carrying out Fourier-Merlin transformation on a production image and a design image of the PCB to obtain a production image after the Fourier-Merlin transformation and a design image after the Fourier-Merlin transformation; and correcting the Fourier-Merlin transformed production image by using the Fourier-Merlin transformed design image based on a phase correlation method to obtain a corrected production image, namely obtaining the change values of the scale and the angle between the production image and the design image and the offset values of the x direction and the y direction, and obtaining the corrected production image according to the change values of the scale and the angle and the offset values of the x direction and the y direction. And performing fine alignment, namely blocking the design image after Fourier Merlin transformation and the corrected production image, and calculating the pixel matching result between each block diagram of the design image and each block diagram of the corresponding production image in parallel. Therefore, when the images are matched, coarse alignment is performed before fine alignment, and matching between the design image and the production image of the PCB can be rapidly and accurately realized.
The following describes a specific implementation procedure of the image matching method with reference to the accompanying drawings.
Referring to fig. 3, a flowchart of another image matching method according to an embodiment of the present application is shown.
S301: respectively downsampling the production image and the design image of the PCB to obtain a downsampled production image and a downsampled design image;
it should be appreciated that downsampling is to further increase the speed of coarse alignment of image matches. The down-sampling multiple may be set according to actual needs, for example, a specific multiple may be set according to the time and accuracy of image matching.
It should be appreciated that the downsampling factor of the production image and the design image may be the same. The downsampling of the production image is described below as an example.
Assuming that the width and height of the initial image of the production image are height, the downsampling multiple is n, n may be an integer or a decimal, and the embodiment of the application is not specifically limited, the calculation formulas of the width width_down and the height height_down of the downsampled image are respectively:
width_down=width/n，height_down=height/n。
s302: performing nearest neighbor interpolation on the downsampled production image and the design image to obtain a production image and a design image before Fourier Merlin transformation;
in order to increase the calculation speed, unnecessary calculation amount is reduced, and the pixel value of the image is not needed to be very accurate in coarse alignment, so that the pixel value of the downsampled image is obtained by adopting a nearest neighbor interpolation method.
S303: and respectively carrying out Fourier-Merlin transformation on the nearest interpolated production image and the downsampled design image to obtain a Fourier-Merlin transformed production image and a Fourier-Merlin transformed design image.
Before correcting the fourier-mellin-transformed production image by using the fourier-mellin-transformed design image based on the phase correlation method, the method further includes:
s304: and carrying out logarithmic polar coordinate transformation on the design image after Fourier-Merlin transformation and the production image after Fourier-Merlin transformation to obtain the design image in the logarithmic polar coordinate and the production image in the logarithmic polar coordinate.
In order to facilitate simpler calculation, the design image after Fourier-Merlin transformation and the production image after Fourier-Merlin transformation are both transferred to a logarithmic polar coordinate domain, and then rotation and scaling are changed into a translation relationship under the logarithmic polar coordinate.
And correcting the production image after Fourier-Merlin transformation by using the design image after Fourier-Merlin transformation based on a phase correlation method to obtain a corrected production image, wherein the corrected production image comprises S305 and S306.
S305: obtaining a scale change value, an angle change value and an xy offset value between a design image in a logarithmic polar coordinate and a production image in the logarithmic polar coordinate based on a phase correlation method;
obtaining xy offset values between the design image in the log polar coordinate and the production image in the log polar coordinate based on the phase correlation method, comprising:
k response values meeting preset conditions in all the response values are obtained based on a phase correlation method, wherein k is an integer greater than or equal to 2; for example, the satisfaction of the preset condition may be that all the response values are sorted from large to small, and the response values located in the first k are sorted as the response values satisfying the preset condition. For example, a series of response values, such as N total response values, are obtained at the spatial position based on the phase correlation method, and the first k larger response values are selected from the N response values, where N is an integer greater than or equal to 2.
And performing non-maximum suppression on the k response values to obtain a plurality of candidate points. Non-maximal suppression is to eliminate invalid candidate points, leaving valid candidate points.
Obtaining a matching value between the design image in the log polar coordinate and the production image in the log polar coordinate for each candidate point; taking a candidate point corresponding to the highest matching value in the plurality of matching values as a final matching position; the aim is to find the best point for the translational position from a plurality of candidate points. In particular, a similarity function may be utilized to obtain a matching value between the design image in log polar coordinates and the production image in log polar coordinates.
And obtaining xy offset values between the design image in the logarithmic polar coordinates and the production image in the logarithmic polar coordinates according to the final matching positions. Specifically, the production image may be shifted based on the final matching position, and offset calibration may be performed on the production image.
The method for obtaining the final matching position provided by the embodiment of the application is more robust, and can improve the accuracy of offset calibration.
It should be appreciated that since both the production image and the design image have been downsampled before, after the xy offset value is obtained, the xy offset value needs to be correspondingly upsampled to reflect the characteristics of the original image.
S306: and correcting the production image in the logarithmic polar coordinates according to the scale change value, the angle change value and the xy offset value to obtain a corrected production image. The specific matching value after coarse alignment can be obtained.
In particular, the corrected production image may be in particular the original production image multiplied by an inverse transformation matrix, the parameters of which are constructed using the scale change values, the angle change values and the xy offset values.
S307: dividing the design image after Fourier Merlin transformation and the corrected production image into a plurality of block diagrams with the same number, and obtaining the block diagram of the design image and the block diagram of the production image; matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result, wherein the matching result comprises the following steps:
and calculating pixel matching results between each block diagram of the design image and each block diagram of the corresponding production image in parallel.
Referring to fig. 4, a schematic diagram of fine alignment is provided according to an embodiment of the present application. The left side of fig. 4 is a design drawing, and the right side of fig. 4 is a production drawing, and it can be seen that the design drawing comprises a plurality of block drawings, and the production drawing comprises a plurality of block drawings; the block diagrams corresponding to the left side C and the block diagrams corresponding to the right side T are correspondingly matched block diagrams.
S308: and performing defect detection on the production image of the PCB by using the matched block diagram of the design image and the corresponding block diagram of the production image to obtain a defect detection result.
The image matching method provided by the embodiment of the application can realize the accurate matching of the design image and the production image of the PCB, and can improve the matching speed due to the coarse matching, and then the precise alignment is carried out on the block diagram in parallel, so that the matching precision can be improved. Since the design image and the production image of the PCB can be corrected in angle, scale and position, respectively, complex scenes of the PCB can be dealt with. The technical scheme provided by the embodiment of the application can realize the precise matching of the oversized image of automatic optical inspection (AOI, automated Optical Inspection), is convenient for the follow-up PCB defect inspection process, and improves the speed and accuracy of the whole PCB defect inspection process.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an image matching device. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of the image matching device or devices provided below may be referred to the limitation of the image matching method hereinabove, and will not be repeated here.
As shown in fig. 5, an embodiment of the present application provides an image matching apparatus 500, including:
a transforming unit 501, configured to perform fourier transform on the production image and the design image of the PCB, respectively, to obtain a production image after fourier transform and a design image after fourier transform;
a correction unit 502, configured to correct the fourier-mellin-transformed production image by using the fourier-mellin-transformed design image based on the phase correlation method, to obtain a corrected production image;
a segmentation unit 503, configured to segment the design image after fourier melin transformation and the corrected production image into a same number of block diagrams, to obtain a block diagram of the design image and a block diagram of the production image;
and the matching unit 504 is used for matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
In some embodiments, the correction unit 502 is specifically configured to, in correcting the fourier transformed production image using the fourier transformed design image based on the phase correlation method, obtain a corrected production image:
obtaining a scale change value, an angle change value and an xy offset value between a design image in a logarithmic polar coordinate and a production image in the logarithmic polar coordinate based on a phase correlation method;
and correcting the production image in the logarithmic polar coordinates according to the scale change value, the angle change value and the xy offset value to obtain a corrected production image.
In some embodiments, the correction unit 502 is specifically configured to, in obtaining xy offset values between the design image in log polar and the production image in log polar based on a phase correlation method:
k response values meeting preset conditions in all the response values are obtained based on a phase correlation method, wherein k is an integer greater than or equal to 2;
performing non-maximum suppression on the k response values to obtain a plurality of candidate points;
obtaining a matching value between the design image in the log polar coordinate and the production image in the log polar coordinate for each candidate point; taking a candidate point corresponding to the highest matching value in the plurality of matching values as a final matching position;
and obtaining xy offset values between the design image in the logarithmic polar coordinates and the production image in the logarithmic polar coordinates according to the final matching positions.
In some embodiments, in terms of performing a one-to-one matching between the block diagram of the design image and the block diagram of the corresponding production image, the matching unit 504 is specifically configured to:
and calculating pixel matching results between each block diagram of the design image and each block diagram of the corresponding production image in parallel.
In some embodiments, the transformation unit 501 is further configured to: and carrying out logarithmic polar coordinate transformation on the design image after Fourier-Merlin transformation and the production image after Fourier-Merlin transformation to obtain the design image in the logarithmic polar coordinate and the production image in the logarithmic polar coordinate.
In some embodiments, the apparatus further comprises a detection unit, wherein:
and the detection unit is used for carrying out defect detection on the production image of the PCB by utilizing the matched block diagram of the design image and the corresponding block diagram of the production image to obtain a defect detection result.
In some embodiments, the apparatus further comprises a sampling unit, wherein:
the sampling unit is used for respectively downsampling the production image and the design image of the PCB to obtain a downsampled production image and a downsampled design image;
in terms of performing fourier-mellin transformation on the production image and the design image of the PCB, respectively, to obtain a production image after fourier-mellin transformation and a design image after fourier-mellin transformation, the transformation unit 501 is specifically configured to:
and respectively carrying out Fourier-Merlin transformation on the downsampled production image and the downsampled design image to obtain a Fourier-Merlin transformed production image and a Fourier-Merlin transformed design drawing.
The respective modules in the above-described image matching apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In some embodiments, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store the design drawings. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement the steps in the image matching method described above.
In some embodiments, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement the steps in the image matching method described above. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer equipment can be a touch layer covered on a display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structures shown in fig. 6 or 7 are merely block diagrams of portions of structures associated with aspects of the application and are not intended to limit the computer device to which aspects of the application may be applied, and that a particular computer device may include more or fewer components than those shown, or may combine certain components, or may have a different arrangement of components.
In some embodiments, a computer device is provided, comprising a memory storing a computer program and a processor implementing the steps of the method embodiments described above when the computer program is executed.
In some embodiments, an internal structural diagram of a computer-readable storage medium is provided as shown in fig. 8, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method embodiments described above.
In some embodiments, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region.
Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (7)
1. An image matching method, comprising:
respectively carrying out Fourier Merlin transformation on the production image and the design image of the PCB to obtain a production image after the Fourier Merlin transformation and a design image after the Fourier Merlin transformation;
carrying out logarithmic polar coordinate transformation on the design image after Fourier-Merlin transformation and the production image after Fourier-Merlin transformation to obtain the design image in the logarithmic polar coordinate and the production image in the logarithmic polar coordinate;
obtaining a scale change value, an angle change value and an xy offset value between a design image in the logarithmic polar coordinate and a production image in the logarithmic polar coordinate based on a phase correlation method; correcting the production image in the logarithmic polar coordinates according to the scale change value, the angle change value and the xy offset value to obtain a corrected production image;
wherein the obtaining the xy offset value between the design image in the log polar coordinate and the production image in the log polar coordinate based on the phase correlation method includes: obtaining k response values meeting preset conditions in all the response values based on a phase correlation method, wherein k is an integer greater than or equal to 2; performing non-maximum suppression on the k response values to obtain a plurality of candidate points; obtaining a matching value between the design image in the logarithmic polar coordinates and the production image in the logarithmic polar coordinates for each of the candidate points; taking the candidate point corresponding to the highest matching value in the plurality of matching values as a final matching position; obtaining xy offset values between the design image in the log polar coordinates and the production image in the log polar coordinates according to the final matching positions;
dividing the design image after Fourier Merlin transformation and the corrected production image into a plurality of block diagrams with the same number to obtain a block diagram of the design image and a block diagram of the production image;
and matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
2. The method according to claim 1, wherein the step of matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result includes:
and calculating pixel matching results between each block diagram of the design image and each corresponding block diagram of the production image in parallel.
3. The method according to claim 2, wherein the method further comprises:
and performing defect detection on the production image of the PCB by using the matched block diagram of the design image and the corresponding block diagram of the production image to obtain a defect detection result.
4. The method of claim 1, wherein before performing fourier-mellin transformation on the production image and the design image of the PCB, respectively, to obtain a fourier-mellin transformed production image and a fourier-mellin transformed design image, the method further comprises:
respectively downsampling the production image and the design image of the PCB to obtain a downsampled production image and a downsampled design image;
the method for performing Fourier-Mellin transformation on the production image and the design image of the PCB respectively to obtain the production image after Fourier-Mellin transformation and the design image after Fourier-Mellin transformation comprises the following steps:
and respectively carrying out Fourier-Merlin transformation on the downsampled production image and the downsampled design image to obtain a Fourier-Merlin transformed production image and a Fourier-Merlin transformed design image.
5. An image matching apparatus, comprising:
the transformation unit is used for respectively carrying out Fourier-Merlin transformation on the production image and the design image of the PCB to obtain a production image after the Fourier-Merlin transformation and a design image after the Fourier-Merlin transformation;
the correction unit is used for carrying out logarithmic polar coordinate transformation on the design image after Fourier-Merlin transformation and the production image after Fourier-Merlin transformation to obtain the design image in the logarithmic polar coordinate and the production image in the logarithmic polar coordinate; obtaining a scale change value, an angle change value and an xy offset value between a design image in the logarithmic polar coordinate and a production image in the logarithmic polar coordinate based on a phase correlation method; correcting the production image in the logarithmic polar coordinates according to the scale change value, the angle change value and the xy offset value to obtain a corrected production image; wherein the obtaining the xy offset value between the design image in the log polar coordinate and the production image in the log polar coordinate based on the phase correlation method includes: obtaining k response values meeting preset conditions in all the response values based on a phase correlation method, wherein k is an integer greater than or equal to 2; performing non-maximum suppression on the k response values to obtain a plurality of candidate points; obtaining a matching value between the design image in the logarithmic polar coordinates and the production image in the logarithmic polar coordinates for each of the candidate points; taking the candidate point corresponding to the highest matching value in the plurality of matching values as a final matching position; obtaining xy offset values between the design image in the log polar coordinates and the production image in the log polar coordinates according to the final matching positions;
the segmentation unit is used for segmenting the design image after Fourier-Merlin transformation and the corrected production image into the same number of block diagrams to obtain the block diagrams of the design image and the block diagrams of the production image;
and the matching unit is used for matching the block diagram of the design image with the corresponding block diagram of the production image one by one to obtain a matching result.
6. A computer device, characterized in that it comprises a processor and a memory, in which computer program instructions are stored, which processor, when executing the computer program instructions, realizes the steps in the method according to any of claims 1-4.
7. A computer readable storage medium, characterized in that it has stored therein computer program instructions which, when executed by a processor, implement the steps in the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310838392.8A CN116563357B (en) | 2023-07-10 | 2023-07-10 | Image matching method, device, computer equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310838392.8A CN116563357B (en) | 2023-07-10 | 2023-07-10 | Image matching method, device, computer equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116563357A CN116563357A (en) | 2023-08-08 |
| CN116563357B true CN116563357B (en) | 2023-11-03 |
Family
ID=87495106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310838392.8A Active CN116563357B (en) | 2023-07-10 | 2023-07-10 | Image matching method, device, computer equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563357B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824086A (en) * | 2014-03-24 | 2014-05-28 | 东方网力科技股份有限公司 | Image matching method and device |
| CN104239882A (en) * | 2013-06-14 | 2014-12-24 | 富士通株式会社 | Image similarity determining device and method and image feature obtaining device and method |
| CN107516322A (en) * | 2017-08-11 | 2017-12-26 | 浙江大学 | A kind of image object size based on logarithm pole space and rotation estimation computational methods |
| CN109767401A (en) * | 2019-01-15 | 2019-05-17 | 深圳看到科技有限公司 | Picture optimization method, device, terminal and corresponding storage medium |
| CN114463391A (en) * | 2022-01-26 | 2022-05-10 | 中国电子科技集团公司第十四研究所 | SAR image registration method using image block matching |
| CN114708151A (en) * | 2022-06-02 | 2022-07-05 | 深圳市先地图像科技有限公司 | PCB image processing method |
| CN114998097A (en) * | 2022-07-21 | 2022-09-02 | 深圳思谋信息科技有限公司 | Image alignment method, device, computer equipment and storage medium |
| CN115239690A (en) * | 2022-08-11 | 2022-10-25 | 成都数之联科技股份有限公司 | PCB line sequence detection method and system based on image recognition |
| CN115965646A (en) * | 2023-03-16 | 2023-04-14 | 深圳思谋信息科技有限公司 | Region division method and device, computer equipment and computer readable storage medium |
| CN116362957A (en) * | 2021-12-27 | 2023-06-30 | 广州镭晨智能装备科技有限公司 | PCB card image alignment method, device, medium and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2296143A1 (en) * | 2000-01-18 | 2001-07-18 | 9071 9410 Quebec Inc. | Optical inspection system |
| US20030185432A1 (en) * | 2002-03-29 | 2003-10-02 | Hong Dezhong | Method and system for image registration based on hierarchical object modeling |
-
2023
- 2023-07-10 CN CN202310838392.8A patent/CN116563357B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104239882A (en) * | 2013-06-14 | 2014-12-24 | 富士通株式会社 | Image similarity determining device and method and image feature obtaining device and method |
| CN103824086A (en) * | 2014-03-24 | 2014-05-28 | 东方网力科技股份有限公司 | Image matching method and device |
| CN107516322A (en) * | 2017-08-11 | 2017-12-26 | 浙江大学 | A kind of image object size based on logarithm pole space and rotation estimation computational methods |
| CN109767401A (en) * | 2019-01-15 | 2019-05-17 | 深圳看到科技有限公司 | Picture optimization method, device, terminal and corresponding storage medium |
| CN116362957A (en) * | 2021-12-27 | 2023-06-30 | 广州镭晨智能装备科技有限公司 | PCB card image alignment method, device, medium and electronic equipment |
| CN114463391A (en) * | 2022-01-26 | 2022-05-10 | 中国电子科技集团公司第十四研究所 | SAR image registration method using image block matching |
| CN114708151A (en) * | 2022-06-02 | 2022-07-05 | 深圳市先地图像科技有限公司 | PCB image processing method |
| CN114998097A (en) * | 2022-07-21 | 2022-09-02 | 深圳思谋信息科技有限公司 | Image alignment method, device, computer equipment and storage medium |
| CN115239690A (en) * | 2022-08-11 | 2022-10-25 | 成都数之联科技股份有限公司 | PCB line sequence detection method and system based on image recognition |
| CN115965646A (en) * | 2023-03-16 | 2023-04-14 | 深圳思谋信息科技有限公司 | Region division method and device, computer equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于傅里叶-梅林变换的图像匹配方法研究;戴宪策 等;《红外技术》;第38卷(第10期);第860-863页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116563357A (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Chen et al. | Accurate light field depth estimation with superpixel regularization over partially occluded regions | |
| CN110096929A (en) | Target detection neural network based | |
| CN111340077A (en) | Disparity map acquisition method and device based on attention mechanism | |
| CN113963072B (en) | Binocular camera calibration method and device, computer equipment and storage medium | |
| CN110163095B (en) | Loop detection method, loop detection device and terminal equipment | |
| CN113516697B (en) | Image registration method, device, electronic equipment and computer readable storage medium | |
| CN116563357B (en) | Image matching method, device, computer equipment and computer readable storage medium | |
| CN112927163A (en) | Image data enhancement method and device, electronic equipment and storage medium | |
| CN115760641B (en) | Remote sensing image cloud and fog removing method and equipment based on multiscale characteristic attention network | |
| CN113298098B (en) | Fundamental matrix estimation method and related product | |
| CN116527908B (en) | Motion field estimation method, motion field estimation device, computer device and storage medium | |
| CN117237347B (en) | PCB defect detection method and device, storage medium and electronic equipment | |
| CN115861429B (en) | Image acquisition equipment calibration method, device, computer equipment and storage medium | |
| CN115965856B (en) | Image detection model construction method, device, computer equipment and storage medium | |
| CN116659520B (en) | Matching positioning method, device and equipment based on bionic polarization vision enhancement | |
| CN115880249B (en) | Image-based object segmentation method, device, equipment and medium | |
| CN115761239B (en) | Semantic segmentation method and related device | |
| CN116402842A (en) | Edge defect detection method, device, computer equipment and storage medium | |
| CN115063408B (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN117635554A (en) | Line defect detection method, device, computer equipment and storage medium | |
| CN117953415A (en) | Article placement detection method, device, computer equipment and storage medium | |
| CN116883491A (en) | Adjustment distance determining method, device, computer equipment and storage medium | |
| CN117764819A (en) | Image processing method, device, electronic equipment and storage medium | |
| CN115760774A (en) | Region detection method, device, computer equipment and computer readable storage medium | |
| CN117974576A (en) | Image quality detection method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |