CN115239628A

CN115239628A - Defect detection method, optical detection device, electronic device, and storage medium

Info

Publication number: CN115239628A
Application number: CN202210716472.1A
Authority: CN
Inventors: 陈龙; 曹沿松
Original assignee: Optima Optics Technology Shenzhen Co ltd
Current assignee: Optima Optics Technology Shenzhen Co ltd
Priority date: 2022-06-21
Filing date: 2022-06-21
Publication date: 2022-10-25

Abstract

The application discloses a defect detection method, an optical detection device, an electronic device and a computer readable storage medium, wherein the defect detection method is applied to the optical detection device, and comprises the following steps: acquiring a circuit board image and information of a first type of defect in the circuit board image; extracting defect relation information among the defects in the first type of defects; and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects. According to the method, the obtained circuit board image is used for extracting the defect information in the circuit board, and the relation information between the defects is extracted according to the defect information, so that partial defects in the circuit board image can be optimized, the detection flow of the circuit board is accelerated, and the consumption of human resources is reduced.

Description

Defect detection method, optical detection device, electronic device, and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a defect detection method, an optical detection device, an electronic device, and a storage medium.

Background

With the wide application of products such as automobile electronics, communication equipment, transformers, inductance devices, power modules and the like in life and the rapid development of electronic information technology and communication technology, the market puts higher requirements on electronic products with high transmission and high voltage. The performance of a Printed Circuit Board (PCB) as a basic carrier of electronic components directly affects the performance of the product after the electronic components are mounted.

Since the PCB inevitably has a large number of defects during the manufacturing process, and the defects are mainly located at the circuit elements of the PCB. Therefore, the circuit elements with defects need to be detected for subsequent PCB repair work; the circuit components on the PCB are generally very small and dense, which is time-consuming and labor-consuming if the circuit components are completely detected manually, and is prone to error. If the circuit element with defects is not detected and is directly put into the next process for manufacturing, the repair cost of the subsequent PCB is higher and higher, and the PCB is more easily scrapped, thereby causing a great deal of waste cost.

Disclosure of Invention

In order to solve the above problems, the present application provides a defect detection method, an optical detection device, an electronic device, and a computer-readable storage medium, which can optimize and reduce some defects in a circuit board image, thereby speeding up a circuit board detection process and reducing human resource consumption.

The technical scheme adopted by the application is as follows: there is provided a defect detection method applied to an optical inspection apparatus, the method including: acquiring a circuit board image and information of a first type of defect in the circuit board image; extracting defect relation information among the defects in the first type of defects; and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

Optionally, the defect relation information includes pitch information; extracting defect relation information among the defects in the first type of defects, comprising the following steps: extracting the position of the central point of each defect in the first type of defects based on the information of the first type of defects; and calculating the distance information among the plurality of first type defects according to the central point position of the first type defects.

Optionally, the defect relation information includes similarity information; extracting defect relation information among the defects in the first type of defects, wherein the defect relation information comprises the following steps: extracting shape features of each defect in the first type of defects based on the information of the first type of defects; and calculating similarity information among a plurality of first-type defects according to the shape characteristics of the first-type defects.

Optionally, generating a second type of defect by using a plurality of first type of defects meeting a preset defect relation condition in the defect relation information, including: and fusing at least part of the first type of defects meeting the preset defect relation condition based on the defect relation information to generate second type of defects.

Optionally, generating a second type of defect by using a plurality of first type of defects meeting a preset defect relation condition in the defect relation information, including: determining a confidence value of the first type of defect based on the information of the first type of defect; selecting at least part of the first type defects meeting the first condition from all the first type defects based on the defect relation information; and according to the confidence degrees of at least part of the first type of defects, selecting the first type of defects with the confidence degrees meeting the second condition to determine the first type of defects as second type of defects.

Optionally, fusing, based on the defect relation information, the first type of defect at least partially meeting a preset defect relation condition to generate a second type of defect, including: determining at least one pair of defects smaller than a preset spacing value based on the defect relation information; and fusing the at least one pair of defects based on the shape characteristics of each defect in the at least one pair of defects to generate a second type of defect.

Optionally, according to the confidence level of at least part of the first type of defects, selecting the first type of defects with the confidence level meeting the second condition to determine as second type of defects, including: obtaining confidence of at least part of the first type of defects; determining the first type of defect with the highest confidence coefficient as a second type of defect; or determining the first type of defect with the confidence coefficient higher than the preset confidence coefficient threshold value as the second type of defect.

Another technical scheme adopted by the application is as follows: there is provided an optical inspection apparatus comprising: the image acquisition module is used for acquiring a circuit board image; the information extraction module is used for extracting information of the first type of defects in the circuit board image and defect relation information among the defects in the first type of defects; and the defect detection module is used for generating a second type of defects by utilizing a plurality of first type of defects which accord with the preset defect relation condition in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

The application adopts another technical scheme that: an electronic device is provided, the electronic device including: the defect detection device comprises a processor and a memory connected with the processor, wherein program data are stored in the memory, and the processor calls the program data stored in the memory to execute the defect detection method.

The application adopts another technical scheme that: there is provided a computer readable storage medium having stored therein program data for implementing the defect detection method as described above when executed by a processor.

Different from the prior art, the defect detection method provided by the application is applied to optical detection equipment, and comprises the following steps: acquiring a circuit board image and information of a first type of defect in the circuit board image; extracting defect relation information among the defects in the first type of defects; and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects. With the above optical inspection apparatus, on the one hand, defect information in the circuit board is extracted using the acquired circuit board image, and relationship information between defects is extracted based on the defect information, so that the production quality of the circuit board and the condition of the defects can be known from the defect relationship information. On the other hand, partial first defects meeting the conditions are generated into second defects by utilizing the defect relation information, so that partial defects in the circuit board image can be optimized and reduced, the detection process of the circuit board is accelerated, and the consumption of human resources is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:

FIG. 1 is a schematic structural diagram of an embodiment of an optical inspection apparatus provided herein;

FIG. 2 is a schematic flow chart diagram illustrating a defect detection method according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating an embodiment of extracting defect relation information according to the present application;

FIG. 4 is a schematic flow chart illustrating another embodiment of extracting defect relation information in the present application;

FIG. 5 is a schematic flow chart diagram illustrating one embodiment of generating a second type of defect in the present application;

FIG. 6 is a schematic flow chart diagram illustrating another embodiment of generating a second type of defect in the present application;

FIG. 7 is a flowchart illustrating an embodiment of step S135 of the present application;

FIG. 8 is a schematic structural diagram of an electronic device provided herein;

FIG. 9 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided in the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Reference in the application to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

The steps in the embodiments of the present application are not necessarily processed according to the described step sequence, and the steps in the embodiments may be optionally rearranged in a random manner, deleted or added as required, and the description of the steps in the embodiments of the present application is only an optional combination of sequences, and does not represent all the combinations of the sequences of the steps in the embodiments of the present application, and the sequence of the steps in the embodiments of the present application cannot be considered as a limitation of the present application.

The term "and/or" in embodiments of the present application refers to any and all possible combinations including one or more of the associated listed items. It is also noted that: when used in this specification, the term "comprises/comprising" specifies the presence of stated features, integers, steps, operations, elements and/or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements and/or components and/or groups thereof.

The terms "first", "second", etc. in this application are used to distinguish between different objects and not to describe a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

In addition, although the terms "first", "second", etc. are used several times in this application to describe various operations (or various elements or various applications or various instructions or various data) and the like, these operations (or elements or applications or instructions or data) should not be limited by these terms. These terms are only used to distinguish one operation (or element or application or instruction or data) from another operation (or element or application or instruction or data). For example, the information of the first type of defect may be referred to as information of the second type of defect, and the information of the second type of defect may also be referred to as information of the first type of defect, only the included ranges of the two are different, without departing from the scope of the present application, and the information of the first type of defect and the information of the second type of defect are both sets of various defect information, only the two are not the same set of defect information.

Referring to fig. 1, fig. 1 is a schematic structural diagram of an embodiment of an optical inspection apparatus 10 provided in the present application, including: an image acquisition module 11, an information extraction module 12 and a defect detection module 13.

In one embodiment, the Optical Inspection apparatus 10 is capable of Automated Optical Inspection (AOI), wherein AOI is an apparatus that detects common defects encountered in solder production based on Optical principles. When AOI is performed, the optical inspection apparatus 10 automatically scans the PCB through the camera, acquires an image, compares the tested welding spot with the qualified parameters in the database, inspects the defects on the PCB through image processing, and displays/marks the defects through the display or the automatic mark for the maintenance personnel to repair.

In one embodiment, the optical inspection device 10 is capable of running a computer program in a user mode to perform one or more specific tasks (e.g., acquiring information about a first type of defect in an image of a circuit board, fusing at least a portion of the first type of defect), which may be interactive with a user, and which has a visual User Interface (UI). The optical detection device 10 may also comprise two parts: graphical User Interface (GUI) and engine (engine), both of which enable a digital client system providing a variety of application services to a user in the form of a user interfaceAnd (4) obtaining the system. Alternatively, the optical detection device 10 may be based on a Liunx (GNU/Linux) system, a Mac (Macintosh, microphone) system, a microsoft system, or the like for program operation, and the optical detection device 10 may also be based on

A platform,

A platform,

Platform or

Platforms, etc. for program applications.

In one embodiment, the user may input corresponding code data or control parameters to the optical detection apparatus 10 through the input device to perform the feature service of the optical detection apparatus 10 in the user mode and to display the application service in the user interface. If a user needs to acquire an image of the circuit board, or the user needs to adjust production parameters of the circuit board, the user operates the circuit board through the input device and displays the circuit board through the display device.

In one embodiment, the image capturing module 11 is used to capture an image of the circuit board.

Alternatively, the image acquisition module 11 may be equipped with an image acquisition device of one of a depth camera, a 3D camera, a monocular camera, a binocular camera, or the like, which may generate corresponding control information according to the input of the user to acquire the acquired image of the circuit board.

Optionally, the image acquisition module 11 may also acquire production parameters of the circuit board. The production parameters are index parameters stored in the storage medium of the optical inspection apparatus 10 for producing circuit boards, and include at least one of the production type of the circuit board, the production area of the circuit component, the production accuracy, and the production quantity. The circuit board image is a circuit board detection image captured by the image acquisition module 11 through AOI, and defect information of each defect existing on the circuit board can be extracted from the detection image, where the defect information includes a defect position, a defect size, a defect type, a defect number, and the like.

Optionally, the circuit board image is a PCB (Printed circuit board) image. The PCB is also called a printed circuit board, and its board surface is divided into a circuit board area and a non-circuit board area. The PCB may be applied to a variety of electronic components including mobile terminals such as a video camera and a video recorder, a mobile phone, a smart phone, a notebook computer, a Personal Digital Assistant (PDA), a tablet computer (PAD), etc., and may also be fixed terminals of a Digital broadcast transmitter, a Digital TV, a desktop computer, a server, etc.

Among them, due to the tension of the medicament, the PCB inevitably has a large number of defects in the production process, and due to the manufacturing process, in the production process of the circuit board, for example, it is inevitable that: holes, rat erosion, open circuit, short circuit, burrs, copper slag and the like.

In an embodiment, the information extraction module 12 is configured to extract information of a first type of defect in the circuit board image and defect relation information between defects in the first type of defect.

Optionally, the information extraction module 12 may compare the circuit board image with the design drawing of the circuit board, so as to obtain information of the first type of defect in the circuit board image. The information of the first type of defects is information of all defects in the circuit board image, and the defect information may include positions of the defects, a total type of the defects, a number of the defects, and the like.

As an example, the information extraction module 12 imports a circuit diagram of a circuit board into software, and automatically generates a design drawing of the circuit board based on the circuit diagram by the software. After the design drawing of the circuit board is obtained, the circuit board which is the same as the design drawing of the circuit board can be generated according to the design drawing of the circuit board. And then, aligning the circuit board image with the design drawing of the circuit board by using an alignment algorithm to further obtain defect information in the circuit board image.

Optionally, the information extraction module 12 may further analyze the defect information of the circuit board image to obtain relationship information (including a position relationship and a similarity relationship between two defects, etc.) of each defect in the circuit board.

Optionally, a fractional prediction model (e.g., a point convolution neural network based on Attention's RNN, LSTM, etc., or a CNN network with point volume base layer + full connectivity) may also be included in the optical inspection device 10. The optical detection device 10 inputs the defect information in the circuit board image into the score prediction model for score prediction to output a corresponding score value of the circuit board. The score value is used for representing the production quality of the circuit board according to the defect score value in the circuit board image, namely the higher the defect score value of the circuit board image is, the lower the production quality of the circuit board is; the lower the defect score value of the circuit board image, the higher the production quality of the circuit board.

The optical detection device 10 may further include a parameter adjustment model (e.g., an autonomic adjustment neural Network in DQN (Deep Q Network)). The optical inspection apparatus 10 inputs the score values of the circuit board images and the production parameters into a parameter adjustment model for parameter recognition and optimization based on the score values to output adjusted production parameters. The purpose of adjusting the production parameters by the parameter adjusting model is to reduce the defect score value in the circuit board image so as to improve the production quality of the circuit board.

In an embodiment, the defect detecting module 13 generates the second type of defects by using a plurality of first type of defects meeting the preset defect relation condition in the defect relation information, where the number of the second type of defects is smaller than that of the plurality of first type of defects.

Optionally, the defect detection module 13 retrieves a plurality of first-type defects meeting a preset defect relation condition through the relation information of the first-type defects, and generates a second-type defect through the plurality of first-type defects, so as to reduce the number of identified defects in the circuit board and improve the importance of remaining defects in the circuit board.

As an example, 500 defects are included in the first type of defect. The defect detection module 13 retrieves 200 first-type defects A1 meeting a condition that the similarity of every two defects is greater than 70% and 150 first-type defects A2 meeting a condition that the distance between every two defect positions is less than 10mm through the relationship information of the first-type defects, and the defect detection module 13 respectively generates 100 second-type defects A3 and 75 second-type defects A4 for the 200 first-type defects A1 and the 150 first-type defects A2.

Different from the prior art, the optical detection apparatus provided in this embodiment includes: the image acquisition module is used for acquiring a circuit board image; the information extraction module is used for extracting information of the first type of defects in the circuit board image and defect relation information among the defects in the first type of defects; and the defect detection module is used for generating a second type of defects by utilizing a plurality of first type of defects which accord with the preset defect relation condition in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects. With the above optical inspection apparatus, on the one hand, defect information in the circuit board is extracted using the acquired circuit board image, and relationship information between defects is extracted based on the defect information, so that the production quality of the circuit board and the condition of the defects can be known from the defect relationship information. On the other hand, partial first defects meeting the conditions are generated into second defects by utilizing the defect relation information, so that partial defects in the circuit board image can be optimized and reduced, the detection process of the circuit board is accelerated, and the consumption of human resources is reduced.

Optionally, the optional embodiments are combined, and further optimization and expansion are performed based on the technical solution, so that an embodiment of the defect detection method provided by the present application can be obtained.

Referring to fig. 2, fig. 2 is a schematic flowchart illustrating an embodiment of a defect detection method provided in the present application. Wherein the method is applied to the optical detection device in the above embodiments to be executed by the optical detection device, and the method includes:

step S11: and acquiring the circuit board image and the information of the first type of defects in the circuit board image.

In one embodiment, the optical detection apparatus may be equipped with an image capturing device of one of a depth camera, a 3D camera, a monocular camera, a binocular camera, or the like, and generate corresponding control information according to the input of the user to acquire the circuit board image.

In another embodiment, the optical inspection apparatus obtains the circuit board image from its own storage medium or by connecting to a third party (e.g., a digital processing platform, a cloud server, an external terminal, etc.).

Optionally, the information about the first type of defect in the circuit board image may be obtained by comparing the circuit board image with a design drawing of the circuit board by the optical inspection device. The design drawing of the circuit board may be a design drawing of the circuit board automatically generated by the software based on the circuit drawing, wherein the circuit drawing of the circuit board is imported into the software.

Exemplarily, the optical detection device scans the circuit board in real time by being connected with the automatic optical detection device, transmits the circuit board to the image acquisition area by sending a motion control command to control the motion of the mechanical table and the motor, and acquires an image of the circuit board through the image acquisition module to finally obtain a detection image of the circuit board. In the process of detecting image acquisition, light can be supplemented to the image acquisition area, for example, the light is supplemented by controlling the on-off and brightness of the light source through the light source control module.

Alternatively, the optical inspection apparatus may input defect information of the circuit board into a fractional prediction model (e.g., a point convolution neural network based on RNN, LSTM, etc. of Attention, or a CNN network in a point volume base layer + full connection) to generate a score value of the circuit board; or, the development engineer manually inputs corresponding parameter data based on defect information displayed on a UI interface of the optical manufacturing apparatus, and the optical manufacturing apparatus obtains a score value of the circuit board.

Optionally, the score value of the circuit board is used for representing the production quality of the circuit board according to the defect score value in the detection image of the circuit board, namely the higher the defect score value of the detection image is, the lower the production quality of the circuit board is; the lower the defect score value of the inspection image, the higher the production quality of the circuit board.

Alternatively, the optical inspection apparatus may input the defect information of the circuit board into the fractional prediction model for defect prediction. Specifically, the score prediction model divides the circuit board into at least one defect area, and the score prediction model predicts the type of defects, the number of defects and the weight of corresponding defects in each defect area.

As an example, in the a defect region, the fractional prediction model predicts three defects A1, A2, and A3, where A1 predicts a correct weight of 20%, A2 predicts a correct weight of 50%, and A3 predicts a correct weight of 30%, and the number of predictions A1 is 8, the number of predictions A2 is 15, and the number of predictions A3 is 5.

Optionally, the optical inspection apparatus inputs the defect information of the circuit board image into the scoring model for feature extraction. The feature matrix and the feature type of the defects in the circuit board image can be extracted through a scoring model by utilizing an AlexNet deep convolution network of an ImageNet image data set.

Specifically, the scoring model may input the RGB circuit board image of each picture 256 × 3 into an AlexNet deep convolution network of the ImageNet image dataset by using a three-channel data matrix, to obtain a feature matrix and a feature type of the trained defect, where the dimension of the trained feature matrix is not limited, and since the fifth layer output dimension of AlexNet is 6 × 256, the trained feature matrix may be expanded into a 96-by-96 feature matrix, and in a specific embodiment, the 96-by-96 feature matrix may be used as the trained feature matrix.

Further, the scoring model divides the plane polar angle of the circuit board image into Ns equal divisions, each equal division is called a sector, and Nr × Ns ring sector grids are obtained; then, a rectangular coordinate system is established by taking the size of the polar angle as a horizontal axis and the size of the polar diameter as a vertical axis, an Nr multiplied by Ns matrix is correspondingly generated, the characteristics of all points in the corresponding grid are stored at each position in the matrix (namely, the characteristic matrix and the corresponding characteristic type are stored), and then the Nr multiplied by Ns matrices are used for establishing a k-dimensional characteristic tree structure of the space through semantic distribution and semantic variance.

Optionally, the scoring model matches the k-dimensional feature tree structure of the detected image with a feature template pre-stored in the corresponding circuit board to obtain a matching result of the circuit board. The characteristic template pre-stored in the circuit board is a reference k-dimensional characteristic tree structure template of the circuit board, the circuit board corresponding to the template is a standard circuit board, and the circuit board has no defects and is also in an ideal state.

Optionally, the matching result is a difference result between the feature tree structure corresponding to the circuit board in real production and the feature tree structure corresponding to the circuit board in an ideal state.

In one embodiment, the matching result includes a difference region and corresponding difference information of the feature tree structure and the feature template; the disparity information includes a defect type, a defect size, and a defect number of a corresponding defect in the disparity area.

Optionally, the optical manufacturing device obtains the defect type, the defect size, and the defect number of the defect in each difference region according to the difference information, and then calculates a penalty score of each difference region according to a product of the score corresponding to each defect type, the score corresponding to the defect size of each defect type, and the score corresponding to the defect number of each defect type.

Optionally, the optical detection device adds the penalty scores of each difference region to obtain a penalty total score of the difference region; and the optical detection equipment subtracts the difference value of the penalty total score from the target score preset by the characteristic template so as to generate a score value of the circuit board. The target score preset by the feature template is a score set by the client, and is not specifically limited herein. Finally, the optical detection equipment adjusts the production parameters of the optical manufacturing equipment based on the section where the score value is located and the type of the circuit board, so that the circuit board is manufactured by utilizing the adjusted production parameters.

Step S12: and extracting defect relation information among the defects in the first type of defects.

In one embodiment, the optical inspection apparatus analyzes the defect information of the circuit board image to obtain relationship information (including a distance relationship, a position relationship, a similarity relationship, etc. between every two defects) of each defect in the circuit board.

Referring to fig. 3, fig. 3 is a schematic flowchart illustrating an embodiment of extracting defect relation information according to the present disclosure. Specifically, step S12 may include the steps of:

step S121: and extracting the position of the center point of each defect in the first type of defects based on the information of the first type of defects.

In one embodiment, the optical detection device extracts the position of the center point of each defect in the first type of defect according to the coordinates of the defect pixel, the size of the defect, the shape of the defect and the outer edge area of the defect in the information of the first type of defect.

Illustratively, in the UI interface of the optical inspection apparatus, defect a is a schematic diagram of a first type of defect in the circuit board image. And the optical detection equipment marks the outer edge of the defect A through a minimum circumscribed circle B according to the defect pixel coordinate, the defect size, the defect shape and the defect outer edge area of the defect A, wherein the midpoint coordinate B1 of the circumscribed circle B is the position of the center point of the defect A.

Step S122: and calculating the distance information among the plurality of first type defects according to the central point position of the first type defects.

In one embodiment, the optical detection device calculates the distance information between all the first type defects in the corresponding defect area according to the central point position between every two defects in the information of the first type defects and the defect area divided by the first type defects.

In one embodiment, the optical detection device divides the acquired circuit board image into a plurality of different image areas based on the type and functional partition of the circuit board, and each image area is divided into a defect area where the first type of defect is located. For example, if the circuit board image is divided into 3 different image areas, the first type of defect may have a first defect area, a second defect area, and a third defect area.

In one embodiment, the optical inspection apparatus calculates the spacing information between all of the first type defects in each defect region. The distance information may be a distance between defect center points between every two defects.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating another embodiment of extracting defect relation information according to the present application. Specifically, step S12 may include the steps of:

step S123: and extracting the shape characteristics of each defect in the first type of defects based on the information of the first type of defects.

In one embodiment, the optical detection device inputs the defect pixel coordinates, the defect size, the defect shape and the defect peripheral region in the information of the first type of defects into the image feature recognition model to extract the shape feature of each defect in the first type of defects.

In an embodiment, the image feature recognition model recognizes shape features which need to be extracted in the direction, position and angle of the corresponding image based on the circuit board image, and then performs feature segmentation on the circuit board image by using a feature extraction network (such as CNN, VGG, resNet, and the like) to segment the shape features corresponding to each defect in the circuit board image.

Step S124: and calculating similarity information among a plurality of first-type defects according to the shape characteristics of the first-type defects.

In one embodiment, the optical inspection apparatus inputs shape features corresponding to each defect in the circuit board image into a convolutional neural network model (e.g., an Attention-based RNN, LSTM, etc.) to extract similarity information between a plurality of first-type defects.

Illustratively, the shape features extracted by the convolutional neural network model are subjected to normalization processing such as translation, rotation, scaling, standard cutting and the like, and feature vectors of the normalized shape features are calculated through a convolutional neural network algorithm. And determining the distance between the feature vectors of the shape features corresponding to each defect by the convolution neural network model through the Eus distance or the cospinning distance. And finally, carrying out relation conversion on the Eus distance or the convolution distance corresponding to the characteristic vector by the convolution neural network model so as to determine the similarity information of the corresponding shape characteristics among a plurality of first-class defects.

Step S13: and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

In one embodiment, the optical detection device retrieves a plurality of first defects meeting a preset defect relation condition through relation information of the first defects, and generates second defects through the plurality of first defects, so that the number of identified defects in the circuit board is reduced, and the importance of defects remaining in the circuit board is improved.

In one embodiment, the optical detection device fuses the first type of defect at least partially meeting the preset defect relation condition based on the defect relation information to generate the second type of defect.

Referring to fig. 5, fig. 5 is a schematic flow chart illustrating an embodiment of generating the second type of defect in the present application. Specifically, step S13 may include the steps of:

step S131: at least one pair of defects smaller than a preset spacing value is determined based on the defect relation information.

In one embodiment, the optical detection device determines at least one pair of defects with a distance smaller than a preset distance value according to the distance between defect center points of the defects in the defect relation information. The at least one pair of defects is two defects in the same defect area in the circuit board, and the distance value between the two defects is smaller than the preset distance value, which is not specifically limited herein, and may be, for example, smaller than 2-20mm, 5mm, 10mm, or 15 mm.

Step S132: and fusing the at least one pair of defects based on the shape characteristics of each defect in the at least one pair of defects to generate a second type of defect.

In one embodiment, the optical detection device fuses at least one pair of defects with a distance smaller than a preset distance value into corresponding at least one second type of defect according to the shape of the defect between the defects in the defect relation information of the first type of defect. The fusion of at least one pair of defects can be performed through an image fusion model, and can also be performed manually in a UI interface of the optical detection device based on a development engineer.

Illustratively, a defect area of the circuit board is displayed in the UI interface of the optical inspection apparatus, and the defect area includes defects A1 and A2, and two pairs of defects A3 and A4 have a defect pitch smaller than a preset pitch value. The optical inspection apparatus fuses 2 the defects A1 and A2 into a new defect A5 in a circumscribed rectangular manner and fuses the defects A3 and A4 into a new defect A6 in a peripheral enclosing curve manner by the image fusion model.

Referring to FIG. 6, FIG. 6 is a schematic flow chart illustrating another embodiment of the present application for generating the second type of defect. Specifically, step S13 may further include the steps of:

step S133: based on the information of the first type of defect, a confidence value for the first type of defect is determined.

In one embodiment, the optical inspection apparatus determines a confidence value for the first type of defect based on the defect pixel coordinates, the defect size, the defect shape, and the defect peripheral region in the information for the first type of defect.

Alternatively, the confidence value of the first type of defect may be obtained by comparing the detected image of the circuit board with the design drawing of the circuit board by the optical detection device. The design drawing of the circuit board may be a design drawing of the circuit board automatically generated by the software based on the circuit drawing, wherein the circuit drawing of the circuit board is imported into the software. After the design drawing of the circuit board is obtained, the circuit board which is the same as the design drawing of the circuit board can be generated according to the design drawing of the circuit board. And then, aligning the circuit board image with the design drawing of the circuit board by using an alignment algorithm so as to obtain a confidence value of each defect in the circuit board image.

Step S134: at least a part of the first type defects meeting the first condition is selected among all the first type defects based on the defect relation information.

In an embodiment, the optical inspection apparatus selecting at least a portion of the first type of defects meeting the first condition may include at least a portion of the first type of defects meeting a defect center point spacing value condition and/or a defect similarity threshold condition.

In one embodiment, the optical detection device selects at least a part of the first type defects with a pitch smaller than a preset pitch value from all the first type defects according to the pitch of the defect center points between the defects in the defect relation information. The at least part of the first type defects are two defects in the same defect area in the circuit board, and the distance value between the two defects is smaller than the preset distance value, which is not specifically limited here, and may be, for example, smaller than 2-20mm, 5mm, 10mm, or 15 mm.

In an embodiment, the optical detection apparatus selects at least a part of the first type defects with similarity greater than a preset value from all the first type defects according to the similarity information between the defects in the defect relation information. The optical detection device may input shape features corresponding to each defect in the circuit board image into a convolutional neural network model (e.g., RNN, LSTM, etc. based on Attention) to extract similarity information between a plurality of first-type defects.

Step S135: and according to the confidence degrees of at least part of the first type of defects, selecting the first type of defects with the confidence degrees meeting the second condition to determine the first type of defects as second type of defects.

Referring to fig. 7, fig. 7 is a schematic flowchart illustrating an embodiment of step S135 in the present application. Specifically, step S135 may include the steps of:

step S1351: a confidence level is obtained for at least a portion of the first type of defect.

In an embodiment, the optical detection apparatus obtains the confidence of at least some of the first type defects meeting the first condition according to the confidence value of the first type defects determined in step S133 and at least some of the first type defects meeting the first condition in step S134.

Further, the optical detection device performs either step S1352 or step S1353 according to the confidence of at least part of the first type of defect to determine the second type of defect according to the first type of defect.

Step S1352: and determining the first type of defect with the highest confidence coefficient as the second type of defect.

In one embodiment, the optical detection device sorts at least some of the first type defects meeting the first condition according to their confidence levels from high to low, and the optical detection device selects the first type defect with the highest confidence level to determine the first type defect as the second type defect.

The first type of defect with the highest confidence is a defect with the highest confidence in a defect region, and the defect with the highest confidence may include one or more defects. For example, if the defect with the highest confidence level in the defect area a is A1, A1 is determined as a second type of defect in the defect area a; the defects with the highest confidence level in the defect region B are B1, B2, and B3, and the three defects have the same and the highest confidence levels in the defect region B, and then B1, B2, and B3 are determined as the second type of defects in the defect region B.

Step S1353: and determining the first type of defects with the confidence coefficient higher than the preset confidence coefficient threshold value as the second type of defects.

In an embodiment, the optical detection device sorts at least a part of the first type defects meeting the first condition according to the confidence coefficient of the first type defects from high to low, and the optical detection device selects the first type defects with the confidence coefficient higher than a preset confidence coefficient threshold value from the first type defects as the second type defects.

The first type of defect with the confidence coefficient higher than the preset confidence coefficient threshold is a defect with the confidence coefficient higher than the preset confidence coefficient threshold in a defect region. For example, if the confidence threshold preset in the defect region C is C0, and the defects satisfying a confidence higher than C0 in the defect region C include C1, C2, C3, and C4, then C1, C2, C3, and C4 are determined as the second type of defects in the defect region C.

Different from the prior art, the optical detection apparatus provided in this embodiment includes: the image acquisition module is used for acquiring a circuit board image; the information extraction module is used for extracting information of the first type of defects in the circuit board image and defect relation information among the defects in the first type of defects; and the defect detection module is used for generating a second type of defects by utilizing a plurality of first type of defects which accord with the preset defect relation condition in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects. With the optical inspection apparatus described above, on the one hand, defect information in the circuit board is extracted using the acquired circuit board image, and relationship information between defects is extracted from the defect information, so that the production quality of the circuit board and the condition of the defects can be understood from the defect relationship information. On the other hand, partial first defects meeting the conditions are generated into second defects by utilizing the defect relation information, so that partial defects in the circuit board image can be optimized and reduced, the detection process of the circuit board is accelerated, and the consumption of human resources is reduced.

Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device 100 provided in the present application, where the electronic device 100 includes a processor 101 and a memory 102 connected to the processor 101, where the memory 102 stores program data, and the processor 101 retrieves the program data stored in the memory 102 to execute the defect detection method.

Optionally, in an embodiment, the processor 101 is applied to an optical detection device; the processor 101 is used to execute the program data stored in the memory 102 to implement the following method: acquiring a circuit board image and information of a first type of defects in the circuit board image; extracting defect relation information among the defects in the first type of defects; and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

By the method, on one hand, the acquired circuit board image is used for extracting the defect information in the circuit board, and the relation information between the defects is extracted according to the defect information, so that the production quality of the circuit board and the conditions of the defects can be known through the defect relation information. On the other hand, partial first defects meeting the conditions are generated into second defects by using the defect relation information, so that partial defects in the circuit board image can be optimized and reduced, the detection process of the circuit board is accelerated, and the consumption of human resources is reduced.

The processor 101 may also be referred to as a Central Processing Unit (CPU). The processor 101 may be an electronic chip having signal processing capabilities. The processor 101 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 102 may be a memory bank, a TF card, etc., and may store all information in the electronic device 100, including the input raw data, the computer program, the intermediate operation result, and the final operation result, all stored in the storage 102. Which stores and retrieves information based on the location specified by the processor 101. With the memory 102, the electronic device 100 has a memory function to ensure normal operation. The storage 102 of the electronic device 100 may be classified into a main storage (internal storage) and an auxiliary storage (external storage) according to the purpose, and there is a classification method into an external storage and an internal storage. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described embodiment of the electronic device 100 is merely illustrative, and for example, the distance information between a plurality of first type defects is calculated according to the positions of the center points of the first type defects; according to the shape characteristics of the first type of defects, similarity information among a plurality of first type of defects is calculated, and the like, which is only in a collection mode, and other division modes can be realized in practice, for example, the first type of defects meeting the first condition and the first type of defects meeting the second condition can be combined or can be collected into another system, or some characteristics can be ignored or not executed.

In addition, functional units (such as an image acquisition module and a defect detection module) in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

Referring to fig. 9, fig. 9 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application, and the computer-readable storage medium 110 stores therein program instructions 111 capable of implementing all the methods described above.

The unit in which the functional units in the embodiments of the present application are integrated may be stored in the computer-readable storage medium 110 if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present application may be substantially implemented or contribute to the prior art, or all or part of the technical solution may be embodied in the form of a software product, and the computer-readable storage medium 110 includes several instructions in a program instruction 111 to enable a computer device (which may be a personal computer, a system server, or a network device, etc.), an electronic device (for example, MP3, MP4, etc., and may also be a mobile terminal such as a mobile phone, a tablet computer, a wearable device, etc., or a desktop computer, etc.), or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application.

Optionally, in an embodiment, the program instructions 111 are applied to an optical detection device; the program instructions 111, when executed by a processor, are configured to implement the method of: acquiring a circuit board image and information of a first type of defect in the circuit board image; extracting defect relation information among the defects in the first type of defects; and generating a second type of defects by utilizing the plurality of first type of defects which accord with the preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

By the method, on one hand, the acquired circuit board image is used for extracting the defect information in the circuit board, and the relation information between the defects is extracted according to the defect information, so that the production quality of the circuit board and the conditions of the defects can be known through the defect relation information. On the other hand, partial first defects meeting the conditions are generated into second defects by utilizing the defect relation information, so that partial defects in the circuit board image can be optimized and reduced, the detection process of the circuit board is accelerated, and the consumption of human resources is reduced.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-readable storage media 110 (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It is to be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by the computer-readable storage medium 110. These computer-readable storage media 110 can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the program instructions 111, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer-readable storage media 110 may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the program instructions 111 stored in the computer-readable storage media 110 produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer-readable storage media 110 may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the program instructions 111 that execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one embodiment, these programmable data processing devices include a processor and memory thereon. The processor may also be referred to as a CPU (Central Processing Unit). The processor may be an electronic chip having signal processing capabilities. The processor may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory may be a memory stick, TF card, etc. that stores and retrieves information based on the location specified by the processor. The memory is classified into a main memory (internal memory) and an auxiliary memory (external memory) according to the purpose, and also into an external memory and an internal memory. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.

The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made according to the content of the present specification and the accompanying drawings, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

Claims

1. A defect detection method applied to an optical detection device is characterized by comprising the following steps:

acquiring a circuit board image and information of a first type of defect in the circuit board image;

extracting defect relation information among the defects in the first type of defects;

and generating a second type of defects by using a plurality of first type of defects which accord with preset defect relation conditions in the defect relation information, wherein the number of the second type of defects is less than that of the plurality of first type of defects.

2. The defect detection method of claim 1, wherein the defect relation information includes pitch information;

the extracting of the defect relation information between the defects in the first type of defect includes:

extracting the position of the central point of each defect in the first type of defects based on the information of the first type of defects;

and calculating the distance information among a plurality of first-type defects according to the central point position of the first-type defects.

3. The defect detection method of claim 1, wherein the defect relationship information includes similarity information;

the extracting of the defect relation information among the defects in the first type of defect includes:

extracting shape features of each defect in the first type of defects based on the information of the first type of defects;

and calculating similarity information among a plurality of first-type defects according to the shape characteristics of the first-type defects.

4. The method according to claim 2 or 3, wherein the generating the second type of defect by using the plurality of first type of defects meeting the preset defect relation condition in the defect relation information comprises:

and fusing at least part of the first type of defects meeting the preset defect relation condition based on the defect relation information to generate the second type of defects.

5. The method according to claim 2 or 3, wherein the generating the second type of defect by using the plurality of first type of defects meeting the preset defect relation condition in the defect relation information comprises:

determining a confidence value of the first type of defect based on the information of the first type of defect;

selecting at least part of first-type defects meeting a first condition from all first-type defects based on the defect relation information;

and according to the confidence degrees of at least part of the first type defects, selecting the first type defects with the confidence degrees meeting the second condition to be determined as the second type defects.

6. The method according to claim 4, wherein the fusing the first type of defects that at least partially satisfy the predetermined defect relationship condition based on the defect relationship information to generate the second type of defects comprises:

determining at least one pair of defects smaller than a preset spacing value based on the defect relation information;

fusing the at least one pair of defects based on shape features of each defect of the at least one pair of defects to generate the second type of defect.

7. The method of claim 5, wherein said selecting the first type of defect with a confidence level meeting a second condition as the second type of defect according to the confidence levels of the at least some first type of defects comprises:

obtaining confidence of at least part of the first type of defects;

determining the first type of defect with the highest confidence coefficient as the second type of defect; or,

and determining the first type of defects with the confidence coefficient higher than a preset confidence coefficient threshold value as the second type of defects.

8. An optical inspection apparatus, characterized in that the optical inspection apparatus comprises:

the image acquisition module is used for acquiring a circuit board image;

the information extraction module is used for extracting information of first type defects in the circuit board image and defect relation information among the defects in the first type defects;

and the defect detection module is used for generating second defects by utilizing a plurality of first defects which accord with preset defect relation conditions in the defect relation information, wherein the number of the second defects is less than that of the first defects.

9. An electronic device, comprising a processor and a memory connected to the processor, wherein the memory stores program data, and the processor retrieves the program data stored in the memory to perform the defect detection method according to any one of claims 1 to 7.

10. A computer readable storage medium having stored therein program instructions, wherein the program instructions are executable to implement a defect detection method as claimed in any one of claims 1-7.