CN116342597A

CN116342597A - Method and system for detecting electroplating processing defects on surface of automobile part

Info

Publication number: CN116342597A
Application number: CN202310609436.XA
Authority: CN
Inventors: 邬伟勇
Original assignee: Shenzhen Minda Technology Co ltd
Current assignee: Shenzhen Minda Technology Co ltd
Priority date: 2023-05-29
Filing date: 2023-05-29
Publication date: 2023-06-27
Anticipated expiration: 2043-05-29
Also published as: CN116342597B

Abstract

The embodiment of the specification discloses a method and a system for detecting the surface electroplating processing defects of an automobile part, wherein the method comprises the following steps: preprocessing a surface image corresponding to the automobile part to be detected to obtain a corresponding plating part image; uniformly dividing a plating part map into a plurality of image blocks, and dividing each image block into a plurality of sub-blocks; determining the image characteristics corresponding to each image block in a sub-block layering mode; determining multiple layers of saliency and defect probability corresponding to each image block based on the image characteristics; screening image blocks with the defect probability larger than the segmentation threshold value as defect image blocks, and calculating distribution anomaly degree corresponding to each sub-block in the defect image blocks; determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree; clustering each defective image block based on the abnormality index and the distribution abnormality degree; and determining defective pixel points in the plating part graph based on the classification result obtained by the clustering process.

Description

Method and system for detecting electroplating processing defects on surface of automobile part

Technical Field

The invention relates to the field of image data processing, in particular to a method and a system for detecting electroplating processing defects on the surface of an automobile part.

Background

Electroplating refers to a method of depositing a layer of metal on the surface of a metal article under the action of electrolysis. The metal product corrosion prevention, abrasion part repair, durability, light reflection, conductivity and the like are all functions, and the metal product corrosion prevention, abrasion part repair, light reflection, conductivity and the like are common processing technologies for processing the surfaces of automobile metal parts.

The electroplating process is affected by factors such as plating solution temperature fluctuation, PH value fluctuation, processing technology and the like, various defects such as uneven film thickness, scaly shape, earthworm grain, bubble holes, local oxidation, lack of welding, scratches, bubbles and the like on the surface of a metal substrate can occur in the electroplating film, the product quality is affected, and the product processing period is prolonged.

The current detection of the electroplating defects of the automobile parts is usually realized by technologies such as visual, optical microscope or metallographic microscope, ultrasonic and the like. The accuracy of visual detection is greatly influenced by subjective factors, and the detection efficiency is low; the detection method using a microscope and ultrasonic waves can only be used in a scene with low requirements on real-time performance, and needs professional instrument operators to carry out, so that the detection requirements in the processing process are difficult to meet.

Based on the above, it is necessary to research a more scientific and effective method for detecting the surface electroplating processing defects of the automobile parts so as to meet the actual production requirements.

Disclosure of Invention

One aspect of the embodiments of the present disclosure provides a method for detecting a defect in a surface plating process of an automotive part, the method comprising: acquiring a surface image corresponding to an automobile part to be detected; denoising and gray-scale processing are carried out on the surface image, and a plating piece diagram corresponding to the automobile part to be detected is obtained; uniformly dividing the plating part map into K image blocks with the area of M, and dividing each image block into subblocks with the size of N; determining image characteristics corresponding to each image block in a sub-block layering mode, wherein the image characteristics comprise inter-block difference degrees and inter-layer difference degrees; determining a plurality of layers of saliency corresponding to each image block based on the image features, wherein the plurality of layers of saliency is used for representing the saliency of the image block in the plating part graph; determining the defect probability corresponding to each image block according to the multi-layer saliency; screening the image blocks with the defect probability larger than the segmentation threshold as defect image blocks, and calculating distribution anomaly corresponding to each sub-block in the defect image blocks, wherein the distribution anomaly is used for reflecting the distribution difference of pixel point texture characteristics among different sub-blocks in the defect image blocks; determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree; determining a measurement distance between different sub-blocks in each defective image block based on the abnormality index and the distribution abnormality degree, and clustering each defective image block based on the measurement distance; and determining defective pixel points in the plating part graph based on the classification result obtained by the clustering process.

In some embodiments, the determining, by using a sub-block layering manner, an image feature corresponding to each of the image blocks includes: for each image block, taking the subblock with the size of N, which is obtained by dividing the image block, as a basic subblock layer of the image block; constructing a plurality of sub-block layers corresponding to the image block in a mode that edge intersection points of four adjacent sub-blocks in the sub-block of the previous layer serve as center points of the sub-block of the next layer, wherein the sub-blocks contained in each sub-block layer have the same size; determining inter-block difference degree between different sub-blocks in each sub-block layer and inter-layer difference degree between two adjacent sub-block layers; and taking the inter-block difference degree and the inter-layer difference degree as image features corresponding to the image blocks.

In some embodiments, the determining the inter-block difference between different sub-blocks in each of the sub-block layers and the inter-layer difference between two adjacent sub-block layers includes: obtaining an LBP value corresponding to each pixel point in the image block; constructing a local histogram corresponding to each sub-block in the plurality of sub-block layers based on the LBP value and the number of pixel points corresponding to the LBP value; determining inter-block difference degree between different sub-blocks in each sub-block layer based on the pasteurization distance between local histograms corresponding to different sub-blocks in each sub-block layer and the average pasteurization distance between sub-blocks corresponding to different sub-blocks in the next sub-block layer; acquiring a first LBP sequence and a second LBP sequence respectively corresponding to the two adjacent sub-block layers; and determining an interlayer difference degree between the two adjacent sub-block layers based on a DTW distance between the first LBP sequence and the second LBP sequence.

In some embodiments, the determining the multi-layer saliency corresponding to each of the image blocks based on the image features includes: and for each target image block, determining the multi-layer saliency corresponding to the target image block based on the inter-block difference degree and the inter-layer difference degree corresponding to the target image block, and the inter-block difference degree mean and the inter-layer difference degree mean corresponding to the plated part graph.

In some embodiments, the determining the defect probability corresponding to each image block according to the multi-layer saliency includes: for each target image block, calculating a first difference value between the multilayer saliency corresponding to the target image block and the minimum value of the multilayer saliency corresponding to the plating image, and a second difference value between the maximum value of the multilayer saliency corresponding to the plating image and the multilayer saliency corresponding to the target image block; and determining the defect probability corresponding to the target image block based on the ratio of the first difference value to the second difference value.

In some embodiments, the calculating the distribution anomaly degree corresponding to each sub-block in the defective image block includes: acquiring HOG feature descriptors corresponding to each pixel point in the defect image block; sequentially calculating cosine similarity between each pixel point in each sub-block in the basic sub-block layer and the HOG feature descriptors corresponding to the central pixel points, and taking the cosine similarity as a feature value of each pixel point; obtaining a distribution diagram corresponding to each sub-block in the defect image block based on the characteristic values; for each target sub-block, determining a distribution association between the target sub-block and other sub-blocks in the base sub-block layer based on the profile; calculating EDR editing distances between the target sub-block and Gaussian distribution curves corresponding to other sub-blocks in the basic sub-block layer; and determining the distribution anomaly degree corresponding to the target sub-block based on the number of sub-blocks in the basic sub-block layer, the distribution association degree and the EDR editing distance.

In some embodiments, the determining a distribution association between the target sub-block and other sub-blocks in the base sub-block layer based on the profile comprises: calculating a first pearson correlation coefficient between sequences of characteristic values located in the same column in distribution diagrams corresponding to the target sub-block and the other sub-blocks, and a second pearson correlation coefficient between sequences of characteristic values located in the same row in distribution diagrams corresponding to the target sub-block and the other sub-blocks; and obtaining the distribution association degree between the target subblock and other subblocks in the basic subblock layer based on the average value of the first pearson correlation coefficient and the average value of the second pearson correlation coefficient.

In some embodiments, the determining the abnormality index of each sub-block in the plating map according to the distribution abnormality comprises: for each target sub-block, calculating a first distribution anomaly mean value corresponding to all sub-blocks in a defect image block where the target sub-block is located and a second distribution anomaly mean value corresponding to all sub-blocks contained in all the defect image block; and determining an abnormality index of the target sub-block in the plating part graph based on a first difference value between the distribution abnormality corresponding to the target sub-block and the first distribution abnormality mean value and a second difference value between the first distribution abnormality mean value and the second distribution abnormality mean value.

In some embodiments, the method further comprises: determining one or more defect areas in the plating graph based on the defect pixel points; determining minimum circumscribed rectangles corresponding to the one or more defect areas respectively; and determining the electroplating qualification rate corresponding to the automobile part to be detected based on the sum of the areas of all the minimum circumscribed rectangles and the total area corresponding to the plating part graph.

Another aspect of the embodiments of the present disclosure also provides a system for detecting a defect in a surface plating process of an automotive part, the system comprising: the acquisition module is used for acquiring a surface image corresponding to the automobile part to be detected; the preprocessing module is used for carrying out denoising treatment and gray scale treatment on the surface image to obtain a plating part diagram corresponding to the automobile part to be detected; the image dividing module is used for uniformly dividing the plating part image into K image blocks with the area of M, and dividing each image block into subblocks with the size of N; the image characteristic determining module is used for determining image characteristics corresponding to each image block in a sub-block layering mode, wherein the image characteristics comprise inter-block difference degree and inter-layer difference degree; the multi-layer saliency determination module is used for determining multi-layer saliency corresponding to each image block based on the image features, wherein the multi-layer saliency is used for representing the saliency of the image block in the plating part graph; the defect probability determining module is used for determining the defect probability corresponding to each image block according to the multi-layer saliency; the distribution anomaly degree determining module is used for screening the image blocks with the defect probability larger than the segmentation threshold value as defect image blocks and calculating the distribution anomaly degree corresponding to each sub-block in the defect image blocks, wherein the distribution anomaly degree is used for reflecting the distribution difference of pixel point texture characteristics among different sub-blocks in the defect image blocks; the abnormality index determining module is used for determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree; the clustering module is used for determining the measurement distance between different sub-blocks in each defective image block based on the abnormality index and the distribution abnormality degree, and carrying out clustering processing on each defective image block based on the measurement distance; and the defective pixel point determining module is used for determining defective pixel points in the plating part graph based on the classification result obtained by the clustering process.

The method and the system for detecting the electroplating processing defects on the surface of the automobile part, provided by the embodiment of the specification, have the beneficial effects that at least: (1) The detection efficiency and the instantaneity of the defect detection can be improved by carrying out surface electroplating processing defect detection on the automobile parts in an image recognition mode; (2) The image characteristics corresponding to each image block in the plating part graph corresponding to the automobile part to be detected are determined in a sub-block layering mode, then the multi-layer saliency corresponding to each image block is determined based on the image characteristics, and the similarity among the sub-blocks and the space relation of the sub-blocks can be fully utilized to represent fine characteristics in the image blocks at any space position in the plating part graph, so that the detection precision of the saliency degree of the image blocks is improved; (3) By constructing the distribution anomaly degree according to the texture distribution characteristics of the pixel points in different sub-blocks in the image block, the local texture distribution of the pixel points in the sub-blocks and the texture characteristics in different directions can be fully considered, so that accurate evaluation indexes can be obtained for the pixel points in the defect area with any size, and the accuracy of subsequent clustering processing can be improved.

Additional features will be set forth in part in the description which follows. As will become apparent to those skilled in the art upon review of the following and drawings, or may be learned by the production or operation of the examples. The features of the present specification can be implemented and obtained by practicing or using the various aspects of the methods, tools, and combinations set forth in the detailed examples below.

Drawings

The present specification will be further described by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. The embodiments are not limiting, in which like numerals represent like structures, wherein:

FIG. 1 is a schematic illustration of an exemplary application scenario of an automotive part surface plating defect detection system according to some embodiments of the present disclosure;

FIG. 2 is an exemplary block diagram of an automotive part surface plating process defect detection system according to some embodiments of the present disclosure;

FIG. 3 is an exemplary flow chart of a method for inspecting an automotive part surface for a plating process defect according to some embodiments of the present disclosure;

FIG. 4 is a schematic diagram of the layering principle of an exemplary sub-block layering method shown in accordance with some embodiments of the present description;

FIG. 5 is an exemplary local histogram shown in accordance with some embodiments of the present description;

FIG. 6 is an exemplary distribution diagram shown according to some embodiments of the present description;

fig. 7 is an exemplary flowchart of a method for detecting defects in an automotive part surface plating process according to other embodiments of the present disclosure.

Detailed Description

In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some examples or embodiments of the present specification, and it is possible for those of ordinary skill in the art to apply the present specification to other similar situations according to the drawings without inventive effort. Unless otherwise apparent from the context of the language or otherwise specified, like reference numerals in the figures refer to like structures or operations.

It should be appreciated that as used in this specification, a "system," "apparatus," "unit" and/or "module" is one method for distinguishing between different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

As used in this specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.

A flowchart is used in this specification to describe the operations performed by the system according to embodiments of the present specification. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.

Automobile parts refer to various parts and devices mounted on an automobile, including engine parts, body parts, interior parts, and the like. Currently, the detection of plating defects in automotive parts is generally performed by visual, optical or metallographic microscopy, ultrasonic techniques, or the like. However, since the accuracy of visual detection is greatly affected by subjective factors, the detection efficiency is low, and the detection method using a microscope or ultrasonic waves can only be used in a scene with low real-time requirements and needs professional instrument operators, the detection means used at present are difficult to meet the detection requirements in the processing process.

Based on the technical problems, the specification provides an automobile accessory surface electroplating processing defect detection method and system based on image recognition, which can improve the detection efficiency and instantaneity of defect detection by detecting the surface electroplating processing defect of an automobile accessory in an image recognition mode. In addition, in the method and the system for detecting the electroplating processing defects on the surface of the automobile part, various image processing algorithms are used for processing the plating part graph corresponding to the automobile part to be detected, so that the recognition accuracy of the image features and the recognition accuracy of the electroplating processing defects are improved.

The method and system provided in the embodiments of the present specification are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic view of an exemplary application scenario of an automotive part surface plating defect detection system according to some embodiments of the present disclosure.

Referring to fig. 1, in some embodiments, an application scenario 100 of an auto-part surface plating defect detection system may include an image acquisition apparatus 110, a storage device 120, a processing device 130, a terminal device 140, and a network 150. The various components in the application scenario 100 may be connected in a variety of ways. For example, the image capturing apparatus 110 may be connected to the storage device 120 and/or the processing device 130 via the network 150, or may be directly connected to the storage device 120 and/or the processing device 130. As another example, the storage device 120 may be directly connected to the processing device 130 or connected via the network 150. For another example, the terminal device 140 may be connected to the storage device 120 and/or the processing device 130 through the network 150, or may be directly connected to the storage device 120 and/or the processing device 130.

The image capturing device 110 may be used to capture a surface image of an electroplated automobile part, where the surface image may reflect the electroplating quality of the automobile part and electroplating defects (e.g., scales, earthworm lines, cells, local oxidation, lack of welding, scratches, bubbles, etc. on the surface of the metal substrate) existing after electroplating. In some embodiments, the surface image may be an RGB image. In some embodiments, the image capture Device 110 may include a CCD (Charge-Coupled Device) camera, a CMOS (Complementary Metal-Oxide-Semiconductor) camera, or the like. In some embodiments, an industrial CCD camera may be preferred in order to ensure the acquisition quality of the surface image. In some embodiments, the image capture device 110 may be positioned in the plating area of the automotive part process and image capture the automotive part after it has been placed in the plating solution for a predetermined period of time (e.g., 5 minutes, 10 minutes, or other time). In some embodiments, the image capturing device 110 may also be disposed in a detection area of the plating process of the automobile accessory, and may capture an image of the automobile accessory to be detected after the plating of the automobile accessory is completed, so as to detect the plating defect in the automobile accessory. In some embodiments, the image capturing apparatus 110 may have an independent power source that may send the surface image captured for the auto-parts to be detected to other components (e.g., the storage device 120, the processing device 130, the terminal device 140) in the application scenario 100 by wired or wireless (e.g., bluetooth, wiFi, etc.). In some embodiments, the application scenario 100 may include a plurality (e.g., two or more) of image capturing devices 110, where the plurality of image capturing devices 110 may capture surface images of an automobile part to be detected from different perspectives.

In some embodiments, the image capturing apparatus 110 may transmit the surface image captured by the image capturing apparatus to the storage device 120, the processing device 130, the terminal device 140, and the like through the network 150. In some embodiments, the surface image acquired by the image acquisition device 110 may be processed by the processing apparatus 130. For example, the processing device 130 may determine, based on the surface image, a defective pixel corresponding to a plating processing defect in a plating drawing of the automobile part to be detected. In some embodiments, the data processed by the processing device 130 may be sent to the storage device 120 for storage, or sent to the terminal device 140 for feedback to the user (e.g., quality inspector or related personnel).

Network 150 may facilitate the exchange of information and/or data. The network 150 may include any suitable network capable of facilitating the exchange of information and/or data of the application scenario 100. In some embodiments, at least one component of the application scenario 100 (e.g., the image acquisition apparatus 110, the storage device 120, the processing device 130, the terminal device 140) may exchange information and/or data with at least one other component in the application scenario 100 via the network 150. For example, the processing device 130 may obtain surface images acquired for the machine room equipment from the image acquisition apparatus 110 and/or the storage device 120 over the network 150. For another example, the processing device 130 may obtain, from the terminal device 140 through the network 150, a user operation instruction, where the exemplary operation instruction may include, but is not limited to, retrieving a surface image collected for the to-be-detected accessory, reading a defective pixel corresponding to a plating processing defect in a plating drawing of the to-be-detected automobile accessory determined based on the surface image, and/or a plating qualification rate corresponding to the to-be-detected automobile accessory.

In some embodiments, network 150 may be any form of wired or wireless network, or any combination thereof. By way of example only, the network 150 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Public Switched Telephone Network (PSTN), a bluetooth network, a ZigBee network, a Near Field Communication (NFC) network, and the like, or any combination thereof. In some embodiments, the network 150 may include at least one network access point through which at least one component of the application scenario 100 may connect to the network 150 to exchange data and/or information.

Storage 120 may store data, instructions, and/or any other information. In some embodiments, the storage device 120 may store data obtained from the image acquisition apparatus 110, the processing device 130, and/or the terminal device 140. For example, the storage device 120 may store the surface image acquired by the image acquisition apparatus 110; for another example, the storage device 120 may store the defect pixel corresponding to the electroplating processing defect in the plating drawing of the to-be-detected automobile part and/or the electroplating qualification rate corresponding to the to-be-detected automobile part calculated by the processing device 130. In some embodiments, the storage device 120 may store data and/or instructions that the processing device 130 uses to perform or use to implement the exemplary methods described in this specification. In some embodiments, the storage device 120 may include mass memory, removable memory, volatile read-write memory, read-only memory (ROM), and the like, or any combination thereof. Exemplary mass storage devices may include magnetic disks, optical disks, solid state disks, and the like. In some embodiments, storage device 120 may be implemented on a cloud platform. For example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, or the like, or any combination thereof.

In some embodiments, the storage device 120 may be connected to the network 150 to communicate with at least one other component (e.g., the image acquisition apparatus 110, the processing device 130, the terminal device 140) in the application scenario 100. At least one component in the application scenario 100 may access data, instructions, or other information stored in the storage device 120 through the network 150. In some embodiments, the storage device 120 may be directly connected or in communication with one or more components (e.g., the image capture apparatus 110, the terminal device 140) in the application scenario 100. In some embodiments, the storage device 120 may be part of the image acquisition apparatus 110 and/or the processing device 130.

The processing device 130 may process data and/or information obtained from the image capture apparatus 110, the storage device 120, the terminal device 140, and/or other components of the application scenario 100. In some embodiments, the processing device 130 may obtain a surface image from any one or more of the image capturing device 110, the storage device 120, or the terminal device 140, process the surface image to determine a defective pixel corresponding to a plating defect in the plating drawing of the auto-part to be detected, and further calculate a plating qualification rate corresponding to the auto-part to be detected. In some embodiments, the processing device 130 may retrieve pre-stored computer instructions from the storage device 120 and execute the computer instructions to implement the auto part surface plating defect detection method described herein.

In some embodiments, the processing device 130 may be a single server or a group of servers. The server farm may be centralized or distributed. In some embodiments, the processing device 130 may be local or remote. For example, the processing device 130 may access information and/or data from the image capture apparatus 110, the storage device 120, and/or the terminal device 140 via the network 150. As another example, the processing device 130 may be directly connected to the image capture apparatus 110, the storage device 120, and/or the terminal device 140 to access information and/or data. In some embodiments, the processing device 130 may be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof.

Terminal device 140 may receive, transmit, and/or display data. The received data may include data collected by the image collecting device 110, data stored in the storage device 120, defective pixel points corresponding to the electroplating processing defect in the plating drawing of the to-be-detected automobile part obtained by processing of the processing device 130, and the like. For example, the data received and/or displayed by the terminal device 140 may include a surface image acquired by the image acquisition device 110, a defective pixel corresponding to a plating processing defect in the plating drawing of the to-be-detected automobile part determined by the processing device 130 based on the surface image, and/or a plating qualification rate corresponding to the to-be-detected automobile part. The transmitted data may include input data and instructions from a user (e.g., a quality inspector or an associated worker), etc. For example, the terminal device 140 may send an operation instruction input by a user to the image capturing apparatus 110 through the network 150, so as to control the image capturing apparatus 110 to perform corresponding image capturing. For another example, the terminal device 140 may transmit the data processing instruction input by the user to the processing device 130 through the network 150.

In some embodiments, terminal device 140 may include a mobile device 141, a tablet computer 142, a laptop computer 143, or the like, or any combination thereof. For example, mobile device 141 may include a mobile telephone, a Personal Digital Assistant (PDA), a dedicated mobile terminal, or the like, or any combination thereof. In some embodiments, terminal device 140 may include input devices (e.g., keyboard, touch screen), output devices (e.g., display, speaker), etc. In some embodiments, the processing device 130 may be part of the terminal device 140.

It should be noted that the above description about the application scenario 100 is only for illustration and description, and does not limit the application scope of the present specification. Various modifications and changes to the application scenario 100 may be made by those skilled in the art under the guidance of the present specification. However, such modifications and variations are still within the scope of the present description. For example, the image capture device 110 may include more or fewer functional components.

FIG. 2 is a block diagram of an automotive part surface plating defect detection system according to some embodiments of the present disclosure. In some embodiments, the plating defect detection system 200 for an automotive part shown in fig. 2 may be applied to the application scenario 100 shown in fig. 1 in a software and/or hardware manner, for example, may be configured in a software and/or hardware manner to the processing device 130 and/or the terminal device 140, so as to process the surface image acquired by the image acquisition device 110, and determine, based on the surface image, a defective pixel corresponding to a plating defect in a plating drawing of the automotive part to be detected and/or a plating qualification rate corresponding to the automotive part to be detected.

Referring to fig. 2, in some embodiments, an auto-part surface plating defect detection system 200 may include an acquisition module 210, a preprocessing module 220, an image partitioning module 230, an image feature determination module 240, a multi-layer saliency determination module 250, a defect probability determination module 260, a distribution anomaly determination module 270, an anomaly index determination module 280, a clustering module 290, and a defective pixel point determination module 2100.

The acquiring module 210 may be configured to acquire a surface image corresponding to the automobile part to be detected.

The preprocessing module 220 may be configured to perform denoising and gray-scale processing on the surface image, so as to obtain a plating image corresponding to the to-be-detected automobile part.

The image dividing module 230 may be configured to uniformly divide the plating image into K image blocks with an area of m×m, and divide each of the image blocks into sub-blocks with a size of n×n.

The image feature determining module 240 may be configured to determine, by using a sub-block hierarchical manner, an image feature corresponding to each of the image blocks, where the image feature includes an inter-block difference degree and an inter-layer difference degree.

The multi-layer saliency determination module 250 may be configured to determine, based on the image features, a multi-layer saliency for each of the image blocks, where the multi-layer saliency is used to characterize a saliency of the image blocks in the plating graph.

The defect probability determining module 260 may be configured to determine a defect probability corresponding to each of the image blocks according to the multi-layer saliency.

The distribution anomaly determination module 270 may be configured to filter, as a defective image block, an image block whose defect probability is greater than a segmentation threshold, and calculate a distribution anomaly corresponding to each sub-block in the defective image block, where the distribution anomaly is used to reflect a distribution difference of pixel point texture features between different sub-blocks in the defective image block.

The abnormality index determination module 280 may be configured to determine an abnormality index for each sub-block in the plating graph based on the distribution anomalies.

The clustering module 290 may be configured to determine a metric distance between different sub-blocks in each of the defective image blocks based on the abnormality index and the distribution abnormality degree, and perform clustering processing on each of the defective image blocks based on the metric distance.

The defective pixel determining module 2100 may be configured to determine defective pixels in the plating graph based on the classification result obtained by the clustering process.

With continued reference to FIG. 2, in some embodiments, the auto part surface plating process defect detection system 200 may further include a defect area determination module 2110 and a plating pass rate determination module 2120. Wherein the defect area determining module 2110 may be configured to determine one or more defect areas in the plating map based on the defect pixel points; the plating pass rate determining module 2120 may be configured to determine a minimum bounding rectangle corresponding to the one or more defect areas, and determine a plating pass rate corresponding to the auto-parts to be detected based on a sum of areas of all the minimum bounding rectangles and a total area corresponding to the plating map.

For more details on the above modules, reference may be made to other positions (e.g. fig. 3-7 and related descriptions) in this specification, and details are not repeated here.

It should be appreciated that the auto part surface plating defect detection system 200 and its modules shown in fig. 2 may be implemented in a variety of ways. For example, in some embodiments, the system and its modules may be implemented in hardware, software, or a combination of software and hardware. Wherein the hardware portion may be implemented using dedicated logic; the software portions may then be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those skilled in the art will appreciate that the methods and systems described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system of the present specification and its modules may be implemented not only with hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but also with software executed by various types of processors, for example, and with a combination of the above hardware circuits and software (e.g., firmware).

It should be noted that the above description of the auto part surface plating defect detection system 200 is provided for illustrative purposes only and is not intended to limit the scope of the present description. It will be appreciated by those skilled in the art from this disclosure that various modules may be combined arbitrarily or constituting a subsystem in connection with other modules without departing from this concept. For example, the acquisition module 210, the preprocessing module 220, the image division module 230, the image feature determination module 240, the multi-layer saliency determination module 250, the defect probability determination module 260, the distribution abnormality determination module 270, the abnormality index determination module 280, the clustering module 290, the defective pixel point determination module 2100, the defective area determination module 2110, and the plating pass rate determination module 2120 described in fig. 2 may be different modules in one system, or may be one module to implement the functions of two or more modules described above. In some embodiments, the foregoing modules may be part of the processing device 130 and/or the terminal device 140.

Fig. 3 is an exemplary flow chart of a method for detecting a surface plating processing defect of an automotive part according to some embodiments of the present disclosure. In some embodiments, method 300 may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (instructions run on a processing device to perform hardware simulation), or the like, or any combination thereof. In some embodiments, one or more operations in flowchart 300 of the auto part surface plating defect detection method shown in fig. 3 may be implemented by processing device 130 and/or terminal device 140 shown in fig. 1. For example, method 300 may be stored in storage device 120 in the form of instructions and invoked and/or executed by processing device 130 and/or terminal device 140. The execution of method 300 is described below using processing device 130 as an example.

Referring to fig. 3, in some embodiments, a method 300 of an auto part surface plating defect detection method may include:

step 310, a surface image corresponding to the automobile part to be detected is obtained. In some embodiments, step 310 may be performed by the acquisition module 210.

In some embodiments, the image capturing device 110 may capture a surface image of the electroplated automobile part to be detected, and store the captured surface image in the storage device 120. The acquiring module 210 may acquire the surface image corresponding to the automobile accessory to be detected from the storage device 120.

It should be noted that, in some embodiments, the same automobile part to be detected may correspond to one or more surface images. In some embodiments, when the same automotive part to be inspected corresponds to multiple surface images, the multiple surface images may be acquired from different perspectives by multiple image acquisition devices 110.

And 320, carrying out denoising treatment and gray scale treatment on the surface image to obtain a plating part diagram corresponding to the automobile part to be detected. In some embodiments, step 320 may be performed by the preprocessing module 220.

In some embodiments, in order to improve the image quality of the surface image corresponding to the automobile accessory to be detected, and eliminate the environmental interference during the electroplating process, the preprocessing module 220 may utilize a bilateral filtering denoising technology to denoise the surface image. The bilateral filtering denoising technology is a known technology, and specific processes thereof are not described in detail in the specification.

Further, the preprocessing module 220 may further perform graying processing on the surface image after the denoising processing, so as to obtain a plating drawing corresponding to the automobile part to be detected. It will be understood that in the present specification, the graying process of the surface image after the denoising process refers to a process of converting an RGB image into a gray image, which can simplify complexity of image analysis and processing, and can also retain main feature information in the image, which is beneficial to subsequent analysis and processing.

Step 330, uniformly dividing the plating image into K image blocks with an area of m×m, and dividing each image block into sub-blocks with a size of n×n. In some embodiments, step 330 may be performed by image partitioning module 230.

In some embodiments, the image dividing module 230 may divide the aforementioned plating image into K image blocks with an area of m×m, and divide each of the image blocks into sub-blocks with a size of n×n, so as to facilitate feature recognition of the plating image in a subsequent process. Wherein K and N may be integers greater than or equal to 2, and M may be an integer greater than or equal to 4.

It should be noted that, in the present specification, the area of the image block may refer to a pixel area, in other words, the image block having the area m×m may refer to the image block including m×m pixels, that is, the image block includes M rows and M columns of pixels, where each row or each column includes M pixels. Similarly, in the present specification, the size of a sub-block may also refer to the pixel size of the sub-block, for example, a sub-block having a size of n×n may refer to the sub-block containing n×n pixels, that is, the sub-block contains N rows and N columns of pixels, where each row or each column contains N pixels. It will be appreciated that in this specification, the aforementioned N value should be less than the M value since the sub-block is part of an image block. In some embodiments, the aforementioned M value may be an integer multiple of the N value.

And step 340, determining the image characteristics corresponding to each image block in a sub-block layering mode. In some embodiments, step 340 may be performed by image feature determination module 240.

Poor treatment before plating of the automobile parts or abnormal proportion of components in the electroplating solution and other factors can cause poor combination of plating layers of the automobile parts and uneven electrodeposition degree, so that phenomena such as fish scales and the like appear, and the uneven electrodeposition degree areas are easily influenced by scouring and the like in the subsequent processing process to form irregular crawling lines or earthworm lines.

In some embodiments, image Pyramid (Image Pyramid) may be used to obtain Image features, but it is considered that when the histogram statistical area of the Pyramid high level is too small, the robustness of the Image description may be poor, and when the histogram statistical area is too large, fine features are easily ignored. Therefore, in some embodiments of the present disclosure, in order to avoid the above problem, after dividing the plating image into K image blocks with an area of m×m, and dividing each image block into sub-blocks with a size of n×n, determining the image feature corresponding to each image block by using a sub-block layering manner.

In this specification, determining the image feature corresponding to each image block by using a sub-block layering manner may be understood as dividing a plurality of sub-blocks included in each image block into a plurality of (e.g. two or more) layers according to a certain rule, and then determining the image feature of the image block based on a feature relationship between the plurality of layers.

Fig. 4 is a schematic diagram of the layering principle of an exemplary sub-block layering method shown in accordance with some embodiments of the present description. Referring to fig. 4, in some embodiments, for each target image block 400, each target image block may be divided into w×w (w×n=m) sub-blocks with size n×n, then w×w sub-blocks with size n×n are used as the basic sub-block layer 410 (gray layer in fig. 4) of the target image block 400, and then the sub-block layers corresponding to the target image block are constructed in such a manner that the edge intersection points 450 of four adjacent sub-blocks in the previous layer sub-block are used as the center points of the next layer sub-blocks. For example, with the edge intersections 450 of four adjacent sub-blocks in the base sub-block layer 410 as the center points, (w-1) intersections may be obtained, after all the intersections are obtained, a sub-block with a size of n×n is constructed with each intersection as the center point to obtain the second sub-block layer 420 (white layer in fig. 4), and similarly, with the edge intersections 450 of four adjacent sub-blocks in the second sub-block layer 420 as the center points, (w-2) intersections may be obtained, after all the intersections are obtained, a sub-block with a size of n×n is constructed with each intersection as the center point to obtain the third sub-block layer 430 (black layer in fig. 4). Similarly, more sub-block layers can be constructed.

It should be noted that, in some embodiments, the subblocks included in each subblock layer may have the same or different sizes, specifically, the sizes of the subblocks in the subblock layers other than the base subblock layer 410 may be n×n, or may be other sizes (for example, greater than n×n or less than n×n).

Further, the image feature determining module 240 may determine the inter-block difference degree between different sub-blocks in each sub-block layer and the inter-layer difference degree between two adjacent sub-block layers, and then use the inter-block difference degree and the inter-layer difference degree as the image feature corresponding to the target image block 400.

It will be appreciated that if there are no defects in the target image block 400, the image characteristics of the pixels in each sub-block layer and in adjacent different sub-block layers may be considered to be nearly uniform, e.g., the gray values are very close in size. In contrast, if the target image block 400 has the defects such as the earthworm line, there will be a certain difference in the image characteristics of the pixels in the sub-block, for example, the variance of the gray value distribution of the pixels is large, and different image textures in different regions will be different.

Based on this, in some embodiments, the image feature determining module 240 may determine an inter-block difference degree between different sub-blocks in each sub-block layer and an inter-layer difference degree between two adjacent sub-block layers, and then reflect the image feature corresponding to the target image block 400 through the inter-block difference degree and the inter-layer difference degree.

Specifically, in some embodiments, to determine the inter-block difference and the inter-layer difference, a LBP (Local Binary Patterns) value corresponding to each pixel in the target image block 400 may be obtained, and then a local histogram corresponding to each sub-block in the multiple sub-block layers may be constructed based on the LBP value and the number of pixels corresponding to each LBP value.

Fig. 5 is an exemplary local histogram shown in accordance with some embodiments of the present description. Referring to FIG. 5, wherein the abscissa represents LBP values corresponding to pixel pointsThe ordinate indicates the number of pixels corresponding to the LBP value.

、

The maximum value and the minimum value of LBP values in all pixel points contained in the sub-block are respectively N2 and N1

And

corresponding pixel number.

Further, the image feature determination module 240 may determine the inter-block variability between different sub-blocks in each sub-block layer based on the pasteurization distance (Bhattacharyya Distance) between the local histograms corresponding to the different sub-blocks in each sub-block layer and the average pasteurization distance between the sub-blocks corresponding to the different sub-blocks in the next sub-block layer. Wherein the sub-block corresponding to the different sub-block in the next sub-block layer refers to the sub-block corresponding to the sub-block center point in the next sub-block layer, for example, the center point of the sub-block a is the sub-block

、

、

、

Intersection of edge lines

、

、

、

Namely, the sub-block corresponding to the sub-block a in the next sub-block layer.

In some embodiments, the aforementioned inter-block variability may be expressed as follows (see equation 1):

，（1）

wherein,,

is the inter-block difference between sub-blocks a, b in the same sub-block layer,

、

the i-th sub-block corresponding to sub-block a and sub-block b in the next sub-block layer, respectively, and n is the number of sub-blocks corresponding to sub-block a and sub-block b in the next sub-block layer (the size of n in this specification may be set to 4 according to an empirical value).

、

、

The sub-blocks a, b respectively,

、

The corresponding local histogram is then used to determine,

is a local histogram

、

The distance between the two pairs of rollers is the pasteurization distance,

is a local histogram

、

The pasteurization distance between them. In the present description of the invention,

the smaller the value of (c) the more similar the image features between sub-blocks a, b.

Further, the image feature determining module 240 may further obtain a first LBP sequence and a second LBP sequence corresponding to two adjacent sub-block layers respectively, and then determine an inter-layer difference degree between the two adjacent sub-block layers based on a DTW (Dynamic Time Warping) distance between the first LBP sequence and the second LBP sequence. The first LBP sequence and the second LBP sequence are sequences formed by arranging LBP values corresponding to pixel points included in each sub-block in the sub-block layer according to position information of the pixel points.

In some embodiments, the inter-layer variability may be expressed as follows (see equation 2):

，（2）

wherein,,

indicating the inter-layer difference between the j-th sub-block layer and the j + 1-th sub-block layer in the picture block,

、

respectively the j sub-block layer and the j+1th sub-block layerThe LBP values corresponding to the pixels included in each sub-block in the sub-block layer are arranged according to the position information of the pixels to form a sequence (i.e. the first LBP sequence and the second LBP sequence),

for the first LBP sequence

And a second LBP sequence

DTW distance between. In the present description of the invention,

the smaller the value of (c) is, the smaller the difference of image characteristics between different sub-block layers in the image block is, and the greater the similarity between pixel points is. In some embodiments, an average value of inter-layer difference degrees between two adjacent sub-block layers in a plurality of sub-blocks corresponding to an image block may be used as the inter-layer difference degree corresponding to the target image block.

And step 350, determining the multi-layer saliency corresponding to each image block based on the image characteristics. In some embodiments, step 350 may be performed by multi-layer saliency determination module 250.

After determining the inter-block difference degree and the inter-layer difference degree, the multi-layer saliency determination module 250 may determine the multi-layer saliency corresponding to each image block based on the inter-block difference degree and the inter-layer difference degree.

Specifically, in some embodiments, the multi-layer saliency corresponding to an image block may be represented as follows (see equation 3):

，（3）

wherein,,

for the multi-layer saliency corresponding to image block a,

for the degree of difference between the aforementioned blocks,

for the degree of difference between the layers as described above,

to be the average value of the inter-block difference degree corresponding to all the image blocks in the plating part graph,

the average value of the interlayer difference degree corresponding to all the image blocks in the plating part graph.

In this specification, the multi-layer saliency may reflect the saliency of each image patch in the plating map. In particular, the more similar the image features between sub-blocks a, b,

The smaller the value of (c), the more similar the image information in the adjacent spaces of the sub-blocks, the smaller the difference between adjacent sub-blocks,

the smaller the value of (i.e.)

The smaller the value of (a), the more similar the image features between the sub-blocks a, b; the smaller the difference of image characteristics between different layers of the image block a, the smaller the difference of pixel points between adjacent layers in the sub-block layering result,

the smaller the value of (2); the greater the difference between the sub-blocks in image block a and the remaining image blocks in the plating drawing,

the larger the value of (c) is,

the larger the value of (2)

The larger the value of (c), the more significant image block a is in the plating.

In the specification, as the multi-layer saliency considers the difference degree between different layers of the image block and between different sub-blocks of the same layer, the similarity between the sub-blocks and the spatial relationship of the sub-blocks can be used for representing the fine features in the image block at any spatial position in the plating part graph, so that the detection precision of the saliency degree of the image block can be improved.

And step 360, determining the defect probability corresponding to each image block according to the multi-layer saliency. In some embodiments, step 360 may be performed by defect probability determination module 260.

Since the multi-layer saliency of an image block containing a defect may be significantly different from that of a normal image block, in some embodiments, the probability of a defect for each image block may be determined based on the multi-layer saliency to which the image block corresponds. The defect probability may reflect a probability of containing a defect in each image block.

In some embodiments, the defect probability may be calculated based on the following formula (see formula 4):

，（4）

wherein,,

for the probability of a defect corresponding to image block a,

and

the maximum value and the minimum value of the multi-layer saliency corresponding to all image blocks in the plating part diagram corresponding to the automobile part to be detected are respectively.

In order to adjust the parameter of the human body,

is to prevent

And (3) with

The equal denominator is 0, which, in some embodiments,

the magnitude of (c) may be set to 0.001 or other values based on empirical values. In the case of the formula 4 of the present invention,

the larger the value of (c) the greater the probability that a defect exists in image block a.

Specifically, in some embodiments, for each target image block A, a multi-layer saliency [ ] corresponding to the target image block A may be calculated

) Minimum value of multi-layer significance corresponding to plating part graph

) And the maximum value of the multi-layer saliency corresponding to the plating part graph

) Multi-layer saliency corresponding to target image block a

) And then based on the ratio of the first difference to the second difference

) Determining defect probability corresponding to target image block A

. By calculating the above formula 4, the defect probabilities corresponding to the K image blocks included in the plating drawing can be obtained respectively.

And 370, screening the image blocks with the defect probability larger than the segmentation threshold as defect image blocks, and calculating the distribution anomaly degree corresponding to each sub-block in the defect image blocks. In some embodiments, step 370 may be performed by the distribution anomaly determination module 270.

In some embodiments, a segmentation threshold of the defect probabilities corresponding to K image blocks included in the plating image may be determined by an Otsu (maximum inter-class variance) algorithm, and then an image block having a defect probability greater than the segmentation threshold is considered as a defective image block in which a defect may exist. The Otsu algorithm is a well-known technology, and specific processes thereof are not described in detail in the present specification.

In some embodiments, for each defective image block that may have a defect, the distribution anomaly determination module 270 may obtain HOG (Histogram of Oriented Gradients) feature descriptors corresponding to each pixel point in the defective image block, then sequentially calculate cosine similarities between each pixel point in each sub-block in the base sub-block layer and HOG feature descriptors corresponding to central pixel points of the sub-blocks, and use values corresponding to the cosine similarities as feature values of each pixel point to obtain a distribution map corresponding to each sub-block in the defective image block.

Fig. 6 is an exemplary distribution diagram according to some embodiments of the present description. Referring to fig. 6, 610 may represent one of the sub-blocks in the defective image block, and 620 may represent a distribution map corresponding to the sub-block 610.

Further, for each target sub-block of the defective image block, the distribution anomaly determination module 270 may determine a distribution association between the target sub-block and other sub-blocks in the base sub-block layer of the defective image block based on the aforementioned distribution map.

In some embodiments, the distribution association may be expressed as follows (see equation 5):

，（5）

Wherein,,

is sub-block a and sub-block bThe distribution association degree between the sub-blocks b is the b sub-block in the same sub-block layer as the sub-block a,

、

the profiles corresponding to sub-block a and sub-block b, respectively, N being the scale of the profile (i.e. the number of rows and columns of eigenvalues contained in the profile),

is the distribution diagram corresponding to the sub-blocks a and b

、

Middle (f)

Pearson correlation coefficients between sequences of column eigenvalues,

is the distribution diagram corresponding to the sub-blocks a and b

、

The h-th line eigenvalue of (c) constitutes the pearson correlation coefficient between the sequences.

Specifically, in the embodiment of the present disclosure, the distribution anomaly determination module 270 may calculate a first pearson correlation coefficient between the target sub-block and a sequence of feature values located in the same column in the distribution map corresponding to other sub-blocks in the same sub-block layer

) And a second pearson correlation coefficient between sequences of eigenvalues located in the same row in the distribution diagram corresponding to other subblocks in the same subblock layer

) Then based on the average value of the first pearson correlation coefficient

) Mean value of the second pearson correlation coefficient

) Obtaining the distribution association degree between the target sub-block and other sub-blocks in the basic sub-block layer

。

Further, the distribution anomaly determination module 270 may also calculate EDR edit distances between the target sub-block and gaussian distribution curves corresponding to other sub-blocks in the base sub-block layer of the defective image block (Edit Distance on Real sequence). Wherein the Gaussian distribution curve can be based on the average value of the characteristic values of all pixel points in the sub-block

Sum of variances

Obtained.

Finally, the distribution anomaly determination module 270 may determine the distribution anomaly corresponding to the target sub-block based on the number of sub-blocks in the base sub-block layer, the distribution association degree, and the EDR editing distance.

In some embodiments, the distribution anomaly corresponding to the target sub-block may be expressed as follows (see equation 6):

，（6）

wherein,,

representing the distribution abnormality of the target sub-block a, m is the number of sub-blocks of the base sub-block layer in the defective image block,

、

the gaussian distribution curves of a target sub-block a and a sub-block b, respectively, which are different sub-blocks located in the same sub-block layer (base sub-block layer) as the target sub-block a,

is a Gaussian distribution curve

、

EDR edit distance between.

In this specification, the distribution abnormality may reflect a distribution difference of pixel texture characteristics between different sub-blocks in the defective image block. The greater the difference between the constituent sequences of each row element or each column element in the profile,

、

The smaller the value of (c) is,

the smaller the value of (2); the larger the distribution difference between the local textures of the pixel points in the sub-blocks is, the lower the corresponding Gaussian distribution curve similarity is,

the smaller the value of (i.e.)

The smaller the value of (c), the smaller the degree of association between the target sub-block a and other sub-blocks in the same sub-block layer. By the above formula 6, the distribution outliers corresponding to all sub-blocks in the defective image block can be obtained.

In the embodiment of the present disclosure, since the distribution abnormality degree considers the local texture distribution of the pixel points in the sub-block and the texture characteristics in different directions, it is possible to obtain a precise evaluation index for the pixel points in the defect area with any size, thereby improving the subsequent classification accuracy.

And step 380, determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree. In some embodiments, step 380 may be performed by the anomaly index determination module 280.

For a defective sub-block, there is a large difference from a normal sub-block, regardless of which type of defect is contained in the sub-block, and all defective sub-blocks are more noticeable in the plated drawing. Based on this, in some embodiments, the abnormality index of each sub-block in the plating graph may be determined based on the aforementioned distribution abnormality.

In some embodiments, the abnormality index may be expressed as follows (see equation 7):

，（7）

wherein,,

is the average value of the distribution outliers corresponding to all the sub-blocks in the defective image block a where the target sub-block a is located,

is the average value of the distribution abnormal degree corresponding to all sub-blocks in the defect image block in the plating part diagram.

The larger the value of (c) is, the greater the degree of abnormality in the plating drawing is indicated for the target sub-block a.

Step 390, determining a metric distance between different sub-blocks in each of the defective image blocks based on the anomaly index and the distribution anomaly degree, and performing clustering processing on each of the defective image blocks based on the metric distance. In some embodiments, step 390 may be performed by the clustering module 290.

After the foregoing abnormality distribution degree and abnormality index are determined, the distance between the different sub-blocks in each defective image block may be determined based on the abnormality index and the distribution abnormality degree. In some embodiments, the distance may be a euclidean distance. Illustratively, in some embodiments, the distance between different sub-blocks p and q in a defective image block may be expressed as follows (see equation 8):

，（8）

wherein,,

for the distance between the different sub-blocks p and q in the defective image block,

、

The abnormality indexes corresponding to the different sub-blocks p and q in the defective image block respectively,

、

the distribution outliers corresponding to the sub-blocks p and q are respectively.

Further, the clustering module 290 may compare the distance

As a measure distance D in the k-means clustering algorithm, the clustering process is performed on each sub-block in the defective image block. In some embodiments, the k value in the k-means clustering algorithm may be set to 2 according to an empirical value, in other words, the number of clusters may be 2 during the clustering process.

It should be noted that the above description of the value of k is merely an exemplary illustration, and in some other embodiments, k may be set to other values. It should be further noted that, in the present specification, the above euclidean distance, EDR edit distance, DTW distance, papanicolaou distance, and the like are parameters for characterizing similarity, and in some embodiments, the distances used in different steps may be replaced with each other, or may be calculated using other distance algorithms.

Step 3100, determining defective pixel points in the plating graph based on the classification result obtained by the clustering process. In some embodiments, step 3100 may be performed by defective pixel point determination module 2100.

After the clustering module 290 performs the clustering process on each sub-block in the defective image block, the pixel points representing the defect in the defective image block may be divided into different clusters from the normal pixel points. The cluster where the maximum value of the abnormal index is located is the cluster corresponding to the defective pixel point. Based on this, the defective pixel point determination module 2100 may determine the pixel point in the cluster where the abnormality index maximum value is located as a defective pixel point in the plating map.

Fig. 7 is an exemplary flowchart of a method for detecting defects in an automotive part surface plating process according to other embodiments of the present disclosure. Referring to fig. 7, in some embodiments, the method 300 of the foregoing method for detecting an automotive part surface plating processing defect may further include:

at step 3110, one or more defective areas in the plating map are determined based on the defective pixel points. In some embodiments, step 3110 may be performed by the defect region determination module 2110.

After determining the defective pixel points in each defective image block through the foregoing steps, the defective area determining module 2110 may obtain all the defective pixel points in all the defective image blocks, and determine one or more defective areas based on the relative positional relationship of each defective pixel point in the plating map. For example, in some embodiments, the defect area determining module 2110 may respectively take an area formed by defective pixels in each defective image block as one defective area. In some embodiments, the defect region determining module 2110 may merge the defect pixels located at adjacent positions in adjacent defect image blocks, thereby merging the defect pixels included in two or more defect image blocks into one defect region.

Step 3120, determining minimum bounding rectangles corresponding to the one or more defect regions, respectively. And 3130, determining the electroplating qualification rate corresponding to the automobile part to be detected based on the sum of all the areas of the minimum circumscribed rectangle and the total area corresponding to the plating part graph. In some embodiments,

steps

3120 and 3130 may be performed by the plating pass rate determination module 2120.

After determining one or more defective areas in the plating map, the plating yield determination module 2120 may determine a minimum bounding rectangle corresponding to each defective area. The minimum bounding rectangle refers to a rectangle that can contain the smallest area (or smallest perimeter) of all defective pixels in the same defective area.

Further, the plating pass rate determining module 2120 may further determine a plating pass rate corresponding to the auto-parts to be detected based on a sum of the areas of all the minimum circumscribed rectangles and a total area corresponding to the plating drawing. In some embodiments, the electroplating qualification rate corresponding to the auto-parts to be tested may be expressed as follows (see equation 9):

，（9）

wherein,,

is the total area corresponding to the L-th automobile part to be detected,

Is the area of the smallest bounding rectangle corresponding to the z-th defect region,

is the number of defective areas in the plating pattern corresponding to the L-th automobile part to be detected.

In some embodiments, the electroplating qualification rate corresponding to the automobile part to be detected can be calculated through surface images acquired from a plurality of different visual angles. When the electroplating qualification rate is onWhen the calculation is performed on too many surface images, the total area in equation 9

May represent the sum of the areas corresponding to the plurality of surface images and, similarly,

may refer to the sum of the number of defective areas in the multiple plating patterns.

Through the above formula 9, the electroplating qualification rate corresponding to each automobile part to be detected can be calculated. In some embodiments, whether the electroplating processing quality of the automobile part to be detected is qualified or not can be judged according to a preset qualification rate threshold value. Illustratively, in some embodiments, the yield threshold may be set to 90%, 95%, 92%, etc., or other values. In some embodiments, for automotive parts that do not meet the plating yield, subsequent processing may be performed, such as removing the failed plating, performing secondary plating, polishing repair, adjusting plating process parameters, and the like.

In summary, the possible benefits of the embodiments of the present disclosure include, but are not limited to: (1) In the method and the system for detecting the surface electroplating processing defects of the automobile parts, provided by some embodiments of the present disclosure, the detection efficiency and the instantaneity of the defect detection can be improved by detecting the surface electroplating processing defects of the automobile parts by using an image recognition mode; (2) In the method and the system for detecting the surface electroplating processing defects of the automobile accessory provided by some embodiments of the present disclosure, by determining the image feature corresponding to each image block in the plating image corresponding to the automobile accessory to be detected in a sub-block layering manner, and then determining the multi-layer saliency corresponding to each image block based on the image feature, the similarity between sub-blocks and the spatial relationship of the sub-blocks can be fully utilized to represent the fine features in the image blocks at any spatial positions in the plating image, so as to improve the detection precision of the saliency degree of the image blocks; (3) In the method and the system for detecting the surface electroplating processing defects of the automobile accessory provided by some embodiments of the present disclosure, the distribution anomaly degree is constructed according to the texture distribution characteristics of the pixel points in different sub-blocks in the image block, so that the local texture distribution of the pixel points in the sub-blocks and the texture characteristics in different directions can be fully considered, and therefore, accurate evaluation indexes can be obtained for the pixel points in the defect area with any size, and the accuracy of the subsequent clustering processing can be improved.

It should be noted that, the advantages that may be generated by different embodiments may be different, and in different embodiments, the advantages that may be generated may be any one or a combination of several of the above, or any other possible advantages that may be obtained.

While the basic concepts have been described above, it will be apparent to those skilled in the art that the foregoing detailed disclosure is by way of example only and is not intended to be limiting. Although not explicitly described herein, various modifications, improvements, and adaptations to the present disclosure may occur to one skilled in the art. Such modifications, improvements, and modifications are intended to be suggested within this specification, and therefore, such modifications, improvements, and modifications are intended to be included within the spirit and scope of the exemplary embodiments of the present invention.

Meanwhile, the specification uses specific words to describe the embodiments of the specification. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the present description. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the present description may be combined as suitable.

Furthermore, those skilled in the art will appreciate that the various aspects of the specification can be illustrated and described in terms of several patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the present description may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the specification may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.

The computer storage medium may contain a propagated data signal with the computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take on a variety of forms, including electro-magnetic, optical, etc., or any suitable combination thereof. A computer storage medium may be any computer readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code located on a computer storage medium may be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or the like, or a combination of any of the foregoing.

The computer program code necessary for operation of portions of the present description may be written in any one or more programming languages, including an object oriented programming language such as Java, scala, smalltalk, eiffel, JADE, emerald, C ++, c#, vb net, python and the like, a conventional programming language such as C language, visual Basic, fortran2003, perl, COBOL2002, PHP, ABAP, a dynamic programming language such as Python, ruby and Groovy, or other programming languages and the like. The program code may execute entirely on the user's computer or as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or processing device. In the latter scenario, the remote computer may be connected to the user's computer through any form of network, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet), or the use of services such as software as a service (SaaS) in a cloud computing environment.

Furthermore, the order in which the elements and sequences are processed, the use of numerical letters, or other designations in the description are not intended to limit the order in which the processes and methods of the description are performed unless explicitly recited in the claims. While certain presently useful inventive embodiments have been discussed in the foregoing disclosure, by way of various examples, it is to be understood that such details are merely illustrative and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements included within the spirit and scope of the embodiments of the present disclosure. For example, while the system components described above may be implemented by hardware devices, they may also be implemented solely by software solutions, such as installing the described system on an existing processing device or mobile device.

Likewise, it should be noted that in order to simplify the presentation disclosed in this specification and thereby aid in understanding one or more inventive embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof. This method of disclosure, however, is not intended to imply that more features than are presented in the claims are required for the present description. Indeed, less than all of the features of a single embodiment disclosed above.

In some embodiments, numbers describing the components, number of attributes are used, it being understood that such numbers being used in the description of embodiments are modified in some examples by the modifier "about," approximately, "or" substantially. Unless otherwise indicated, "about," "approximately," or "substantially" indicate that the number allows for a 20% variation. Accordingly, in some embodiments, numerical parameters set forth in the specification and claims are approximations that may vary depending upon the desired properties sought to be obtained by the individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and employ a method for preserving the general number of digits. Although the numerical ranges and parameters set forth herein are approximations that may be employed in some embodiments to confirm the breadth of the range, in particular embodiments, the setting of such numerical values is as precise as possible.

Each patent, patent application publication, and other material, such as articles, books, specifications, publications, documents, etc., referred to in this specification is incorporated herein by reference in its entirety. Except for application history documents that are inconsistent or conflicting with the content of this specification, documents that are currently or later attached to this specification in which the broadest scope of the claims to this specification is limited are also. It is noted that, if the description, definition, and/or use of a term in an attached material in this specification does not conform to or conflict with what is described in this specification, the description, definition, and/or use of the term in this specification controls.

Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments of this specification. Other variations are possible within the scope of this description. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present specification may be considered as consistent with the teachings of the present specification. Accordingly, the embodiments of the present specification are not limited to only the embodiments explicitly described and depicted in the present specification.

Claims

1. The method for detecting the electroplating processing defects on the surface of the automobile part is characterized by comprising the following steps of:

Acquiring a surface image corresponding to an automobile part to be detected;

denoising and gray-scale processing are carried out on the surface image, and a plating piece diagram corresponding to the automobile part to be detected is obtained;

uniformly dividing the plating part map into K image blocks with the area of M, and dividing each image block into subblocks with the size of N;

determining image characteristics corresponding to each image block in a sub-block layering mode, wherein the image characteristics comprise inter-block difference degrees and inter-layer difference degrees;

determining a plurality of layers of saliency corresponding to each image block based on the image features, wherein the plurality of layers of saliency is used for representing the saliency of the image block in the plating part graph;

determining the defect probability corresponding to each image block according to the multi-layer saliency;

screening the image blocks with the defect probability larger than the segmentation threshold as defect image blocks, and calculating distribution anomaly corresponding to each sub-block in the defect image blocks, wherein the distribution anomaly is used for reflecting the distribution difference of pixel point texture characteristics among different sub-blocks in the defect image blocks;

determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree;

Determining a measurement distance between different sub-blocks in each defective image block based on the abnormality index and the distribution abnormality degree, and clustering each defective image block based on the measurement distance;

and determining defective pixel points in the plating part graph based on the classification result obtained by the clustering process.

2. The method of claim 1, wherein determining the image feature corresponding to each of the image blocks using sub-block layering comprises:

for each of the image blocks in question,

the subblocks with the size of N, which are obtained by dividing the image blocks, are used as basic subblock layers of the image blocks;

constructing a plurality of sub-block layers corresponding to the image block in a mode that edge intersection points of four adjacent sub-blocks in the sub-block of the previous layer serve as center points of the sub-block of the next layer, wherein the sub-blocks contained in each sub-block layer have the same size;

determining inter-block difference degree between different sub-blocks in each sub-block layer and inter-layer difference degree between two adjacent sub-block layers;

and taking the inter-block difference degree and the inter-layer difference degree as image features corresponding to the image blocks.

3. The method of claim 2, wherein the determining the inter-block difference between different sub-blocks in each of the sub-block layers and the inter-layer difference between adjacent two sub-block layers comprises:

obtaining an LBP value corresponding to each pixel point in the image block;

constructing a local histogram corresponding to each sub-block in the plurality of sub-block layers based on the LBP value and the number of pixel points corresponding to the LBP value;

determining inter-block difference degree between different sub-blocks in each sub-block layer based on the pasteurization distance between local histograms corresponding to different sub-blocks in each sub-block layer and the average pasteurization distance between sub-blocks corresponding to different sub-blocks in the next sub-block layer;

acquiring a first LBP sequence and a second LBP sequence respectively corresponding to the two adjacent sub-block layers;

and determining an interlayer difference degree between the two adjacent sub-block layers based on a DTW distance between the first LBP sequence and the second LBP sequence.

4. The method of claim 3, wherein said determining a multi-layer saliency for each of said image blocks based on said image features comprises:

For each of the target image blocks,

and determining the multi-layer saliency corresponding to the target image block based on the inter-block difference degree and the inter-layer difference degree corresponding to the target image block, and the inter-block difference degree mean and the inter-layer difference degree mean corresponding to the plating image.

5. The method of claim 1, wherein said determining a probability of a defect for each of said image blocks based on said multi-layer saliency comprises:

for each of the target image blocks,

calculating a first difference value between the multilayer saliency corresponding to the target image block and the minimum value of the multilayer saliency corresponding to the plating part graph, and a second difference value between the maximum value of the multilayer saliency corresponding to the plating part graph and the multilayer saliency corresponding to the target image block;

and determining the defect probability corresponding to the target image block based on the ratio of the first difference value to the second difference value.

6. The method of claim 2, wherein calculating the distribution anomaly for each sub-block in the defective image block comprises:

acquiring HOG feature descriptors corresponding to each pixel point in the defect image block;

Sequentially calculating cosine similarity between each pixel point in each sub-block in the basic sub-block layer and the HOG feature descriptors corresponding to the central pixel points, and taking the cosine similarity as a feature value of each pixel point;

obtaining a distribution diagram corresponding to each sub-block in the defect image block based on the characteristic values;

for each of the target sub-blocks,

determining a distribution association between the target sub-block and other sub-blocks in the base sub-block layer based on the distribution map;

calculating EDR editing distances between the target sub-block and Gaussian distribution curves corresponding to other sub-blocks in the basic sub-block layer;

and determining the distribution anomaly degree corresponding to the target sub-block based on the number of sub-blocks in the basic sub-block layer, the distribution association degree and the EDR editing distance.

7. The method of claim 6, wherein the determining a distribution association between the target sub-block and other sub-blocks in the base sub-block layer based on the profile comprises:

calculating a first pearson correlation coefficient between sequences of characteristic values located in the same column in distribution diagrams corresponding to the target sub-block and the other sub-blocks, and a second pearson correlation coefficient between sequences of characteristic values located in the same row in distribution diagrams corresponding to the target sub-block and the other sub-blocks;

And obtaining the distribution association degree between the target subblock and other subblocks in the basic subblock layer based on the average value of the first pearson correlation coefficient and the average value of the second pearson correlation coefficient.

8. The method of claim 6, wherein said determining an abnormality index for each sub-block in the plating map based on the distribution abnormalities comprises:

for each of the target sub-blocks,

calculating first distribution anomaly averages corresponding to all sub-blocks in a defect image block where the target sub-block is located and second distribution anomaly averages corresponding to all sub-blocks contained in all the defect image block;

and determining an abnormality index of the target sub-block in the plating part graph based on a first difference value between the distribution abnormality corresponding to the target sub-block and the first distribution abnormality mean value and a second difference value between the first distribution abnormality mean value and the second distribution abnormality mean value.

9. The method of any one of claims 1-8, wherein the method further comprises:

determining one or more defect areas in the plating graph based on the defect pixel points;

Determining minimum circumscribed rectangles corresponding to the one or more defect areas respectively;

and determining the electroplating qualification rate corresponding to the automobile part to be detected based on the sum of the areas of all the minimum circumscribed rectangles and the total area corresponding to the plating part graph.

10. An automotive part surface electroplating processing defect detection system, which is characterized by comprising:

the acquisition module is used for acquiring a surface image corresponding to the automobile part to be detected;

the preprocessing module is used for carrying out denoising treatment and gray scale treatment on the surface image to obtain a plating part diagram corresponding to the automobile part to be detected;

the image dividing module is used for uniformly dividing the plating part image into K image blocks with the area of M, and dividing each image block into subblocks with the size of N;

the image characteristic determining module is used for determining image characteristics corresponding to each image block in a sub-block layering mode, wherein the image characteristics comprise inter-block difference degree and inter-layer difference degree;

the multi-layer saliency determination module is used for determining multi-layer saliency corresponding to each image block based on the image features, wherein the multi-layer saliency is used for representing the saliency of the image block in the plating part graph;

The defect probability determining module is used for determining the defect probability corresponding to each image block according to the multi-layer saliency;

the distribution anomaly degree determining module is used for screening the image blocks with the defect probability larger than the segmentation threshold value as defect image blocks and calculating the distribution anomaly degree corresponding to each sub-block in the defect image blocks, wherein the distribution anomaly degree is used for reflecting the distribution difference of pixel point texture characteristics among different sub-blocks in the defect image blocks;

the abnormality index determining module is used for determining an abnormality index of each sub-block in the plating part graph according to the distribution abnormality degree;

the clustering module is used for determining the measurement distance between different sub-blocks in each defective image block based on the abnormality index and the distribution abnormality degree, and carrying out clustering processing on each defective image block based on the measurement distance;

and the defective pixel point determining module is used for determining defective pixel points in the plating part graph based on the classification result obtained by the clustering process.