CN109753979B - Fault detection method and device based on image characteristics - Google Patents
Fault detection method and device based on image characteristics Download PDFInfo
- Publication number
- CN109753979B CN109753979B CN201711061292.XA CN201711061292A CN109753979B CN 109753979 B CN109753979 B CN 109753979B CN 201711061292 A CN201711061292 A CN 201711061292A CN 109753979 B CN109753979 B CN 109753979B
- Authority
- CN
- China
- Prior art keywords
- image
- equipment
- parameters
- product
- equipment parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a fault detection method and device based on image characteristics. Wherein, the method comprises the following steps: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; and analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type. The invention solves the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is lower.
Description
Technical Field
The invention relates to the field of fault detection, in particular to a fault detection method and device based on image characteristics.
Background
In the electronic industry, the demand for silicon wafers is mainly expressed in the semiconductor industries such as solar photovoltaic power generation and integrated circuits, and the silicon wafer cutting is an upstream key technology of silicon wafer production, which is a very important link in the semiconductor industry chain of solar photovoltaic power generation and integrated circuits, wherein the silicon wafer cutting process refers to a process of cutting a polycrystalline silicon or monocrystalline silicon ingot into silicon wafers by operating a slicing machine (as shown in fig. 1, the slicing machine comprises an electric control cabinet, a winding chamber, a mortar system, a cutting area, a crystal bar, a sand discharge groove, a base frame, a pad foot and the like) for several hours.
Because the slicing process takes a long time, the slicing operation is complex, and the silicon wafer cutting equipment cannot meet the precision requirement of the cutting process, the cut silicon wafer is often poor in quality (for example, the silicon wafer is uneven in thickness and line marks), so that the economic value of the silicon wafer is reduced, and even the silicon wafer becomes a waste wafer, and therefore, the phenomenon of poor quality of the cut silicon wafer needs to be reduced, so that the economic value of the silicon wafer is improved.
The existing solutions are as follows: one is to take precautionary measures in advance to avoid the condition of poor silicon wafer quality; and the other method is to find out the reason causing the poor quality of the silicon wafer in time after the poor quality of the silicon wafer is achieved, namely, fault diagnosis of a silicon wafer cutting system and equipment is carried out, so that the adverse effect on the silicon wafer in the next production batch is avoided.
However, the existing fault diagnosis method mainly relies on manual experience to diagnose faults through information such as silicon wafer images and machine alarms based on expert experience of process technicians, the diagnosis process is time-consuming and labor-consuming, the experience requirements of the process technicians are high, and the accuracy and the sustainability of diagnosis results are difficult to guarantee.
In view of the above problems that the conventional fault diagnosis method is time-consuming and labor-consuming in the diagnosis process, and it is difficult to ensure the accuracy and sustainability of the diagnosis result, an effective solution has not been proposed at present.
Disclosure of Invention
The embodiment of the invention provides a fault detection method and device based on image characteristics, and aims to at least solve the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is low.
According to an aspect of the embodiments of the present invention, there is provided a fault detection method based on image features, including: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
According to another aspect of the embodiments of the present invention, there is also provided an image feature-based fault detection apparatus, including: the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring an image of a product to be detected and equipment parameters, and the equipment parameters are parameters of equipment for manufacturing the product to be detected; the extraction module is used for extracting image characteristics of the image; the determining module is used for analyzing the image characteristics and the equipment parameters based on a first model and determining the fault type of the equipment, wherein the first model is obtained through machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
According to another aspect of the embodiments of the present invention, there is also provided a computer terminal, including: a processor; and a memory coupled to the processor for providing instructions to the processor for processing the following processing steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein when the program is executed, a device on which the storage medium is located is controlled to execute the following steps: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image features and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image features, the equipment parameters and the fault type.
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes the following steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
In the embodiment of the invention, a fault detection mode based on image characteristics is adopted, and the image and the equipment parameters of a product to be detected are obtained; extracting image features of the image; the image characteristics and the equipment parameters are analyzed based on a first model, and then the fault type of the equipment is determined, wherein the first model is obtained through machine learning training based on at least one group of data, the at least one group of data comprises the image characteristics, the equipment parameters and the fault type, the purposes of reducing personnel participation and improving the efficiency and the accuracy of diagnosing the fault type are achieved, the technical effect of ensuring the accuracy and the sustainability of a diagnosis result is achieved, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is low are solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and do not constitute a limitation of the invention. In the drawings:
FIG. 1 is a schematic view of a microtome for cutting a silicon wafer according to the prior art;
fig. 2 is a flowchart of a fault detection method based on image features according to embodiment 1 of the present invention;
FIG. 3 is a schematic diagram of preprocessing an image of a product to be inspected according to embodiment 1 of the present invention;
FIG. 4 is a flowchart of an alternative image feature-based fault detection method according to embodiment 1 of the present invention;
FIG. 5 is a flow chart of an alternative image feature based fault detection method according to embodiment 1 of the present invention;
fig. 6 is a schematic structural diagram of an image feature-based failure detection apparatus according to embodiment 2 of the present invention; and
fig. 7 is a block diagram of a hardware configuration of a computer terminal according to embodiment 3 of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:
silicon chip: refers to a thin slice cut from polycrystalline or single crystal silicon.
Charge-coupled Device (CCD): a semiconductor device, also called a CCD image sensor or an image controller, capable of converting an optical image into an electrical signal.
Digital Signal Processing (DSP): it refers to a technical subject for researching analysis, transformation, filtering, detection, modulation, demodulation and fast algorithm of signals by a digital method.
Example 1
There is also provided, in accordance with an embodiment of the present invention, an embodiment of a method for image feature based fault detection, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be implemented in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
It should be noted that the fault detection method based on image features provided in the embodiments of the present application can be used in a silicon wafer cutting process, and a fault self-diagnosis model is generated based on a machine learning algorithm, so that a silicon wafer cutting fault type can be quickly and effectively diagnosed and analyzed by obtaining image information and device parameters of a silicon wafer.
With the popularization of green energy in various countries around the world and the ultra-conventional development of the semiconductor industry in recent years, the supply and demand of the silicon wafer market are greatly unbalanced, the backward cutting processing capacity and the serious shortage of the capacity form the bottleneck of an industrial chain, the silicon wafer cutting is used as an upstream key technology of silicon wafer production, and the cutting quality and the scale directly influence the subsequent production of the whole semiconductor industrial chain.
Because the slicing process takes a long time, the slicing operation is complex, and the existing silicon wafer cutting equipment is difficult to meet the precision requirement of the cutting process, the situation that the quality of the cut silicon wafer is poor (for example, the thickness of the silicon wafer is uneven and line marks) often occurs, so that the economic value of the silicon wafer is reduced, and even the silicon wafer becomes a waste wafer. Therefore, the yield (the proportion of the silicon wafers with good quality) of silicon wafer cutting is improved, so that the production cost is reduced, the economic value of the silicon wafers is improved, and the silicon wafers become core requirements of semiconductor industries such as photovoltaic enterprises, integrated circuits and the like.
During the silicon wafer cutting process, detecting and locating faults (i.e., fault diagnosis) of silicon wafer cutting systems and equipment have become the most fundamental and challenging problem in the silicon wafer manufacturing industry.
Specifically, the application provides a fault detection method based on image characteristics as shown in fig. 2. Fig. 2 is a flowchart of a method for detecting a fault based on image features according to embodiment 1 of the present invention, as shown in fig. 2, the method includes the following method steps:
step S102, obtaining an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected.
Optionally, the product to be detected may be, but is not limited to, a silicon wafer, and the apparatus for manufacturing the product to be detected may be, but is not limited to, a slicing machine, for example, a silicon wafer laser cutting machine, a laser dicing machine, and may also be other similar or similar products.
In an alternative embodiment, the device parameter is a parameter of a device for manufacturing the product to be detected, and includes at least one of the following: temperature, vibration, rotational speed.
In addition, in an alternative embodiment, the image of the product to be detected may be acquired in the following alternative ways: and acquiring an image of the product to be detected by a charge coupled device image sensor.
Alternatively, the CCD image sensor may be a CCD image sensor.
In an alternative embodiment, the process of capturing an image of the product to be inspected by the ccd image sensor is as follows: the reflected light of the silicon chip passes through a lens of a CCD image sensor, the CCD image sensor converts the collected light signal into an electric signal, the electric signal is compressed by a DSP image and then stored, and the obtained silicon chip CCD image is collected.
Step S104, extracting image characteristics of the image.
In an alternative embodiment, the image feature includes at least one of: the size of the product, the position of the line mark, the thickness of the line mark, the length of the line mark and the unfilled corner of the product.
Optionally, the product may be a cut silicon wafer.
In an alternative embodiment, the image features of the image may be extracted in the following alternative ways:
preprocessing the image; and extracting image features from the pre-processed image.
As an alternative embodiment, the preprocessing the image includes: the silicon wafer after the preprocessing is performed on the image by graying and denoising, as shown in fig. 3.
The image features may be extracted through an image processing algorithm, for example, the extraction of the silicon wafer image features is realized through an algorithm such as image segmentation and edge extraction.
And S106, analyzing the image characteristics and the equipment parameters based on the first model, and determining the fault type of the equipment.
The first model is obtained through machine learning training based on at least one group of data, and the at least one group of data comprises image characteristics, equipment parameters and fault types.
Optionally, the first model is a fault self-diagnosis model, and the model supports automatic periodic update and is used for analyzing image features and device parameters to determine a fault type of the device.
Optionally, the above fault types may include, but are not limited to: guide wheel radial shake, steel wire fault, product impurities and the like, it should be noted that according to any one of the fault types, the equipment generating the fault can be determined.
It should be noted that, for a training sample reaching a certain scale, supervised machine learning may be performed, and usually, a machine classification algorithm is used for learning first, so that a relatively accurate prediction model may be obtained and used for predicting a fault type of a certain generated batch in the future.
According to the fault self-diagnosis model, the prediction of the fault type label can be realized by inputting the image characteristics and the equipment parameters of the production batch, and the automatic diagnosis of the fault type is completed.
In an alternative implementation, fig. 4 is a flowchart of an alternative image feature-based fault detection method according to embodiment 1 of the present invention, and as shown in fig. 4, the first model may be established through the following method steps:
step S1062, acquiring a sample image, an apparatus parameter, and a fault type of the training sample, where the apparatus parameter of the training sample is a parameter of an apparatus for manufacturing the training sample.
It should be noted that the fault type of the training sample may be obtained according to the sample image and the device alarm information.
In an alternative embodiment, a process technology expert may perform comprehensive diagnosis of the fault through information such as silicon wafer images and machine alarms, and recovery measures after the fault occurs, so as to accurately form a tag library of fault type descriptions of each production lot, and correspond to the silicon wafers produced by each production lot.
It should be noted that, during the silicon wafer cutting process, the fault types corresponding to a production lot are the same or similar, and therefore, the granularity of the training sample may be set by taking the production lot as the granularity. Meanwhile, in machine learning, a training sample for supervised learning can be formed by adding a target variable to the number of samples of the training sample.
In addition, it should be noted that when sample images, device parameters and fault label data of a silicon wafer of a certain scale are accumulated, input sample library data of a conventional machine learning algorithm can be obtained, and since fault diagnosis can be abstracted into classification problems, sample data is classified and modeled through machine learning, and then the fault self-diagnosis model can be obtained.
For example, the fault tag data may be fault tag data such as "guide wheel radial jitter", and the fault tag data is used as a tag of a production lot to obtain a fault type corresponding to the production lot.
In step S1064, image features of the sample image are extracted.
Optionally, in step S1064, the image features of the sample image include, but are not limited to: and the silicon chip image characteristics and the silicon chip non-image characteristics are associated by marking the samples to form a complete training sample.
Step S1066, modeling the image characteristics, the equipment parameters and the fault types of the sample images by a machine learning classification algorithm to obtain a first model.
Optionally, the machine learning classification algorithm may be, but is not limited to: decision trees, SVM, NB, or deep learning correlation algorithms.
Based on the scheme defined by the embodiment, the image and the equipment parameters of the product to be detected can be obtained, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image characteristics of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
It is easy to notice that since the image information such as the shape of the line mark, the position of the line mark and the like, the parameter information such as the temperature, the vibration and the rotating speed of the slicer and the like can be obtained by obtaining the image and the equipment parameters of the product to be detected, the effective image characteristics in the image information are extracted through an image processing algorithm, and the image characteristics and the equipment parameters are analyzed based on the first model, the diagnosis and analysis of the fault type can be further realized, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is lower are solved.
In addition, according to the embodiment of the application, the fault self-diagnosis model is generated based on the machine learning algorithm, manual experience is not mainly relied on, manual intervention is less in the diagnosis process, and the workload and the participation degree of personnel can be reduced.
Through the scheme provided by the embodiment of the application, the purposes of reducing personnel participation and improving the efficiency and accuracy of diagnosing the fault type are achieved, the technical effects of ensuring the accuracy and sustainability of the diagnosis result are achieved, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is low are solved.
In order to understand the above embodiments of the present application, the following examples are given as an example, but not as a limitation of the present application. Fig. 5 is a flowchart of an alternative image feature-based fault detection method according to embodiment 1 of the present invention, and as shown in fig. 5, the image feature-based fault detection method provided in the present application may be implemented by the following method steps:
step S701: and acquiring an image and equipment parameters of the product to be detected.
Optionally, the product to be detected may be, but is not limited to, a silicon wafer, and the apparatus for manufacturing the product to be detected may be, but is not limited to, a slicing machine, for example, a silicon wafer laser cutting machine, a laser dicing machine, or other similar or analogous products.
In the step S701, the device parameter is a parameter of a device for manufacturing a product to be detected, and in an optional implementation manner, the device parameter is a parameter of a device for manufacturing a product to be detected, and includes at least one of the following: temperature, vibration, rotational speed.
Step S702: and preprocessing the image.
Optionally, in the step S702, the preprocessing the image includes: and carrying out graying processing and denoising processing on the image.
Step S703: and extracting image features from the preprocessed image.
Optionally, in step S703, the image features may be, but are not limited to: line mark features, thickness features, profile features, and the like.
More specifically, in an alternative embodiment, the image feature includes at least one of: the size of the product, the position of the line mark, the thickness of the line mark, the length of the line mark and the unfilled corner of the product.
The image features may be extracted through an image processing algorithm, for example, the extraction of the silicon wafer image features is realized through an algorithm such as image segmentation and edge extraction.
Step S704: and fault diagnosis is carried out by a fault expert.
Optionally, in step S704, a fault expert (e.g., a process engineering expert) performs comprehensive diagnosis of the fault through information such as a silicon wafer image and a machine alarm, and a recovery measure after the fault is generated, so as to accurately form a tag library of fault type descriptions for each production lot, and to correspond to the silicon wafers produced in each production lot.
Step S705: and obtaining the fault type according to the generated fault label.
As an optional embodiment, the failure tag data is used as a tag of the production lot, and the failure type corresponding to the production lot is further obtained.
Step S706: modeling is performed by using a machine classification algorithm.
In an alternative embodiment, a machine classification algorithm is typically used for learning, and a relatively accurate prediction model can be obtained and used for predicting the fault type of a future generation batch.
Step S707: and obtaining a fault self-diagnosis model.
It should be noted that the fault self-diagnosis model supports automatic periodic update, and may be used to analyze image features and device parameters, and further determine the fault type of the device.
Step S708: a fault type of the device is determined.
In an alternative embodiment, the image features and the device parameters may be analyzed using a fault self-diagnosis model to determine the type of fault of the device.
After obtaining sample images, equipment parameters and fault label data of the silicon wafer, input sample library data of a conventional machine learning algorithm can be obtained, classification problems can be abstracted due to fault diagnosis, sample data is classified and modeled through machine learning, and then the fault self-diagnosis model can be obtained.
It should be noted that for simplicity of description, the above-mentioned method embodiments are shown as a series of combinations of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method according to the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
According to an embodiment of the present invention, there is further provided an apparatus for implementing the foregoing fault detection method based on image features, and fig. 6 is a schematic structural diagram of a fault detection apparatus based on image features according to embodiment 2 of the present invention, as shown in fig. 6, the apparatus includes: a first acquisition module 80, a first extraction module 82, and a determination module 84, wherein,
the first obtaining module 80 is configured to obtain an image of a product to be detected and device parameters, where the device parameters are parameters of a device for manufacturing the product to be detected; a first extraction module 82, configured to extract image features of an image; and a determining module 84, configured to analyze the image features and the device parameters based on a first model, and determine a fault type of the device, where the first model is obtained through machine learning training based on at least one set of data, and the at least one set of data includes the image features, the device parameters, and the fault type.
It should be noted here that the first obtaining module 80, the first extracting module 82 and the determining module 84 correspond to steps S102 to S108 in embodiment 1, and the two modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the above modules as a part of the apparatus may be operated in the computer terminal 9 provided in embodiment 3.
Based on the scheme defined in the above embodiment, it can be known that the first obtaining module is configured to obtain an image of a product to be detected and an apparatus parameter, where the apparatus parameter is a parameter of an apparatus for manufacturing the product to be detected; the first extraction module is used for extracting image features of the image; the determining module is used for analyzing the image characteristics and the equipment parameters based on a first model and determining the fault type of the equipment, wherein the first model is obtained through machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
It is easy to notice that the shape of the line mark, the image information such as the position of the line mark and the like, the parameter information such as the temperature, the vibration and the rotating speed of the slicer and the like can be obtained by obtaining the image and the equipment parameters of the product to be detected, effective image characteristics in the image information are extracted through an image processing algorithm, and the image characteristics and the equipment parameters are analyzed based on the first model, so that the diagnosis and analysis of the fault type can be realized, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is lower are solved.
In addition, according to the embodiment of the application, the fault self-diagnosis model is generated based on the machine learning algorithm, manual experience is not mainly relied on, manual intervention is less in the diagnosis process, and the workload and the participation degree of personnel can be reduced.
Through the scheme provided by the embodiment of the application, the purposes of reducing personnel participation and improving the efficiency and accuracy of diagnosing the fault type are achieved, the technical effects of ensuring the accuracy and sustainability of the diagnosis result are achieved, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of the diagnosis result is low are solved.
In an alternative embodiment, as shown in fig. 6, the determining module 84 includes: a second acquisition module 841, a second extraction module 843, and a modeling module 845, wherein,
a second obtaining module 841, configured to obtain a sample image, an apparatus parameter, and a fault type of a training sample, where the apparatus parameter of the training sample is a parameter of an apparatus for manufacturing the training sample; a second extraction module 843, configured to extract image features of the sample image; and the modeling module 845 is used for modeling the image characteristics, the equipment parameters and the fault types of the sample images through a machine learning classification algorithm to obtain a first model.
It should be noted here that the second obtaining module 841, the second extracting module 843 and the modeling module 845 correspond to steps S1082 to S1088 in the embodiment 1, and the two modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the above modules may be operated in the computer terminal 9 provided in embodiment 3 as a part of the apparatus.
In an alternative embodiment, the type of failure of the training sample is derived from the sample image and the device alarm information.
In an alternative embodiment, as shown in fig. 6, the first extraction module 82 includes: a processing unit 821 and an extraction unit 823, wherein,
a processing unit 821 for preprocessing an image; an extraction unit 823 is configured to extract image features from the preprocessed image.
In an optional embodiment, the processing unit 821 is further configured to perform graying and denoising on the image.
In an alternative embodiment, the first acquiring module 80 is further configured to acquire an image through a ccd image sensor.
In an alternative embodiment, the image features include at least one of: the size of the product, the position of the line mark, the thickness of the line mark, the length of the line mark and the unfilled corner of the product; the device parameters include at least one of: temperature, vibration, rotational speed.
It should be noted that, reference may be made to the relevant description in embodiment 1 for a preferred implementation of this embodiment, and details are not repeated here.
Example 3
The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.
Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.
In this embodiment, the computer terminal may execute the program code of the following steps in the image feature-based failure detection method of the application program: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image characteristics of the image; and analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
Fig. 7 shows a block diagram of a hardware configuration of a computer terminal. As shown in fig. 7, the computer terminal 9 may include one or more (shown as 92a, 92b, \ 8230; \8230;, 92 n) processors 92 (the processors 92 may include, but are not limited to, processing devices such as microprocessor MCUs or programmable logic devices FPGAs), a memory 94 for storing data, and a transmission device 96 for communication functions. In addition, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 7 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 9 may also include more or fewer components than shown in FIG. 7, or have a different configuration than shown in FIG. 7.
It should be noted that the one or more processors 92 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Furthermore, the data processing circuit may be a single stand-alone processing module or any of the other elements incorporated in whole or in part into the computer terminal 9. As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).
The processor 92 may invoke the memory-stored information and the application program via the transmission means to perform the following steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; and analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
The memory 94 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the image feature-based fault detection method in the embodiment of the present application, and the processor 92 executes various functional applications and data processing by running the software programs and modules stored in the memory 94, so as to implement the above-mentioned image feature-based fault detection method. The memory 94 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 94 may further include memory located remotely from the processor 92, which may be connected to the computer terminal 9 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 96 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 9. In one example, the transmission device 96 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 96 may be a Radio Frequency (RF) module, which is used to communicate with the internet by wireless means.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with the user interface of the computer terminal 9.
It should be noted here that in some alternative embodiments, the computer terminal 9 shown in fig. 7 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that fig. 7 is only one example of a specific example and is intended to illustrate the types of components that may be present in the above-described computer terminal 9.
In this embodiment, the computer terminal may execute program codes of the following steps in the cloud platform-based data sharing method for the application program: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image characteristics of the image; and analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; and analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
Optionally, the processor may further execute the program code of the following steps: acquiring a sample image, equipment parameters and a fault type of a training sample, wherein the equipment parameters of the training sample are parameters of equipment for manufacturing the training sample; extracting image characteristics of a sample image; and modeling the image characteristics, the equipment parameters and the fault types of the sample images by a machine learning classification algorithm to obtain a first model.
Optionally, the processor may further execute the program code of the following steps: preprocessing the image; and extracting image features from the preprocessed image.
Optionally, the processor may further execute the program code of the following steps: and carrying out graying processing and denoising processing on the image.
Optionally, the processor may further execute the program code of the following steps: images are acquired by a charge coupled device image sensor.
The embodiment of the invention provides a fault detection scheme based on image characteristics. Acquiring an image and equipment parameters of a product to be detected; extracting image features of the image; the image characteristics and the equipment parameters are analyzed based on a first model, and then the fault type of the equipment is determined, wherein the first model is obtained through machine learning training based on at least one group of data, the at least one group of data comprises the image characteristics, the equipment parameters and the fault type, the purposes of reducing personnel participation and improving the efficiency and the accuracy of fault type diagnosis are achieved, and the technical problems that the existing fault diagnosis method depends on manual experience, the diagnosis process is time-consuming and labor-consuming, and the accuracy of a diagnosis result is low are solved.
It should be understood by those skilled in the art that the structure shown in fig. 7 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 7 is a diagram illustrating a structure of the electronic device. For example, the computer terminal 9 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 7, or have a different configuration than shown in FIG. 7.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, read-Only memories (ROMs), random Access Memories (RAMs), magnetic or optical disks, and the like.
Example 4
The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store the program code executed by the image feature-based fault detection method provided in the first embodiment.
Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image characteristics of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a sample image, equipment parameters and a fault type of a training sample, wherein the equipment parameters of the training sample are parameters of equipment for manufacturing the training sample; extracting image characteristics of a sample image; and modeling the image characteristics, the equipment parameters and the fault types of the sample images by a machine learning classification algorithm to obtain a first model.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: preprocessing the image; and extracting image features from the preprocessed image.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and carrying out graying processing and denoising processing on the image.
Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: and acquiring an image through the charge coupled device image sensor.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is only a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that it is obvious to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and these modifications and improvements should also be considered as the protection scope of the present invention.
Claims (11)
1. A fault detection method based on image features is characterized by comprising the following steps:
acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected;
extracting image features of the image;
analyzing the image features and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image features, the equipment parameters and the fault type.
2. The method of claim 1, wherein the method of building the first model comprises:
acquiring a sample image, equipment parameters and a fault type of a training sample, wherein the equipment parameters of the training sample are parameters of equipment for manufacturing the training sample;
extracting image features of the sample image;
and modeling the image characteristics, the equipment parameters and the fault types of the sample images through a machine learning classification algorithm to obtain the first model.
3. The method of claim 2, wherein the type of fault for the training sample is derived from the sample image and equipment alarm information.
4. The method of claim 1, wherein the extracting image features of the image comprises:
preprocessing the image;
and extracting the image features from the preprocessed image.
5. The method of claim 1, wherein the pre-processing the image comprises:
and carrying out graying processing and denoising processing on the image.
6. The method of claim 1, wherein said obtaining an image of a product to be detected comprises:
the image is acquired by a charge coupled device image sensor.
7. The method of claim 1,
the image features include at least one of: the size of the product, the position of the line mark, the thickness of the line mark, the length of the line mark and the unfilled corner of the product;
the device parameter includes at least one of: temperature, vibration, rotational speed;
the fault type includes at least one of: radial shaking of guide wheels, steel wire faults and product impurities.
8. An image feature-based failure detection apparatus, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring an image of a product to be detected and equipment parameters, and the equipment parameters are parameters of equipment for manufacturing the product to be detected;
the extraction module is used for extracting the image characteristics of the image;
the determining module is used for analyzing the image features and the equipment parameters based on a first model and determining the fault type of the equipment, wherein the first model is obtained through machine learning training based on at least one group of data, and the at least one group of data comprises the image features, the equipment parameters and the fault type.
9. A computer terminal, comprising:
a processor; and
a memory coupled to the processor for providing instructions to the processor for processing the following processing steps: acquiring an image of a product to be detected and equipment parameters, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image features and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image features, the equipment parameters and the fault type.
10. A storage medium comprising a stored program, wherein the program, when executed, controls an apparatus on which the storage medium is located to perform the steps of: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
11. A processor, for running a program, wherein the program when run performs the steps of: acquiring an image and equipment parameters of a product to be detected, wherein the equipment parameters are parameters of equipment for manufacturing the product to be detected; extracting image features of the image; analyzing the image characteristics and the equipment parameters based on a first model, and determining the fault type of the equipment, wherein the first model is obtained by machine learning training based on at least one group of data, and the at least one group of data comprises the image characteristics, the equipment parameters and the fault type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711061292.XA CN109753979B (en) | 2017-11-01 | 2017-11-01 | Fault detection method and device based on image characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711061292.XA CN109753979B (en) | 2017-11-01 | 2017-11-01 | Fault detection method and device based on image characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109753979A CN109753979A (en) | 2019-05-14 |
| CN109753979B true CN109753979B (en) | 2022-11-15 |
Family
ID=66399167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711061292.XA Active CN109753979B (en) | 2017-11-01 | 2017-11-01 | Fault detection method and device based on image characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109753979B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110831116A (en) * | 2019-11-05 | 2020-02-21 | 珠海格力电器股份有限公司 | Household appliance network distribution method, storage medium and terminal |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4710495B2 (en) * | 2005-09-06 | 2011-06-29 | 富士ゼロックス株式会社 | Failure diagnosis system, image forming apparatus, and failure diagnosis method |
| JP4631865B2 (en) * | 2007-03-13 | 2011-02-16 | 富士ゼロックス株式会社 | Electronic circuit device, failure diagnosis device, failure diagnosis system, and failure diagnosis program. |
| CN102467089B (en) * | 2010-11-10 | 2013-09-11 | 北京北方微电子基地设备工艺研究中心有限责任公司 | Process control method of semiconductor technology and system thereof |
| US9349660B2 (en) * | 2011-12-01 | 2016-05-24 | Taiwan Semiconductor Manufacturing Company, Ltd. | Integrated circuit manufacturing tool condition monitoring system and method |
| US8607169B2 (en) * | 2011-12-28 | 2013-12-10 | Elitetech Technology Co., Ltd. | Intelligent defect diagnosis method |
| JP6075240B2 (en) * | 2013-08-16 | 2017-02-08 | 富士ゼロックス株式会社 | Predictive failure diagnosis apparatus, predictive failure diagnosis system, predictive failure diagnosis program, and predictive failure diagnosis method |
| CN106739485B (en) * | 2016-12-13 | 2019-01-04 | 北京印刷学院 | A kind of printing machine longitudinal direction register on-line checking and method for diagnosing faults and device |
| CN106951197A (en) * | 2017-01-19 | 2017-07-14 | 北京印刷学院 | A kind of printing equipment remote fault diagnosis method and system based on signal testing |
-
2017
- 2017-11-01 CN CN201711061292.XA patent/CN109753979B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109753979A (en) | 2019-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113918376B (en) | Fault detection method, device, equipment and computer readable storage medium | |
| CN110163852B (en) | Conveying belt real-time deviation detection method based on lightweight convolutional neural network | |
| CN113419298B (en) | Multi-parameter hydrological meteorological data acquisition device | |
| CN103366176B (en) | Optical element defect batch automatic identification equipment and method | |
| CN102654463A (en) | Watermelon quality NDT (non-destructive testing) method and device | |
| CN114881989A (en) | Small sample based target object defect detection method and device, and electronic equipment | |
| CN113934720A (en) | Data cleaning method and equipment and computer storage medium | |
| CN112258507A (en) | Target object detection method and device of internet data center and electronic equipment | |
| CN109753979B (en) | Fault detection method and device based on image characteristics | |
| CN112883929A (en) | Online video abnormal behavior detection model training and abnormal detection method and system | |
| CN102610002B (en) | Machine station state detection system and method | |
| CN110827263B (en) | Magnetic shoe surface defect detection system and detection method based on visual identification technology | |
| CN116641855B (en) | Wind generating set operation monitoring method, system, equipment and storage medium | |
| CN112585551A (en) | Server and control method thereof | |
| TWM550465U (en) | Semiconductor wafer analyzing system | |
| CN114322446A (en) | Cooling system fault early warning method and device, cooling system and operation machine | |
| CN104964989A (en) | Method and system for visual inspection of photovoltaic module | |
| CN112101375A (en) | Intelligent substation equipment state detection method and system based on deep learning | |
| CN111047550A (en) | Product abnormity detection method, device and equipment based on machine vision | |
| CN112860348A (en) | Intelligent diagnosis and analysis system of injection molding machine | |
| CN116958153B (en) | Unsupervised defect detection method, system and medium based on slice matching | |
| CN112073544A (en) | Method, computing device, and computer storage medium for processing sensor data | |
| CN116880365B (en) | Method, system and readable storage medium for monitoring equipment production process | |
| CN116295855B (en) | Fault positioning and diagnosing system based on wireless communication network | |
| CN114897810A (en) | Image detection method, image detection device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |