CN116993704A - Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer - Google Patents
Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer Download PDFInfo
- Publication number
- CN116993704A CN116993704A CN202311006630.5A CN202311006630A CN116993704A CN 116993704 A CN116993704 A CN 116993704A CN 202311006630 A CN202311006630 A CN 202311006630A CN 116993704 A CN116993704 A CN 116993704A
- Authority
- CN
- China
- Prior art keywords
- defect
- image
- reverse osmosis
- osmosis membrane
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 207
- 238000001514 detection method Methods 0.000 title claims abstract description 113
- 239000012528 membrane Substances 0.000 title claims abstract description 101
- 238000001223 reverse osmosis Methods 0.000 title claims abstract description 99
- 238000003860 storage Methods 0.000 title claims abstract description 9
- 238000013136 deep learning model Methods 0.000 claims abstract description 37
- 238000013135 deep learning Methods 0.000 claims abstract description 30
- 238000011068 loading method Methods 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims abstract description 11
- 238000005520 cutting process Methods 0.000 claims abstract description 8
- 238000012549 training Methods 0.000 claims description 41
- 238000002372 labelling Methods 0.000 claims description 20
- 230000000007 visual effect Effects 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 230000002146 bilateral effect Effects 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000004519 manufacturing process Methods 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 239000000047 product Substances 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 239000011248 coating agent Substances 0.000 description 2
- 238000000576 coating method Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 239000000084 colloidal system Substances 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000010612 desalination reaction Methods 0.000 description 1
- 239000003651 drinking water Substances 0.000 description 1
- 235000020188 drinking water Nutrition 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 239000010842 industrial wastewater Substances 0.000 description 1
- 244000005700 microbiome Species 0.000 description 1
- 230000003204 osmotic effect Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 239000012466 permeate Substances 0.000 description 1
- 239000011148 porous material Substances 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000013535 sea water Substances 0.000 description 1
- 239000012498 ultrapure water Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
- 238000004065 wastewater treatment Methods 0.000 description 1
- 230000037303 wrinkles Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a reverse osmosis membrane defect detection system, a detection method, a storage medium and a computer, wherein the reverse osmosis membrane defect detection method comprises a defect detection stage B, and the defect detection stage B comprises the following steps: loading a deep learning model; collecting reverse osmosis membrane images; cutting out an image to be detected from the image of the reverse osmosis membrane; performing defect detection on the image to be detected based on the loaded deep learning model to obtain a detection result; and B5, carrying out target tracking on the defects detected in the step B5, assigning a unique ID for each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of the target, the boundary box and the corresponding ID. Compared with the prior art, the method adopts the deep learning technology to detect the defects of the reverse osmosis membrane, thereby improving the accuracy, the instantaneity and the anti-interference capability of the detection.
Description
[ field of technology ]
The invention belongs to the field of continuous product defect real-time detection, and particularly relates to a reverse osmosis membrane defect detection system, a reverse osmosis membrane defect detection method, a reverse osmosis membrane defect detection storage medium and a reverse osmosis membrane defect detection computer based on deep learning.
[ background Art ]
The reverse osmosis membrane is an artificial semipermeable membrane with certain filtering characteristics, which is made of a simulated biological semipermeable membrane, and is a core component of the reverse osmosis technology. The principle of reverse osmosis is to separate other substances from water under the action of osmotic pressure higher than that of solution according to the fact that the substances cannot permeate through a semipermeable membrane. The reverse osmosis membrane has very small pore diameter, so that dissolved salts, colloid, microorganisms, organic matters and the like in water can be effectively removed, and the reverse osmosis membrane has the advantages of good water quality, low energy consumption, no pollution, simple process, simple and convenient operation and the like, and the reverse osmosis membrane has wide application in industries such as direct drinking water, food and beverage, industrial high-purity water, industrial wastewater treatment, sea water desalination and the like. In the reverse osmosis membrane production process, various links such as coating and carrying can damage the reverse osmosis membrane, and defects such as holes, uneven coating, wrinkles and the like are generated. These defects can greatly affect the quality of the product and reduce the qualification rate of the product. The existing defect detection method for the reverse osmosis membrane mainly depends on a manual visual mode for defect screening, is simpler, but requires longer time of worker training, and has the advantages of low sampling rate, low accuracy, poor instantaneity, low efficiency, high labor intensity, large influence by manual experience and subjective factors and great waste of more human resources.
Therefore, a new solution is needed to solve the above problems.
[ invention ]
The invention aims to provide a reverse osmosis membrane defect detection system, a detection method, a storage medium and a computer based on deep learning, which adopt the deep learning technology to detect defects of a reverse osmosis membrane, so that the detection accuracy, the real-time performance and the anti-interference capability are improved.
According to one aspect of the present invention, there is provided a reverse osmosis membrane defect detection method comprising a defect detection stage B, the defect detection stage B comprising: step B1, loading a deep learning model; step B2, collecting reverse osmosis membrane images; step B3, cutting out an image to be detected from the image of the reverse osmosis membrane; step B5, performing defect detection on the image to be detected based on the loaded deep learning model to obtain a detection result; and B7, carrying out target tracking on the defects detected in the step B5, distributing a unique ID to each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of a target, a boundary box and a corresponding ID.
According to another aspect of the present invention, there is provided a reverse osmosis membrane defect detection system comprising a defect detection module, the defect detection module performing the steps of: step B1, loading a deep learning model; step B2, collecting reverse osmosis membrane images; step B3, cutting out an image to be detected from the image of the reverse osmosis membrane; step B5, performing defect detection on the image to be detected based on the loaded deep learning model to obtain a detection result; and B7, carrying out target tracking on the defects detected in the step B5, distributing a unique ID to each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of a target, a boundary box and a corresponding ID.
According to another aspect of the present invention, there is provided a storage medium storing program instructions, the program execution being executed to perform the reverse osmosis membrane defect detection method according to the present invention.
According to another aspect of the present invention, there is provided a computer comprising a processor and a memory, the memory having stored therein program instructions, the processor executing the program instructions to perform the reverse osmosis membrane defect detection method according to the present invention.
Compared with the prior art, the method adopts the deep learning technology to detect the defects of the reverse osmosis membrane, thereby improving the accuracy, the instantaneity and the anti-interference capability of the detection.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a flow chart of a deep learning based reverse osmosis membrane defect detection method in one embodiment of the invention;
FIG. 2 is a schematic diagram of a deep learning-based reverse osmosis membrane defect detection system in accordance with one embodiment of the present invention;
FIG. 3 is a system installation diagram of a deep learning based reverse osmosis membrane defect detection system in one embodiment of the invention;
FIG. 4 is an interface diagram of a main interface monitoring zone of a deep learning based reverse osmosis membrane defect detection system in one embodiment of the present invention;
fig. 5 is an interface diagram of a quality information statistics module of a deep learning-based reverse osmosis membrane defect detection system in an embodiment of the invention.
[ detailed description ] of the invention
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic may be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Unless specifically stated otherwise, the terms coupled, connected, or connected, as used herein, mean either direct or indirect connection, such as a and B, and include both direct electrical connection of a and B, and connection of a to B through electrical components or circuitry.
In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element in question must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.
Referring to fig. 1, a flow chart of a reverse osmosis membrane defect detection method based on deep learning according to an embodiment of the invention is shown. The reverse osmosis membrane defect detection method based on deep learning shown in fig. 1 comprises a model training stage a and a defect detection stage B.
The model training stage A comprises the following steps:
step A1, collecting reverse osmosis membrane defect images; and acquiring reverse osmosis membrane defect images in the production process by using an industrial camera in the image acquisition module.
Step A2, data enhancement is carried out: and C, carrying out random operation on all the reverse osmosis membrane defect images obtained in the step A1 to obtain a random generation image of the reverse osmosis membrane defect images. Wherein the type of random operation includes any one or a combination of flipping, rotating, panning, adding noise, and scaling.
Step A3, labeling a sample: and marking the reverse osmosis membrane defect image obtained in the step A1 and the randomly generated image obtained in the step A2 by using a Labe1Img tool (which is an image marking tool), wherein the defect part in the image is marked, and pixel-level marking is performed by drawing a boundary box, so that txt files (which can be called marking files) which are in one-to-one correspondence with the image are obtained as training samples. the txt file records labeling information of the corresponding image, wherein the labeling information comprises defect types and corresponding defect boundary frame coordinates.
Step A4, dividing the data set: training samples are divided into training sets, validation sets, test sets at a ratio of 8:1:1 (which can be other ratios as well) as data sets.
Step A5, training a neural network: and (3) training the data set obtained in the step (A4) by using a neural network model to obtain a training model suitable for reverse osmosis membrane defect detection. In a specific embodiment of the present invention, the neural network model is a YOLOv5s model, and the corresponding step A5 specifically includes: firstly, configuring parameters of a YOLOv5s model, including input image size, category number, training batch size, learning rate and the like; and then, training the model by using a Pytorch deep learning framework, evaluating the trained model by using a verification set, and evaluating the performance of the model by calculating indexes such as the precision, recall rate, mAP and the like of the model.
Step A6, storing a deep learning model: after training in step A5, a round of training model with the best effect is selected as a final use model (i.e., a deep learning model).
The defect detection stage B comprises the following steps:
step B1, loading a model: and D, loading the deep learning model obtained in the step A.
Step B2, collecting reverse osmosis membrane images: and acquiring reverse osmosis membrane images of the production line in real time by using an industrial camera in the image acquisition module.
Step B3, ROI region (i.e. region of interest) extraction: and B2, cutting out an image to be detected from the reverse osmosis membrane image obtained in the step B2. In a specific embodiment of the present invention, a detection area is selected at a system interface by using a mouse event, and an image to be detected is cut out from the reverse osmosis membrane image obtained in the step B2 through a program process.
Step B4, image preprocessing: and filtering the image to be detected by using a bilateral filtering function of an open source visual library of the OpenCV computer to reduce noise interference, and sharpening the image to be detected by using a filter2D function to strengthen image edge characteristics.
Step B5, deep learning defect detection: and B, performing defect detection on the image to be detected obtained in the step B4 after the image preprocessing based on the loaded deep learning model to obtain a detection result, wherein the detection result comprises the position and the defect type of the image defect. In a specific embodiment of the present invention, the image to be detected after the image preprocessing obtained in step B4 is inferred by using the onnxuntime deep learning inference framework, and the image defect is detected, and correspondingly, step B5 includes: b, converting the deep learning model obtained in the step A into an ONNX format, and storing the ONNX format as an ONNX file; loading the ONNX format file which is converted by using an ONNXRuntime library; b4, inputting the image to be detected after the image preprocessing obtained in the step B4 into a loaded deep learning model, and reasoning by using ONNXRuntime to obtain a detection result; and analyzing the reasoning result according to the output format of the deep learning model, acquiring the position and defect type information of the image defect, and visualizing the detection result on the corresponding reverse osmosis membrane image by using an open source visual library of an OpenCV computer.
In one embodiment of the present invention, step B5 may also be said to include: converting the deep learning model into a preset format and storing the deep learning model into a file in the preset format; loading a file in a predetermined format of the converted deep learning model by using a predetermined library; inputting the image to be detected into the loaded deep learning model, and reasoning by using a preset library to obtain a detection result; and analyzing the reasoning result according to the output format of the loaded deep learning model, acquiring the position and category information of the defect, and visualizing the detection result by using a computer open source visual library. Wherein the predetermined library may be onnxuntime; the predetermined format may be an ONNX format; the file of the predetermined format may be an onnx file; the computer open source visual library may be OpenCV.
And B6, sending the detection result of the step B5 to a PLC (Programmable Logic Controller, namely a programmable logic controller), and if the step B5 detects that the image has defects, enabling the PLC to control the motion control module to cut off the corresponding defect positions in the reverse osmosis membrane.
Step B7, target tracking: and B5, carrying out target tracking on the defects detected in the step B5, assigning a unique ID for each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of the target, the boundary box and the corresponding ID. In a specific embodiment of the present invention, the target tracking is performed on the defects obtained in step B5 by using a Bytetrack algorithm, so that each defect has only a unique ID in the detection field of view, and step B7 specifically includes: b5, detecting defects in the video frame by frame, acquiring initial defect positions and boundary frames, and initializing a target tracker; in each frame, tracking the target through a Bytetrack algorithm, and updating the position and the boundary box of the target according to the characteristic information and the tracking algorithm of the target; assigning a unique ID to each defect according to the tracking result of the defect; and outputting defect target tracking information according to the tracking result of the target, wherein the defect target tracking information comprises the position of the target, the bounding box, the corresponding ID and other information.
It can also be said that, in one embodiment of the present invention, the defect in the video is detected frame by frame using step B5, and correspondingly, the step B7 includes: initializing a target tracker based on the initial position of the defect and the bounding box; in each frame, tracking the target defect through a preset algorithm, updating the position and the boundary box of the target defect, and distributing a unique ID to each defect according to the tracking result of the defect. The predetermined algorithm may be a Bytetrack algorithm.
And B8, quality information statistics: and B7, finishing defect statistics according to the defect target tracking information output in the step B7. For example, the defect target tracking information output in the step B7 is sent to a quality information statistics module to complete defect statistics.
According to another aspect of the invention, the invention provides a reverse osmosis membrane defect detection system based on deep learning. Referring to fig. 2, a schematic structural diagram of a reverse osmosis membrane defect detection system based on deep learning according to an embodiment of the invention is shown. The deep learning-based reverse osmosis membrane defect detection system shown in fig. 2 comprises a hardware system (not identified), a software system (not identified) and a communication module 210, wherein the hardware system comprises a motion control module 220 and an image acquisition module 230; the software system includes a defect detection module 240 and a quality information statistics module 250; the communication module 210 is used for connecting a hardware system and a software system.
The motion control module 220 is used to control the speed and feed distance of the reverse osmosis membrane. In one embodiment of the present invention, the motion control module 212 may control the motion, tension, and adjustment of the speed and advance distance of the membrane by the PLC.
The image acquisition module 230 comprises a camera and a light source, wherein the camera can acquire reverse osmosis membrane images in real time by adopting a color lens of an Abbe medium light Astra mini s small-sized depth camera, and the acquired images are clear and have no smear by setting a proper acquisition frame rate. The camera can adopt a high-speed data transmission interface of USB3.0 as a data transmission hub between the PC (or an upper computer) and the camera, thereby ensuring high-speed picture transmission between the PC and the camera. The light source can adopt an LED light source, and the light reflection effect on the surface of the reverse osmosis membrane is reduced by using a backlight lighting mode, so that a clear image is obtained.
The defect detection module 240 includes a model training stage a and a defect detection stage B, and refer to the description of the training stage a and the defect detection stage B.
The model training stage A is a training generation process of a deep learning model, and samples are collected on site; labeling the sample by using a LabelImg labeling tool, and labeling the position and the category of the defect; and building a YOLOv5s deep learning network through the Pytorch deep learning platform to train the sample, and finally obtaining a defect detection model (or a deep learning model).
The defect detection stage B loads a defect detection model (or a deep learning model) obtained in the training stage when the system is initialized, an image is input from the image acquisition module 230, the image is subjected to ROI region extraction and image preprocessing, the reasoning module is used for reasoning, a detection result is sent to the PLC, if the defect PLC is used for controlling the motion control module 220 to cut out a defective position, defect information is transmitted to the target tracking module, the target is tracked, and the defect is guaranteed to have a unique ID.
The quality information statistics module 250 stores the batch number, defect type, defect location and detection time of the reverse osmosis membrane to MySQL (which is a relational database management system) database, and completes statistics and display of defects.
The communication module 210 adopts ModbusTCP protocol, and completes communication between the hardware system and the software system by reading and writing the holding register of the PLC.
Referring to fig. 3, a system installation diagram of a reverse osmosis membrane defect detection system based on deep learning in an embodiment of the present invention can be seen from the system installation diagram, wherein a software system is installed in a host computer 310, which is responsible for the defect detection program operation, detects the system state, and the rest is a hardware system. The hardware system is the main place of product production, and the industrial camera 320 is installed directly above reverse osmosis membrane production line, takes charge of gathering the image. The PLC330 is responsible for controlling the hardware system to adjust the tension and the advancing speed of the reverse osmosis membrane, and simultaneously receives the defect instruction sent by the upper computer 310, and controls the hardware to finish the defect part rejection.
Referring to fig. 4, an interface diagram of a main interface monitoring area of a reverse osmosis membrane defect detection system based on deep learning in an embodiment of the present invention is shown, and connection, detection operation and stop of a PLC can be controlled through an interface control, and meanwhile, communication with the PLC is completed through a communication module, and information such as feeding distance, feeding speed, counting, etc. is displayed.
Referring to fig. 5, an interface diagram of a quality information statistics module of a reverse osmosis membrane defect detection system based on deep learning according to an embodiment of the invention is shown, which stores information of detected defects and statistical display.
The method for using the main interface monitoring area of the reverse osmosis membrane defect detection system based on deep learning shown in fig. 4 is as follows:
clicking a 'connect PLC' button of the main interface monitoring area, connecting the PLC of the hardware system, and prompting successful connection by an interface message box after successful connection, otherwise prompting connection failure.
Clicking an operation button, opening a camera to start to collect images, inputting the collected images into a defect detection module for detection, and simultaneously, carrying out real-time visual display on the detected images in a main interface monitoring area.
The defect detection module detects by using the YOLOv5s model obtained in the training stage, when detecting that the defect exists on the surface of the reverse osmosis membrane, the main interface can jump out of the popup window to prompt that the defect exists, and the communication module sends a signal to the PLC, and meanwhile, the detected defect is subjected to target tracking and then is sent to the quality information statistics module for recording.
After receiving the signal sent by the software system, the PLC controls the hardware system to cut the defect, and after finishing the processing, sends a processing end signal to the software system, and after receiving the signal, the software system continues to finish the detection work.
After the production line is stopped, clicking a stop button of a main interface monitoring area to finish detection, disconnecting the PLC, interrupting the acquisition of image data, and simultaneously exiting the system to wait for the next production.
According to another aspect of the present invention, there is provided a storage medium storing program instructions that are executed to perform the deep learning-based reverse osmosis membrane defect detection method as described in the foregoing of the present invention.
According to another aspect of the present invention, there is provided a computer comprising a processor and a memory, the memory having stored therein program instructions, the processor executing the program instructions to perform the deep learning based reverse osmosis membrane defect detection method as described herein before.
In summary, the invention discloses a reverse osmosis membrane defect detection system, a detection method, a storage medium and a computer based on deep learning, wherein the defect detection system is constructed by combining C++ language with QT framework and by means of deep learning technology, defects caused in the production process are detected in real time, classification and positioning of the defects are completed, and statistics and display of the defects are completed by combining a target tracking technology and a MySQL database. The system mainly comprises a motion control module of a hardware system, an image acquisition module, a defect detection module of a software system, a quality information statistics module and a communication module between the hardware system and the software system. The invention adopts the deep learning technology to detect the defects of the reverse osmosis membrane, and solves the problems of low accuracy, poor real-time performance, weak anti-interference capability and the like in the prior art. The visual interface is designed, the operation is easy, the defects of the reverse osmosis membrane are automatically detected, and the detection efficiency is improved.
Compared with the prior art, the invention has the beneficial effects that:
(1) The operation is simple and convenient, and the defect detection can be performed by starting the industrial personal computer to run the upper computer software;
(2) The visual, upper computer interface is simple and clear, and can observe the defect detection process directly, mark the position of defect category and other parameters;
(3) The compatibility is good, the expansibility is strong, and the system can directly run in Windows, linux and other different operating systems;
(4) The anti-interference capability is strong, the light source and vibration are insensitive, and even if the external hardware causes image vibration, the detection result is not affected;
(5) The invention has high precision and wide detection range, and the defect detection precision of the invention reaches 97 percent, and the detection range can reach 1060mm;
(6) The real-time performance is strong, and 50 frames of images can be acquired and detected every second.
It should be noted that any modifications to the specific embodiments of the invention may be made by those skilled in the art without departing from the scope of the invention as defined in the appended claims. Accordingly, the scope of the claims of the present invention is not limited to the foregoing detailed description.
It should be noted that any modifications to the specific embodiments of the invention may be made by those skilled in the art without departing from the scope of the invention as defined in the appended claims. Accordingly, the scope of the claims of the present invention is not limited to the foregoing detailed description.
Claims (10)
1. A reverse osmosis membrane defect detection method, characterized in that the method comprises a defect detection stage B, wherein the defect detection stage B comprises:
step B1, loading a deep learning model;
step B2, collecting reverse osmosis membrane images;
step B3, cutting out an image to be detected from the image of the reverse osmosis membrane;
step B4, performing image preprocessing on the image to be detected;
step B5, performing defect detection on the image to be detected based on the loaded deep learning model to obtain a detection result;
and B7, carrying out target tracking on the defects detected in the step B5, distributing a unique ID to each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of a target, a boundary box and a corresponding ID.
2. The method for detecting defects of a reverse osmosis membrane according to claim 1,
the step B3 comprises the following steps: b, selecting a detection area on a system interface by using a mouse event, and cutting out an image to be detected from the reverse osmosis membrane image obtained in the step B2 through program processing;
the step B4 comprises the following steps: filtering the image to be detected by using a bilateral filter function of an open source visual library of an OpenCV computer to reduce noise interference, and sharpening the image to be detected by using a filter2D function to strengthen image edge characteristics;
the step B5 comprises the following steps: converting the deep learning model into a preset format and storing the deep learning model into a file in the preset format; loading a file in a predetermined format of the deep learning model which has been converted by using a predetermined library; inputting the image to be detected into the loaded deep learning model, and reasoning by using a preset library to obtain a detection result; analyzing the reasoning result according to the loaded output format of the deep learning model, acquiring the position and category information of the defect, and visualizing the detection result by using a computer open source visual library; the preset library is ONNXRuntime; the preset format is ONNX format; the file with the preset format is an onnx file; the computer open source visual library is OpenCV; and/or
Detecting defects in the video frame by frame using step B5, said step B7 comprising: initializing a target tracker based on the initial position of the defect and the bounding box; in each frame, tracking the target defect by a preset algorithm, updating the position and the boundary box of the target defect, and distributing a unique ID to each defect according to the tracking result of the defect, wherein the preset algorithm is a Bytetrack algorithm.
3. The method for detecting defects in a reverse osmosis membrane according to claim 1, further comprising:
step B6, sending the detection result of the step B5 to a PLC, and if the step B5 detects that the image has defects, enabling the PLC to control the motion control module to cut off the corresponding defect positions in the reverse osmosis membrane;
and B8, finishing defect statistics according to the defect target tracking information output in the step B7.
4. The method for detecting defects of a reverse osmosis membrane according to any one of claims 1 to 3, further comprising a model training stage,
the model training phase comprises:
step A1, collecting reverse osmosis membrane defect images;
a2, carrying out random operation on the reverse osmosis membrane defect image to obtain a random generation image of the reverse osmosis membrane defect image;
a3, performing defect labeling on the reverse osmosis membrane defect image and the randomly generated image by using an image labeling tool to obtain labeling files corresponding to the images one by one, wherein the labeling files are used as training samples, and labeling information of the corresponding images is recorded in the labeling files, and comprises defect types and corresponding defect boundary frame coordinates;
step A4, dividing the training sample into a training set, a verification set and a test set according to a preset proportion, and taking the training sample as a data set;
step A5, training the data set by using a neural network model to obtain a training model suitable for reverse osmosis membrane defect detection;
and A6, selecting one training model with the best effect from a plurality of training models as the deep learning model.
5. The method for detecting defects of a reverse osmosis membrane according to claim 4,
the type of random operation comprises any one or more of turning, rotating, translating, adding noise and zooming;
the image labeling tool is a Labe1Img tool, and the labeling file is a txt file;
the neural network model is a YOLOv5s model;
the step A5 comprises the following steps: configuring parameters of the YOLOv5s model, wherein the parameters comprise input image size, category number, training batch size and learning rate; model training is carried out by using a Pytorch deep learning framework; and evaluating the trained model by using the verification set, and calculating the precision, recall rate and mAP of the model to evaluate the performance of the model.
6. A reverse osmosis membrane defect detection system, comprising a defect detection module that performs the steps of:
step B1, loading a deep learning model;
step B2, collecting reverse osmosis membrane images;
step B3, cutting out an image to be detected from the image of the reverse osmosis membrane;
step B5, performing defect detection on the image to be detected based on the loaded deep learning model to obtain a detection result;
and B7, carrying out target tracking on the defects detected in the step B5, distributing a unique ID to each defect according to the tracking result of the defects, and outputting defect target tracking information, wherein the defect target tracking information comprises the position of a target, a boundary box and a corresponding ID.
7. The reverse osmosis membrane defect detection system of claim 6, wherein the defect detection module further performs the steps of:
a step B4 arranged between the step B3 and the step B5, wherein the step B4 carries out image preprocessing on the image to be detected;
step B6, sending the detection result of the step B5 to a PLC, and if the step B5 detects that the image has defects, enabling the PLC to control the motion control module to cut off the corresponding defect positions in the reverse osmosis membrane;
step B8, completing defect statistics according to the defect target tracking information output in the step B7,
the step B3 comprises the following steps: b, selecting a detection area on a system interface by using a mouse event, and cutting out an image to be detected from the reverse osmosis membrane image obtained in the step B2 through program processing;
the step B4 comprises the following steps: filtering the image to be detected by using a bilateral filtering function of an open source visual library of an OpenCV computer to reduce noise interference, sharpening the image to be detected by using a filter2D function to strengthen image edge characteristics,
the step B5 comprises the following steps:
converting the deep learning model into a preset format and storing the deep learning model into a file in the preset format;
loading a file in a predetermined format of the deep learning model which has been converted by using a predetermined library;
inputting the image to be detected into the loaded deep learning model, and reasoning by using a preset library to obtain a detection result;
analyzing the reasoning result according to the loaded output format of the deep learning model, obtaining the position and category information of the defect, visualizing the detection result by using a computer open source visual library,
the preset library is ONNXRuntime;
the preset format is ONNX format;
the file with the preset format is an onnx file;
the computer open source visual library is OpenCV.
Defects in the video are detected frame by frame using step B5,
the step B7 comprises the following steps:
initializing a target tracker based on the initial position of the defect and the bounding box;
in each frame, tracking the target defect by a predetermined algorithm, updating the position and the boundary box of the target defect, assigning a unique ID to each defect according to the tracking result of the defect,
the predetermined algorithm is a Bytetrack algorithm.
8. The reverse osmosis membrane defect detection system of claim 6, further comprising a model training module,
the model training module performs the steps of:
step A1, collecting reverse osmosis membrane defect images;
a2, carrying out random operation on the reverse osmosis membrane defect image to obtain a random generation image of the reverse osmosis membrane defect image;
a3, performing defect labeling on the reverse osmosis membrane defect image and the randomly generated image by using an image labeling tool to obtain labeling files corresponding to the images one by one, wherein the labeling files are used as training samples, and labeling information of the corresponding images is recorded in the labeling files, and comprises defect types and corresponding defect boundary frame coordinates;
step A4, dividing the training sample into a training set, a verification set and a test set according to a preset proportion, and taking the training sample as a data set;
step A5, training the data set by using a neural network model to obtain a training model suitable for reverse osmosis membrane defect detection;
and A6, selecting one training model with the best effect from a plurality of training models as the deep learning model.
9. A storage medium having stored thereon program instructions that are executed to perform the reverse osmosis membrane defect detection method of any one of claims 1-5.
10. A computer comprising a processor and a memory, wherein the memory has stored therein program instructions that are executed by the processor to perform the reverse osmosis membrane defect detection method of any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311006630.5A CN116993704A (en) | 2023-08-10 | 2023-08-10 | Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311006630.5A CN116993704A (en) | 2023-08-10 | 2023-08-10 | Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116993704A true CN116993704A (en) | 2023-11-03 |
Family
ID=88528198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311006630.5A Pending CN116993704A (en) | 2023-08-10 | 2023-08-10 | Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116993704A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473179A (en) * | 2019-07-30 | 2019-11-19 | 上海深视信息科技有限公司 | A kind of film surface defects detection method, system and equipment based on deep learning |
| CN115100149A (en) * | 2022-06-27 | 2022-09-23 | 佛山市南海区广工大数控装备协同创新研究院 | Aluminum film defect detection method based on deep learning and related equipment |
| CN115690778A (en) * | 2022-11-07 | 2023-02-03 | 广西大学 | Method for detecting, tracking and counting mature fruits based on deep neural network |
| CN116256372A (en) * | 2023-05-16 | 2023-06-13 | 征图新视(江苏)科技股份有限公司 | Optical film front-end Cheng Juancai full-detection device and detection method thereof |
| CN116342540A (en) * | 2023-03-24 | 2023-06-27 | 歌尔股份有限公司 | Packaging film defect detection method, device, equipment and storage medium |
| CN116485734A (en) * | 2023-04-10 | 2023-07-25 | 西安理工大学 | Transformer substation equipment defect detection and tracking method based on improved yolov5+bytetrack |
-
2023
- 2023-08-10 CN CN202311006630.5A patent/CN116993704A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473179A (en) * | 2019-07-30 | 2019-11-19 | 上海深视信息科技有限公司 | A kind of film surface defects detection method, system and equipment based on deep learning |
| CN115100149A (en) * | 2022-06-27 | 2022-09-23 | 佛山市南海区广工大数控装备协同创新研究院 | Aluminum film defect detection method based on deep learning and related equipment |
| CN115690778A (en) * | 2022-11-07 | 2023-02-03 | 广西大学 | Method for detecting, tracking and counting mature fruits based on deep neural network |
| CN116342540A (en) * | 2023-03-24 | 2023-06-27 | 歌尔股份有限公司 | Packaging film defect detection method, device, equipment and storage medium |
| CN116485734A (en) * | 2023-04-10 | 2023-07-25 | 西安理工大学 | Transformer substation equipment defect detection and tracking method based on improved yolov5+bytetrack |
| CN116256372A (en) * | 2023-05-16 | 2023-06-13 | 征图新视(江苏)科技股份有限公司 | Optical film front-end Cheng Juancai full-detection device and detection method thereof |
Non-Patent Citations (3)
| Title |
|---|
| 刘国华: "机器视觉技术", 30 November 2021, 华中科技大学出版社, pages: 303 - 308 * |
| 谢佳标: "Keras深度学习入门、实战与进阶", 31 October 2021, 机械工业出版社, pages: 110 - 112 * |
| 高敬鹏 等: "机器学习 基于OpenCV和Python的智能图像处理", 31 May 2020, 机械工业出版社, pages: 122 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111553929B (en) | Mobile phone screen defect segmentation method, device and equipment based on converged network | |
| CN104077773A (en) | Image edge detection method, and image target identification method and device | |
| KR20220111634A (en) | Online offline combined multidimensional education AI school system | |
| CN101706875B (en) | Hand-held laser code-jetting character recognizer under complex background | |
| CN113160123B (en) | Leather defect detection method, system and device based on YOLOv5 | |
| CN110220908A (en) | A kind of on-line detecting system and its online test method of floatation clean coal ash content | |
| CN113421230B (en) | Visual detection method for defects of vehicle-mounted liquid crystal display light guide plate based on target detection network | |
| CN1806501B (en) | Marine phytoplankton automatic distinguishing method and apparatus | |
| CN109886928A (en) | A kind of target cell labeling method, device, storage medium and terminal device | |
| CN110728681A (en) | Mura defect detection method and device | |
| CN106770330A (en) | A kind of body surface scratch detection device and its detection method | |
| CN113628159A (en) | Full-automatic training method and device based on deep learning network and storage medium | |
| CN113506281B (en) | Bridge crack detection method based on deep learning framework | |
| CN113139507B (en) | Automatic capturing method and system for drainage pipeline defect photos | |
| CN103175840B (en) | Based on offset plate surface detection method and the system of machine vision | |
| CN116993704A (en) | Reverse osmosis membrane defect detection system, reverse osmosis membrane defect detection method, storage medium and computer | |
| CN113361487A (en) | Foreign matter detection method, device, equipment and computer readable storage medium | |
| CN111783900A (en) | Training method of oil smoke concentration detection model and control method of range hood gear | |
| CN117016430A (en) | Broiler chicken breeding system based on intelligent monitoring | |
| CN110363245B (en) | Online classroom highlight screening method, device and system | |
| CN116309343A (en) | Defect detection method and device based on deep learning and storage medium | |
| CN206387735U (en) | A kind of body surface scratch detection device | |
| CN115861160A (en) | Method and device for detecting surface defects of power interface of mobile phone and storage medium | |
| CN114387564A (en) | Head-knocking engine-off pumping-stopping detection method based on YOLOv5 | |
| Xu et al. | Development and experiment on automatic grading equipment for kiwi. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |