CN117911409B - Mobile phone screen bad line defect diagnosis method based on machine vision - Google Patents

Mobile phone screen bad line defect diagnosis method based on machine vision Download PDF

Info

Publication number
CN117911409B
CN117911409B CN202410309774.6A CN202410309774A CN117911409B CN 117911409 B CN117911409 B CN 117911409B CN 202410309774 A CN202410309774 A CN 202410309774A CN 117911409 B CN117911409 B CN 117911409B
Authority
CN
China
Prior art keywords
bad line
image
data
mobile phone
bad
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410309774.6A
Other languages
Chinese (zh)
Other versions
CN117911409A (en
Inventor
蔡薇
许雅静
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Kutong Xiaoyang Technology Co ltd
Original Assignee
Shenzhen Kutong Xiaoyang Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Kutong Xiaoyang Technology Co ltd filed Critical Shenzhen Kutong Xiaoyang Technology Co ltd
Priority to CN202410309774.6A priority Critical patent/CN117911409B/en
Publication of CN117911409A publication Critical patent/CN117911409A/en
Application granted granted Critical
Publication of CN117911409B publication Critical patent/CN117911409B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Processing (AREA)

Abstract

The invention relates to the technical field of image diagnosis, in particular to a mobile phone screen bad line defect diagnosis method based on machine vision. The method comprises the following steps: acquiring a mobile phone screen image; performing ROI region segmentation on the mobile phone screen image to generate a standard mobile phone screen ROI region image; carrying out local region pixel characteristic matching on the standard mobile phone screen ROI region image to generate bad line characteristic matching result data; carrying out bad line pattern matching on the bad line characteristic matching result data to generate bad line pattern matching result data; carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data; and carrying out bad line classification prediction on the bad line positioning position data to generate bad line classification prediction data. The invention carries out classification prediction while positioning the bad wire, namely judges the type and severity of the bad wire, and improves the precision and efficiency of bad wire diagnosis.

Description

Mobile phone screen bad line defect diagnosis method based on machine vision
Technical Field
The invention relates to the technical field of image diagnosis, in particular to a mobile phone screen bad line defect diagnosis method based on machine vision.
Background
Early, the bad line diagnosis of the mobile phone screen mainly relies on manual visual inspection, and the efficiency is low and the error is easy to occur. With the development of electronic technology, automatic optical detection technology has been developed. With the continuous progress of technology, image processing-based methods are becoming mature. By digitizing the screen image, an image processing algorithm can be used to detect line defects on the screen. These algorithms can identify the brightness and color changes of the pixel points to accurately locate and mark the defective line. However, this approach requires a large amount of computational resources and complex algorithms, limiting its application in mass production. With the rise of artificial intelligence technology, deep learning and neural network become a new generation technology for mobile phone screen line defect diagnosis. With a large amount of training data, the deep learning model can automatically learn and identify different types of line defects without explicit programming. The method has higher accuracy and efficiency, so that a handset manufacturer can quickly detect and locate screen line defects on a production line, and the product quality and the production efficiency are improved. However, the conventional bad line diagnosis of the mobile phone screen at present cannot well and accurately position the bad line area on the bad line of the mobile phone screen, so that the bad line diagnosis is low in accuracy and efficiency.
Disclosure of Invention
Based on this, it is necessary to provide a method for diagnosing a defective line of a mobile phone screen based on machine vision, so as to solve at least one of the above technical problems.
In order to achieve the above purpose, a method for diagnosing a defective line of a mobile phone screen based on machine vision, the method comprises the following steps:
Step S1: acquiring a mobile phone screen image; performing ROI region segmentation on the mobile phone screen image to generate a standard mobile phone screen ROI region image;
Step S2: carrying out local region pixel characteristic matching on the standard mobile phone screen ROI region image to generate bad line characteristic matching result data; carrying out bad line pattern matching on the bad line characteristic matching result data to generate bad line pattern matching result data;
Step S3: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data; carrying out bad line classification prediction on the bad line positioning position data to generate bad line classification prediction data;
step S4: carrying out bad line severity assessment on the bad line classification prediction data to generate bad line severity assessment data; and integrating the defective line positioning position data, the defective line classification prediction data and the defective line severity evaluation data to generate a defective line defect diagnosis report of the mobile phone screen.
According to the invention, the ROI region is segmented on the mobile phone screen image, and diagnosis is performed by focusing on the screen region, so that interference factors possibly introduced during the whole image analysis are avoided. The pixel characteristic matching and bad line pattern matching of the local area further improve the accuracy of diagnosis, and bad lines can be accurately positioned and identified. Bad line feature matching and pattern matching can effectively distinguish real bad lines from other possible interference factors in images, so that the probability of false alarm is reduced. This helps the manufacturer avoid unnecessary repair or replacement. The automated process greatly improves the efficiency of diagnosis. The whole diagnosis process can be completed in a shorter time without relying on manual visual inspection, and the method is beneficial to rapidly positioning and treating the problem of broken wires of a screen. Bad line location, classification prediction and severity assessment in steps S3 and S4 make diagnosis more comprehensive and multi-level. Not only can the position of the bad wire be determined, but also the type and degree of the bad wire can be predicted, and more information can be provided for manufacturers to take appropriate maintenance measures. The mobile phone screen bad line defect diagnosis report integrates the data of each step and provides clear diagnosis results and suggestions. Such reporting may help manufacturers to better understand the problems with the screen, with targeted repairs and improvements. Therefore, the invention performs the region segmentation of the interested region on the mobile phone screen image, adopts the methods of pixel characteristic matching and pattern matching, performs classification prediction while positioning the bad line, namely judges the type and the severity of the bad line, and improves the precision and the efficiency of bad line diagnosis.
The invention has the beneficial effects that through the subdivision steps, the accuracy of final defect diagnosis can be obviously improved through concentration and accurate processing of each step from image acquisition, ROI region segmentation, pixel feature matching, bad line pattern matching and classification prediction. Especially, the pixel characteristic matching and bad line pattern matching of the local area can accurately identify the fine bad line characteristic, and misdiagnosis and missed diagnosis are reduced. The whole flow can realize high automation, reduce manual intervention and improve diagnosis speed and efficiency. The automated process can process a large amount of data, and is suitable for the rapid detection requirement of a production line or a maintenance center. The generated defective diagnosis report of the defective wire of the mobile phone screen not only contains the position information of the defective wire, but also integrates the type, the density and the severity of the defective wire, provides comprehensive and detailed defect information for maintenance personnel, and is beneficial to making a more accurate and effective maintenance plan. By accurate defect diagnosis, it is better to decide whether to perform screen repair or replacement, thereby optimizing the cost expenditure. For minor problems, this may be solved by simple maintenance, while for severe wire defects, the screen may need to be replaced. The screen broken line problem can be accurately and rapidly diagnosed and solved, and the satisfaction degree of users on products and services can be remarkably improved. This is important to maintain brand reputation and user loyalty. The generated detailed diagnostic report is not only used for current defect repair, but also can be used as precious data resource for quality control and product improvement. By analyzing this data, manufacturers can find potential problems in the product design or manufacturing process and optimize accordingly. Therefore, the invention performs the region segmentation of the interested region on the mobile phone screen image, adopts the methods of pixel characteristic matching and pattern matching, performs classification prediction while positioning the bad line, namely judges the type and the severity of the bad line, and improves the precision and the efficiency of bad line diagnosis.
Drawings
FIG. 1 is a schematic flow chart of steps of a method for diagnosing bad line defects of a mobile phone screen based on machine vision;
FIG. 2 is a detailed flowchart illustrating the implementation of step S1 in FIG. 1;
FIG. 3 is a flowchart illustrating the detailed implementation of step S14 in FIG. 2;
FIG. 4 is a flowchart illustrating the detailed implementation of step S2 in FIG. 1;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
The following is a clear and complete description of the technical method of the present patent in conjunction with the accompanying drawings, and it is evident that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.
Furthermore, the drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. The functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor methods and/or microcontroller methods.
It will be understood that, although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
In order to achieve the above objective, please refer to fig. 1 to 4, a method for diagnosing a defective line of a mobile phone screen based on machine vision, the method comprises the following steps:
Step S1: acquiring a mobile phone screen image; performing ROI region segmentation on the mobile phone screen image to generate a standard mobile phone screen ROI region image;
Step S2: carrying out local region pixel characteristic matching on the standard mobile phone screen ROI region image to generate bad line characteristic matching result data; carrying out bad line pattern matching on the bad line characteristic matching result data to generate bad line pattern matching result data;
Step S3: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data; carrying out bad line classification prediction on the bad line positioning position data to generate bad line classification prediction data;
step S4: carrying out bad line severity assessment on the bad line classification prediction data to generate bad line severity assessment data; and integrating the defective line positioning position data, the defective line classification prediction data and the defective line severity evaluation data to generate a defective line defect diagnosis report of the mobile phone screen.
According to the invention, the ROI region is segmented on the mobile phone screen image, and diagnosis is performed by focusing on the screen region, so that interference factors possibly introduced during the whole image analysis are avoided. The pixel characteristic matching and bad line pattern matching of the local area further improve the accuracy of diagnosis, and bad lines can be accurately positioned and identified. Bad line feature matching and pattern matching can effectively distinguish real bad lines from other possible interference factors in images, so that the probability of false alarm is reduced. This helps the manufacturer avoid unnecessary repair or replacement. The automated process greatly improves the efficiency of diagnosis. The whole diagnosis process can be completed in a shorter time without relying on manual visual inspection, and the method is beneficial to rapidly positioning and treating the problem of broken wires of a screen. Bad line location, classification prediction and severity assessment in steps S3 and S4 make diagnosis more comprehensive and multi-level. Not only can the position of the bad wire be determined, but also the type and degree of the bad wire can be predicted, and more information can be provided for manufacturers to take appropriate maintenance measures. The mobile phone screen bad line defect diagnosis report integrates the data of each step and provides clear diagnosis results and suggestions. Such reporting may help manufacturers to better understand the problems with the screen, with targeted repairs and improvements. Therefore, the invention performs the region segmentation of the interested region on the mobile phone screen image, adopts the methods of pixel characteristic matching and pattern matching, performs classification prediction while positioning the bad line, namely judges the type and the severity of the bad line, and improves the precision and the efficiency of bad line diagnosis.
In the embodiment of the present invention, as described with reference to fig. 1, the method for diagnosing a defective line of a mobile phone screen based on machine vision according to the present invention includes the following steps:
Step S1: acquiring a mobile phone screen image; performing ROI region segmentation on the mobile phone screen image to generate a standard mobile phone screen ROI region image;
in the embodiment of the invention, the images of the mobile phone screen are acquired by using a camera or other image acquisition equipment. This may be done by connecting the handset to the computer, using a handset camera, or by other suitable means. And preprocessing the acquired mobile phone screen image, including removing noise, adjusting brightness and contrast, and the like. This helps to improve the accuracy and stability of subsequent processing steps. The cell phone screen image is segmented into regions of interest (Region of Interest, ROIs) using image processing techniques such as edge detection, thresholding, etc. The ROI is the area of the cell phone screen that contains the actual display content and is extracted for subsequent processing. And extracting the ROI area obtained by segmentation, and generating a standard mobile phone screen ROI area image through necessary adjustment and standardization. This may include resizing the image, color space, etc. to ensure consistency and accuracy of subsequent processing. And saving the generated standard mobile phone screen ROI area image for later steps.
Step S2: carrying out local region pixel characteristic matching on the standard mobile phone screen ROI region image to generate bad line characteristic matching result data; carrying out bad line pattern matching on the bad line characteristic matching result data to generate bad line pattern matching result data;
In the embodiment of the invention, the characteristic extraction is carried out on the local area in the standard mobile phone screen ROI area image. This may use various image processing techniques such as local binary pattern (Local Binary Patterns), gray Co-occurrence matrix (GRAY LEVEL Co-occurrence Matrix), harris corner detection, etc. The selection of a suitable feature extraction method depends on the characteristics and shape of the bad wire. And performing similar feature extraction on the image to be detected by using the features obtained by the feature extraction, and performing local region pixel feature matching by using a matching algorithm such as a feature matching algorithm (e.g. SIFT, SURF, ORB and the like). This will produce a set of matching results representing local areas in the image to be detected that are similar to those in the standard image. The pattern defining the bad line may be a series of specific pixel arrangements or shapes. This may involve formulating rules, templates, or pattern learning using machine learning techniques. And further matching the pixel characteristic matching result data of the local area by using the defined bad line mode. This may include using techniques such as template matching, morphological operations, and the like. The result of the match will identify the region in the image that matches the defined bad line pattern. And combining the local area pixel characteristic matching result data and the bad line pattern matching result to generate bad line pattern matching result data. This may include information on the position, shape, number, etc. of the bad wire.
Step S3: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data; carrying out bad line classification prediction on the bad line positioning position data to generate bad line classification prediction data;
In the embodiment of the invention, the bad line pattern matching result data generated in the step S2 is loaded or read into the processing environment. And preprocessing the bad line pattern matching result data, such as removing noise, filtering and the like, so as to enhance the characteristics of the bad line. And carrying out bad line positioning on the mobile phone screen image according to the bad line pattern matching result data by utilizing image processing and computer vision technology. This may involve edge detection, morphological operations, region growing, etc. And determining the position of the bad line on the mobile phone screen, and converting the position information into a coordinate form. And carrying out bad line classification prediction on the bad line positioning position data by using a pre-trained machine learning model or a deep learning model. These models can be classified based on the shape, length, color, etc. characteristics of the bad line. And converting the classification prediction result of the model on the bad line into a standard format for subsequent processing and analysis. And integrating the bad wire positioning position data and the bad wire classification prediction data together to form a complete bad wire information data set. And storing or outputting the generated bad line positioning position data and bad line classification prediction data for subsequent steps.
Step S4: carrying out bad line severity assessment on the bad line classification prediction data to generate bad line severity assessment data; and integrating the defective line positioning position data, the defective line classification prediction data and the defective line severity evaluation data to generate a defective line defect diagnosis report of the mobile phone screen.
In the embodiment of the invention, the bad line classification prediction data is evaluated by using a predefined evaluation standard or index. These evaluation criteria may be based on factors such as length, density, location, etc. of the bad wire. The severity of bad line is quantitatively evaluated by using expert experience or machine learning model. This may involve grading or scoring the bad wire. And integrating the bad wire positioning position data, the bad wire classification prediction data and the bad wire severity evaluation data together to form a complete defect data set. The defect data set is ensured to contain information such as the position, the type, the severity and the like of each bad line. And generating a bad line defect diagnosis report of the mobile phone screen according to the integrated defect data set. Reports may include, but are not limited to, the following: the position and type of the bad wire; severity assessment of each bad line; counting the number of bad wires; other relevant information such as picture examples, etc. The report may be output in text form, or may contain images and charts to provide visual information.
Preferably, step S1 comprises the steps of:
Step S11: acquiring a mobile phone screen image;
Step S12: image denoising is carried out on the mobile phone screen image, and a mobile phone screen denoising image is obtained; contrast enhancement is carried out on the denoising image of the mobile phone screen, and an enhanced image of the mobile phone screen is generated;
Step S13: performing image edge detection on the enhanced image of the mobile phone screen to generate image edge detection data of the mobile phone screen; dividing an image core area of the enhanced image of the mobile phone screen according to the edge detection data of the image of the mobile phone screen to generate an image of the core area of the mobile phone screen;
step S14: extracting a mobile phone screen ROI based on mobile phone screen image edge detection data and a mobile phone screen core region image, and generating a mobile phone screen ROI region image;
Step S15: and carrying out size normalization on the mobile phone screen ROI area image so as to generate a standard mobile phone screen ROI area image.
The invention can acquire the mobile phone screen image to be processed by the step of acquiring the mobile phone screen image which is the beginning of the whole flow, and provides input data for subsequent processing. The image denoising can remove noise interference in the image, so that the image is clearer, and the accuracy of subsequent processing steps is improved. The contrast enhancement can enhance the contrast of the image, so that details in the image are clearer and more prominent, and the visual effect and the recognition of the image are improved. Image edge detection can effectively identify edge information in an image, which has important significance in identifying the shape, outline and the like of an object. The image is subjected to core region division according to the edge detection data, and main content in the image can be separated from the background, so that subsequent processing is more concentrated in an important region, and the processing efficiency and the processing precision are improved. ROI extraction may extract regions of interest from the image according to predefined rules or features, which helps to focus on resources and attention to critical regions, improving the efficiency and accuracy of subsequent processing. The size normalization can unify ROI area images of different sizes to the same size, which helps to eliminate the effect of size differences on subsequent processing, making the processing process more stable and reliable.
As an example of the present invention, referring to fig. 2, the step S1 in this example includes:
Step S11: acquiring a mobile phone screen image;
In the embodiment of the invention, the image data of the mobile phone screen is obtained by directly shooting or recording the mobile phone screen by using the image pickup equipment (such as a camera and a video camera). This method can adjust the photographing effect by controlling parameters of the photographing apparatus (such as exposure, focal length, white balance, etc.). The images to be acquired are displayed on the cell phone screen and then saved as a picture file using a screen capture function (typically by pressing a specific combination key or by system settings). The method is suitable for acquiring static images, such as intercepting application interfaces, webpage contents and the like. The mobile phone camera is accessed through a software application interface (such as CAMERA API in Android or AVFoundation framework in iOS), and image data captured by the camera is acquired in real time. The method can realize the real-time acquisition of the mobile phone screen image, and can control the camera parameters and the image processing process through software. Video of the mobile phone screen is recorded on a computer using screen recording software (such as QuickTime Player, OBS Studio, etc.), and then a required image frame is extracted from the video as a mobile phone screen image. The method is suitable for a scene requiring dynamic display of mobile phone screen operation.
Step S12: image denoising is carried out on the mobile phone screen image, and a mobile phone screen denoising image is obtained; contrast enhancement is carried out on the denoising image of the mobile phone screen, and an enhanced image of the mobile phone screen is generated;
In the embodiment of the invention, the image can be smoothed and noise can be reduced by using Gaussian blur. This can be achieved by applying a gaussian filter, the size of which and standard deviation parameters can be adjusted according to the noise level of the image. The median filter is a nonlinear filter, and takes the median of the pixel values in the neighborhood of each pixel value in the image, so that noise types such as salt and pepper noise are effectively removed. The bilateral filter considers the similarity between the spatial distance and the pixel value, so that the edge and detail information of the image can be kept while removing noise. Histogram equalization is a common method of enhancing contrast by redistributing the pixel values of an image to extend the dynamic range of the image, thereby enhancing the contrast of the image. Unlike global histogram equalization, adaptive histogram equalization performs histogram equalization based on local regions of the image, thereby avoiding the generation of excessive enhanced noise in the image. Contrast stretching enhances the contrast of an image by linearly stretching the gray scale range of the image and uniformly distributing the gray scale values of the image throughout the dynamic range.
Step S13: performing image edge detection on the enhanced image of the mobile phone screen to generate image edge detection data of the mobile phone screen; dividing an image core area of the enhanced image of the mobile phone screen according to the edge detection data of the image of the mobile phone screen to generate an image of the core area of the mobile phone screen;
In the embodiment of the invention, the image edge detection algorithm, such as Sobel, canny and the like, is selected appropriately. These algorithms can highlight edges in the image. And inputting the enhanced image of the mobile phone screen into a selected algorithm, and obtaining image edge detection data. Based on the image edge detection data, a core region in the image is determined. This may be achieved by thresholding, region segmentation, or the like techniques. The use of morphological operations (e.g., dilation, erosion) can be considered to further optimize the edge detection results to ensure that an accurate core region is obtained. And according to the determined core area, masking the non-core area in the enhanced image of the mobile phone screen to make the non-core area transparent or replace the non-core area with background color. The obtained result is the image of the core area of the mobile phone screen. And (3) performing experiments and tests, and adjusting parameters of an image edge detection algorithm to obtain the best effect. The performance optimization when processing a large number of images is considered, so that the algorithm has good execution efficiency in practical application. The image processing flow is integrated into a mobile phone screen enhancement system, so that good cooperation with other modules is ensured. And verifying the performance and effect of the whole system, and carrying out necessary adjustment and optimization.
Step S14: extracting a mobile phone screen ROI based on mobile phone screen image edge detection data and a mobile phone screen core region image, and generating a mobile phone screen ROI region image;
In the embodiment of the invention, the interested edge area is identified and extracted by utilizing the mobile phone screen image edge detection data. This may be achieved by applying a threshold, edge detection algorithm, etc. technique. And determining the core part of the region of interest by using the mobile phone screen core region image so as to ensure that the extracted ROI meets the design requirement. And combining the interested edge area with the core area to obtain the ROI area image of the mobile phone screen. This may be achieved by subjecting the two images to a logical operation (e.g. an AND operation) to preserve their intersection portions. And optimizing the algorithm parameters for extracting the ROI so as to ensure that the region of interest of the mobile phone screen is accurately extracted. The execution efficiency of the algorithm is considered, the processing steps are optimized, and good performance in practical application is ensured.
Step S15: and carrying out size normalization on the mobile phone screen ROI area image so as to generate a standard mobile phone screen ROI area image.
In the embodiment of the invention, the target size of the ROI area image of the standard mobile phone screen is determined. This may be a predefined standard size or a size customized to the application requirements. The image of the ROI area of the cell phone screen is adjusted to the target size using an image processing library or algorithm, such as OpenCV, PIL, etc. Interpolation methods, such as bilinear interpolation, nearest neighbor interpolation, etc., can be used to ensure that image quality is not significantly lost. Ensuring that the adjusted image maintains the aspect ratio of the original image. This can be achieved by maintaining the aspect ratio while resizing, avoiding image distortion. If the adjusted image does not exactly match the target size, a fill or crop operation may be considered. Filling: pixels are added around the image to reach the target size. Cutting: the excess is cropped from the image to the target size. The choice of filling or cropping depends on the retention requirements for the image content. And outputting the adjusted image as a standard mobile phone screen ROI area image for subsequent processing or analysis.
Preferably, step S14 includes the steps of:
step S141: performing core area edge point dispersion on the mobile phone screen core area image based on the mobile phone screen image edge detection data to generate core area edge points; performing edge linking on the edge points of the core area by using a Hough transformation technology to generate edge line segments of the core area;
Step S142: performing morphological filling on the mobile phone screen core region image according to the core region edge line segments to generate a mobile phone screen core region filling ROI image, wherein the morphological filling comprises expansion and corrosion;
Step S143: performing ROI contour extraction on the mobile phone screen core region form filling image according to the core region edge line segments to generate a mobile phone screen ROI contour image;
Step S144: filling the ROI image and the ROI outline image of the mobile phone screen through the core area of the mobile phone screen, and carrying out region-of-interest image segmentation on the core area image of the mobile phone screen to generate a ROI segmented image of the mobile phone screen; and performing ROI region verification on the mobile phone screen ROI segmentation image to generate a mobile phone screen ROI region image.
According to the invention, through edge point dispersion and edge linking, accurate edge characteristics of a core area are extracted, and the shape of a mobile phone screen is better represented. Morphological filling makes the edge feature more complete through expansion and corrosion operations, facilitating subsequent processing and analysis. The extraction of the ROI contour is helpful for understanding the overall shape of the region of interest of the mobile phone screen, and key information is provided for further processing. Through the steps of segmentation and verification, the finally generated ROI area image is ensured to accurately represent the region of interest of the mobile phone screen.
As an example of the present invention, referring to fig. 3, the step S14 in this example includes:
step S141: performing core area edge point dispersion on the mobile phone screen core area image based on the mobile phone screen image edge detection data to generate core area edge points; performing edge linking on the edge points of the core area by using a Hough transformation technology to generate edge line segments of the core area;
In the embodiment of the invention, the edge information is extracted from the mobile phone screen image by using a proper edge detection algorithm (such as Sobel, canny and the like). These algorithms can detect edges from gray level variations between pixels. And determining the core area of the mobile phone screen according to the specific shape and design of the mobile phone screen. This may require predefined rules or be implemented using image segmentation techniques. For the core region image, a discretization method may be employed to extract edge points. This can be achieved by applying a thresholding process to the edge detection result and selecting pixels that meet the condition. For the extracted edge points, the points are linked into edge line segments using a hough transform technique. The hough transform can convert the point set into a parameter space and accomplish edge linking by detecting straight lines or curves in the parameter space. The straight line segment or curve segment in the image can be determined by the parameters obtained by the Hough transform. These segments are edge segments of the core region. After the edge segments are obtained, some optimization and filtering operations may be performed, such as removing segments that are too short or too long, or screening segments that conform to the shape of the core region according to certain rules. The optimized and filtered edge line segments are combined to form a complete edge line segment of the core region.
Step S142: performing morphological filling on the mobile phone screen core region image according to the core region edge line segments to generate a mobile phone screen core region filling ROI image, wherein the morphological filling comprises expansion and corrosion;
In the embodiment of the present invention, the edge line segments of the core area obtained in step S141 are converted into a binary image, wherein the pixel values of the edge line segments are foreground (white), and the pixel values of the rest areas are background (black). Morphological filling is an image processing technique that involves two basic operations, expansion and etching, typically used to fill holes or join approaching objects. The dilation operation may expand the boundary of the foreground object towards the background pixels, making the object larger. Here, the edge segments of the core region will be filled with an inflation operation to ensure that the edge segments are completely closed and cover the entire core region. The erosion operation is opposite to the expansion, which tends to shrink the boundary of the foreground object inward. Here, the inflated image will be corrected using an etching operation to maintain the accuracy of the shape of the core region while eliminating unnecessary expansion that may be introduced by inflation. After the expansion and corrosion operations, the obtained image will be a cell phone screen core area filled ROI image, wherein all pixels of the core area are filled to form a closed area.
Step S143: performing ROI contour extraction on the mobile phone screen core region form filling image according to the core region edge line segments to generate a mobile phone screen ROI contour image;
In the embodiment of the present invention, the image is filled by the morphology of the core region generated according to step S142, which is the result of morphological filling of the core region. Using the core region edge line segments generated in step S141 as references, these line segments represent edge features of the core region. Contour information in the core region morphology filling image may be extracted using a contour extraction algorithm based on edge detection results, such as findContours functions in OpenCV. For each detected contour, it is determined whether it is inside the edge line segment of the core region. If the contour is contained inside an edge line segment, it is marked as an ROI contour. The portion marked as the ROI contour is left in the final ROI contour image, with the rest set as the background. Thus, the ROI outline image of the mobile phone screen is obtained.
Step S144: filling the ROI image and the ROI outline image of the mobile phone screen through the core area of the mobile phone screen, and carrying out region-of-interest image segmentation on the core area image of the mobile phone screen to generate a ROI segmented image of the mobile phone screen; and performing ROI region verification on the mobile phone screen ROI segmentation image to generate a mobile phone screen ROI region image.
In the embodiment of the invention, the mobile phone screen core region filling ROI image and the mobile phone screen ROI outline image are combined, and the mobile phone screen core region image can be segmented by using an image segmentation technology, such as threshold segmentation, region growing or edge-based segmentation, so as to obtain an ROI segmented image. For the generated ROI segmented image, verification is required to ensure that the extracted region is the correct ROI. This may be accomplished by a series of verification steps, such as: and (3) verifying the size of the region: verifying whether the size of the ROI area meets the expectations, excluding too small or too large areas. And (3) shape verification: it is verified whether the shape of the ROI area corresponds to the expected shape, for example, whether the shape of the cell phone screen is met. And (3) position verification: and verifying whether the position of the ROI area is reasonable or not, and whether the ROI area is positioned at the center of a mobile phone screen or other predefined positions or not. Other feature verification: depending on the specific application requirements, further feature verification may be performed, such as color features, texture features, etc. The verified ROI area is marked as a mobile phone screen ROI area according to the ROI area, and the ROI area is contained in a final ROI area image.
Preferably, step S2 comprises the steps of:
Step S21: performing corner detection on the ROI region image of the standard mobile phone screen to obtain the feature data of the ROI region corner;
step S22: generating local region pixel descriptors on the ROI region angular point characteristic data to obtain an ROI region pixel local characteristic image;
Step S23: performing feature matching on the local feature image of the pixel of the ROI based on a nearest neighbor matching algorithm to generate bad line feature matching result data; performing feature vector conversion on the bad line feature matching result data to generate a bad line feature vector set; combining the bad line feature vector sets, and constructing a bad line feature vector map for the standard mobile phone screen ROI region image by utilizing the combined bad line feature vector sets to generate a bad line feature vector map;
step S24: and carrying out bad line pattern matching on the local feature images of the pixels of the ROI based on the bad line feature vector map, and generating bad line pattern matching result data.
The invention carries out corner detection on the standard mobile phone screen ROI region image. Corner points are regions of abrupt change in an image, generally corresponding to important feature points in the image. By detecting the corner points on the mobile phone screen, the potential bad line position or change can be determined. The feature data of these corner points may provide key information required for subsequent processing steps. And generating local region pixel descriptors for the corner points detected in the ROI region. A local feature descriptor is a description of a local region of an image, typically used to represent local structure and texture information of the image. Generating these descriptors helps to capture local features on the cell phone screen, providing data support for subsequent feature matching. And performing feature matching on the local feature images of the pixels of the ROI area by using a nearest neighbor matching algorithm so as to identify possible bad lines. By this step, similar feature points in the image can be matched together, so that the position and shape of the bad line can be determined. And further performing feature vector conversion on the bad line feature matching result data, and combining the feature vector conversion to construct a bad line feature vector map. This map can be considered as a collection of bad line patterns that may appear on the cell phone screen, providing a reference for subsequent pattern matching. And carrying out bad line pattern matching on the pixel local feature image of the ROI based on the generated bad line feature vector map. The purpose of this step is to identify bad line patterns on the cell phone screen and generate corresponding matching result data. And by comparing with the map, the bad line on the mobile phone screen can be detected and repaired.
As an example of the present invention, referring to fig. 4, the step S2 in this example includes:
Step S21: performing corner detection on the ROI region image of the standard mobile phone screen to obtain the feature data of the ROI region corner;
In the embodiment of the invention, the image data of the ROI (region of interest) of the standard mobile phone screen is acquired. This may be captured by a cell phone camera or other imaging device, ensuring that the image quality and resolution are sufficient for efficient corner detection. The image is preprocessed, including operations such as graying and denoising. Converting an image into a gray-scale image can simplify the process and facilitate the extraction of corner points. A corner detection algorithm, such as Harris corner detection algorithm, is applied. The Harris corner detection algorithm identifies possible corners by calculating the intensity variation of the area around each pixel in the image. The result of the detection is the coordinates of the corner points. And obtaining coordinate information of the corner detected in the ROI according to the output of the corner detection algorithm. These coordinate data may be used to represent corner features of the ROI area.
Step S22: generating local region pixel descriptors on the ROI region angular point characteristic data to obtain an ROI region pixel local characteristic image;
In the embodiment of the present invention, a local area (for example, a window of a fixed size centered on the corner point) is defined for each detected corner point. In each local region, pixel descriptors are computed using a selected feature description algorithm. These descriptors should be features that uniquely represent each local region for subsequent matching. And integrating pixel descriptors of all local areas into a local feature image of the pixels of the ROI area. This may be a matrix, wherein each row represents a corner point and its corresponding descriptor.
Step S23: performing feature matching on the local feature image of the pixel of the ROI based on a nearest neighbor matching algorithm to generate bad line feature matching result data; performing feature vector conversion on the bad line feature matching result data to generate a bad line feature vector set; combining the bad line feature vector sets, and constructing a bad line feature vector map for the standard mobile phone screen ROI region image by utilizing the combined bad line feature vector sets to generate a bad line feature vector map;
In the embodiment of the invention, the descriptors in the local feature images of the pixels of the region of interest (ROI) are matched with the descriptors in other images (possibly training data or other acquired images) by using a nearest neighbor matching algorithm (such as KNN, nearest neighbor algorithm). This will generate bad line feature matching result data, i.e. matches the descriptors in each ROI region pixel local feature image with the most similar descriptors. The descriptors in the match result data are converted into feature vectors, which may be the descriptors themselves or encoded as feature vectors in some way, in order to better represent the match result. And combining the feature vector sets of all the matching results to form a comprehensive bad line feature vector set. Thus, bad line characteristics in a plurality of images can be comprehensively considered. And constructing a bad line feature vector map for the standard mobile phone screen ROI area image by using the combined bad line feature vector set. This means that the feature vectors in the feature vector set are applied to the standard mobile phone screen ROI region image to generate a bad line feature vector map, which contains various bad line features possibly existing in the region.
Step S24: and carrying out bad line pattern matching on the local feature images of the pixels of the ROI based on the bad line feature vector map, and generating bad line pattern matching result data.
In the embodiment of the invention, the bad line pattern matching is carried out on the local feature image of the pixels of the ROI region by using the feature vectors in the bad line feature vector map. This can be achieved by calculating the similarity between the feature vector of the ROI area pixel local feature image and each feature vector in the bad line feature vector map. And comparing the local feature image of each pixel of the ROI with the feature vector in the bad line feature vector map, and recording the bad line feature vector and the similarity score which are most similar to the local feature image of each pixel of the ROI. These data may be considered as the result data of bad line pattern matching. It may be desirable to set a similarity threshold to filter out bad line feature vectors that are not similar to the ROI area pixel local feature image. Only matching results with similarity scores above the threshold are considered valid matches. And analyzing and visualizing the generated bad line pattern matching result data. The matching result may be compared to the original image to intuitively understand which areas have bad lines, and further actions may be taken based on the matching result, such as repairing bad lines or performing other processing.
Preferably, step S22 includes the steps of:
step S221: performing pixel color analysis on the standard mobile phone screen ROI region image according to the ROI region angular point characteristic data to generate pixel color data; performing adjacent pixel color difference analysis on the pixel color data to obtain adjacent pixel color difference data;
Step S222: comparing the color difference data of the adjacent pixels with a preset standard pixel color difference threshold value, and when the color difference data of the adjacent pixels is larger than or equal to the preset standard pixel color difference threshold value, marking the corresponding pixels by suspicious bad lines to generate suspicious bad line pixels;
Step S223: carrying out brightness variation fluctuation analysis on suspicious bad line pixels by using a pixel brightness variation analysis formula to generate pixel brightness fluctuation data; converting the fluctuation gradient map of the pixel brightness fluctuation data according to the filter to generate a brightness fluctuation gradient map;
Step S224: carrying out local extremum point marking on the brightness fluctuation gradient map to generate brightness fluctuation local extremum key point data, wherein the brightness fluctuation local extremum key point data comprises pixel position data, gradient amplitude data and brightness fluctuation direction data; carrying out fixed local area definition on the brightness fluctuation local extremum key points to generate a local brightness fluctuation area;
step S225: performing local description sub-calculation on the local brightness fluctuation area according to the pixel position data, the gradient amplitude data and the brightness fluctuation direction data to generate a local description sub-set; and carrying out image collection on the local description subset to generate the local feature image of the pixels of the ROI.
According to the invention, through carrying out pixel color analysis and adjacent pixel color difference analysis on the corner characteristics of the ROI area, the system can identify the pixels possibly having problems and generate suspicious bad line pixels. This helps to mark potential bad line areas, making the subsequent analysis more accurate. The system can more fully understand the fluctuation condition of the pixel brightness by utilizing the pixel brightness change analysis and the brightness fluctuation gradient map generated by the filter. This helps detect and analyze the abnormal brightness change that may be caused by the broken wire. And marking local extremum points in the brightness fluctuation gradient map to generate brightness fluctuation local extremum key point data, then calculating local descriptors, and finally generating the local feature image of the pixels of the ROI. This helps to capture local textures and features in the image, providing more information for subsequent bad line pattern matching. The whole process enables the system to accurately locate the area where the bad line may exist in the image, and the mode and the feature of the bad line can be identified more accurately through the generated local feature image. This is critical for further analysis, repair or alarm. Through setting the threshold value and analyzing the color difference of adjacent pixels, the system can control the condition of marking the suspicious bad line, thereby reducing the false alarm rate. Only when the color difference data is larger than a preset threshold value, the color difference data can be marked as suspicious bad lines, and false alarms caused by normal changes can be eliminated.
In the embodiment of the invention, the corner feature data is extracted from the ROI area image of the standard mobile phone screen by using a corner detection algorithm (such as Harris corner detection). For the extracted corner feature data, pixel color data of corresponding positions are acquired, and color space conversion (such as RGB to HSV) can be adopted to represent colors. The adjacent pixels are traversed, the color differences between them are computed, and Euclidean distance or other color difference metrics may be used. And comparing the color difference data of adjacent pixels with a preset standard pixel color difference threshold value. And marking pixels with color differences exceeding a threshold value, and generating a suspicious bad line pixel set. And carrying out pixel brightness variation fluctuation analysis on the suspicious bad line pixels, and calculating pixel brightness fluctuation data by using a formula. The pixel brightness fluctuation data is processed by a filter, possibly using a gaussian filter or the like, to smooth the data. And converting the processed pixel brightness fluctuation data to generate a brightness fluctuation gradient map. And carrying out local extremum point marking on the brightness fluctuation gradient map to obtain brightness fluctuation local extremum key point data. And fixedly defining a local area according to the brightness fluctuation local extremum key point data, and generating a local brightness fluctuation area. And calculating a local descriptor of each local brightness fluctuation area according to the pixel position data, the gradient amplitude data and the brightness fluctuation direction data. And combining the calculated local description subsets into an image set. And generating a local feature image of the pixels of the ROI area through the combined local descriptors.
Preferably, the pixel brightness change analysis formula in step S223 is specifically as follows:
In the method, in the process of the invention, Expressed as pixel in position (/ >)) And time/>Luminance of/>Represented as input pixel in position (/ >)) And input time/>Luminance of/>Expressed as the degree of blurring in control space,/>Expressed as the degree of ambiguity over control time,/>Expressed as pixel abscissa,/>Expressed as pixel ordinate,/>Expressed as time points,/>Expressed as input pixel abscissa,/>Expressed as input pixel ordinate,/>Represented as the input time point.
The invention analyzes and integrates a pixel brightness change analysis formula, wherein two Gaussian kernel functions in the formula respectively act on the changes in space and time. Spatial Gaussian kernelThe spatial smoothness is controlled. It represents the distance (/ >) from the current pixel position) The closer the pixel (/ >)) The greater the impact on the current pixel, and the less the pixel weight is further away. Temporal Gaussian kernel/>The temporal smoothness is controlled. It means that the closer to the current time/>Is/are > ofThe greater the impact on the current pixel, and the more distant the point in time the less the pixel weight. The integration operation in the formula achieves a weighted average of surrounding pixels in space and time. This weighted average results in a final pixel brightness/>Is a weighted sum of the surrounding pixel intensities, the weights being determined by spatial and temporal gaussian kernels. /(I)And/>The parameters controlling the shape of the Gaussian kernel function directly influence the fluctuation analysis result of the brightness of the pixel. Greater/>And/>The values will result in a larger range of pixels and a longer range of pixel effects on the current pixel, thereby smoothing out the variations in pixel brightness. This helps to remove noise and small-scale luminance variations, highlighting the trend of luminance variations over a large-scale, long-term range. The pixel in position (/ >) can be obtained using pixel brightness variation analysis formulas conventional in the art) And time/>By applying the pixel brightness change analysis formula provided by the invention, the pixel position (/ >) can be calculated more accurately) And time/>Is a luminance of (a) a light source. This smoothing method helps to remove random noise from individual pixels, as the weights of the pixels and time points are reduced farther away, making the result more stable. For large scale, long time range luminance variations, this approach can be better emphasized and analyzed because of their greater weight in the weighted average. For bad line pixels, the final luminance result will reduce their impact as their impact will be smoothed in a weighted average. The pixel brightness change analysis method based on the Gaussian kernel function can effectively smooth brightness change, remove noise and highlight large-scale features, so that the image processing and analysis effects are improved.
Preferably, step S24 includes the steps of:
step S241: carrying out bad line angle analysis on the local characteristic image of the pixels of the ROI area to generate bad line angle data;
Step S242: comparing the bad line angle data with a preset bad line angle range threshold, and marking the corresponding local characteristic image of the pixels of the ROI as a single horizontal bad line mode when the bad line angle data is larger than the bad line angle range threshold; when the bad line angle data is within the bad line angle range threshold, marking the corresponding local characteristic image of the pixels of the ROI as a single oblique bad line mode; when the bad line angle data is smaller than the bad line angle range threshold value, marking the corresponding local characteristic image of the pixels of the ROI area as a single vertical bad line mode;
Step S243: carrying out bad line distribution shape construction based on a single horizontal bad line mode, a single inclined bad line mode and a single vertical bad line mode to generate bad line distribution shape data; grid matching is carried out on the bad line distribution shape data, and a local grid bad line mode is generated;
step S244: carrying out bad line mode integration on a single horizontal bad line mode, a single inclined bad line mode, a single vertical bad line mode and a local grid bad line mode to generate a bad line judging mode; and carrying out pattern matching on the pixel local feature image of the ROI region and the bad line judging pattern based on the bad line feature vector map, and generating bad line pattern matching result data.
The invention determines the angle of a bad line possibly existing in the partial characteristic image of the pixels of the ROI through analyzing the partial characteristic image of the pixels of the ROI. The angle of the bad line is analyzed, so that the trend of the bad line can be determined, and a basis is provided for subsequent classification and identification. And comparing the bad line angle data with a preset threshold value to determine the type of the bad line. Different types of bad lines, such as horizontal, diagonal, vertical, are marked to provide directions for further processing. And constructing distribution shape data of the bad wire based on the marked bad wire mode. Grid matching of the bad line distribution shape data may be performed by matching the shape data with a predefined grid pattern. The distribution situation of the bad line can be more accurately positioned and described by constructing the shape data of the bad line and matching. And integrating the single horizontal, oblique line, vertical line modes and the local grid bad line modes to form a bad line judging mode. And matching the local characteristic image of the pixels of the ROI region with a bad line judging mode based on the bad line characteristic vector map. And different types of bad wire modes are integrated and matched, so that the bad wires can be more accurately identified and positioned, and an accurate data basis is provided for further processing.
In the embodiment of the invention, by determining a region of interest (ROI) from the image, it is possible to be a specific part of the product or a certain region of the image. And (5) carrying out pixel-level analysis on the ROI area, and extracting a local characteristic image. The angle analysis is performed on the local characteristic image of the pixels of the ROI area, which may involve edge detection, hough transformation and other technologies, so as to obtain the angle data of the bad line. And comparing the obtained bad line angle data with a preset bad line angle range threshold value. And marking the local characteristic image of the pixels of the ROI area as a single horizontal, oblique line or vertical bad line mode according to the comparison result. Based on the marked bad line mode, the distribution shape data of the bad line is constructed, and the method such as morphological operation or contour extraction can be involved. And carrying out grid matching on the bad line distribution shape data to obtain a local grid bad line mode. And integrating the single horizontal, oblique line and vertical bad line modes and the local grid bad line modes to form a bad line judging mode. And carrying out pattern matching on the pixel local feature image of the ROI region and the bad line judging pattern based on the bad line feature vector map so as to generate bad line pattern matching result data.
Preferably, step S3 comprises the steps of:
Step S31: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data;
Step S32: carrying out data set division on the bad line positioning position data to obtain a model training set and a model testing set;
Step S33: model training is carried out on the model training set based on a convolutional neural network algorithm, and a bad line classification training model is generated; carrying out model test on the bad line classification training model through a model test set to generate a bad line classification prediction model;
Step S34: and importing the bad line locating position data into a bad line classification prediction model to perform bad line classification prediction, and generating bad line classification prediction data.
The invention can accurately determine the position of the bad wire by positioning the bad wire pattern matching result data, thereby helping subsequent processing and repairing work. The model training set and the test set are obtained by carrying out data set division on the bad line positioning position data, which is helpful for guaranteeing the generalization capability and accuracy of the model. And training the model training set based on a convolutional neural network algorithm to generate a bad line classification training model. And testing through a model test set to obtain a bad line classification prediction model. The bad wires can be effectively classified, and the automation degree of bad wire detection is improved. And importing the bad line positioning position data into a bad line classification prediction model to perform classification prediction, and generating bad line classification prediction data. The method can help to distinguish and identify the bad wires of different types, and improves the classification accuracy. The deep learning algorithm such as convolutional neural network is utilized for training and prediction, so that the processing speed and efficiency can be improved, and the bad line detection and classification process is faster and real-time.
In the embodiment of the invention, the bad line pattern matching result data is collected and can be data obtained by a sensor, an image processing system or other detection equipment. And processing the bad wire pattern matching result data by using a proper algorithm to determine the position information of the bad wire. And (5) arranging the position information of the bad wire into a proper data format for later use. And preprocessing the bad line positioning position data, including removing noise, filling missing values and the like. The preprocessed data is divided into a model training set and a model testing set, and the model training set and the model testing set are generally divided according to a certain proportion so as to ensure the representativeness and the reliability of the training set and the testing set. Model training set data is input into algorithms such as Convolutional Neural Networks (CNNs). And training the model training set by using a deep learning algorithm such as CNN and the like so as to learn the characteristics and classification rules of the bad line. In the training process, the optimal bad line classification training model is obtained through iterative optimization. And importing the bad line locating position data into a trained bad line classification prediction model. And carrying out classification prediction on the bad wire by using the trained model to obtain bad wire classification prediction data. And analyzing and explaining the classification prediction result, and taking corresponding measures to repair the bad wire or perform other subsequent treatments according to the needs.
Preferably, step S4 comprises the steps of:
step S41: carrying out bad line density calculation on bad line classification prediction data according to a bad line density calculation formula to obtain bad line density data;
step S42: carrying out bad line dense map generation on bad line classification prediction data and bad line positioning position data based on bad line density data to obtain a bad line dense detection map; carrying out bad line severity assessment on the bad line dense detection graph to generate bad line severity assessment data;
Step S43: and integrating the defective line positioning position data, the defective line classification prediction data and the defective line severity evaluation data to generate a defective line defect diagnosis report of the mobile phone screen.
According to the invention, through bad line density calculation, the bad line density on the mobile phone screen can be quantitatively estimated, and the bad line distribution concentration degree can be analyzed. The generation of the bad wire density data provides more accurate quantitative information, is beneficial to judging the distribution condition of bad wires, and provides reference basis for subsequent diagnosis and repair. And a bad line dense detection graph is generated based on bad line density data, so that the distribution condition of the bad lines is clear at a glance, and the observation and analysis are convenient. The severity evaluation is carried out on the bad wire dense detection graph, so that the severity of bad wire problems can be intuitively evaluated, and the priority determination and the treatment strategy formulation are facilitated. And integrating the bad wire positioning position data, the bad wire classification prediction data and the bad wire severity evaluation data to form complete defect data. A defective diagnosis report of the defective line of the mobile phone screen is generated, a systematic diagnosis result is provided, and the information including the defective line position, the type, the density, the severity and the like is included, so that a reference basis is provided for maintenance personnel or users.
In the embodiment of the invention, the bad line classification prediction data on the mobile phone screen is collected, which may be obtained through an image processing algorithm or other technologies. And processing the bad line classification prediction data according to a predefined bad line concentration calculation formula to calculate bad line concentration data. This formula may take into account the number of bad wires, the distribution density, and other relevant factors. And storing or outputting the calculated bad line density data for the subsequent steps. And drawing a broken line dense detection graph by using the broken line density data. This may involve converting the intensity information into a visual image representation to visually demonstrate the distribution of bad lines. And evaluating the severity of the bad wires by observing the generated bad wire dense detection graph. This may include quantization indicators such as length, density, etc. of bad lines. The evaluation result will generate bad line severity evaluation data. And integrating the bad line positioning position data, the bad line classification prediction data and the bad line severity evaluation data to form a complete defect data set. And generating a bad line defect diagnosis report of the mobile phone screen by utilizing the integrated data. The report may include information such as the location, type, density, and severity of the bad wire so that a user or service personnel can fully understand the bad wire condition and take appropriate action to repair or process.
Preferably, the bad line density calculation formula in step S41 is specifically as follows:
In the method, in the process of the invention, Expressed as in position/>Bad line concentration at site,/>Expressed as the number of dead pixels,/>Expressed as the rate of increase in concentration near the bad line location,/>Expressed as the growth rate near the bad line location,/>Expressed as bad line concentration adjustment coefficient,/>Expressed as/>Bad line location of data points,/>Expressed as the average of bad line positions,/>Expressed as standard deviation of bad line position,/>Expressed as time points,/>Represented as the abscissa of bad line,/>Represented as the ordinate of the bad line.
The invention analyzes and integrates a bad line concentration calculation formula, and the principle of the formula is that the concentration of the bad line position is expressed as the weighted sum of two parts: the first part uses Sigmoid function to represent the increase in density as the bad line position approaches the mean. The second part uses a cumulative density function of Gaussian distribution to represent the distribution of bad line positions. In the formulaExpressed as the rate at which the concentration of the control Sigmoid function increases as it approaches the bad line location. Greater/>The value will cause steeper slope of the Sigmoid function near the bad line position and faster increase of the density; smaller/>The value then makes the slope more gradual. /(I)Represented as controlling the growth rate of the Sigmoid function around the bad line location. Greater/>The value means that the Sigmoid function grows more rapidly and the bad line concentration will approach the maximum more rapidly; smaller/>The value will cause the Sigmoid function to grow more slowly. /(I)The contribution degree of the control Gaussian integral term to the overall bad line density is shown. Larger sizeThe value is such that the Gaussian integral term is more and less in overall concentrationThe value reduces its impact on the overall density. The position/>, can be obtained when using a bad line concentration calculation formula conventional in the artThe bad line concentration at the position can be calculated more accurately by applying the bad line concentration calculation formula provided by the inventionBad line concentration at the location. By adjusting/>And/>The density can be more sensitive or smoothly changed near the bad line position, and the actual distribution situation of the bad line is better reflected. /(I)The adjustment of the (a) can make bad wires at different positions have different contributions in the overall density, and the condition of the overall bad wires can be expressed more accurately.
The invention has the beneficial effects that through the subdivision steps, the accuracy of final defect diagnosis can be obviously improved through concentration and accurate processing of each step from image acquisition, ROI region segmentation, pixel feature matching, bad line pattern matching and classification prediction. Especially, the pixel characteristic matching and bad line pattern matching of the local area can accurately identify the fine bad line characteristic, and misdiagnosis and missed diagnosis are reduced. The whole flow can realize high automation, reduce manual intervention and improve diagnosis speed and efficiency. The automated process can process a large amount of data, and is suitable for the rapid detection requirement of a production line or a maintenance center. The generated defective diagnosis report of the defective wire of the mobile phone screen not only contains the position information of the defective wire, but also integrates the type, the density and the severity of the defective wire, provides comprehensive and detailed defect information for maintenance personnel, and is beneficial to making a more accurate and effective maintenance plan. By accurate defect diagnosis, it is better to decide whether to perform screen repair or replacement, thereby optimizing the cost expenditure. For minor problems, this may be solved by simple maintenance, while for severe wire defects, the screen may need to be replaced. The screen broken line problem can be accurately and rapidly diagnosed and solved, and the satisfaction degree of users on products and services can be remarkably improved. This is important to maintain brand reputation and user loyalty. The generated detailed diagnostic report is not only used for current defect repair, but also can be used as precious data resource for quality control and product improvement. By analyzing this data, manufacturers can find potential problems in the product design or manufacturing process and optimize accordingly. Therefore, the invention performs the region segmentation of the interested region on the mobile phone screen image, adopts the methods of pixel characteristic matching and pattern matching, performs classification prediction while positioning the bad line, namely judges the type and the severity of the bad line, and improves the precision and the efficiency of bad line diagnosis.
The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
The foregoing is only a specific embodiment of the invention to enable those skilled in the art to understand or practice the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (5)

1. The mobile phone screen bad line defect diagnosis method based on machine vision is characterized by comprising the following steps of:
Step S1: acquiring a mobile phone screen image; performing ROI region segmentation on the mobile phone screen image to generate a standard mobile phone screen ROI region image;
Step S2: carrying out local region pixel characteristic matching on the standard mobile phone screen ROI region image to generate bad line characteristic matching result data; carrying out bad line pattern matching on the bad line characteristic matching result data to generate bad line pattern matching result data; step S2 comprises the steps of:
Step S21: performing corner detection on the ROI region image of the standard mobile phone screen to obtain the feature data of the ROI region corner;
Step S22: generating local region pixel descriptors on the ROI region angular point characteristic data to obtain an ROI region pixel local characteristic image; step S22 includes the steps of:
step S221: performing pixel color analysis on the standard mobile phone screen ROI region image according to the ROI region angular point characteristic data to generate pixel color data; performing adjacent pixel color difference analysis on the pixel color data to obtain adjacent pixel color difference data;
Step S222: comparing the color difference data of the adjacent pixels with a preset standard pixel color difference threshold value, and when the color difference data of the adjacent pixels is larger than or equal to the preset standard pixel color difference threshold value, marking the corresponding pixels by suspicious bad lines to generate suspicious bad line pixels;
Step S223: carrying out brightness variation fluctuation analysis on suspicious bad line pixels by using a pixel brightness variation analysis formula to generate pixel brightness fluctuation data; converting the fluctuation gradient map of the pixel brightness fluctuation data according to the filter to generate a brightness fluctuation gradient map; the pixel brightness variation analysis formula in step S223 is as follows:
In the method, in the process of the invention, Expressed as pixel in position (/ >)) And time/>Luminance of/>Represented as input pixel in position (/ >)) And input time/>Luminance of/>Expressed as the degree of blurring in control space,/>Expressed as the degree of ambiguity over control time,/>Expressed as pixel abscissa,/>Expressed as pixel ordinate,/>Expressed as time points,/>Expressed as input pixel abscissa,/>Expressed as input pixel ordinate,/>Represented as an input time point;
Step S224: carrying out local extremum point marking on the brightness fluctuation gradient map to generate brightness fluctuation local extremum key point data, wherein the brightness fluctuation local extremum key point data comprises pixel position data, gradient amplitude data and brightness fluctuation direction data; carrying out fixed local area definition on the brightness fluctuation local extremum key points to generate a local brightness fluctuation area;
step S225: performing local description sub-calculation on the local brightness fluctuation area according to the pixel position data, the gradient amplitude data and the brightness fluctuation direction data to generate a local description sub-set; carrying out image collection on the local description subset to generate a local feature image of pixels of the ROI;
Step S23: performing feature matching on the local feature image of the pixel of the ROI based on a nearest neighbor matching algorithm to generate bad line feature matching result data; performing feature vector conversion on the bad line feature matching result data to generate a bad line feature vector set; combining the bad line feature vector sets, and constructing a bad line feature vector map for the standard mobile phone screen ROI region image by utilizing the combined bad line feature vector sets to generate a bad line feature vector map;
step S24: carrying out bad line pattern matching on the local feature images of the pixels of the ROI region based on the bad line feature vector map, and generating bad line pattern matching result data;
Step S3: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data; carrying out bad line classification prediction on the bad line positioning position data to generate bad line classification prediction data;
Step S4: carrying out bad line severity assessment on the bad line classification prediction data to generate bad line severity assessment data; carrying out defect data integration on the bad wire positioning position data, the bad wire classification prediction data and the bad wire severity evaluation data, thereby generating a bad wire defect diagnosis report of the mobile phone screen; step S4 comprises the steps of:
Step S41: carrying out bad line density calculation on bad line classification prediction data according to a bad line density calculation formula to obtain bad line density data; the bad line density calculation formula in step S41 is as follows:
In the method, in the process of the invention, Expressed as in position/>Bad line concentration at site,/>Expressed as the number of dead pixels,/>Expressed as the rate of increase in concentration near the bad line location,/>Expressed as the growth rate near the bad line location,/>Expressed as bad line concentration adjustment coefficient,/>Expressed as/>Bad line location of data points,/>Expressed as the average of bad line positions,/>Expressed as standard deviation of bad line position,/>Expressed as time points,/>Represented as the abscissa of bad line,/>An ordinate indicated as bad line;
step S42: carrying out bad line dense map generation on bad line classification prediction data and bad line positioning position data based on bad line density data to obtain a bad line dense detection map; carrying out bad line severity assessment on the bad line dense detection graph to generate bad line severity assessment data;
Step S43: and integrating the defective line positioning position data, the defective line classification prediction data and the defective line severity evaluation data to generate a defective line defect diagnosis report of the mobile phone screen.
2. The machine vision-based mobile phone screen bad line defect diagnosis method according to claim 1, wherein the step S1 comprises the steps of:
Step S11: acquiring a mobile phone screen image;
Step S12: image denoising is carried out on the mobile phone screen image, and a mobile phone screen denoising image is obtained; contrast enhancement is carried out on the denoising image of the mobile phone screen, and an enhanced image of the mobile phone screen is generated;
Step S13: performing image edge detection on the enhanced image of the mobile phone screen to generate image edge detection data of the mobile phone screen; dividing an image core area of the enhanced image of the mobile phone screen according to the edge detection data of the image of the mobile phone screen to generate an image of the core area of the mobile phone screen;
step S14: extracting a mobile phone screen ROI based on mobile phone screen image edge detection data and a mobile phone screen core region image, and generating a mobile phone screen ROI region image;
Step S15: and carrying out size normalization on the mobile phone screen ROI area image so as to generate a standard mobile phone screen ROI area image.
3. The machine vision-based mobile phone screen bad line defect diagnosis method according to claim 2, wherein the step S14 comprises the steps of:
step S141: performing core area edge point dispersion on the mobile phone screen core area image based on the mobile phone screen image edge detection data to generate core area edge points; performing edge linking on the edge points of the core area by using a Hough transformation technology to generate edge line segments of the core area;
Step S142: performing morphological filling on the mobile phone screen core region image according to the core region edge line segments to generate a mobile phone screen core region filling ROI image, wherein the morphological filling comprises expansion and corrosion;
Step S143: performing ROI contour extraction on the mobile phone screen core region form filling image according to the core region edge line segments to generate a mobile phone screen ROI contour image;
Step S144: filling the ROI image and the ROI outline image of the mobile phone screen through the core area of the mobile phone screen, and carrying out region-of-interest image segmentation on the core area image of the mobile phone screen to generate a ROI segmented image of the mobile phone screen; and performing ROI region verification on the mobile phone screen ROI segmentation image to generate a mobile phone screen ROI region image.
4. The machine vision-based mobile phone screen bad line defect diagnosis method according to claim 1, wherein the step S24 comprises the steps of:
step S241: carrying out bad line angle analysis on the local characteristic image of the pixels of the ROI area to generate bad line angle data;
Step S242: comparing the bad line angle data with a preset bad line angle range threshold, and marking the corresponding local characteristic image of the pixels of the ROI as a single horizontal bad line mode when the bad line angle data is larger than the bad line angle range threshold; when the bad line angle data is within the bad line angle range threshold, marking the corresponding local characteristic image of the pixels of the ROI as a single oblique bad line mode; when the bad line angle data is smaller than the bad line angle range threshold value, marking the corresponding local characteristic image of the pixels of the ROI area as a single vertical bad line mode;
Step S243: carrying out bad line distribution shape construction based on a single horizontal bad line mode, a single inclined bad line mode and a single vertical bad line mode to generate bad line distribution shape data; grid matching is carried out on the bad line distribution shape data, and a local grid bad line mode is generated;
step S244: carrying out bad line mode integration on a single horizontal bad line mode, a single inclined bad line mode, a single vertical bad line mode and a local grid bad line mode to generate a bad line judging mode; and carrying out pattern matching on the pixel local feature image of the ROI region and the bad line judging pattern based on the bad line feature vector map, and generating bad line pattern matching result data.
5. The machine vision-based mobile phone screen bad line defect diagnosis method according to claim 1, wherein the step S3 comprises the steps of:
Step S31: carrying out bad line positioning on the bad line pattern matching result data to generate bad line positioning position data;
Step S32: carrying out data set division on the bad line positioning position data to obtain a model training set and a model testing set;
Step S33: model training is carried out on the model training set based on a convolutional neural network algorithm, and a bad line classification training model is generated; carrying out model test on the bad line classification training model through a model test set to generate a bad line classification prediction model;
Step S34: and importing the bad line locating position data into a bad line classification prediction model to perform bad line classification prediction, and generating bad line classification prediction data.
CN202410309774.6A 2024-03-19 2024-03-19 Mobile phone screen bad line defect diagnosis method based on machine vision Active CN117911409B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410309774.6A CN117911409B (en) 2024-03-19 2024-03-19 Mobile phone screen bad line defect diagnosis method based on machine vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410309774.6A CN117911409B (en) 2024-03-19 2024-03-19 Mobile phone screen bad line defect diagnosis method based on machine vision

Publications (2)

Publication Number Publication Date
CN117911409A CN117911409A (en) 2024-04-19
CN117911409B true CN117911409B (en) 2024-05-31

Family

ID=90692516

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410309774.6A Active CN117911409B (en) 2024-03-19 2024-03-19 Mobile phone screen bad line defect diagnosis method based on machine vision

Country Status (1)

Country Link
CN (1) CN117911409B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101715050A (en) * 2009-11-20 2010-05-26 北京中星微电子有限公司 Method and system for detecting dead pixels of image sensor
CN104749184A (en) * 2013-12-31 2015-07-01 研祥智能科技股份有限公司 Automatic optical detection method and automatic optical detection system
CN105301810A (en) * 2015-11-24 2016-02-03 上海斐讯数据通信技术有限公司 Screen defect detecting method and screen defect detecting device
CN110276759A (en) * 2019-06-28 2019-09-24 东北大学 A kind of bad line defect diagnostic method of Mobile phone screen based on machine vision
CN110648626A (en) * 2019-09-30 2020-01-03 深圳市奥拓电子股份有限公司 Method and system for correcting bright and dark lines of LED display screen and storage medium thereof
CN111524101A (en) * 2020-04-10 2020-08-11 苏州赛腾精密电子股份有限公司 Electronic screen defect detection method based on machine vision technology
CN111724375A (en) * 2020-06-22 2020-09-29 中国科学院大学 Screen detection method and system
CN115100207A (en) * 2022-08-26 2022-09-23 北京恒新天创科技有限公司 Detection system and detection method based on machine vision
CN116664551A (en) * 2023-07-21 2023-08-29 深圳市长荣科机电设备有限公司 Display screen detection method, device, equipment and storage medium based on machine vision
CN117132797A (en) * 2023-09-12 2023-11-28 苏州佳智彩光电科技有限公司 POLMARK positioning detection method and system for LCD display screen
CN117314826A (en) * 2023-08-28 2023-12-29 广州千筱母婴用品有限公司 Performance detection method of display screen
CN117670886A (en) * 2024-02-01 2024-03-08 深圳市欧灵科技有限公司 Display screen defect detection method, device, equipment and storage medium

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101715050A (en) * 2009-11-20 2010-05-26 北京中星微电子有限公司 Method and system for detecting dead pixels of image sensor
CN104749184A (en) * 2013-12-31 2015-07-01 研祥智能科技股份有限公司 Automatic optical detection method and automatic optical detection system
CN105301810A (en) * 2015-11-24 2016-02-03 上海斐讯数据通信技术有限公司 Screen defect detecting method and screen defect detecting device
CN110276759A (en) * 2019-06-28 2019-09-24 东北大学 A kind of bad line defect diagnostic method of Mobile phone screen based on machine vision
CN110648626A (en) * 2019-09-30 2020-01-03 深圳市奥拓电子股份有限公司 Method and system for correcting bright and dark lines of LED display screen and storage medium thereof
CN111524101A (en) * 2020-04-10 2020-08-11 苏州赛腾精密电子股份有限公司 Electronic screen defect detection method based on machine vision technology
CN111724375A (en) * 2020-06-22 2020-09-29 中国科学院大学 Screen detection method and system
CN115100207A (en) * 2022-08-26 2022-09-23 北京恒新天创科技有限公司 Detection system and detection method based on machine vision
CN116664551A (en) * 2023-07-21 2023-08-29 深圳市长荣科机电设备有限公司 Display screen detection method, device, equipment and storage medium based on machine vision
CN117314826A (en) * 2023-08-28 2023-12-29 广州千筱母婴用品有限公司 Performance detection method of display screen
CN117132797A (en) * 2023-09-12 2023-11-28 苏州佳智彩光电科技有限公司 POLMARK positioning detection method and system for LCD display screen
CN117670886A (en) * 2024-02-01 2024-03-08 深圳市欧灵科技有限公司 Display screen defect detection method, device, equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
以CCD为基础的手机液晶屏缺陷检测;董洪涛;;通讯世界;20171225(第24期);全文 *

Also Published As

Publication number Publication date
CN117911409A (en) 2024-04-19

Similar Documents

Publication Publication Date Title
CN113592845A (en) Defect detection method and device for battery coating and storage medium
CN110148130B (en) Method and device for detecting part defects
WO2019104767A1 (en) Fabric defect detection method based on deep convolutional neural network and visual saliency
CN107909081B (en) Method for quickly acquiring and quickly calibrating image data set in deep learning
CN113160192A (en) Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background
CN111507426B (en) Non-reference image quality grading evaluation method and device based on visual fusion characteristics
CN111968095B (en) Product surface defect detection method, system, device and medium
CN114926407A (en) Steel surface defect detection system based on deep learning
CN116664565A (en) Hidden crack detection method and system for photovoltaic solar cell
CN110689003A (en) Low-illumination imaging license plate recognition method and system, computer equipment and storage medium
CN113537037A (en) Pavement disease identification method, system, electronic device and storage medium
CN115830004A (en) Surface defect detection method, device, computer equipment and storage medium
CN117635615B (en) Defect detection method and system for realizing punching die based on deep learning
CN117314826A (en) Performance detection method of display screen
CN117496532A (en) Intelligent recognition tool based on 0CR
CN117036314A (en) Method for detecting oxidation area of high-density flexible IC substrate
CN110188693B (en) Improved complex environment vehicle feature extraction and parking discrimination method
CN117911409B (en) Mobile phone screen bad line defect diagnosis method based on machine vision
CN113920068B (en) Body part detection method and device based on artificial intelligence and electronic equipment
CN116129417A (en) Digital instrument reading detection method based on low-quality image
CN115131355A (en) Intelligent method for detecting abnormality of waterproof cloth by using data of electronic equipment
CN114494142A (en) Mobile terminal middle frame defect detection method and device based on deep learning
CN117474916B (en) Image detection method, electronic equipment and storage medium
CN117975175B (en) Plastic pipeline appearance defect detection method based on machine vision
CN117934453B (en) Method and system for diagnosing defects of backlight foreign matters of mobile phone screen

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant