CN110286126B - Wafer surface defect regional detection method based on visual image - Google Patents
Wafer surface defect regional detection method based on visual image Download PDFInfo
- Publication number
- CN110286126B CN110286126B CN201910524321.4A CN201910524321A CN110286126B CN 110286126 B CN110286126 B CN 110286126B CN 201910524321 A CN201910524321 A CN 201910524321A CN 110286126 B CN110286126 B CN 110286126B
- Authority
- CN
- China
- Prior art keywords
- image
- wafer
- contour
- defect
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/94—Investigating contamination, e.g. dust
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/9501—Semiconductor wafers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
- G01N21/9501—Semiconductor wafers
- G01N21/9505—Wafer internal defects, e.g. microcracks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8861—Determining coordinates of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8867—Grading and classifying of flaws using sequentially two or more inspection runs, e.g. coarse and fine, or detecting then analysing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8874—Taking dimensions of defect into account
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Testing Or Measuring Of Semiconductors Or The Like (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a wafer surface defect regional detection method based on visual images, and belongs to the technical field of semiconductor defect detection and image processing. Firstly, dividing a color wafer surface image into regions respectively including an outer contour region, an oxide film region and a grain region, and individually detecting the defects of each region; the defects which can be detected in the outer contour area comprise edge breakage and corner breakage defects; the defects which can be detected by the oxide film area comprise oxide film deletion, oxide film cross area and oxide film sawtooth; the defects which can be detected in the crystal grain area comprise stains, red ink, crystal grain loss, scratches, large-area oxide film defects in the area and texture defects; the invention realizes that a sample set does not need to be trained, is particularly suitable for surface defect detection of specific types of wafers in the initial stage and small batch of wafer production, has obviously improved efficiency compared with manpower, and has important guiding significance for specific type detection.
Description
Technical Field
The invention relates to the technical field of semiconductor defect detection and image processing, in particular to a wafer surface defect regional detection method based on visual images.
Background
The wafer is a thin wafer cut from a single crystal silicon, and is a semi-finished product, which is subjected to deep processing to obtain chips that can be used for actual production, mainly for producing integrated circuits. China is the biggest semiconductor consuming country today, accounting for about 45% of the semiconductors worldwide, only 15% of which are autonomous in China, and is expected to increase to 55% by year 2020, with countries planning to invest billions of dollars for semiconductor production in the next decade. Meanwhile, the self-support of the chip plays a strategic role in the development of the country, particularly the wafer technology blockage of China in the recent United states, and the chip-related products are in more urgent need. Today, wafer defect detection is mainly divided into 4 types: pure manual detection; detecting by combining manpower and images; template matching and machine learning detection, and defect detection is mainly carried out in China in an artificial mode. Therefore, it is significant to develop and design an accurate and rapid wafer detection algorithm, wherein the core technology is an image recognition and judgment algorithm. The manual judgment easily causes the false detection condition due to the influence of subjective factors, and meanwhile, the efficiency is not high for 1 hour, only about 0.5 ten thousand chips can be detected, and the long-time work cannot be realized. The template matching method comprises the steps of matching the area threshold of a defect region; matching a target image gray level histogram; matching a target image color histogram; histogram correlation matching method; histogram chi-square matching method; histogram intersection matching method and histogram Bhattacharyya distance matching method. The histogram matching is widely applied to matching of images, wherein the main characteristic of the matching is the number of pixel points of different gray values of the images. The algorithm has the main characteristic that the detection method is simple and easy to implement, but the situation that the matching result is high in precision but the two pictures are very different can occur. The perimeter and area are the most widely and mature methods, the principle of perimeter calculation is to connect the contour lines of the wafers, compare the contour lines with a set threshold, and if the length of the measured contour line is within a specified range, the wafer is intact. The judgment precision of the two algorithms is obviously improved compared with that of histogram statistics, the general precision can be met, but the detection precision is often very low if the wafer has a tiny edge breakage condition. Since the number of wafer defects is as many as 19, the accuracy of template matching is often low, but the method based on machine learning has been well applied in the field of cloth inspection and the like for 60 defects, but it is generally impractical to accurately inspect the wafer defects with such many types of defects by using a large number of training set samples. The machine learning method is far superior to template matching in accuracy, and is higher in efficiency than manual processing in combination with automation. Theoretically, the defect detection method based on artificial intelligence recognition can achieve about 2-5 thousands of defects per hour, meanwhile, the equipment can operate in 24 hours under all working conditions, and the recognition efficiency can be greatly improved. In combination with the advantages and disadvantages of existing methods, a novel algorithm for wafer surface defect detection is designed herein that satisfies this type of wafer micro-defect.
The size of the wafer researched by the patent is 1mm-5mm, the wafer is mainly used for manufacturing voltage stabilizing and ballasting chips, and generally the wafer has about 20% -30% of defective rate. The defect types of the chips are about 15, which respectively correspond to three different areas of the chips, and since the defects of the chips can appear at any positions of the three areas and the sizes of the areas are random, the three areas are separately researched, and the detection and classification of the wafers can be realized by detecting the defects of each area.
Disclosure of Invention
The invention aims to provide a visual image-based wafer surface defect regional detection method, which aims to solve the problem of low accuracy in wafer multi-type defect detection in the prior art and provide an instructive detection method for any similar defect and working condition detection.
A wafer surface defect regional detection method based on visual images comprises the following steps:
obtaining a rectangular wafer original color image based on a macro prime lens continuous exposure and global scanning mode, wherein the wafer is a white background;
dividing the original color image of the wafer into 3 research areas, namely an outer contour area, an oxide film area and a grain area, according to the characteristics of the defect types of the wafer; the defect detection of the outer contour area comprises rough detection and fine detection;
the outer contour rough detection step: background removing operation of the original color image of the wafer, firstly removing isolated interference points by adopting median filtering, then traversing all pixels of the background to record a gray value corresponding to the minimum point in RGB gray values, and setting a linear transformation coefficient according to the corresponding gray value;
converting the RGB color wafer image into a gray image, then carrying out binarization to obtain a binary image, carrying out outmost layer contour extraction, simultaneously calculating the area surrounded by the outmost layer contour, and carrying out fine inspection if the area is within a set threshold value range;
the fine inspection method comprises the steps of determining the inclination angle of a wafer and the geometric center of the wafer in an image based on a binary image in rough inspection, rotating around the geometric center of the wafer in the image to enable the vertical edge of the wafer to be perpendicular to the bottom edge of the image, extracting the profile while extracting the horizontal edge and the vertical edge, calculating the lengths of the horizontal edge and the vertical edge while comparing with a threshold, and if the lengths of the horizontal edge and the vertical edge are smaller than the threshold, the outer contour has no defects;
detecting the oxide film defect, carrying out global contour extraction on the binary image obtained by rough detection based on the contour line perimeter surrounded by the oxide film, cutting the global contour image based on the coordinates of the corner points according to the four corner points of the wafer without the outer contour defect to obtain an image only with the oxide film contour line as the outermost contour, solving the surrounding perimeter of the outermost contour of the contour image and comparing the surrounding perimeter with a threshold value to judge the oxide film defect;
and detecting the defects of the grain regions, extracting the global contour of the binary image obtained by rough detection, cutting the global contour image based on coordinates of the corner points according to the four corner points of the wafer without the outer contour defects to obtain a contour image only containing the features of the grain regions, calculating the surrounding perimeter of the global outer contour of the image, and comparing the surrounding perimeter with a threshold value to judge whether the defect of the grain regions exists.
The invention carries out independent detection research on the defect types of the three research areas, thereby achieving the purpose of detection. For the detection of the outer contour area, the main corresponding defect types comprise edge breakage and corner breakage, the adopted method is based on a two-step screening mode, namely, the area surrounded by the outermost contour is compared with a standard threshold value for initial detection, and if the detected wafer has defects, the defects are directly judged. If the detected wafer has no defects, the second step of fine detection is carried out, the fine detection can detect tiny edge breakage and corner breakage defects, the precision is higher compared with an area mode, and the actual production requirements can be met.
And secondly, judging based on the standard straight line lengths corresponding to four edges of the wafer, if a defect exists in the middle of one edge of the wafer, dividing a complete straight line into the defective parts, and if a corner is missing, enabling the total length of the corresponding line to be smaller than a threshold value. The side length of the accurate measurement result is an important prerequisite, and meanwhile, large deviation cannot exist.
Further, in the detection of the oxide film defect, the global contour image is cut based on the coordinates of the corner points, namely the horizontal and vertical coordinates of the point a at the upper left corner are plus 15; the lower left corner b has the abscissa +15 and the ordinate-15; the upper right corner point c has the abscissa of-15 and the ordinate of + 15; the value of the point d at the lower right corner, the abscissa is-15, and the ordinate is-15, can be adjusted according to different batches; in the defect detection of the grain region, a global contour image is cut based on the coordinates of the corner points, wherein the coordinates of the point a at the upper left corner are plus 75; point b at the lower left corner has abscissa +75 and ordinate-75; the upper right corner point c has the abscissa of-75 and the ordinate of + 75; the lower right-hand point d, abscissa-75 and ordinate-75, can be adjusted in value according to the batch.
Further, performing median filtering on the color background, and traversing all pixels after the median filtering to find a value pixel (x, y) with the minimum gray value in 3 channels; the quotient of pixel (x, y) and 255 is obtained to obtain a contrast enhancement coefficient as shown in formula (2), global processing is carried out based on a linear processing transformation formula g (x, y) in image processing, wherein the transformation formula g (x, y) is alpha f (x, y) + beta, as shown in formula (1),
IBGR(i,j)＝(fB(i,j),fG(i,j),fR(i,j)) (4)
wherein f isb(i,j)、fg(i,j)、fr(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the original image; f. ofB(i,j)、fG(i,j)、fR(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the processed image;
representing the contrast enhancement factor.
Furthermore, in the outline defect detection rough detection, outline extraction is based on gradient, and a Ca nny edge detection operator and a first-order gradient operator used by Canny in a two-dimensional space are adopted
For two-dimensional continuous data, if there is a function f (x, y), partial derivatives g in x and y directions can be obtained respectivelyx、gy：
The magnitude and direction of the gradient can be expressed as:
the magnitude and direction of the gradient calculation can be simplified as:
M(x,y)≈|gx|+|gy| (9)
the direction is as follows:
the contour extraction is based on gradient, a large number of high-frequency regions exist at the edge of the outer contour of the wafer, and if s obel and robert operators are adopted, the edge with the width larger than one pixel point appears, so that high-precision calculation is not facilitated. The Canny edge detection operator with non-maximum suppression is adopted, and the Canny edge detection operator only has the width of one pixel point, so that calculation and judgment are facilitated.
Furthermore, in the outer contour defect detection and fine detection, the center of rotation is the center of the wafer in the image but not the center of the image; extracting horizontal and vertical edges of the wafer based on morphological operation, wherein structural elements of the horizontal edge are of a horizontal structure, and structural elements of the vertical edge are of a vertical structure; respectively solving the length of each extracted independent straight line segment, and comparing the length with a standard threshold value; and if the corresponding length of the obtained horizontal side and the vertical side is smaller than a fixed threshold value, the outer contour is intact and has no defects.
Further, the implementation step of the verticalization of the image at any angle comprises the following steps:
the top edge a1 of the wafer can be calculated according to any input image, the coordinates of 11 points which are equidistantly distributed on the a1 side are obtained to be b1, b2, b3, b4, b5, b6, b7, b8, b9, b10 and b11 respectively, a straight line equation of the corresponding side is fitted according to the points, the inclination angle alpha 1 epsilon [0,90 ] of the corresponding straight line of the a1 side is obtained according to the equation, and the straight line equations of the corresponding sides of the other three sides a2, a3 and a4 are respectively obtained to be la2，la3andla4，
la1＝fitting(b1,b2,b3,b4,b5,b6,b7,b7,b8,b9,b10,b11) (11)
According to la1，la3And la4，la2，la3And la4The linear equation respectively calculates the coordinate x and the value y of the intersection points, the intersection points are marked as points A, B, C and D in a sequence from small to large, the coordinate of the rotation geometric center point of the wafer is calculated according to the coordinates of the point A and the point C and is marked as O1, and the solving mode of the x and y coordinates of the geometric center point is as follows:
xO1＝(yC-yA)/2+yA (13)
yO1＝(xC-xA)/2+xA (14)
the rotation direction can be determined as the positive direction by rotating counterclockwise and the negative direction by rotating clockwise,
(1)0°≤α1≤45°
Angle of rotation＝-α1 (15)
(2)45°＜α1＜90°
Angle of rotation＝(90-α1)。 (16)
further, the defect detecting step detects the defects of the oxide film region only under the condition that the outer contour region has no defects, and detects the defects of the crystal grain region only under the condition that the oxide film region has no defects; and the fine inspection is performed only under the condition that the coarse inspection is qualified.
Further, binarization processing in the outline defect detection rough inspection adopts fixed threshold binarization, and median filtering adopts structural elements with the length of 3 and the width of 1 to carry out median filtering.
Further, in the coarse detection of the outer contour defect, the contour information extracted from the outer contour includes an outermost contour, an oxide film contour and a contour of a defect feature in a grain region, the oxide film contour can be directly used for detecting the defect in the oxide film region, and the contour of the defect in the grain region can be directly used for detecting the defect in the grain region.
The fixed threshold value can ensure the extraction precision, because the gray values of the background pixel points are all 255, the extraction cannot be influenced. And (3) carrying out median filtering by adopting the structural elements with the length of 3 and the width of 1 to eliminate the interference of isolated points in the binary image.
Further, the RGB color wafer image is converted into the grayscale image by 3-channel averaging, i.e., I (x, y) ═ 1/3 × R (x, y) +1/3 × G (x, y) +1/3 × B (x, y).
The defects which can be detected in the outer contour area comprise edge breakage and corner breakage defects; the defects which can be detected by the oxide film area comprise oxide film deletion, oxide film cross area and oxide film sawtooth; the defects which can be detected in the crystal grain area comprise stains, red ink, crystal grain loss, scratches, large-area oxide film defects in the area and texture defects; the invention realizes that a sample set does not need to be trained, is particularly suitable for surface defect detection of specific types of wafers in the initial stage and small batch of wafer production, has obviously improved efficiency compared with manpower, and has important guiding significance for specific type detection.
Drawings
FIG. 1 is a division of a wafer analysis area;
FIG. 2 is a background effect removing diagram;
FIG. 3 is a view of a binarized wafer surface;
FIG. 4 extraction effect of the contours;
FIG. 5 is a schematic diagram of rotation;
FIG. 6 is a rotation effect diagram;
FIG. 7 extraction of the line corresponding to the outermost profile;
FIG. 8 is a segmentation of the oxide film region contour;
fig. 9 is a division of the outline of the die region.
Detailed Description
The technical scheme of the invention is further explained by combining the drawings in the specification.
1 computer hardware and software configuration: the WINSOWS 764 bit system is adopted, and the processor is AMD FX-8350, the memory DDR 312G, the hard disk SSD 850 and the graphics card sapphire R9390. The image analysis processing software is based on Visual Studio 15 as a platform, and the OpenCV version of the image processing library is 3.4.4. The camera is a large constant image, a global exposure mode and an annular LED lamp light source.
2 dividing the wafer surface image into three areas according to the structural characteristics of the wafer surface image, wherein each area has a corresponding type of defect type and respectively comprises an outer contour area, an oxide film area and a crystal grain area. And the defect types of the three areas are independently researched and analyzed, so that the purpose of detection is achieved. The main basis for the segmentation is the correspondence between the four corners of the same wafer size and the image coordinate system of each region. The division form of the wafer analysis area is shown in fig. 1.
3 outer contour region detection
The method is based on a two-step screening mode, namely, comparing the area surrounded by the outline of the outermost layer with a standard threshold value for initial detection, and directly judging whether the detected wafer has defects. If the detected wafer has no defects, the second step of fine detection is carried out, the fine detection can detect tiny edge breakage and corner breakage defects, the precision is higher compared with an area mode, and the actual production requirements can be met. And secondly, judging based on the standard straight line lengths corresponding to four edges of the wafer, if a defect exists in the middle of one edge of the wafer, dividing a complete straight line into the defective parts, and if a corner is missing, enabling the total length of the corresponding line to be smaller than a threshold value. However, it is an important prerequisite to obtain an accurate side length, and at the same time, there is no large deviation.
The first step of the initial detection of the outer contour area comprises the following steps:
the processing mode is that firstly, discrete salt and pepper noise is removed by adopting an edge-preserving algorithm, namely a median filtering algorithm, so that the phenomenon that the whole image of the wafer is bright when isolated points take values is prevented. And traversing all the pixels of the background, finding the minimum value point corresponding to the gray value of the color image 3 channel, and recording the specific gray value. The quotient of the value and 255 is obtained as a contrast enhancement coefficient, and global processing is performed based on a linear processing transformation formula g (x, y) ═ α f (x, y) + β in image processing, so that pixel values of all points of the background can be transformed into 255, and the contrast of the whole image can be improved to a certain extent. The principle of background removal is to remove background noise without obscuring wafer profile information. When the wafer image is shot, a white background, namely BGR (255 ) is adopted, and when noise interference exists, the value of one or more channels is smaller than 255, so that the minimum value pixel (x, y) of the channel gray scale in the background pixel is found, and the enhancement coefficient is determined based on the minimum value, as shown in formula 1. The pixel points of the whole image are operated as formula 2, and the purpose of removing the background noise is achieved. The effect is shown in figure 2.
IBGR(i,j)＝(fB(i,j),fG(i,j),fR(i,j)) (4)
Wherein f isb(i,j)、fg(i,j)、fr(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the original image; f. ofB(i,j)、fG(i,j)、fR(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the processed image;
representing a contrast enhancement coefficient; i isBGRAnd (i, j) represents the gray value of the corresponding pixel point.
The gray scale of the wafer color image can be converted into the gray scale image by using a tie value method and a different coefficient method, namely, the method that the tie value is used in the gray scale image is that I (x, y) ═ 1/3 × R (x, y) +1/3 × G (x, y) +1/3 × B (x, y).
And (3) binarization processing: the binarization mode adopts fixed threshold binarization, and the fixed threshold can ensure the extraction precision, because the gray values of the background pixel points are 255 completely, the extraction cannot be influenced. Corresponding implementation code threshold (pic6, pic7,220,255, CV _ THRESH _ BINARY);
and (3) binary image filtering processing: and (3) carrying out median filtering by adopting the structural elements with the length of 3 and the width of 1 to eliminate the interference of isolated points. Corresponding implementation code media blue (pic7, pic8,3), the effect after processing is as shown in FIG. 3
Extracting a binary image contour: contour extraction is based on gradient, a large number of high-frequency regions exist at the edge of the outer contour of the wafer, if sobel and robert operators are adopted, the edge with the width larger than one pixel point appears, and the precision of a calculation method is not facilitated. The Canny edge detection operator with non-maximum suppression is adopted, and the Canny detected edge only has the width of one pixel point, so that calculation and judgment are facilitated. Since the initial inspection is performed by only calculating the profile of the outermost layer. First order gradient operator used by Canny in two-dimensional space
For two-dimensional continuous data, if there is a function f (x, y), partial derivatives g in x and y directions can be obtained respectivelyx、gy：
The magnitude and direction of the gradient can be expressed as:
the magnitude and direction of the gradient calculation can be simplified as:
M(x,y)≈|gx|+|gy| (9)
the direction is as follows:
the corresponding implementation code Canny (pic8, pic9,100,255,3), the extraction effect of the outline is as shown in fig. 4.
Solving for the enclosed area of the outermost profile image: and obtaining the area surrounded by the outermost layer outline based on a solving area function con tourArea in OpenCV, and comparing and judging the area with a standard threshold value.
And a second step of implementing the outer contour region fine inspection: and rotating the binary image obtained in the first step, wherein the rotation center of the image is positioned at the geometric center of the wafer in the image, and the determination of the rotation center is based on the center corresponding to the minimum outline moment line surrounded by the wafer in the image. The vertical edge of the rotated wafer in the image is perpendicular to the bottom edge of the image, and the rotated holes are filled with corresponding gray values of 255.
Realization of any-angle image verticality: the top edge of the wafer, namely the edge a1 in the graph, can be calculated according to any input image, the coordinates of 11 points which are equidistantly distributed on the edge a1 can be obtained as b1, b2, b3, b4, b5, b6, b7, b8, b9, b10 and b11, a straight line equation of the corresponding edge is fitted according to the points, the inclination angle alpha 1 epsilon [0,90 ] of the corresponding straight line of the edge a1 is obtained according to the equation, and the straight line equations of the corresponding edges a2, a3 and a4 are respectively la2，la3andla4The rotation principle is explained in connection with fig. 5.
la1＝fitting(b1,b2,b3,b4,b5,b6,b7,b7,b8,b9,b10,b11) (11)
According to la1，la3And la4，la2，la3And la4The linear equation respectively calculates the coordinate x and the value y of the intersection points, the intersection points are marked as points A, B, C and D in a sequence from small to large, the coordinate of the rotation geometric center point of the wafer is calculated according to the coordinates of the point A and the point C and is marked as O1, and the solving mode of the x and y coordinates of the geometric center point is as follows:
xO1＝(yC-yA)/2+yA (13)
yO1＝(xC-xA)/2+xA (14)
let the counterclockwise rotation be a positive direction and the clockwise rotation be a negative direction, the rotation direction can be determined as shown in fig. 5:
(1)0°≤α1≤45°
Angle of rotation＝-α1 (15)
(2)45°＜α1＜90°
Angle of rotation＝(90-α1) (16)
the rotated holes are filled with corresponding gray value 255, and the rotation effect is shown in fig. 6.
Extracting horizontal and vertical edges of the wafer based on morphological operation, wherein structural elements of the horizontal edges are in a horizontal structure, and the length is cols/2 single-pixel width; the structural elements of the vertical side are vertical structures, and the length is rows/2 single-pixel width. The effect of the extraction is shown in fig. 7.
And solving the length of each extracted independent straight line segment respectively, and comparing the length with a standard threshold value. If the total number of the obtained effective edges of the horizontal edge and the vertical edge is 4 and the corresponding length error is smaller than a fixed threshold value, the outer contour is intact and has no defects.
4, detection of defects of the oxide film region: the detection of the oxidation film area is based on the premise that the detection of the outline is complete, and the detection of the oxidation film area is not needed if the detection of the outline is defective. Therefore, when the oxide film is detected, the corresponding wafer image has no edge breakage or corner breakage defect, and four corner points exist. In the wafer processing process, the oxide film isolation region is usually accompanied by saw teeth, the performance of the wafer is not affected by the saw teeth in a small range, but the performance of the wafer is affected by the appearance of the saw teeth in a continuous large range, which is not only unfavorable for detection. The circumference surrounded by the oxide films of the same batch of wafers is relatively fixed, so the method adopts the contour surrounding length of the oxide films as a basis for judgment.
For the first initial detection of the outer contour area, the obtained binary image can be directly used for detecting the defects of the oxide film area. Firstly, extracting all profile information of the wafer based on the global Canny profile detection, wherein the corresponding profile information comprises three parts: outermost profile, oxide film profile, profile of grain region defect features. If the die region is free of defects, the corresponding die region will not have a profile.
The way of extracting the outline of the oxide film region: obtaining an outermost layer contour findCon points (pic9, continours 21, hierarchy21, RETR _ EXTERNAL, CHAIN _ APPROX _ NONE and Point ()) of the binary image of the wafer, and cutting the global contour of the wafer according to the coordinate position relation of the outermost layer contour in the image.
Respectively carrying out algebraic operation by four corner points corresponding to the outermost layer outline, namely the horizontal and vertical coordinates of the point a at the upper left corner + 15; the lower left corner b has the abscissa +15 and the ordinate-15; the upper right corner point c has the abscissa of-15 and the ordinate of + 15; the lower right hand point d has the abscissa-15 and the ordinate-15. Since the detection of the outer contour is complete, coordinates of four corner points according to which the oxide film defect exists.
And cutting the global contour image according to the coordinates of the four corner points. A profile image containing only the oxide region and the grain region features is obtained, as shown in fig. 8.
The length of the curve enclosed by the outermost contour is solved for the image, and the OpenCV function used is arcLength. The outermost contour here corresponds to the contour of the oxide film.
The length is compared with a set threshold value, and the defect of the oxide film area can be detected.
5, detecting the defects of the grain regions:
the grain region is detected based on the premise that the outer contour region and the oxide film region are detected completely, and if the oxide film region is detected to be defective, the grain region is not detected. Therefore, when the die region is detected, the corresponding wafer image has no edge breakage or corner breakage defect, and four corner points exist. For the detection of the defects in the grain regions, when the perimeter surrounded by the features of the grain regions exceeds the set threshold 0, the corresponding grain regions have defects.
For the first initial detection of the outer contour area, the obtained binary image can be directly used for detecting the defects of the grain area. Firstly, extracting all profile information of the wafer based on the global Canny profile detection, wherein the corresponding profile information comprises three parts: outermost profile, oxide film profile, profile of grain region defect features.
The method for extracting the characteristic outline of the grain region comprises the following steps: obtaining an outermost layer contour findCon points (pic9, continours 21, hierarchy21, RETR _ EXTERNAL, CHAIN _ APPROX _ NONE and Point ()) of the binary image of the wafer, and cutting the global contour of the wafer according to the coordinate position relation of the outermost layer contour in the image. Respectively carrying out algebraic operation on four corner points corresponding to the outermost layer outline, namely the horizontal and vertical coordinates of the point a at the upper left corner plus 75; point b at the lower left corner has abscissa +75 and ordinate-75; the upper right corner point c has the abscissa of-75 and the ordinate of + 75; the lower right hand point d has the abscissa-75 and the ordinate-75. Since the detection of the outer contour is complete, coordinates of four corner points according to which the oxide film defect exists.
And cutting the global contour image according to the coordinates of the four corner points. A profile image containing only the features of the die region is obtained, as shown in fig. 9, if the die region is intact, there are no features.
And solving the length of a curve surrounded by a global contour for the image, and using an OpenCV function as arcLength, wherein the global contour corresponds to the contour of the defect characteristics in the grain region. The length is compared with a set threshold value to judge whether the grain region is defective or not.
Claims (8)
1. A wafer surface defect regional detection method based on visual images is characterized by comprising the following steps:
obtaining a rectangular wafer original color image based on a macro prime lens continuous exposure and global scanning mode, wherein the background of the wafer image is white;
dividing the original color image of the wafer into 3 research areas, namely an outer contour area, an oxide film area and a grain area, according to the characteristics of the defect types of the wafer; the defect detection of the outer contour area comprises rough detection and fine detection;
the outer contour rough detection step: background removing operation of the original color image of the wafer, firstly removing isolated interference points by adopting median filtering, then traversing all pixels of the background to record a gray value corresponding to the minimum point in RGB gray values, and setting a linear transformation coefficient according to the corresponding gray value;
converting the RGB color wafer image into a gray image, then carrying out binarization to obtain a binary image, carrying out outmost layer contour extraction, simultaneously calculating the area surrounded by the outmost layer contour, and carrying out fine inspection if the area is within a set threshold value range;
the fine inspection method comprises the steps of determining the inclination angle of a wafer and the geometric center of the wafer in an image based on a binary image in rough inspection, rotating around the geometric center of the wafer in the image to enable the vertical edge of the wafer to be perpendicular to the bottom edge of the image, extracting the profile while extracting the horizontal edge and the vertical edge, calculating the lengths of the horizontal edge and the vertical edge while comparing with a threshold, and if the lengths of the horizontal edge and the vertical edge are smaller than the threshold, the outer contour has no defects;
detecting the oxide film defect, carrying out global contour extraction on the binary image obtained by rough detection based on the contour perimeter surrounded by the oxide film, cutting the global contour image based on the coordinates of the corner points according to the four corner points of the wafer without the outer contour defect to obtain an image only with the oxide film contour line as the outermost contour, solving the surrounding perimeter of the outermost contour of the contour image and comparing the surrounding perimeter with a threshold value to judge the oxide film defect;
carrying out defect detection on the grain region, carrying out global contour extraction on the binary image obtained by rough detection, cutting the global contour image based on coordinates of the corner points according to the four corner points of the wafer without the outer contour defect to obtain a contour line image only containing the characteristics of the grain region, and judging whether the image has the defects of the grain region or not by calculating the surrounding perimeter of the global outer contour of the image and comparing the surrounding perimeter with a threshold value;
performing median filtering on the color image, traversing all pixels after the median filtering to find a value pixel (x, y) with the minimum gray value in 3 channels; obtaining a contrast enhancement coefficient as shown in formula (2), performing global processing based on a linear processing transformation formula g (x, y) ═ α f (x, y) + β in image processing, as shown in formula (1),
IBGR(i,j)＝(fB(i,j),fG(i,j),fR(i,j)) (4)
wherein f isb(i,j)、fg(i,j)、fr(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the original image; f. ofB(i,j)、fG(i,j)、fR(i, j) respectively representing the blue, green and red channel gray values of the corresponding pixel values in the processed image;
representing a contrast enhancement coefficient;
in the outline defect detection rough detection, outline extraction is based on gradient, a Canny edge detection operator and a first-order gradient operator used by Canny in a two-dimensional space are adopted
For two-dimensional continuous data, if there is a function f (x, y), partial derivatives g in x and y directions can be obtained respectivelyx、gy：
The magnitude and direction of the gradient can be expressed as:
the magnitude and direction of the gradient calculation can be simplified as:
M(x,y)≈|gx|+|gy| (9)
the direction is as follows:
in the outer contour defect detection and fine detection, the center of rotation is the center of the wafer in the image instead of the center of the image; the method for realizing the verticality of the image at any angle comprises the following steps:
the top edge a1 of the wafer can be calculated according to any input image, the coordinates of 11 points which are equidistantly distributed on the a1 side are obtained to be b1, b2, b3, b4, b5, b6, b7, b8, b9, b10 and b11 respectively, a straight line equation of the corresponding side is fitted according to the points, the inclination angle alpha 1 epsilon [0,90 ] of the corresponding straight line of the a1 side is obtained according to the equation, and the straight line equations of the corresponding sides of the other three sides a2, a3 and a4 are respectively obtained to be la2，la3 and la4，
la1＝fitting(b1,b2,b3,b4,b5,b6,b7,b7,b8,b9,b10,b11) (11)
According to la1，la3And la4，la2，la3And la4The linear equation respectively calculates the intersection point coordinates x and y, the intersection point coordinates are sorted into points A, B, C and D according to clockwise or counterclockwise, the rotation geometric center point coordinate of the wafer is calculated according to the coordinates of the point A and the point C and is recorded as O1, and the solving mode of the x and y coordinates of the geometric center point is as follows:
xO1＝(yC-yA)/2+yA (13)
yO1＝(xC-xA)/2+xA (14)
the rotation direction can be determined as the positive direction by rotating counterclockwise and the negative direction by rotating clockwise,
(1)0°≤α1≤45°
frame rotation angle-alpha 1 (15)
(2)45°＜α1＜90°
The frame rotation angle is (90- α 1). (16)
2. The wafer surface defect regional detection method based on the visual image according to claim 1, wherein in the detection of the oxide film defect, the global contour image is cut based on coordinates of an angular point, that is, a horizontal and vertical coordinates of a point a at the upper left corner is + 15; the lower left corner b has the abscissa +15 and the ordinate-15; the upper right corner point c has the abscissa of-15 and the ordinate of + 15; the value of the point d at the lower right corner, the abscissa is-15, and the ordinate is-15, can be adjusted according to different batches; in the defect detection of the grain region, a global contour image is cut based on the coordinates of the corner points, wherein the coordinates of the point a at the upper left corner are plus 75; point b at the lower left corner has abscissa +75 and ordinate-75; the upper right corner point c has the abscissa of-75 and the ordinate of + 75; the lower right-hand point d, abscissa-75 and ordinate-75, can be adjusted in value according to the batch.
3. The method for detecting the defect regions on the surface of the wafer based on the visual image as claimed in claim 1, wherein the calculation of the magnitude and direction of the gradient can be simplified as follows:
M(x,y)≈|gx|+|gy| (9)
the direction is as follows:
4. the method of claim 1, wherein in the outline defect inspection, the center of rotation is the center of the wafer in the image and not the center of the image; extracting horizontal and vertical edges of the wafer based on morphological operation, wherein structural elements of the horizontal edge are of a horizontal structure, and structural elements of the vertical edge are of a vertical structure; respectively solving the length of each extracted independent straight line segment, and comparing the length with a standard threshold value; and if the corresponding length of the obtained horizontal side and the vertical side is smaller than a fixed threshold value, the outer contour is intact and has no defects.
5. The method according to claim 1, wherein the step of detecting the defects comprises detecting the defects in the oxide film only when the outer contour region has no defects, and detecting the defects in the die region only when the oxide film has no defects; and the fine inspection is performed only under the condition that the coarse inspection is qualified.
6. The visual-image-based wafer surface defect regional detection method according to claim 1, wherein a fixed threshold binarization is adopted in the binarization processing in the outer contour defect detection rough inspection, and the median filtering is performed by adopting structural elements with a length of 3 and a width of 1.
7. The method as claimed in claim 1, wherein in the rough outer contour defect inspection, the contour information extracted from the outer contour includes an outermost contour, an oxide film contour and a grain region defect feature contour, the oxide film contour can be directly used for detecting the oxide film defect, and the grain region defect contour can be directly used for detecting the grain region defect.
8. The method as claimed in claim 1, wherein the RGB color wafer image is converted into the gray scale image by 3-channel averaging, i.e. I (x, y) ═ 1/3 × R (x, y) +1/3 × G (x, y) +1/3 × B (x, y).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910524321.4A CN110286126B (en) | 2019-06-17 | 2019-06-17 | Wafer surface defect regional detection method based on visual image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910524321.4A CN110286126B (en) | 2019-06-17 | 2019-06-17 | Wafer surface defect regional detection method based on visual image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110286126A CN110286126A (en) | 2019-09-27 |
| CN110286126B true CN110286126B (en) | 2021-09-14 |
Family
ID=68005177
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910524321.4A Active CN110286126B (en) | 2019-06-17 | 2019-06-17 | Wafer surface defect regional detection method based on visual image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110286126B (en) |
Families Citing this family (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110672620B (en) * | 2019-10-08 | 2022-08-26 | 英特尔产品（成都）有限公司 | Chip defect detection method and system |
| CN111340753B (en) * | 2019-12-30 | 2023-06-02 | 上海集成电路研发中心有限公司 | Wafer detection device, data processing method and storage medium |
| CN111415878B (en) * | 2020-03-30 | 2023-09-12 | 英特尔产品（成都）有限公司 | Wafer-level automatic detection method, equipment and system |
| CN111489338A (en) * | 2020-04-07 | 2020-08-04 | 中铁工程服务有限公司 | Nondestructive testing method for internal defects of hydraulic pipeline of shield tunneling machine |
| CN111665261A (en) * | 2020-06-04 | 2020-09-15 | 安徽安视智能科技有限公司 | Chip crystal grain detection method based on machine vision |
| CN111815565B (en) * | 2020-06-11 | 2024-06-18 | 上海华虹宏力半导体制造有限公司 | Wafer backside detection method, equipment and storage medium |
| CN112014407A (en) * | 2020-09-03 | 2020-12-01 | 浙江长芯光电科技有限公司 | Method for detecting surface defects of integrated circuit wafer |
| CN112070766B (en) * | 2020-11-16 | 2021-02-19 | 深圳中科飞测科技有限公司 | Defect detection method and device, detection equipment and readable storage medium |
| CN112748116A (en) * | 2020-12-23 | 2021-05-04 | 郑州金惠计算机***工程有限公司 | Medical gauze surface defect online detection method and device |
| CN112718530B (en) * | 2020-12-30 | 2023-07-25 | 马鞍山芯乔科技有限公司 | Soft package heat seal inspection mechanism and system thereof |
| CN113034474A (en) * | 2021-03-30 | 2021-06-25 | 无锡美科微电子技术有限公司 | Test method for wafer map of OLED display |
| CN113077449B (en) * | 2021-04-08 | 2022-04-08 | 仲恺农业工程学院 | Image detection method for corner defects of rectangular wafer |
| CN113706468A (en) * | 2021-07-27 | 2021-11-26 | 河北光兴半导体技术有限公司 | Glass defect detection method based on BP neural network |
| CN114066917B (en) * | 2021-11-11 | 2022-08-05 | 深圳市云鼠科技开发有限公司 | Cleaning method, cleaning device, electronic equipment and storage medium |
| CN114663416B (en) * | 2022-03-31 | 2023-05-09 | 河北鹰眼智能科技有限公司 | Copper bar cold-break defect detection method and system based on computer vision |
| CN114897881B (en) * | 2022-06-10 | 2024-06-07 | 合肥工业大学 | Grain defect detection method based on edge characteristics |
| CN114972317B (en) * | 2022-06-22 | 2024-04-05 | 合肥工业大学 | Square grain placement planning method |
| CN115274486B (en) * | 2022-09-27 | 2022-12-13 | 南通睿禧智能科技有限公司 | Semiconductor surface defect identification method |
| CN116106226B (en) * | 2023-02-13 | 2023-08-29 | 山东师范大学 | Target area analysis system and method for biological detection |
| CN116485787B (en) * | 2023-06-15 | 2023-08-22 | 东莞市立时电子有限公司 | Method for detecting appearance defects of data line molding outer die |
| CN116973367B (en) * | 2023-07-05 | 2024-01-30 | 广东汇发塑业科技有限公司 | Method for inspecting degradation defect position of biodegradable film |
| CN117269179B (en) * | 2023-11-23 | 2024-02-02 | 平方和(北京)科技有限公司 | High-precision detection method and system for edge defects of contact lens based on machine vision |
| CN117455897B (en) * | 2023-11-30 | 2024-04-30 | 魅杰光电科技(上海)有限公司 | Wafer scratch detection method, device, equipment and storage medium |
| CN117871416A (en) * | 2024-03-11 | 2024-04-12 | 视睿（杭州）信息科技有限公司 | Grain coordinate sorting method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101839690A (en) * | 2010-04-13 | 2010-09-22 | 河海大学常州校区 | Visual inspection method for chip electronic component position error based on edge fitting |
| CN103345743A (en) * | 2013-06-18 | 2013-10-09 | 宁波成电泰克电子信息技术发展有限公司 | Image segmentation method for intelligent flaw detection of cell tail end |
| CN106651837A (en) * | 2016-11-14 | 2017-05-10 | 中国科学院自动化研究所 | White glass plate surface edge breakage defect detecting method |
| CN108802051A (en) * | 2018-07-10 | 2018-11-13 | 华南理工大学 | A kind of flexibility IC substrate straight path bubbles and folding line defect detecting system and method |
| CN109859160A (en) * | 2018-12-10 | 2019-06-07 | 湖南航天天麓新材料检测有限责任公司 | Almag internal defect in cast image-recognizing method based on machine vision |
-
2019
- 2019-06-17 CN CN201910524321.4A patent/CN110286126B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101839690A (en) * | 2010-04-13 | 2010-09-22 | 河海大学常州校区 | Visual inspection method for chip electronic component position error based on edge fitting |
| CN103345743A (en) * | 2013-06-18 | 2013-10-09 | 宁波成电泰克电子信息技术发展有限公司 | Image segmentation method for intelligent flaw detection of cell tail end |
| CN106651837A (en) * | 2016-11-14 | 2017-05-10 | 中国科学院自动化研究所 | White glass plate surface edge breakage defect detecting method |
| CN108802051A (en) * | 2018-07-10 | 2018-11-13 | 华南理工大学 | A kind of flexibility IC substrate straight path bubbles and folding line defect detecting system and method |
| CN109859160A (en) * | 2018-12-10 | 2019-06-07 | 湖南航天天麓新材料检测有限责任公司 | Almag internal defect in cast image-recognizing method based on machine vision |
Non-Patent Citations (1)
| Title |
|---|
| Optimized self-adapting contrast enhancement algorithm for wafer contour extraction;Zhiyong Yu et.al;《Multimedia Tools and Applications》;20190729;第78卷;32087-32108 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110286126A (en) | 2019-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110286126B (en) | Wafer surface defect regional detection method based on visual image | |
| CN109978839B (en) | Method for detecting wafer low-texture defects | |
| CN110286124B (en) | Machine vision-based refractory brick measuring system | |
| CN115791822A (en) | Visual detection algorithm and detection system for wafer surface defects | |
| CN115345885A (en) | Method for detecting appearance quality of metal fitness equipment | |
| CN115035122B (en) | Artificial intelligence-based integrated circuit wafer surface defect detection method | |
| CN113034464B (en) | Visual real-time detection method for defects of liquid crystal display under multiple backgrounds | |
| CN115375686B (en) | Glass edge flaw detection method based on image processing | |
| CN109540925B (en) | Complex ceramic tile surface defect detection method based on difference method and local variance measurement operator | |
| WO2021109697A1 (en) | Character segmentation method and apparatus, and computer-readable storage medium | |
| CN110070523B (en) | Foreign matter detection method for bottle bottom | |
| CN106780526A (en) | A kind of ferrite wafer alligatoring recognition methods | |
| CN110648330B (en) | Defect detection method for camera glass | |
| CN110674812B (en) | Civil license plate positioning and character segmentation method facing complex background | |
| CN114331986A (en) | Dam crack identification and measurement method based on unmanned aerial vehicle vision | |
| CN112308854A (en) | Automatic detection method and system for chip surface flaws and electronic equipment | |
| CN114910480A (en) | Wafer surface defect detection method based on machine vision | |
| CN107220946B (en) | Real-time removing method for bad block degree images on rock conveyer belt | |
| CN114897881B (en) | Grain defect detection method based on edge characteristics | |
| CN115078365A (en) | Soft package printing quality defect detection method | |
| CN116773548A (en) | Wafer surface defect detection method and system | |
| CN113129323A (en) | Remote sensing ridge boundary detection method and system based on artificial intelligence, computer equipment and storage medium | |
| CN112802022B (en) | Method for intelligently detecting defective glass image, electronic device and storage medium | |
| CN116012292A (en) | Wafer appearance defect detection method based on machine vision | |
| CN117495856B (en) | Wafer surface detection method, device, equipment and medium based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |