CN111310611B - Method for detecting cell view map and storage medium - Google Patents
Method for detecting cell view map and storage medium Download PDFInfo
- Publication number
- CN111310611B CN111310611B CN202010075316.2A CN202010075316A CN111310611B CN 111310611 B CN111310611 B CN 111310611B CN 202010075316 A CN202010075316 A CN 202010075316A CN 111310611 B CN111310611 B CN 111310611B
- Authority
- CN
- China
- Prior art keywords
- network
- training
- classification
- model
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
- G06V20/698—Matching; Classification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method for detecting a cell view map, which comprises an acquisition step, a training step and a classification step. And training a cascade network using an end-to-end detection network and classification network combination, integrating the characteristic information of the abnormality of the field view level reflected by the detection network into the classification network, and avoiding the loss of the information. The two networks are trained simultaneously, so that the detection network and the classification network supervise each other and promote each other, and the detection false positive of an abnormal region is reduced while the classification precision is ensured.
Description
Technical Field
The present invention relates to the field of cell image detection, and more particularly, to a method for detecting a cell visual field map and a storage medium.
Background
In the prior art for detecting abnormal cell areas in an abnormal visual field map, the visual field map is not further classified, but the acquisition of the position and classification information of abnormal cells is completed. The regional regression and tag frame classification of two networks are typically combined using a combination of the fast-RCNN based detection method and the R-FCN detection method. The specific process is that for a view diagram, a certain number of candidate frames are generated through a feature extractor, then through an RPN (candidate region generation network), approximately 2000 mark frames are generated, and regression and classification of the mark frames are respectively carried out by utilizing the regional properties of the R-FCN, so that a final detection result is obtained. However, the resulting test results are not applied to the final classification or diagnostic task. And the detected result is inaccurate, and false positive results can exist.
In addition, the current abnormal cell view map classification mainly depends on cell classification, and in cell classification, the current most advanced method uses a classification model based on map convolution, firstly uses Densenet to extract characteristics of cells, then uses a K-Means method to cluster the cells, finally uses a map convolution method to iteratively update the characteristics to obtain final characteristics, and carries out final cell classification.
The main purpose of the present invention is to solve the above three problems 1, and not to fuse the detected results into the model for judging the visual field map classification online, we obtain the abnormal detection information of the visual field map, which includes a part of prior information that can be used for the visual field map judgment, but the part of information is not included in the data stream for the visual field map classification judgment, resulting in the waste and loss of the prior information. 2. After passing through the detection module, the view map marks a part of areas where the network thinks that abnormality exists, but the areas may be disputed false positive areas, namely, areas where the normal areas are detected by network false marks, and obviously, the areas should not appear. However, in the case of input data of an abnormal view map, when detecting some abnormal areas and detecting normal areas of false positive, we can consider that the judgment on the abnormal view map is greatly influenced, but if a normal view map has a plurality of abnormal areas or even only one abnormal area, the judgment on the view map is unreasonable, the judgment on the view map is extremely unfriendly, and the abnormal areas are detected in the normal view map which is least abnormal, so that the standard and the robustness of the method for judging whether the view map is abnormal or not are reduced, and the accuracy of a classification network and a detection network is greatly reduced. 3. The existing models do not adopt an end-to-end training method, do not allow the network to simultaneously consider the information of two tasks, train a detection network and train a classification network, and can not regenerate a candidate frame and reduce abnormal false positives, so that the two tasks are trained simultaneously, and the purposes that the classification network is constrained on the detection network and the detection network is helpful to the classification network are achieved.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method for detecting a cell view map, comprising: an acquisition step of acquiring a cell view map and preparing a sample set;
training, namely inputting the sample set and the classification labels, training a first-level network to obtain a cascading model, wherein the cascading network consists of a Retinonet network and a CNN network, the Retinonet network is used for obtaining a detection model through training, and outputting a region judgment result of a visual field diagram; the CNN is used for obtaining a classification model through training and outputting a classification result of the visual field diagram; and a classification step, namely inputting the view diagram to be identified into a trained cascade network to obtain a classification result of the whole view diagram and a region judgment result of the view diagram.
Further, the training step comprises a Retinanet network training step, and specifically comprises the following steps: dividing the sample set into a training set and a testing set; the first training step, inputting the training set, training the Retinonet to obtain a first network model and a plurality of feature graphs; a first output step of inputting the test set to the first network model to obtain a first judgment result; and a first optimization step, namely comparing the first judging result with the correct result, calculating the difference value of the first judging result and the correct result, reversely transmitting the difference value, and optimizing a first network model to obtain a detection model.
Further, the training step includes a CNN network training step, specifically including: a second training step of inputting the feature map and the classification labels and training a CNN (computer network) to obtain a second network model; a second output step of inputting the test set and outputting a second determination result; and a second optimizing step, namely comparing the second judging result with the correct result, calculating the difference value of the second judging result and the correct result, reversely transmitting the difference value, and optimizing the classification model of the second network model, wherein the classification model and the detection model form a cascade model.
Further, the Retinanet network includes a convolutional layer, a pooling layer, and an activation layer.
Further, the CNN network includes a convolutional layer, a pooling layer, an activation layer, and a fully-connected layer.
Further, in the classifying step, the view region determination result includes positions of a plurality of abnormal marker frames and marker frame category information.
Further, in the first training step, the feature map is obtained by training a feature extraction network of the Retinanet network.
Further, the feature extraction network comprises a feature pyramid network.
Further, the CNN network includes a Resnet network and a dense network; the classification tag includes a normal view map or an abnormal view map.
The present invention also provides a storage medium storing a computer program for executing the abnormality detection method of the view map.
The beneficial effects of the invention are as follows: the invention provides a method for detecting a cell visual field map and a storage medium, wherein a cascade network using an end-to-end detection network and classification network combination is used for training, and characteristic information of abnormality of the visual field map level reflected by the detection network is integrated into the classification network, so that the loss of the information is avoided. The two networks are trained simultaneously, so that the detection network and the classification network supervise each other and promote each other, and the detection false positive of an abnormal region is reduced while the classification precision is ensured.
Drawings
The technical solution and other advantageous effects of the present invention will be made apparent by the following detailed description of the specific embodiments of the present invention with reference to the accompanying drawings.
FIG. 1 is a flow chart of a cell field of view diagram provided by the present invention.
Fig. 2 is a block diagram of a cascade network according to the present invention.
Fig. 3 is a flowchart of the detection step provided in the present invention.
Fig. 4 is a flowchart of a second classification step provided in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the invention. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples, which are for the purpose of brevity and clarity, and which do not themselves indicate the relationship between the various embodiments and/or arrangements discussed. In addition, the present invention provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.
As shown in FIG. 1, the invention provides a method for detecting a cell field map, which comprises S1-S5.
S1, acquiring a cell view map and manufacturing a sample set.
S2, training, namely inputting the sample set and the classification labels, and training a cascade network to obtain a cascade model.
As shown in fig. 2, the cascade network is composed of a Retinanet network (dashed line box in fig. 2) and a CNN network, where the Retinanet network is used to obtain a detection model through training, and output a region determination result of the view map; the CNN network is used for obtaining a classification model through training and outputting a classification result of the visual field diagram.
The classification tag includes a normal view map or an abnormal view map.
The training step comprises a Retinonet training step and a CNN network training step.
As shown in fig. 3, the Retinanet network training step specifically includes: S201-S204.
S201, dividing the sample set into a training set and a testing set.
S202, inputting the training set, and training the Retinonet to obtain a first network model and a plurality of feature graphs. The Retinanet network includes a convolutional layer, a pooling layer, and an activation layer.
In the first training step, the feature map is obtained by training a feature extraction network of the Retinonet network.
The feature extraction network includes a feature pyramid network.
S203, a first output step, namely inputting the test set into the first network model to obtain a first judgment result.
S204, a first optimization step, namely comparing the first judging result with the correct result, calculating the difference value of the first judging result and the correct result, reversely transmitting the difference value, and optimizing a first network model to obtain a detection model.
As shown in fig. 4, the training step includes a CNN network training step, specifically including: s301 to S303.
S301, inputting the feature map and the classification labels, and training a CNN (computer network) to obtain a second network model; the CNN network includes a convolutional layer, a pooling layer, an activation layer, and a fully-connected layer.
The CNN network comprises a Resnet network, and particularly the Resnet50 network has the best training effect. The CNN network also includes a dense network.
S302, a second output step, namely inputting the test set and outputting a second judging result.
S303, comparing the second judging result with the correct result, calculating the difference value of the second judging result and the correct result, reversely transmitting the difference value, and optimizing the classification model of the second network model, wherein the classification model and the detection model form a cascade model.
S3, inputting the view diagram to be identified into a trained cascade network to obtain a classification result of the whole view diagram and a region judgment result of the view diagram.
In the classifying step, the view region determination result includes positions of a plurality of abnormal marker frames and marker frame category information.
The present invention provides a storage medium storing a computer program for executing the method for detecting a cell view map according to claim.
The invention provides a method for detecting a cell visual field map, which trains a cascade network combined by an end-to-end detection network and a classification network, integrates characteristic information of abnormality of the visual field map level reflected by the detection network into the classification network, and avoids the loss of the information. The two networks are trained simultaneously, so that the detection network and the classification network supervise each other and promote each other, and the detection false positive of an abnormal region is reduced while the classification precision is ensured.
The cascade network synchronously corrects false positive data generated by abnormal detection results, reduces the detection rate of abnormal visual field images of normal category visual field images, is end-to-end, does not need to output detection network results as input of a classification network, and has higher efficiency. The result of abnormality detection is optimized, so that false positive is reduced, the burden of doctors is reduced, and the diagnosis efficiency of the visual field map and the utilization of medical resources are improved; the detection rate of the abnormal region of the normal view map is reduced, the secondary detection of the normal view map is avoided, the diagnosis precision is improved, the medical cost is further reduced, and the social resource waste is reduced.
In practical application, for a view map with a normal classification result, the detection network should not generate an abnormal marking frame, but the previous model has no way to ensure that the abnormal marking frame does not appear in the view map classified as normal, because the simple detection network has no way to restrict the output of the normal view map. However, after a classification network is added, the output of the normal view map can be constrained forcefully, and the abnormal detection rate of the normal view map is reduced. And the generation of abnormal areas is enhanced after the classified information is added to the abnormal view map. And finally, training of end-to-end detection network and abnormal classification is realized.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of the above examples is only for aiding in understanding the technical solution of the present invention and its core ideas; those of ordinary skill in the art will appreciate that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (8)
1. A method for detecting a cell view, comprising:
an acquisition step of acquiring a cell view map and preparing a sample set;
training, namely inputting the sample set and the classification labels, training a first-level network to obtain a cascading model, wherein the cascading network consists of a Retinonet network and a CNN network, the Retinonet network is used for obtaining a detection model through training, and outputting a region judgment result of a visual field diagram; the CNN is used for obtaining a classification model through training and outputting a classification result of the visual field diagram; and
a classification step, namely inputting the view diagram to be identified into a trained cascade network to obtain a classification result of the whole view diagram and a region judgment result of the view diagram;
the training step comprises a Retinonet network training step, and specifically comprises the following steps:
dividing the sample set into a training set and a testing set;
the first training step, inputting the training set, training the Retinonet to obtain a first network model and a plurality of feature graphs;
a first output step of inputting the test set to the first network model to obtain a first judgment result; and
a first optimization step of comparing the first judgment result with the correct result, calculating the difference value of the first judgment result and the correct result, reversely transmitting the difference value, and optimizing a first network model to obtain the detection model;
the training step comprises a CNN network training step, and specifically comprises the following steps:
a second training step of inputting the feature map and the classification labels and training a CNN (computer network) to obtain a second network model;
a second output step of inputting the test set and outputting a second determination result;
and a second optimizing step, namely comparing the second judging result with the correct result, calculating the difference value of the second judging result and the correct result, reversely transmitting the difference value, and optimizing the second network model to the classification model, wherein the classification model and the detection model form a cascade model.
2. The method for detecting a cell view according to claim 1, wherein,
the Retinanet network includes a convolutional layer, a pooling layer, and an activation layer.
3. The method for detecting a cell view according to claim 1, wherein,
the CNN network includes a convolutional layer, a pooling layer, an activation layer, and a fully-connected layer.
4. The method for detecting a cell view according to claim 1, wherein,
in the step of classifying the objects in the object of the present invention,
the view map region determination result includes positions of a plurality of abnormal marker frames and marker frame category information.
5. The method for detecting a cell view according to claim 1, wherein,
in the first training step, the feature map is obtained by training a feature extraction network of the Retinonet network.
6. The method for detecting a cell view according to claim 5, wherein
The feature extraction network includes a feature pyramid network.
7. The method for detecting a cell view according to claim 1, wherein,
the CNN network comprises a Resnet network and a dense network;
the classification tag includes a normal view map or an abnormal view map.
8. A storage medium storing a computer program for executing the abnormality detection method of a view map according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010075316.2A CN111310611B (en) | 2020-01-22 | 2020-01-22 | Method for detecting cell view map and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010075316.2A CN111310611B (en) | 2020-01-22 | 2020-01-22 | Method for detecting cell view map and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111310611A CN111310611A (en) | 2020-06-19 |
| CN111310611B true CN111310611B (en) | 2023-06-06 |
Family
ID=71161616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010075316.2A Active CN111310611B (en) | 2020-01-22 | 2020-01-22 | Method for detecting cell view map and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310611B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113838008B (en) * | 2021-09-08 | 2023-10-24 | 江苏迪赛特医疗科技有限公司 | Abnormal cell detection method based on attention-introducing mechanism |
| CN116977905B (en) * | 2023-09-22 | 2024-01-30 | 杭州爱芯元智科技有限公司 | Target tracking method, device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448090A (en) * | 2018-11-01 | 2019-03-08 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN110110799A (en) * | 2019-05-13 | 2019-08-09 | 广州锟元方青医疗科技有限公司 | Cell sorting method, device, computer equipment and storage medium |
| CN110210362A (en) * | 2019-05-27 | 2019-09-06 | 中国科学技术大学 | A kind of method for traffic sign detection based on convolutional neural networks |
| CN110287927A (en) * | 2019-07-01 | 2019-09-27 | 西安电子科技大学 | Based on the multiple dimensioned remote sensing image object detection method with context study of depth |
| CN110334565A (en) * | 2019-03-21 | 2019-10-15 | 江苏迪赛特医疗科技有限公司 | A kind of uterine neck neoplastic lesions categorizing system of microscope pathological photograph |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10282589B2 (en) * | 2017-08-29 | 2019-05-07 | Konica Minolta Laboratory U.S.A., Inc. | Method and system for detection and classification of cells using convolutional neural networks |
-
2020
- 2020-01-22 CN CN202010075316.2A patent/CN111310611B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448090A (en) * | 2018-11-01 | 2019-03-08 | 北京旷视科技有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN110334565A (en) * | 2019-03-21 | 2019-10-15 | 江苏迪赛特医疗科技有限公司 | A kind of uterine neck neoplastic lesions categorizing system of microscope pathological photograph |
| CN110110799A (en) * | 2019-05-13 | 2019-08-09 | 广州锟元方青医疗科技有限公司 | Cell sorting method, device, computer equipment and storage medium |
| CN110210362A (en) * | 2019-05-27 | 2019-09-06 | 中国科学技术大学 | A kind of method for traffic sign detection based on convolutional neural networks |
| CN110287927A (en) * | 2019-07-01 | 2019-09-27 | 西安电子科技大学 | Based on the multiple dimensioned remote sensing image object detection method with context study of depth |
Non-Patent Citations (1)
| Title |
|---|
| Mask-guided Contrastive Attention Model for Person Re-Identification;Chunfeng Song 等;2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition;1179-1188 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111310611A (en) | 2020-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200387785A1 (en) | Power equipment fault detecting and positioning method of artificial intelligence inference fusion | |
| WO2018108129A1 (en) | Method and apparatus for use in identifying object type, and electronic device | |
| CN111444939B (en) | Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field | |
| CN112581463A (en) | Image defect detection method and device, electronic equipment, storage medium and product | |
| CN107423278B (en) | Evaluation element identification method, device and system | |
| CN108875602A (en) | Monitor the face identification method based on deep learning under environment | |
| CN111861978A (en) | Bridge crack example segmentation method based on Faster R-CNN | |
| CN111401419A (en) | Improved RetinaNet-based employee dressing specification detection method | |
| US20200402221A1 (en) | Inspection system, image discrimination system, discrimination system, discriminator generation system, and learning data generation device | |
| US20230360390A1 (en) | Transmission line defect identification method based on saliency map and semantic-embedded feature pyramid | |
| CN111310611B (en) | Method for detecting cell view map and storage medium | |
| CN112330631B (en) | Railway wagon brake beam pillar rivet pin collar loss fault detection method | |
| CN105930836A (en) | Identification method and device of video text | |
| CN108986142A (en) | Shelter target tracking based on the optimization of confidence map peak sidelobe ratio | |
| CN112613428B (en) | Resnet-3D convolution cattle video target detection method based on balance loss | |
| CN113836850A (en) | Model obtaining method, system and device, medium and product defect detection method | |
| CN112419268A (en) | Method, device, equipment and medium for detecting image defects of power transmission line | |
| CN111368824B (en) | Instrument identification method, mobile device and storage medium | |
| CN110717602B (en) | Noise data-based machine learning model robustness assessment method | |
| CN110175519B (en) | Method and device for identifying separation and combination identification instrument of transformer substation and storage medium | |
| CN110751138A (en) | Pan head identification method based on yolov3 and CNN | |
| CN110765963A (en) | Vehicle brake detection method, device, equipment and computer readable storage medium | |
| CN114494823A (en) | Commodity identification, detection and counting method and system in retail scene | |
| JP2022139174A (en) | Apparatus, method, and program for classifying defects | |
| CN115620083B (en) | Model training method, face image quality evaluation method, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |