CN103440619B - A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method - Google Patents
A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method Download PDFInfo
- Publication number
- CN103440619B CN103440619B CN201310231478.0A CN201310231478A CN103440619B CN 103440619 B CN103440619 B CN 103440619B CN 201310231478 A CN201310231478 A CN 201310231478A CN 103440619 B CN103440619 B CN 103440619B
- Authority
- CN
- China
- Prior art keywords
- sampling
- resolution
- grid
- mode
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000005070 sampling Methods 0.000 title claims abstract description 123
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000003384 imaging method Methods 0.000 claims abstract description 35
- 244000007853 Sarothamnus scoparius Species 0.000 claims abstract description 32
- 239000013598 vector Substances 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012937 correction Methods 0.000 abstract description 6
- 238000012428 routine sampling Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 14
- 238000003491 array Methods 0.000 description 8
- 230000010354 integration Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 241000764238 Isis Species 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 108091008695 photoreceptors Proteins 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method, to realize the reconstruction of high spatial resolution satellite remote sensing images, the above method includes：Cell coordinate system on oblique pattern sampling grid is mapped to the cell coordinate system on high-resolution routine sampling grid；In space coordinates, triangulation is carried out to the pixel point of oblique pattern sampling；For the unknown pixel in fine-resolution meshes, as the known pixel value of the triangular apex where it, the unknown pixel value in fine-resolution meshes is calculated using super resolution ratio reconstruction methods such as polynomial interopolations；Method disclosed by the invention, using the oblique pattern sampling imaging of single line battle array, avoids the problem of two rows of sensor linear array manufactures are difficult, registration accuracy is unmanageable；At the same time by controlling direction sampling interval of pushing broom to carry out image data acquiring, it is ensured that oblique pattern sampling institute is square into spatial sampling grid, avoids loss of significance problem caused by extra geometric correction and gray-level interpolation.
Description
Technical Field
The invention belongs to the technical field of super-resolution image reconstruction, and particularly relates to an oblique mode sampling modeling and super-resolution reconstruction method.
Background
The French SPOT-5 optical remote sensing satellite launched in 2002 adopts an over-mode sampling technology for the first time, two rows of linear array sensors with staggered half pixels are used for acquiring images, and the spatial resolution is improved from 5m to 2.5m under the condition that the size of a photosensitive element of the sensor is not changed. Successful realization of the sub-pixel sampling technology on the SPOT-5 satellite introduces a new idea for the research of improving the spatial resolution of the remote sensing image. Super-resolution reconstruction based on oblique mode sampling is a method for improving spatial resolution by combining software and hardware under the support of a specific imaging mode, and the effective resolution is improved from hardware sampling by changing the imaging mode of a sensor linear array. The method starts to realize the super-resolution reconstruction of the remote sensing image from two aspects, on one hand, a new sensor sampling mode is designed on hardware, and on the other hand, the mathematical model and the algorithm for super-resolution image reconstruction are designed on software by comprehensively considering the hardware sampling characteristic and various image degradation factors, so as to make up the limitation of a pure software method. The imaging linear array direction is vertical to the broom pushing direction under the conventional mode sampling, and the oblique mode sampling is to incline the imaging linear array by a certain angle and control the sampling interval in the broom pushing direction to perform imaging.
In order to improve the spatial resolution of the satellite remote sensing image, the french national space agency successively provides a method for improving the spatial resolution of the remote sensing image through spatial sampling technologies such as a high mode, a super mode, an oblique mode and the like. High-mode and super-mode sampling mainly reduce the spectrum aliasing effect by improving the spatial sampling frequency of an imaging linear array. The ultra-mode sampling and the high-mode sampling are both carried by two imaging linear arrays which are mutually shifted to be half array elements on a remote sensing platform, the difference is that the integral time of the ultra-mode sampling in the broom pushing direction is the same as that of the conventional mode, while the integral time of the high-mode sampling in the broom pushing direction is 1/2 of the conventional mode, and the spatial sampling density is improved by improving the time sampling frequency. For these two sampling modes, the spatial sampling densities of the hyper-mode and the hyper-mode are 2 times and 4 times, respectively, that of the regular mode. However, the problem with these two sampling modes is: 1) two rows of imaging linear arrays increase the complexity of the platform; 2) when the satellite side beats, in order to ensure that two rows of linear arrays stagger half array elements all the time, additional hardware equipment is required to be added.
The oblique mode sampling only uses a single imaging linear array, the linear array direction is not perpendicular to the broom pushing direction any more, but is inclined by a certain angle, and meanwhile, the sampling interval in the broom pushing direction is controlled to acquire image data, so that the spatial resolution of the image is improved under the condition of not changing the size of an array element. Because the oblique mode sampling only uses one row of imaging linear arrays, the manufacturing cost of the imaging equipment is reduced, and the problems that the two rows of imaging linear arrays are difficult to manufacture and the registration precision is difficult to control are avoided. The method is limited by the high sensitivity of the technology, and the detailed reports of the related technology are rarely made abroad. China is still in the theoretical verification and imaging equipment principle prototype development stage in the aspect of oblique mode remote sensing satellite design, and only a few documents relate to the method. A cycle peak and the like provide an oblique mode sampling mode with a space sampling grid being a parallelogram, and then an image under an original space relation is obtained after simple geometric correction is carried out on an oblique mode sampling image, wherein the sampling mode has the problems that: 1) gray interpolation causes geometric distortion; 2) the inclination angle of the imaging linear array simultaneously restricts the sampling density and the field width.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an oblique mode sampling modeling and super-resolution reconstruction method, which avoids the problem of precision loss caused by additional geometric correction and gray interpolation and ensures the sampling density and the field width.
In order to achieve the purpose, the invention adopts the technical scheme that:
a method for modeling oblique mode sampling and reconstructing super-resolution comprises the following steps:
based on a linear array broom pushing mode, adopting a single linear array to perform oblique mode data acquisition;
mapping a pixel coordinate system on the oblique mode sampling grid to a pixel coordinate system on the high-resolution conventional sampling grid;
triangulating pixel points sampled in an oblique mode in a space coordinate system;
and for the unknown pixel on the high-resolution grid, calculating the value of the unknown pixel on the high-resolution grid by using a super-resolution reconstruction method according to the known pixel value of the vertex of the triangle where the unknown pixel is located.
In the data acquisition process, the imaging linear array and the push broom direction form a certain angle, and the sampling interval in the push broom direction is controlled, so that the spatial sampling grid formed by the oblique mode sampling is square.
The angle is 26.56 degrees, and the sampling interval of the imaging linear array along the broom pushing direction isPicking in the direction perpendicular to the push broomSample spacing ofOr,
the angle is 63.44 degrees, and the sampling interval of the imaging linear array along the broom pushing direction isThe sampling interval along the direction vertical to the push broom is
In pixel coordinate system mapping, oblique mode sampling grid is used as basis vector Continuation, or, by basis vectorAnd (5) continuation.
In a space coordinate system, Delaunay triangulation is carried out on pixel points sampled in an oblique mode.
In super-resolution reconstruction, a polynomial interpolation is adopted to calculate an unknown pixel value on a high-resolution grid.
Compared with the prior art, the single linear array inclined mode sampling provided by the invention inclines the linear array sensor by a certain angle, thereby avoiding the problems that the manufacture of two rows of linear arrays of sensors is difficult and the registration precision is difficult to control. Meanwhile, image data acquisition is carried out by controlling the sampling interval in the broom pushing direction, so that a space sampling grid formed by oblique mode sampling is ensured to be square, and the problem of precision loss caused by additional geometric correction and gray interpolation is solved.
Drawings
FIG. 1 is a general flow diagram of the present invention.
FIG. 2 is a diagram of a diagonal mode sampling model according to the present invention.
FIG. 3 is a diagram of another diagonal mode sampling model of the present invention.
Fig. 4 is a coordinate system representation of the diagonal pattern sampling grid of the present invention.
FIG. 5 is a coordinate system representation of a square sampling grid of the present invention.
Fig. 6 is a schematic diagram of the Delaunay triangulation process of the present invention.
FIG. 7 is a schematic diagram of the present invention for diagonal mode sampling interpolation.
Fig. 8 is a schematic diagram of the oblique mode sampling light integration of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the drawings and examples.
As shown in fig. 1, the overall process of the present invention comprises the following steps: linear array data acquisition, coordinate system mapping, triangulation and super-resolution reconstruction. The data acquisition adopts a push broom imaging mode based on a linear array, the data are acquired line by line, the imaging linear array and the push broom direction form a certain angle, and the sampling interval in the push broom direction is controlled, so that the spatial sampling grid formed by the oblique mode sampling is square.
The linear array data acquisition adopts a single linear array inclined mode, c is set to indicate the size of a sensor array element,representing the distance between the picture elements in the resolution grid. In order to avoid the problems caused by geometric deformation, the space sampling grid formed by the oblique mode sampling is ensured to be square, namely, the points requiring the oblique mode sampling fall on the high-resolution grid to be interpolated and are expressed by a mathematical relation formulaIn this relation, the angle of inclinationFIG. 2 shows an oblique-mode sampling model established by the present invention, in which the imaging linear array and the broom pushing direction form an angle of 26.56 degrees for sampling, and the sampling interval of the imaging linear array along the broom pushing direction isThe sampling interval along the direction vertical to the push broom isAlternatively, as shown in fig. 3, the imaging linear arrays are sampled at an angle of 63.44 degrees with respect to the broom pushing direction, and the sampling intervals of the imaging linear arrays along the broom pushing direction are equal toThe sampling interval along the direction vertical to the push broom is
And s is set as the sampling interval of the imaging linear array in the broom pushing direction, and s is equal to upsilont t, wherein upsilon is the broom pushing speed of the imaging linear array, and t is the integration time. Obviously, under the condition of constant broom pushing speed, the purpose of reducing the pixel sampling interval can be achieved only by reducing the integration time, but the reduction of the integration time can cause the problems of insufficient exposure of a photosensitive element, reduction of the signal-to-noise ratio of an image and the like. Therefore, in order to ensure the integration time of the linear array data acquisition, the imaging linear array needs to have a larger sampling interval for two times of continuous imaging in the broom pushing direction.
The oblique mode sampling image is defined inThe regular sampling grid formed for basis vector extension can be expressed as
Wherein, { e1，e1Is the space domainThe basis vector of (2). In particular, the diagonal pattern sampling grid shown in FIG. 3 isIs a basis vector continuation.
The most commonly used basis vectors for a square sampling grid areThe square sampling grid can be represented asFIG. 4 shows a coordinate system representation of a diagonal pattern sampling grid, with the coordinate convention being (n)1，n2) FIG. 5 shows a coordinate system representation of a square sampling grid, with the coordinate convention given as (m)1，m2). And in the oblique mode sampling reconstruction, the unknown pixel points on the square sampling grid are estimated by the known pixel points acquired by oblique mode sampling, and the oblique mode sampling image is reconstructed into a high-resolution image.
The embodiment adopts a method of spatial scatter interpolation to acquire a high-resolution image. The interpolation is a super-resolution reconstruction method which has real-time performance and is suitable for hardware implementation, but the embodiment does not limit the adoption of the interpolation method, and other more complex super-resolution reconstruction methods can also be adopted. The interpolation problem in this embodiment can be summarized as the triangulation problem of the planar scatter. And triangulating the spatial scatter points on a two-dimensional plane, then constructing an interpolation surface on a triangular domain, and estimating the value of the unknown point on the basis of the interpolation surface.
Voronoi first mathematically defined each separation in 1908The effective scope of the scatter data, i.e., the scope in which it effectively reflects the region information, defines a Voronoi diagram on a two-dimensional plane. Delaunay evolved from Voronoi diagrams in 1934 to Delaunay triangulation, which was easier to analyze for applications. Voronoi diagrams and Delaunay triangulation become powerful tools for analyzing regional discrete data. The Delaunay triangulation is in duality with the Voronoi diagram, growth centers of all adjacent Voronoi polygons are connected, and the duality Delaunay triangulation can be formed through the Voronoi diagram. An important property of the Delaunay triangulation is the property of an empty circumcircle, i.e. for any Delaunay triangle, the inner region of the circumcircle of the triangle does not contain any other point, all Delaunay triangles do not overlap and completely cover the whole spatial domain. Fig. 6 shows a schematic diagram of Delaunay triangulation of known pixel points sampled in diagonal mode, where the solid origin represents a known pixel point and the cross is an unknown pixel point.
For each triangle T of the Delaunay triangulationkThe linear interpolation method is characterized in that the pixel values f of 3 vertexes of a known triangle are usedi(xi，yi) I-0, 1, 2, forming a plane：
In the formula,andis the coefficient of a polynomial function. According to each triangle TkPixel values f of 3 vertices ofi(xi，yi)，iA linear system of equations is established to solve the polynomial coefficients simultaneously for 0, 1, 2. Then, the pixel value at an arbitrary point (x, y) is obtained by using equation (2).
Cubic interpolation method constructs a cubic surface from 6 points of neighborhood. Linear interpolation has continuity, but it is not once derivable, the derivatives at the boundary lines of all triangles producing jumps. Cubic interpolation is once instructive, but requires higher computation time. FIG. 7 shows a schematic diagram of linear interpolation and cubic interpolation of diagonal mode sampling, in the diagonal mode sampling model established by the present invention, a line segment P0M1＝P1P2For such a sampling grid with uniform spacing, linear interpolation can be simplified in practice to be expressed as M1＝(P1+P2)/2，M0＝(P0+M1)/2。
Assuming that the point spread function of the imaging system is a uniform function, the downsampling model of the present invention simulates light intensity integration of the photoreceptor surface by performing double integration on the high resolution image. Let f (x)1，x2) Representing a real continuous image, fig. 8 shows an oblique-mode sampling light integral diagram, and the light intensity ^ integral of any photosensitive elementΩf(x1，x2)dx1dx2It is divided into three plane areas omega1、Ω2And Ω3The three plane areas x1Axis and x2The upper and lower limits of the integral of the axis are
Wherein,the light intensity collected by a single photosensing element can be modeled asWherein, c2Indicating the area covered by the cell.
The spatial sampling grid formed by the oblique mode sampling mode is a parallelogram, so that the formed image is oblique in space, and the used super-resolution reconstruction method only performs geometric correction on the image according to the linear array inclination angle. The serious problem brought by the oblique mode imaging mode is that the inclination angle of the imaging linear array can simultaneously influence the sampling density and the field of view width, and the two influences are mutually restricted, so that the spatial sampling density is increased along with the increase of the inclination angle of the imaging linear array, but the field of view width is reduced at the same time. In the invention, the spatial sampling grid formed by the oblique mode sampling is square, so that the formed image does not need additional geometric correction and gray interpolation. In order to ensure that the spatial sampling grid formed by the oblique mode sampling is square, the sampling interval in the broom pushing direction must be controlled to acquire image data. The distance between two times of data acquisition of the imaging linear array along the broom pushing direction is set as。
The spatial resolution of an image is typically measured in dots per inch (dpi), which in effect measures the sampling density. Sampling density r of spatial gridnomIs defined as
In the equation, ρ (Г) represents the sampling density of the spatial grid Г, i.e., the number of sampling points per unit area.The spatial resolution is measured in dpi. Nominal resolution (Nominal resolution) RnomIs defined as
The nominal resolution represents the distance between adjacent sample points.
Spatial resolution is typically measured using an expression of array element size c. The spatial grid basis vector of the regular pattern sampling is e1＝(c，0)T、e2＝(0，c)TSampling density rnom＝c-2Nominal resolution RnomC. Spatial grid basis vector of hyper-modal sampling is e1＝(c，0)T、Density of sampling rnom＝2c-2Nominal resolutionThe spatial grid basis vector of the high-mode sampling isDensity of sampling rnom＝4c-2Nominal resolutionThe spatial grid basis vectors for the diagonal mode sampling are Density of samplesNominal resolutionThe diagonal mode sampling density is 1.25 times that of the conventional mode sampling, but the diagonal mode sampling does not increase the spatial sampling frequency compared to the high mode and super mode sampling. In the sampling technologies, the ground pixel resolution of the remote sensing image is improved by oblique mode sampling under the condition of not changing the size of an array element.
Claims (1)
1. A method for modeling by oblique mode sampling and reconstructing super-resolution is characterized by comprising the following steps:
based on the push broom imaging mode of the linear array, the single linear array is adopted to carry out oblique mode data acquisition, in the data acquisition process, the imaging linear array and the push broom direction form an angle of 26.56 degrees or 63.44 degrees, and the oblique mode sampling grid is sampled by a basis vector e by controlling the sampling interval in the push broom direction to be 4p or 2p1＝(4p,0)T、e2＝(2p,p)TContinuation, or, by base vector e1＝(2p,0)T、e2＝(p,2p)TContinuation;
mapping a pixel coordinate system on an oblique mode sampling grid to a pixel coordinate system on a high-resolution conventional sampling grid, wherein the distance between pixels in the grid is p;
in a space coordinate system, performing Delaunay triangulation on pixel points sampled in an oblique mode;
and for the unknown pixel on the high-resolution grid, calculating the value of the unknown pixel on the high-resolution grid by adopting polynomial interpolation according to the known pixel value of the vertex of the triangle where the unknown pixel is located.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310231478.0A CN103440619B (en) | 2013-06-09 | 2013-06-09 | A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310231478.0A CN103440619B (en) | 2013-06-09 | 2013-06-09 | A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103440619A CN103440619A (en) | 2013-12-11 |
| CN103440619B true CN103440619B (en) | 2018-04-17 |
Family
ID=49694311
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310231478.0A Expired - Fee Related CN103440619B (en) | 2013-06-09 | 2013-06-09 | A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103440619B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3033071B1 (en) * | 2015-02-25 | 2017-02-17 | Centre Nat D'etudes Spatiales (Cnes) | METHOD FOR DETERMINING A SET OF MESHES FOR IMAGING A GEOGRAPHICAL AREA |
| CN105973205B (en) * | 2016-05-13 | 2019-04-02 | 中国人民解放军装备学院 | It is a kind of to intersect oblique sampling configuration high-resolution imaging method |
| CN106791405B (en) * | 2016-12-26 | 2019-05-21 | 首都师范大学 | A kind of method and system obtaining Displacement sequence image |
| CN106920213B (en) * | 2017-01-19 | 2021-02-12 | 首都师范大学 | Method and system for acquiring high-resolution image |
| CN107220936A (en) * | 2017-05-26 | 2017-09-29 | 首都师范大学 | A kind of image super-resolution reconstructing method and system |
| CN110401968B (en) * | 2019-05-09 | 2020-05-22 | 温州曼昔维服饰有限公司 | Wireless communication system based on density analysis |
| CN110849816A (en) * | 2019-12-16 | 2020-02-28 | 西南交通大学 | Ground sampling grid device applied to low-altitude unmanned remote sensing inversion of soil humidity |
-
2013
- 2013-06-09 CN CN201310231478.0A patent/CN103440619B/en not_active Expired - Fee Related
Non-Patent Citations (4)
| Title |
|---|
| 传输型光学遥感器超模式斜采样新方法研究;周峰 等;《航天返回与遥感》;20050930;第26卷(第3期);43-46页 * |
| 基于最佳倒易晶胞的斜采样遥感图像复原方法;王静，夏德深;《宇航学报》;20111130;第32卷(第11期);2451-2456页 * |
| 空域自适应滤波方法及其在斜模式遥感图像复原中的应用;郑钰辉;《中国博士学位论文全文数据库 信息科技辑》;20100715(第7期);图6.8 * |
| 超分辨率图像重建及其应用;赵荣椿;《测控技术 增刊》;20071231;第26卷;3.4部分 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103440619A (en) | 2013-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103440619B (en) | A kind of tiltedly pattern sampling modeling and super resolution ratio reconstruction method | |
| CN105303616B (en) | Embossment modeling method based on single photo | |
| CN107945217B (en) | Image characteristic point pair rapid screening method and system suitable for automatic assembly | |
| CN114283203B (en) | Calibration method and system of multi-camera system | |
| JP2005308553A (en) | Three-dimensional image measuring device and method | |
| CN106249291A (en) | A kind of high precision seismic data re-establishing method based on two-dimentional non-homogeneous warp wavelet | |
| CN107451956A (en) | A kind of reconstructing method of code aperture spectrum imaging system | |
| CN111127613B (en) | Image sequence three-dimensional reconstruction method and system based on scanning electron microscope | |
| CN109146787A (en) | A kind of real-time reconstruction method of the double camera spectrum imaging system based on interpolation | |
| CN103824323B (en) | A kind of three-dimensional rebuilding method based on single width two dimensional image | |
| CN104574338A (en) | Remote sensing image super-resolution reconstruction method based on multi-angle linear array CCD sensors | |
| CN114863258A (en) | Method for detecting small target based on visual angle conversion in sea-sky-line scene | |
| CN113920240B (en) | Three-dimensional imaging method for laboratory temperature field | |
| KR100965838B1 (en) | An Implicit Geometric Regularization of Building Polygon Using LiDAR Data | |
| CN107941241B (en) | Resolution board for aerial photogrammetry quality evaluation and use method thereof | |
| CN106920213B (en) | Method and system for acquiring high-resolution image | |
| CN116068468B (en) | MPI reconstruction method for time domain system matrix combined with x-space | |
| CN104809688A (en) | Affine Transform registration algorithm-based sheep body measuring method and system | |
| Kim et al. | Digital elevation modeling via curvature interpolation for LiDAR data | |
| CN111612844A (en) | Three-dimensional laser scanner and camera calibration method based on sector features | |
| Singh et al. | Comparison between square pixel structure and hexagonal pixel structure in digital image processing | |
| CN109191557B (en) | Image texture mapping method and device for stereoscopic topographic map | |
| CN104361630A (en) | Acquiring method of optical field of surface of human face | |
| CN110070604B (en) | Three-dimensional reconstruction method based on single revolving body image cultural relics | |
| CN117953174B (en) | Three-dimensional modeling method in geological investigation process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20180417 Termination date: 20210609 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |