CN109547798A - A kind of quick HEVC inter-frame mode selecting method - Google Patents
A kind of quick HEVC inter-frame mode selecting method Download PDFInfo
- Publication number
- CN109547798A CN109547798A CN201811540849.2A CN201811540849A CN109547798A CN 109547798 A CN109547798 A CN 109547798A CN 201811540849 A CN201811540849 A CN 201811540849A CN 109547798 A CN109547798 A CN 109547798A
- Authority
- CN
- China
- Prior art keywords
- mode
- cost
- ssd
- skip
- diff
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a kind of quick HEVC inter-frame mode selecting methods.It judges whether to skip Rect mode first in advance: if Skip prediction mode is better than Merge mode and 2Nx2N mode, using Skip prediction mode as current CU layers of optimal inter-frame mode, to skip Rect mode；Secondly predict the direction of Rect and skip second Rect mode in advance: existing method successively seeks the rate distortion costs of Nx2N mode and 2NxN mode, and other modes compare by them and 2Nx2N etc., select the mode with minimum rate distortion costs as optimal prediction mode, the present invention is then before encoding Rect mode, predict the block characteristic of current coded unit, Nx2N or 2NxN is first sought in determination, once rate distortion costs striked by first mode meet some requirements, that is, it can skip second mode.The beneficial effects of the present invention are: the speed of optimization Rect model selection promotes whole coding rate to save a large amount of time.
Description
Technical field
The present invention relates to Video coding correlative technology fields, refer in particular to a kind of quick inter mode decision side HEVC
Method.
Background technique
Video encoding standard HEVC of new generation is improved relative to the compression efficiency of previous generation video encoding standard H.264
50%, a large amount of network bandwidth can be saved in the case where providing same video quality, is very suitable to the videos such as 4K, 3D blue light
The popularization of content.
The basic premise of Video coding compression is that have correlation between pixel, has many neighborhood pixels inside a frame image
Data be it is similar or identical, here it is spatial redundancy, and the data in video between contiguous frames there is also many similar or
It repeats, here it is temporal redundancy, coding compression is exactly to remove these redundancies in data, and infra-prediction techniques can be gone
Except spatial redundancy, and inter-frame prediction techniques can then remove temporal redundancy.
One frame image is divided into several encoding blocks, and try using existing correlation between contiguous frames by inter-prediction
The each piece of corresponding match block in contiguous frames is searched out, the relative displacement of spatial position between the two is calculated, obtains
Relative displacement be exactly MV (Motion Vector, motion vector), and the process for obtaining MV is referred to as estimation, movement
Estimation procedure is complex, and the calculating time accounts for 40% or more in entire encoder.
Every frame image segmentation is several CU (Coding Unit, coding unit) by HEVC, and uses quaternary tree form pair
CU carries out Recursion process, successively traverses the CU of this 4 kinds of levels of 64x64,32x32,16x16 and 8x8.Inter-prediction is encoding each layer
When CU, successively to the inter-frame modes such as Skip, Merge, 2Nx2N, Rect (Nx2N and 2NxN) and Intra2Nx2N, IntraNxN this
Two kinds of frame modes are traversed, and are selected with the smallest mode of rate distortion costs as optimal prediction modes, wherein rate
The calculating of distortion cost RDcost are as follows:
RDcost=λ R+D
Wherein λ is Lagrange factor, and R represents number of coded bits, and D represents coded predictors relative to original video data
The distortion of value.
In each prediction mode, 2Nx2N and Rect prediction mode is since it is desired that find reference frame by estimation
In match block, computation complexity is very big.Rect prediction mode is the further division to 2Nx2N block, as Nx2N predicts mould
Formula is that 2Nx2N is divided to the block different for left and right two motion vectors, and 2NxN prediction mode is then that be divided to 2Nx2N be upper and lower two
The different block of motion vector, therefore Nx2N and 2NxN mode has different directionality.Conventional method adopts Rect prediction mode
With fixed step, successively seek the rate distortion costs of Nx2N and 2NxN, do not make full use of current CU layers traversal Skip,
Texture information acquired in Merge and 2Nx2N mode reduces the calculating process of Rect.
Summary of the invention
The present invention is in order to overcome the above deficiencies in the prior art, to provide and a kind of promote binary encoding speed
Quick HEVC inter-frame mode selecting method.
To achieve the goals above, the invention adopts the following technical scheme:
A kind of quick HEVC inter-frame mode selecting method, specific implementation step are as follows:
(1) rate distortion costs for successively seeking Skip, Merge, 2Nx2N in CU layers each, are respectively labeled as cost_
Skip, cost_merge, cost_2Nx2N, if cost_skip is less than cost_merge and cost_skip is less than cost_
2Nx2N is executed step (5) then using Skip prediction mode as current CU layers of optimal inter-frame forecast mode；It is no to then follow the steps
(2)；
(2) by the prediction block that 2Nx2N prediction mode is sought be divided into 4 sizes be NxN sub-block, be respectively labeled as a, b, c,
D block, and seek their squared differences and SSD value between original pixels block respectively, successively labeled as ssd_a, ssd_b,
Ssd_c, ssd_d, and diff_ver, diff_hor and diff_ratio are defined, if meeting diff_ratio > TH1 and diff_
Ver > diff_hor is thened follow the steps (3), no to then follow the steps (4)；
(3) rate distortion costs of 2NxN prediction mode are sought, cost_2NxN is labeled as, is less than when meeting cost_2NxN
TH2*cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of Nx2N prediction mode, is labeled as cost_Nx2N,
And execute step (5)；
(4) rate distortion costs of Nx2N prediction mode are sought, cost_Nx2N is labeled as, is less than when meeting cost_Nx2N
TH2*cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of 2NxN prediction mode, is labeled as cost_2NxN,
And execute step (5)；
(5) rate distortion costs of each mode are compared, determine optimal inter-frame forecast mode, and and
Intra2Nx2N, IntraNxN mode are compared, and further determine that current CU layers of optimal prediction modes.
Existing HEVC coding method calculates complicated height, and Rect mode is wherein occupying for 40% time.The present invention is abundant
Using Skip, Merge and 2Nx2N prediction mode done before coding Rect mode, predict that the texture of current coded unit is special
Property, optimize the speed of Rect model selection, to promote whole coding rate.Specifically, judgement is the present invention in advance first
It is no to skip Rect mode: if Skip prediction mode be better than Merge mode and 2Nx2N mode, using Skip prediction mode as
Current CU layers of optimal inter-frame mode, to skip Rect mode；Secondly it predicts the direction of Rect and skips second in advance
Rect mode: existing method successively seeks the rate distortion costs of Nx2N mode and 2NxN mode, and by they and 2Nx2N etc. other
Mode compares, and selects the mode with minimum rate distortion costs as optimal prediction mode, the present invention is then encoding
Before Rect mode, the block characteristic of current coded unit is predicted, Nx2N or 2NxN is first sought in determination, once first mode
Striked rate distortion costs meet some requirements, that is, can skip second mode, to save a large amount of time.
Preferably, defining the calculating of diff_ver, diff_hor and diff_ratio in step (2) are as follows:
Diff_ver=| (ssd_a+ssd_b)-(ssd_c+ssd_d) |
Diff_hor=| (ssd_a+ssd_c)-(ssd_b+ssd_d) |
Wherein c is constant, value 0.05；TH1 is constant, and value range is 1.0~10.0.
Preferably, TH2 is constant, and value range is 0.5~1.0 in step (3) and step (4).
The beneficial effects of the present invention are: the speed of optimization Rect model selection is promoted whole to save a large amount of time
The coding rate of body.
Detailed description of the invention
Fig. 1 is flow chart of the invention.
Specific embodiment
The present invention will be further described with reference to the accompanying drawings and detailed description.
In embodiment as described in Figure 1, a kind of quick HEVC inter-frame mode selecting method, specific implementation step is as follows:
(1) rate distortion costs for successively seeking Skip, Merge, 2Nx2N in CU layers each, are respectively labeled as cost_
Skip, cost_merge, cost_2Nx2N, if cost_skip is less than cost_merge and cost_skip is less than cost_
2Nx2N is executed step (5) then using Skip prediction mode as current CU layers of optimal inter-frame forecast mode；It is no to then follow the steps
(2)；
In step (1) using Skip prediction mode as the accuracy rate of current CU layers of optimal inter-frame forecast mode be 82%,
The optimization model that the optimization model determined using step (1) and conventional method are determined is identical, that is, it is accurate to think.
(2) by the prediction block that 2Nx2N prediction mode is sought be divided into 4 sizes be NxN sub-block, be respectively labeled as a, b, c,
D block, and seek their squared differences and SSD value between original pixels block respectively, successively labeled as ssd_a, ssd_b,
Ssd_c, ssd_d, and diff_ver, diff_hor and diff_ratio are defined, it calculates are as follows:
Diff_ver=| (ssd_a+ssd_b)-(ssd_c+ssd_d) |
Diff_hor=| (ssd_a+ssd_c)-(ssd_b+ssd_d) |
Wherein c is constant, value 0.05；TH1 is constant, and value range is 1.0~10.0；If meeting diff_ratio
> TH1 and diff_ver > diff_hor are thened follow the steps (3), no to then follow the steps (4)；
Judge that the accuracy rate of the priority of Nx2N and 2NxN is related to the selection of TH1 value in step (2), specifically refers to
Table 1, if step (2), which determines, first seeks Nx2N, and the cost_Nx2N that conventional method is sought is less than cost_2NxN, that is, thinks quasi-
Really, if step (2), which determines, first seeks 2NxN, and the cost_2NxN that conventional method is sought is less than cost_Nx2N, that is, it is accurate to think.
(3) rate distortion costs of 2NxN prediction mode are sought, cost_2NxN is labeled as, is less than when meeting cost_2NxN
TH2*cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of Nx2N prediction mode, is labeled as cost_Nx2N,
And execute step (5)；Wherein TH2 is constant, and value range is 0.5~1.0；
Judge whether that the accuracy rate for executing step (5) in advance is related to the selection of TH2 value in step (3), specifically refers to
Table 2, if the cost_2NxN that step (3) determines is less than TH2*cost_2Nx2N, and the cost_2NxN that conventional method is sought is less than
Cost_Nx2N, that is, it is accurate to think.
(4) rate distortion costs of Nx2N prediction mode are sought, cost_Nx2N is labeled as, is less than when meeting cost_Nx2N
TH2*cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of 2NxN prediction mode, is labeled as cost_2NxN,
And execute step (5)；Wherein TH2 is constant, and value range is 0.5~1.0；
Judge whether that the accuracy rate for executing step (5) in advance is related to the selection of TH2 value in step (4), specifically refers to
Table 2, if the cost_Nx2N that step (4) determines is less than TH2*cost_2Nx2N, and the cost_Nx2N that conventional method is sought is less than
Cost_2NxN, that is, it is accurate to think.
(5) rate distortion costs of each mode are compared, determine optimal inter-frame forecast mode, and and
Intra2Nx2N, IntraNxN mode are compared, and further determine that current CU layers of optimal prediction modes.
Existing HEVC coding method calculates complicated height, and Rect mode is wherein occupying for 40% time.The present invention is abundant
Using Skip, Merge and 2Nx2N prediction mode done before coding Rect mode, predict that the texture of current coded unit is special
Property, optimize the speed of Rect model selection, to promote whole coding rate.Specifically, judgement is the present invention in advance first
It is no to skip Rect mode: if Skip prediction mode be better than Merge mode and 2Nx2N mode, using Skip prediction mode as
Current CU layers of optimal inter-frame mode, to skip Rect mode；Secondly it predicts the direction of Rect and skips second in advance
Rect mode: existing method successively seeks the rate distortion costs of Nx2N mode and 2NxN mode, and by they and 2Nx2N etc. other
Mode compares, and selects the mode with minimum rate distortion costs as optimal prediction mode, the present invention is then encoding
Before Rect mode, the block characteristic of current coded unit is predicted, Nx2N or 2NxN is first sought in determination, once first mode
Striked rate distortion costs meet some requirements, that is, can skip second mode, to save a large amount of time.This hair
It is bright to make HEVC video encoder in the improvement situation of encoder bit rate and coding rate such as table 3.
The selection and the accuracy rate of the priority of Rect of table 1.TH1 value
| TH1 | Accuracy rate |
| 2 | 72% |
| 4 | 76% |
The selection of table 2.TH2 value and the accuracy rate for skipping second Rect in advance
| TH2 | Accuracy rate |
| 0.95 | 85% |
| 0.98 | 81% |
The selection and coding result of table 3.TH1 and TH2 value
| TH1 | TH2 | Encoder bit rate | Coding rate |
| 2 | 0.95 | + 0.4% | + 21% |
| 2 | 0.98 | + 0.5% | + 22% |
| 4 | 0.95 | + 0.3% | + 18% |
| 4 | 0.98 | + 0.4% | + 19% |
Claims (3)
1. a kind of quick HEVC inter-frame mode selecting method, characterized in that specific implementation step is as follows:
(1) rate distortion costs for successively seeking Skip, Merge, 2Nx2N in CU layers each, be respectively labeled as cost_skip,
Cost_merge, cost_2Nx2N, if cost_skip is less than cost_merge and cost_skip is less than cost_2Nx2N,
Using Skip prediction mode as current CU layers of optimal inter-frame forecast mode, execute step (5)；It is no to then follow the steps (2)；
(2) prediction block that 2Nx2N prediction mode is sought is divided into the sub-block that 4 sizes are NxN, is respectively labeled as a, b, c, d block,
And seek their squared differences and SSD value between original pixels block respectively, successively labeled as ssd_a, ssd_b, ssd_c,
Ssd_d, and define diff_ver, diff_hor and diff_ratio, if meet diff_ratio > TH1 and diff_ver >
Diff_hor is thened follow the steps (3), no to then follow the steps (4)；
(3) rate distortion costs of 2NxN prediction mode are sought, cost_2NxN is labeled as, are less than TH2* when meeting cost_2NxN
Cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of Nx2N prediction mode, is labeled as cost_Nx2N, and hold
Row step (5)；
(4) rate distortion costs of Nx2N prediction mode are sought, cost_Nx2N is labeled as, are less than TH2* when meeting cost_Nx2N
Cost_2Nx2N executes step (5), otherwise seeks the rate distortion costs of 2NxN prediction mode, is labeled as cost_2NxN, and hold
Row step (5)；
(5) rate distortion costs of each mode are compared, determine optimal inter-frame forecast mode, and with Intra2Nx2N,
IntraNxN mode is compared, and further determines that current CU layers of optimal prediction modes.
2. a kind of quick HEVC inter-frame mode selecting method according to claim 1, characterized in that in step (2),
Define the calculating of diff_ver, diff_hor and diff_ratio are as follows:
Diff_ver=| (ssd_a+ssd_b)-(ssd_c+ssd_d) |
Diff_hor=| (ssd_a+ssd_c)-(ssd_b+ssd_d) |
Wherein c is constant, value 0.05；TH1 is constant, and value range is 1.0~10.0.
3. a kind of quick HEVC inter-frame mode selecting method according to claim 1, characterized in that in step (3) and
In step (4), TH2 is constant, and value range is 0.5~1.0.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811540849.2A CN109547798B (en) | 2018-12-17 | 2018-12-17 | Rapid HEVC inter-frame mode selection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811540849.2A CN109547798B (en) | 2018-12-17 | 2018-12-17 | Rapid HEVC inter-frame mode selection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109547798A true CN109547798A (en) | 2019-03-29 |
| CN109547798B CN109547798B (en) | 2020-09-18 |
Family
ID=65855091
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811540849.2A Active CN109547798B (en) | 2018-12-17 | 2018-12-17 | Rapid HEVC inter-frame mode selection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109547798B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110611819A (en) * | 2019-08-01 | 2019-12-24 | 杭州当虹科技股份有限公司 | Coding method for improving quality of B frame in H.264 |
| WO2021238546A1 (en) * | 2020-05-25 | 2021-12-02 | 腾讯科技（深圳）有限公司 | Video encoding method, video playing back method, related devices and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1753501A (en) * | 2005-10-31 | 2006-03-29 | 连展科技(天津)有限公司 | Method of selecting H.264/AVC frame to frame motion estimation mode |
| CN101141647A (en) * | 2007-08-24 | 2008-03-12 | 上海广电（集团）有限公司中央研究院 | AVS video coding based fast intraframe predicting mode selecting method |
| CN107343198A (en) * | 2017-05-08 | 2017-11-10 | 上海大学 | A kind of quick decision method of AVS2 inter-frame forecast modes |
| WO2018175409A1 (en) * | 2017-03-20 | 2018-09-27 | Qualcomm Incorporated | Enhanced deblocking filtering design in video coding |
-
2018
- 2018-12-17 CN CN201811540849.2A patent/CN109547798B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1753501A (en) * | 2005-10-31 | 2006-03-29 | 连展科技(天津)有限公司 | Method of selecting H.264/AVC frame to frame motion estimation mode |
| CN101141647A (en) * | 2007-08-24 | 2008-03-12 | 上海广电（集团）有限公司中央研究院 | AVS video coding based fast intraframe predicting mode selecting method |
| WO2018175409A1 (en) * | 2017-03-20 | 2018-09-27 | Qualcomm Incorporated | Enhanced deblocking filtering design in video coding |
| CN107343198A (en) * | 2017-05-08 | 2017-11-10 | 上海大学 | A kind of quick decision method of AVS2 inter-frame forecast modes |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110611819A (en) * | 2019-08-01 | 2019-12-24 | 杭州当虹科技股份有限公司 | Coding method for improving quality of B frame in H.264 |
| WO2021238546A1 (en) * | 2020-05-25 | 2021-12-02 | 腾讯科技（深圳）有限公司 | Video encoding method, video playing back method, related devices and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109547798B (en) | 2020-09-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103873861B (en) | Coding mode selection method for HEVC (high efficiency video coding) | |
| CN111147867B (en) | Multifunctional video coding CU partition rapid decision-making method and storage medium | |
| CN110087087B (en) | VVC inter-frame coding unit prediction mode early decision and block division early termination method | |
| CN106961606B (en) | HEVC intra-frame coding mode selection method based on texture division characteristics | |
| CN108347616B (en) | Depth prediction method and device based on optional time domain motion vector prediction | |
| CN102984521B (en) | High-efficiency video coding inter-frame mode judging method based on temporal relativity | |
| CN104378643B (en) | A kind of 3D video depths image method for choosing frame inner forecast mode and system | |
| CN108781284A (en) | The method and device of coding and decoding video with affine motion compensation | |
| CN107371022B (en) | Inter-frame coding unit rapid dividing method applied to HEVC medical image lossless coding | |
| CN103997645B (en) | Quick HEVC intra-frame coding unit and pattern decision method | |
| CN103327327B (en) | For the inter prediction encoding unit selection method of high-performance video coding HEVC | |
| CN105141954A (en) | HEVC interframe coding quick mode selection method | |
| CN105208387A (en) | HEVC intra-frame prediction mode fast selection method | |
| CN110557646B (en) | Intelligent inter-view coding method | |
| CN105430415A (en) | Fast intraframe coding method of 3D-HEVC depth videos | |
| CN105791826A (en) | Data mining-based HEVC inter-frame fast mode selection method | |
| CN106604035B (en) | A method of the estimation for Video coding and compression | |
| Chen et al. | A novel fast intra mode decision for versatile video coding | |
| CN108076347A (en) | A kind of acquisition methods and device for encoding starting point | |
| CN110446052A (en) | The quick CU depth selection method of depth map in a kind of 3D-HEVC frame | |
| CN109660811B (en) | Rapid HEVC inter-frame coding method | |
| CN101895761B (en) | Quick intraframe prediction algorithm | |
| CN109547798A (en) | A kind of quick HEVC inter-frame mode selecting method | |
| CN106878754B (en) | A kind of 3D video depth image method for choosing frame inner forecast mode | |
| CN103595991B (en) | The Pixel-level Forecasting Methodology of deep video coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |