CN109640087B - Intra-frame prediction mode judgment method, device and equipment - Google Patents
Intra-frame prediction mode judgment method, device and equipment Download PDFInfo
- Publication number
- CN109640087B CN109640087B CN201811644745.6A CN201811644745A CN109640087B CN 109640087 B CN109640087 B CN 109640087B CN 201811644745 A CN201811644745 A CN 201811644745A CN 109640087 B CN109640087 B CN 109640087B
- Authority
- CN
- China
- Prior art keywords
- intra
- prediction mode
- frame prediction
- angle
- coded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a method for judging an intra-frame prediction mode, which comprises the steps of analyzing the effectiveness of adjacent macro blocks of a current macro block to be coded and determining the effective intra-frame prediction mode of the macro block to be coded; if the number of the effective intra-frame prediction modes exceeds a preset threshold value, calculating the direction angle of image texture in the macro block to be coded; according to a preset matching rule, determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from effective intra-frame prediction modes related to all angles of a macro block to be coded; and respectively carrying out intra-frame prediction on the macro block to be coded in the determined intra-frame prediction mode and the DC prediction mode, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded. The method can reduce the number of the intra-frame prediction modes related to the angle to be traversed, thereby reducing the calculated amount in the pre-analysis process and improving the intra-frame prediction efficiency. The invention also discloses a device, equipment and a computer readable storage medium based on the method.
Description
Technical Field
The present invention relates to the field of intra prediction technologies, and in particular, to a method, an apparatus, a device, and a computer readable storage medium for intra prediction mode decision.
Background
AVS is the second generation source coding standard with independent intellectual property rights in China. The AVS + is an extended version based on the AVS, is mainly used for the standard of the broadcasting and television industry, and has obviously improved compression efficiency.
The intra-frame prediction is one of core technologies of AVS and AVS + video coding standards, and means that the correlation of a video spatial domain is utilized, and the coded pixels of the current image are used for predicting the current pixels so as to achieve the purpose of removing video spatial redundancy. The intra 8x8 prediction modes of each macroblock (Macro Block, MB) of AVS, AVS + are up to 5, including DC mode and 4 angle dependent modes. Wherein, each image frame is divided into a plurality of macroblocks with the same size, the 8x8 mode means that the brightness component of each macroblock is divided into 8x8 brightness points, each macroblock is used as a basic unit of intra-frame prediction, and prediction and coding are performed on each macroblock.
At present, in the AVS or AVS + encoding process, if intra-frame prediction is desired, the 5 intra-frame prediction modes need to be traversed first, then one intra-frame prediction mode with the lowest distortion rate is determined as a final intra-frame prediction mode, and then intra-frame prediction is completed according to the final intra-frame prediction mode. In the process, all intra-frame prediction modes need to be traversed in advance for pre-analysis, so that the calculation amount of the final intra-frame prediction mode is large, and the efficiency of the intra-frame prediction mode is low.
Therefore, how to provide a method, an apparatus, a device and a computer readable storage medium for intra prediction mode decision with small computation amount and high efficiency is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide an intra-frame prediction mode judgment method which can reduce the number of intra-frame prediction modes related to angles needing to be traversed, thereby reducing the calculated amount in the pre-analysis process and improving the intra-frame prediction efficiency; the invention further aims to provide a device, equipment and a computer readable storage medium based on the method.
In order to solve the above technical problem, the present invention provides an intra prediction mode decision method, including:
analyzing the effectiveness of adjacent macro blocks of a current macro block to be coded, and determining the effective intra-frame prediction mode of the macro block to be coded;
judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold value, and if so, calculating the direction angle of image texture in the macro block to be coded; the preset threshold is a positive integer greater than 2;
according to a preset matching rule, determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from effective intra-frame prediction modes related to all angles of the macro block to be coded;
and respectively carrying out intra-frame prediction on the macro block to be coded in the determined intra-frame prediction mode and the DC prediction mode, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
Preferably, if the number of the effective intra-frame prediction modes does not exceed a preset threshold, the intra-frame prediction of the macroblock to be encoded is performed in each effective intra-frame prediction mode of the macroblock to be encoded, and the prediction mode with the best intra-frame prediction result is selected as the final intra-frame prediction mode of the macroblock to be encoded.
Preferably, the process of determining, according to a preset matching rule, an intra prediction mode with an angle closest to a direction angle of the image texture from among the effective intra prediction modes associated with the respective angles of the macroblock to be encoded includes:
and respectively calculating the difference value between the direction angle of the image texture and the angle of the effective intra-frame prediction mode related to each angle of the macro block to be coded, and taking the effective intra-frame prediction mode with the minimum difference value as the intra-frame prediction mode matched with the direction angle of the image texture.
The process of determining the intra prediction mode with the angle closest to the direction angle of the image texture from the effective intra prediction modes related to all angles of the macro block to be coded according to the preset matching rule comprises the following steps:
and comparing the direction angle of the image texture with a preset angle range interval corresponding to the effective intra-frame prediction mode related to each angle, and taking the effective intra-frame prediction mode corresponding to the angle range interval to which the direction angle of the image texture belongs as the intra-frame prediction mode matched with the direction angle of the image texture.
Preferably, the method further comprises the following steps:
if the direction angle of the image texture exceeds the angle range of the effective intra-frame prediction mode related to each angle of the macro block to be coded, the process of determining the intra-frame prediction mode with the angle closest to the direction angle of the image texture from the effective intra-frame prediction modes related to each angle of the macro block to be coded according to a preset matching rule comprises the following steps:
preferably, the macroblock to be encoded is applied to an AVS or AVS + encoding process, and the preset threshold is 5.
Preferably, the process of calculating the direction angle of the image texture in the macroblock to be coded according to the gradient value is as follows:
calculating gradient values of all brightness points in the macro block to be coded;
calculating the direction angles of all brightness points in the macro block to be coded according to the gradient value and the direction angle relational expression; the direction angle relation is as follows:
θ＝arctan(yGradient/xGradient),
wherein, theta is a direction angle, xGradient is a gradient value in the horizontal direction, and yGradient is a gradient value in the vertical direction;
and calculating the mean value of the direction angles of all the brightness points in the macro block to be coded, namely the direction angle of the image texture.
In order to solve the above technical problem, the present invention further provides an intra prediction mode decision device, including:
the mode number analysis module is used for analyzing the effectiveness of adjacent macro blocks of a current macro block to be coded and determining the effective intra-frame prediction mode of the macro block to be coded;
the threshold value judging module is used for judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold value or not, and if so, the angle calculating module is triggered; the preset threshold is a positive integer greater than 2;
the angle calculation module is used for calculating the direction angle of the image texture in the macro block to be coded;
the matching module is used for determining the intra-frame prediction mode with the angle closest to the direction angle of the image texture from the effective intra-frame prediction modes related to all angles of the macro block to be coded according to a preset matching rule;
and the fast prediction module is used for performing intra-frame prediction on the macro block to be coded under the determined intra-frame prediction mode and the DC prediction mode respectively, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
In order to solve the above technical problem, the present invention further provides an intra prediction mode decision device, including:
a memory for storing a computer program;
a processor for implementing the steps of the intra prediction mode decision method as claimed in any of the above when executing the computer program.
To solve the above technical problem, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when being executed by a processor, implements the steps of the intra prediction mode decision method according to any one of the above.
The invention provides an intra-frame prediction mode decision method, firstly determining the effective intra-frame prediction modes of a current macro block to be coded, if the number of the current effective intra-frame prediction modes exceeds a preset threshold (the preset threshold is more than 2, namely the number of the current effective intra-frame prediction modes at least comprises a DC mode and 2 angle-related modes), namely the number of the current effective intra-frame prediction modes is larger, if all traversals are subjected to pre-analysis, the calculated amount is larger, therefore, the intra-frame prediction mode with the angle closest to the direction angle of an image texture is selected in the effective intra-frame prediction modes related to all angles of the macro block to be coded by calculating the direction angle of the image texture of the macro block to be coded; and then, pre-analyzing the selected angle-related intra-frame prediction mode and the DC prediction mode, and taking the optimal result as the final intra-frame prediction mode. Therefore, when the number of the effective intra-frame prediction modes related to the angle is not less than 2, in order to reduce the workload during pre-analysis, the effective intra-frame prediction modes related to the angle are primarily screened in a direction angle matching mode of direction angle image textures, so that the number of the intra-frame prediction modes related to the angle needing to be traversed is only 1 in the subsequent pre-analysis process, the number of the intra-frame prediction modes related to the angle needing to be traversed is reduced, the calculation amount in the pre-analysis process is greatly reduced, the calculation amount in the intra-frame prediction mode selection process is further reduced, and the intra-frame prediction efficiency is improved. The invention also provides a device, equipment and a computer readable storage medium based on the method.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed in the prior art and the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a flowchart illustrating a process of an intra prediction mode decision method according to the present invention;
FIG. 2 is a flow chart of another method for determining intra prediction mode according to the present invention;
FIG. 3 illustrates 4x4 intra prediction modes and prediction directions;
FIG. 4 is a diagram illustrating a mode 0 vertical prediction mode;
FIG. 5 is a schematic diagram of mode 1 horizontal prediction modes;
FIG. 6 is a diagram of mode 3 lower left prediction modes;
FIG. 7 is a diagram of mode 4 lower right prediction modes;
FIG. 8 is a schematic diagram of a first angular range of direction angles;
FIG. 9 is a schematic diagram of a second angular range of direction angles;
FIG. 10 is a schematic view of a third angular range of azimuth angles;
FIG. 11 is a fourth angular range diagram of the azimuth angle;
FIG. 12 is a representation of the gradient computation of X, Y, exemplified by the Roberts operator;
fig. 13 is a schematic structural diagram of an intra prediction mode decision device according to the present invention.
Detailed Description
The core of the invention is to provide an intra-frame prediction mode judgment method, which can reduce the number of angle-related intra-frame prediction modes needing traversal, thereby reducing the calculated amount in the pre-analysis process and improving the intra-frame prediction efficiency; the invention also provides a device, equipment and a computer readable storage medium based on the method.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims of the present application and in the drawings described above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In addition, technical solutions between the embodiments may be combined with each other, but must be based on the realization of the technical solutions by a person skilled in the art, and when the technical solutions are contradictory to each other or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
The present invention provides a method for deciding an intra-frame prediction mode, as shown in fig. 1, fig. 1 is a flowchart of a process of a method for deciding an intra-frame prediction mode according to the present invention; the method comprises the following steps:
step s1: analyzing the effectiveness of adjacent macro blocks of the current macro block to be coded, and determining the effective intra-frame prediction mode of the macro block to be coded;
it can be understood that, because the front-back correlation exists between the continuous pixel points in the image of one frame, the image prediction can be performed by adopting a prediction coding mode, and at this time, the output after the prediction coding is no longer the value of the original pixel point, but is the difference between the predicted value and the actual value of the pixel point. The starting point of the design of the predictive coding is that due to the fact that the front and the back are related, a large number of identical or similar phenomena exist in adjacent pixel points, and the data volume for storing and transmitting the value of the original pixel point can be greatly reduced by calculating the difference value of the adjacent pixel points. The predictive coding comprises inter-frame prediction and intra-frame prediction, wherein the intra-frame prediction refers to comparison between pixel points in the same frame of image, and finally output coding is obtained according to the difference value. In addition, the intra prediction includes both luminance and chrominance portions, and although the sizes of luminance and chrominance are different, the principles of the intra prediction are consistent. The present invention is directed to intra prediction of luminance.
Since the intra prediction is to predict pixels that are not encoded later using already encoded pixels, the prediction modes include a DC prediction mode and an angle-dependent prediction mode, wherein the angle-dependent prediction mode refers to: the predicted value of each pixel point is obtained by predicting the value of the adjacent pixel point in the corresponding angle direction in the mode. The DC prediction mode refers to representing the prediction value of the entire pixel to be predicted by the direction angles of the image textures of the upper and left neighboring pixels. In one image, because of different pixel positions, direction angles of pixels which are already coded around the image relative to a current pixel to be predicted are different, and therefore, for a part of macroblocks, some angle-related prediction modes cannot be used, the invention needs to analyze the prediction modes first and determine which effective intra-frame prediction modes corresponding to the current macroblock to be coded are respectively.
Step s2: judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold value, and if so, entering a step s3; the preset threshold value is a positive integer greater than 2;
the invention aims to solve the problem that the calculation amount is large because all intra-frame prediction modes need to be traversed when the final intra-frame prediction mode is determined in the pre-analysis stage at present, and the method is used for screening angle-related effective intra-frame prediction modes, reducing the angle-related effective intra-frame prediction modes into 1 type and then performing subsequent traversal. Therefore, when the number of effective intra prediction modes is only 2, that is, the effective intra prediction modes only include the DC prediction mode and one angle-dependent prediction mode, the process of subsequently screening the angle-dependent effective intra prediction modes is not meaningful, and the purpose of reducing the amount of calculation cannot be achieved. Therefore, the invention only screens the angle-related effective intra-frame prediction modes to reduce the number of the effective intra-frame prediction modes when the number of the effective intra-frame prediction modes is larger than a preset threshold value and the preset threshold value is larger than 2, namely the angle-related effective intra-frame prediction modes are more than one, and adopts a direct traversal mode when the angle-related effective intra-frame prediction modes are only 1 or not, thereby ensuring that the calculation amount of the intra-frame prediction mode judgment process adopts a mode of minimum calculation amount for any macro block as far as possible.
And step s3: calculating the direction angle of image texture in the macro block to be coded;
the direction angle of the image texture of the direction angles of all the luminance points is the average angle of the macro block to be coded in the direction trend of the image texture.
It can be understood that, when performing intra prediction, the angular dependent intra prediction mode predicts the current pixel according to the encoded pixels in the angular direction of the intra prediction mode. Therefore, the angle of the intra prediction mode associated with the most suitable angle of a macroblock to be coded is associated with, or matches, the image texture direction of the macroblock to be coded. Therefore, the present invention obtains the angle information of the image texture of the macroblock to be encoded (i.e. the direction angle of the image texture of the direction angle) by calculating the gradient value of the luminance point in the macroblock to be encoded, and then the intra-frame prediction mode with the angle closest to the direction angle of the image texture is the intra-frame prediction mode with the prediction encoding direction closest to the image texture of the macroblock to be encoded, i.e. the intra-frame prediction mode related to the most suitable angle of the macroblock to be encoded.
And step s4: according to a preset matching rule, determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from effective intra-frame prediction modes related to all angles of a macro block to be coded;
and step s5: and respectively carrying out intra-frame prediction on the macro block to be coded in the determined intra-frame prediction mode and the DC prediction mode, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
It is understood that the DC prediction mode refers to representing the prediction value of the entire pixel to be predicted by the direction angles of the image textures of the upper and left neighboring pixels. Since the macroblock performs intra prediction along the direction from left to right and from top to bottom, the effective intra prediction mode of the macroblock must include the DC prediction mode under normal conditions. Therefore, after the most suitable one of the angle-dependent intra prediction modes is selected from the plurality of angle-dependent intra prediction modes, the intra prediction is pre-analyzed in the angle-dependent intra prediction mode and the DC prediction mode, and the one with the best result (i.e., the lowest distortion rate) is used as the final intra prediction mode.
After the final intra-frame prediction mode is selected, intra-frame prediction coding can be directly carried out on the macro block to be coded according to the final intra-frame prediction mode; or the process can be continuously repeated for the next macro block to be coded until all the macro blocks in the image frame to be coded determine the final intra-frame prediction mode, and then the intra-frame prediction coding is uniformly carried out on all the macro blocks in the image frame to be coded. The present invention is not limited to the specific method.
The invention provides an intra-frame prediction mode judging method, firstly determining the effective intra-frame prediction modes of a current macro block to be coded, if the number of the current effective intra-frame prediction modes exceeds a preset threshold (the preset threshold is more than 2, namely the number of the current effective intra-frame prediction modes at least comprises a DC mode and 2 angle-related modes), namely the number of the current effective intra-frame prediction modes is larger, if all traversals are subjected to pre-analysis, the calculated amount is larger, therefore, the invention obtains the direction angles of image textures of the direction angles of all brightness points in the macro block to be coded by calculating the gradient value of the brightness points in the macro block to be coded, and then, in the effective intra-frame prediction modes related to all angles of the macro block to be coded, the intra-frame prediction mode with the angle closest to the direction angle of the image texture is selected; and then, pre-analyzing the selected angle-related intra-frame prediction mode and the DC prediction mode, and taking the optimal result as the final intra-frame prediction mode. Therefore, when the number of the effective intra-frame prediction modes related to the angle is not less than 2, in order to reduce the workload during pre-analysis, the effective intra-frame prediction modes related to the angle are primarily screened in a direction angle matching mode of direction angle image textures, so that the number of the intra-frame prediction modes related to the angle needing to be traversed is only 1 in the subsequent pre-analysis process, the number of the intra-frame prediction modes related to the angle needing to be traversed is reduced, the calculation amount in the pre-analysis process is greatly reduced, the calculation amount in the intra-frame prediction mode selection process is further reduced, and the intra-frame prediction efficiency is improved.
In a preferred embodiment, referring to fig. 2, fig. 2 is a flowchart of a process of another intra prediction mode decision method provided by the present invention; the method further comprises the following steps: if the number of the effective intra-frame prediction modes does not exceed the preset threshold, entering step s6;
and step s6: and respectively carrying out intra-frame prediction on the macro block to be coded under each effective intra-frame prediction mode of the macro block to be coded, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
It can be understood that when the number of intra prediction modes is small, if the direction angle is still calculated additionally and the direction angle matching operation is performed, the overall calculation amount is larger, and therefore, in this case, the present invention adopts the original mode of traversing the effective intra prediction modes to determine the intra prediction modes. So that the determination of the intra prediction mode can be performed in a computationally as small as possible manner in each case, thereby reducing the overall amount of computation in determining the intra prediction mode.
In a preferred embodiment, the process of step s4 comprises:
and respectively calculating the difference between the direction angle of the image texture and the angle of the effective intra-frame prediction mode related to each angle of the macro block to be coded, and taking the effective intra-frame prediction mode with the minimum difference as the intra-frame prediction mode matched with the direction angle of the image texture.
In this embodiment, since the purpose of step s4 is to determine the intra prediction mode with the angle closest to the direction angle of the image texture, the difference between the direction angle of the image texture and the effective intra prediction mode related to each angle can be directly made, and the intra prediction mode with the smallest difference is the closest intra prediction mode.
In a preferred embodiment, the process of step s4 comprises:
and comparing the direction angle of the image texture with a preset angle range interval corresponding to the effective intra-frame prediction mode related to each angle, and taking the effective intra-frame prediction mode corresponding to the angle range interval to which the direction angle of the image texture belongs as the intra-frame prediction mode matched with the direction angle of the image texture.
It can be understood that, since the angle corresponding to each angle-dependent intra prediction mode is fixed, in the present embodiment, a preset angle range section corresponding to each angle-dependent intra prediction mode is divided in advance, and the preset angle range section refers to a direction angle range of the image texture matched with the corresponding intra prediction mode. For example, in the same coordinate axis, if the pattern 1 corresponds to 0 °, the preset angle range interval corresponding to the pattern 1 is-22.5 ° to 22.5 °, that is, the direction angles of the image texture belonging to the range are all matched with the pattern 1. It should be noted that, when the image texture is located on the boundary line of the preset angle range interval, it indicates that the matching degree of the two intra-frame prediction modes corresponding to the direction angle of the image texture and the boundary line is the same, in this case, any one of the intra-frame prediction modes can be selected as the matching degree of the direction angle of the image texture; alternatively, both of these modes may be used as intra prediction modes for matching the direction angles of the image texture. The present invention is not limited in this regard, as it is specifically selected. The setting mode of the embodiment only needs to check the angle range interval to which the direction angle of the image texture belongs, and does not need to calculate the difference between the direction angle of the image texture and the angle corresponding to each intra-frame prediction mode, so that the calculation amount of the direction angle matching process of the image texture is reduced, and the efficiency of intra-frame prediction mode judgment is improved.
It should be noted that, in order to ensure the correctness of the angle difference, the angle values of the effective intra prediction modes related to the direction angle of the image texture and each angle obtained by the present invention need to be based on the absolute angle value of the same coordinate axis, so that the angle difference value obtained by calculation can accurately reflect the matching degree between the direction angle of the image texture and the effective intra prediction modes related to each angle.
Preferably, the coordinate axis here may be a coordinate axis with the abscissa axis being the horizontal direction and the ordinate axis being the vertical direction, and 0 ° being the positive half axis of the abscissa axis. In addition, if the macroblock to be encoded in the present invention is applied to the AVS or AVS + encoding process, since the AVS or AVS + encoding only includes 5 prediction modes (DC +4 angle-dependent prediction modes), as shown in fig. 3 to 7 and table 1, the upper and left pixels c and r of the 8 × 8 luminance block are already encoded and reconstructed pixels and are used as prediction reference pixels in the codec. Pixels which are not marked with numbers in the blank spaces are pixels to be predicted, and the pixels are realized by using pixel values of r rows and c columns and corresponding intra-frame prediction modes. The angular range span of the 4 angular-dependent prediction modes is 180 °, so that the angles of all intra-frame prediction modes can be normalized by taking the mode 3 as a reference; i.e. the angular position of mode 3 is considered to be 0 deg., the remaining angles increase in rotation in the counter-clockwise direction, and the normalized results are shown in table 2. Then, after the direction angle of the image texture is calculated according to the normal coordinate axis angle (namely, the abscissa axis is the horizontal direction, the ordinate axis is the coordinate axis in the vertical direction, and 0 degree is the positive half axis of the abscissa axis), the direction angle of the image texture is converted into an angle value in the coordinate axis taking the mode 3 as the reference, and then the comparison is carried out according to the converted angle and the angle of the effective intra-frame prediction mode related to each angle. Of course, the above are only some specific implementation manners, and the present invention is not limited to the arrangement manner of the coordinate axes.
TABLE 1 Direction description of Intra prediction modes of AVS, AVS +
| Mode(s) | Description of the invention |
| Mode 0 (vertical) | In the vertical direction |
| Mode 1 (horizontal) | In the horizontal direction |
| Mode 2 (DC) | ———— |
| Mode 3 (lower left diagonal) | Vertically deviated from left by 45 degrees |
| Mode 4 (lower right diagonal) | Vertically inclined to the right by 45 degrees |
TABLE 2 Angle information normalization of AVS, AVS +
| Intra mode | Original angle information | Normalized |
| 3 | Vertically deviated from left by 45 |
0 |
| 0 | Is perpendicular to | 45 |
| 4 | Vertically inclined to the right by 45 degrees | 90 |
| 1 | In the horizontal direction | 135 degree |
Preferably, step s4 further includes:
if the direction angle of the image texture exceeds the angle range of the effective intra prediction mode associated with each angle of the macroblock to be coded, step s6 is entered.
It can be understood that when the macroblock to be encoded is applied to the AVS or AVS + encoding process, the angle range corresponding to the angle-dependent prediction mode included in the AVS or AVS + encoding is 180 ° instead of 360 °. In this case, if the pattern 1 is 0 °, the angle range corresponding to the intra prediction mode in AVS or AVS + encoding is only-125 ° to 45 °, and the other angles exceed the angle range, and in this case, if the direction angle of the image texture exceeds the angle range, the effective intra mode for direction angle matching of the image texture cannot be determined by the angle matching method, and therefore, the procedure must proceed to step s6 to perform prediction by the full traversal method. By the method, the situation that the intra-frame prediction mode is wrongly matched when the direction angle of the image texture exceeds the angle range of the intra-frame prediction mode can be avoided, and the accuracy of the intra-frame prediction mode judgment is improved.
Preferably, the preset threshold is 5 when the macroblock to be encoded is applied to the AVS or AVS + encoding process.
It can be understood that, in the AVS or AVS + encoding process, only 5 intra prediction modes are included, and therefore, in the present embodiment, when the effective intra prediction mode corresponding to the macroblock to be encoded is 5, that is, when the intra prediction modes related to the angle are 4, the intra prediction mode screening is performed. It should be noted that, since the present invention also needs a certain amount of computation for calculating the direction angle of the image texture, when the preset threshold is set to be small, the advantages of the present invention are not obvious, but when the preset threshold is set to be large, such as 5, the present invention can simplify the process of traversing 4 angle-related intra prediction modes to only traverse 1 angle-related intra prediction mode, thereby greatly simplifying the process of determining the intra prediction mode. Of course, the present invention is not limited to the specific setting value of the preset threshold.
Wherein, the macroblock to be coded is an 8 × 8 macroblock.
It can be understood that, because the macroblock partition is too small, the number of macroblocks in one frame to be encoded is large, which results in a large amount of calculation when the whole frame to be encoded performs intra-frame prediction; if the macroblock partition is too large, the image contained in one macroblock may contain different textures, that is, the part that needs to be predicted by different intra prediction modes originally is the macroblock, because the unit of intra prediction is the macroblock due to too large macroblock partition, the selected intra prediction mode cannot be well applied to different parts of one macroblock, and the intra prediction result is affected. On the other hand, the size division of the 8x8 macro block is more suitable than that of the 4x4 or 16x16 macro block, and the above problem can be avoided as much as possible. Of course, the macro blocks may be 4x4 or 16x16, which is not limited in the present invention.
Wherein, the process of calculating the direction angle of the image texture in the macro block to be coded according to the gradient value comprises the following steps:
calculating gradient values of all brightness points in a macro block to be coded;
calculating the direction angles of all brightness points in the macro block to be coded according to the gradient value and the direction angle relational expression; the direction angle relation is as follows:
θ＝arctan(yGradient/xGradient),
wherein, theta is a direction angle, xGradient is a gradient value in the horizontal direction, and yGradient is a gradient value in the vertical direction;
and calculating the mean value of the direction angles of all the brightness points in the macro block to be coded, namely the direction angle of the image texture.
It will be appreciated that the gradient of the image function f (x, y) at point (x, y) is a vector having a magnitude and direction, set to G x And G y The gradients in the x-direction and the y-direction are indicated in this order. The gradient direction is the direction in which the function f (x, y) changes most rapidly, when an edge exists in the image, a larger gradient value is certain, and conversely, when a smoother part exists in the image, the gray value change is smaller, and the corresponding gradient is also smaller. The vector of gradients can be expressed as:
the magnitude of this vector is:
the direction angle is:
specifically, for a two-dimensional digital image f (x, y), partial differentiation in two directions of X, Y needs to be performed, which is expressed by the following formula:
for discrete images, the mathematical expression of the first order differential corresponds to the difference between two adjacent pixels, and the effect may be different, but the rationale does not change, depending on the gradient operator chosen. The most common operator is the Roberts operator, others are also common, as are Sobel, prewitt, etc. Gradient calculation demonstration of X, Y, exemplified by the Roberts operator, is shown in fig. 12, where the gradient of point (x, y) is equivalent to the gradient of a two-dimensional discrete function, as follows:
G x ＝f(x,y)-f(x-1,y)
G y ＝f(x,y)-f(x,y-1)
according to the above formula, the direction angle relation in the present invention can be obtained:
θ＝arctan(yGradient/xGradient),
further, according to the formula, the following classification between angles can be obtained:
when xGradient is more than or equal to 0, yGradient is more than or equal to 0,
as shown in fig. 8;
xGradient>0，yGradient<at the time of 0, the number of the first,
as shown in fig. 9;
xGradient<when 0, yGradient is more than or equal to 0,
as shown in fig. 10;
xGradient<0，yGradient<at the time of 0, the number of the first electrode,
as shown in fig. 11.
Fig. 13 shows a schematic structural diagram of an intra prediction mode decision device provided by the present invention, and fig. 13 shows a schematic structural diagram of an intra prediction mode decision device provided by the present invention. The device includes:
the mode number analysis module 1 is used for analyzing the effectiveness of adjacent macro blocks of the current macro block to be coded and determining the effective intra-frame prediction mode of the macro block to be coded;
the threshold judging module 2 is used for judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold, and if so, the angle calculating module 3 is triggered; the preset threshold value is a positive integer greater than 2;
the angle calculation module 3 is used for calculating the direction angle of the image texture in the macro block to be coded;
the matching module 4 is used for determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from effective intra-frame prediction modes related to all angles of a macro block to be coded according to a preset matching rule;
and the fast prediction module 5 is configured to perform intra prediction on the macroblock to be encoded in the determined intra prediction mode and the DC prediction mode, respectively, and select the prediction mode with the best intra prediction result as the final intra prediction mode of the macroblock to be encoded.
Preferably, the threshold value judging module 2 triggers the traversal predicting module 6 when judging that the number of the effective intra-frame prediction modes does not exceed the preset threshold value;
and the traversal prediction module 6 is used for performing intra-frame prediction on the macro block to be coded under each intra-frame prediction mode in which the macro block to be coded is effective, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
The above modules may or may not be physically separate, may be located in one place, or may be distributed over multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
The present invention also provides an intra prediction mode decision device, including:
a memory for storing a computer program;
a processor for implementing the steps of the intra prediction mode decision method as any of the above when executing a computer program.
Wherein the memory includes at least one type of readable storage medium including flash memory, hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory may in some embodiments be an internal storage unit of the intra prediction mode decision device, e.g. a hard disk of the intra prediction mode decision device. The memory may also be an external storage device of the device in other embodiments, such as a plug-in hard disk provided on the intra prediction mode decision device, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. Further, the memory may also include both an internal storage unit of the intra prediction mode decision device and an external storage device. The memory may be used not only to store application software installed in the intra prediction mode decision device and various kinds of data, such as a code of an intra prediction mode decision program, etc., but also to temporarily store data that has been output or is to be output.
The processor may be, in some embodiments, a Central Processing Unit (CPU), controller, microcontroller, microprocessor or other data Processing chip configured to execute program code stored in a memory or process data, such as executing an intra-prediction mode decision process.
The bus is used for connecting the processor and the memory, and the bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.
Further, the intra prediction mode decision device may further include a network interface for receiving an input video to be compressed and outputting compressed video data and the like. The network interface may optionally include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication link between the device and other electronic devices.
Optionally, the device may further comprise a user interface, which may comprise a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the intra prediction mode decision device and for displaying a visualized user interface, among others.
In this embodiment, the intra prediction mode decision device may be a PC (Personal Computer), or may also be a terminal device such as a smart phone, a tablet Computer, a palmtop Computer, or a portable Computer. Preferably, the device is a node constituting a content delivery network CDN network or a blockchain network.
The present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the intra prediction mode decision method as in any one of the above.
The modules in the intra prediction mode decision apparatus provided in the foregoing description of the present invention may be stored in a computer readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The present invention also provides a computer program product comprising computer instructions which, when run on a computer, cause the computer to perform the method for intra prediction mode decision as described in any of the above.
The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device, such as a server, a data center, etc., that is integrated with one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be electrical, mechanical or other forms.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product.
Claims (10)
1. An intra prediction mode decision method, comprising:
analyzing the effectiveness of adjacent macro blocks of a current macro block to be coded, and determining the effective intra-frame prediction mode of the macro block to be coded;
judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold value, and if so, calculating the direction angle of image texture in the macro block to be coded; the preset threshold is a positive integer greater than 2;
according to a preset matching rule, determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from the effective intra-frame prediction modes related to all angles of the macro block to be coded;
and respectively carrying out intra-frame prediction on the macro block to be coded in the determined intra-frame prediction mode and the DC prediction mode, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
2. The method according to claim 1, wherein if the number of the effective intra prediction modes does not exceed a preset threshold, intra prediction of the macroblock to be encoded is performed in each effective intra prediction mode of the macroblock to be encoded, and a prediction mode with a best intra prediction result is selected as a final intra prediction mode of the macroblock to be encoded.
3. The method according to claim 2, wherein the step of determining the intra prediction mode with the angle closest to the direction angle of the image texture from the effective intra prediction modes associated with the respective angles of the macroblock to be encoded according to the preset matching rule comprises:
and respectively calculating the difference between the direction angle of the image texture and the angle of the effective intra-frame prediction mode related to each angle of the macro block to be coded, and taking the effective intra-frame prediction mode with the minimum difference as the intra-frame prediction mode matched with the direction angle of the image texture.
4. The method according to claim 2, wherein the step of determining the intra prediction mode with the angle closest to the direction angle of the image texture from the effective intra prediction modes associated with the respective angles of the macroblock to be encoded according to the preset matching rule comprises:
and comparing the direction angle of the image texture with a preset angle range interval corresponding to the effective intra-frame prediction mode related to each angle, and taking the effective intra-frame prediction mode corresponding to the angle range interval to which the direction angle of the image texture belongs as the intra-frame prediction mode matched with the direction angle of the image texture.
5. The method of claim 3 or 4, further comprising:
if the direction angle of the image texture exceeds the angle range of the effective intra-frame prediction mode related to each angle of the macro block to be coded, the intra-frame prediction of the macro block to be coded is carried out under each effective intra-frame prediction mode of the macro block to be coded, and the prediction mode with the best intra-frame prediction result is selected as the final intra-frame prediction mode of the macro block to be coded.
6. The method according to claim 1 or 2, wherein the predetermined threshold is 5 when the macroblock to be encoded is applied to an AVS or AVS + encoding process.
7. The method according to claim 1 or 2, wherein the process of calculating the direction angle of the image texture in the macroblock to be coded is:
calculating gradient values of all brightness points in the macro block to be coded;
calculating the direction angles of all brightness points in the macro block to be coded according to the gradient value and the direction angle relational expression; the direction angle relation is as follows:
θ＝arctan(yGradient/xGradient),
wherein, theta is a direction angle, xGradi is a gradient value in the horizontal direction, and yGradi is a gradient value in the vertical direction;
and calculating the mean value of the direction angles of all the brightness points in the macro block to be coded, namely the direction angle of the image texture.
8. An intra prediction mode decision device, comprising:
the mode number analysis module is used for analyzing the effectiveness of adjacent macro blocks of a current macro block to be coded and determining the effective intra-frame prediction mode of the macro block to be coded;
the threshold judging module is used for judging whether the number of the effective intra-frame prediction modes exceeds a preset threshold, and if so, the angle calculating module is triggered; the preset threshold is a positive integer greater than 2;
the angle calculation module is used for calculating the direction angle of the image texture in the macro block to be coded;
the matching module is used for determining an intra-frame prediction mode with the angle closest to the direction angle of the image texture from effective intra-frame prediction modes related to all angles of the macro block to be coded according to a preset matching rule;
and the fast prediction module is used for performing intra-frame prediction on the macro block to be coded under the determined intra-frame prediction mode and the DC prediction mode respectively, and selecting the prediction mode with the best intra-frame prediction result as the final intra-frame prediction mode of the macro block to be coded.
9. An intra prediction mode decision device, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the intra prediction mode decision method as claimed in any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the intra prediction mode decision method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811644745.6A CN109640087B (en) | 2018-12-30 | 2018-12-30 | Intra-frame prediction mode judgment method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811644745.6A CN109640087B (en) | 2018-12-30 | 2018-12-30 | Intra-frame prediction mode judgment method, device and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109640087A CN109640087A (en) | 2019-04-16 |
| CN109640087B true CN109640087B (en) | 2022-12-09 |
Family
ID=66054828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811644745.6A Active CN109640087B (en) | 2018-12-30 | 2018-12-30 | Intra-frame prediction mode judgment method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109640087B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113489976A (en) * | 2021-07-29 | 2021-10-08 | 维沃移动通信有限公司 | Predictive coding method and device and electronic equipment |
| CN115550645A (en) * | 2022-12-02 | 2022-12-30 | 杭州未名信科科技有限公司 | Method and device for determining intra-frame prediction mode, storage medium and electronic equipment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101247525B (en) * | 2008-03-24 | 2010-06-02 | 北京邮电大学 | Method for improving image intraframe coding velocity |
| CN102186081B (en) * | 2011-05-11 | 2013-09-18 | 北京航空航天大学 | H.264 intra-frame mode selection method based on gradient vector |
| US9344715B2 (en) * | 2011-11-07 | 2016-05-17 | Futurewei Technologies, Inc. | Angular table for improving intra prediction |
| CN103384326A (en) * | 2013-03-20 | 2013-11-06 | 张新安 | Quick intra-frame prediction mode selection method for AVS-M video coding |
| US11032550B2 (en) * | 2016-02-25 | 2021-06-08 | Mediatek Inc. | Method and apparatus of video coding |
| CN106231302B (en) * | 2016-07-28 | 2019-03-19 | 同观科技（深圳）有限公司 | A kind of determination method and system of optimal intra prediction mode |
| CN109040754A (en) * | 2018-11-06 | 2018-12-18 | 深圳市网心科技有限公司 | A kind of intra prediction mode of HEVC determines method, equipment, system and medium |
-
2018
- 2018-12-30 CN CN201811644745.6A patent/CN109640087B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109640087A (en) | 2019-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109005408B (en) | Intra-frame prediction method and device and electronic equipment | |
| CN107071416B (en) | HEVC intra-frame prediction mode rapid selection method | |
| WO2016180129A1 (en) | Prediction mode selection method, apparatus and device | |
| US11363276B2 (en) | Intra-frame prediction method and apparatus, video coding device, and storage medium | |
| CN109640087B (en) | Intra-frame prediction mode judgment method, device and equipment | |
| CN109040754A (en) | A kind of intra prediction mode of HEVC determines method, equipment, system and medium | |
| US20190253709A1 (en) | Method and apparatus for determining division of coding unit, computing device, and readable storage medium | |
| CN109495742B (en) | Video frame coding method, device and equipment | |
| CN110324631A (en) | A kind of image parallel processing method, device and electronic equipment | |
| CN111428740A (en) | Detection method and device for network-shot photo, computer equipment and storage medium | |
| CN109462758B (en) | Intra-frame prediction mode determination method, electronic device, system and storage medium | |
| CN114596294A (en) | Circuit element missing detection method based on difference image, electronic device and storage medium | |
| Tun et al. | Low complexity mode selection for H. 266/VVC intra coding | |
| CN111654696B (en) | Intra-frame multi-reference-line prediction method and device, storage medium and terminal | |
| US10136156B2 (en) | Directional motion vector filtering | |
| CN112907433A (en) | Digital watermark embedding method, digital watermark extracting device, digital watermark embedding apparatus, digital watermark extracting apparatus, and digital watermark extracting medium | |
| WO2023155751A1 (en) | Video coding control method, apparatus, device, and storage medium | |
| CN108765503B (en) | Skin color detection method, device and terminal | |
| CN111918061B (en) | Coding block dividing method and device, storage medium and electronic equipment | |
| US20140254664A1 (en) | Methods to perform intra prediction for compression of cfa data | |
| CN108737836A (en) | A kind of interframe prediction encoding method, device and electronic equipment | |
| CN115118990A (en) | Chroma prediction method, device, encoding device, decoding device and storage medium | |
| CN109587502B (en) | Method, device, equipment and computer readable storage medium for intra-frame compression | |
| CN114697650A (en) | Intra-frame division method based on down-sampling, related device equipment and medium | |
| CN112738522A (en) | Video coding method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |