CN112463387B - Method for identifying deep learning model on local server based on GPU space-time resource consumption - Google Patents
Method for identifying deep learning model on local server based on GPU space-time resource consumption Download PDFInfo
- Publication number
- CN112463387B CN112463387B CN202011427759.XA CN202011427759A CN112463387B CN 112463387 B CN112463387 B CN 112463387B CN 202011427759 A CN202011427759 A CN 202011427759A CN 112463387 B CN112463387 B CN 112463387B
- Authority
- CN
- China
- Prior art keywords
- deep learning
- data
- local server
- gpu
- learning model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013136 deep learning model Methods 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000012360 testing method Methods 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims abstract description 5
- 238000013135 deep learning Methods 0.000 claims abstract description 4
- 238000002474 experimental method Methods 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims abstract description 4
- 238000010606 normalization Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract description 4
- 238000012549 training Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000013145 classification model Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 235000000332 black box Nutrition 0.000 description 1
- 238000005336 cracking Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 230000034994 death Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005670 electromagnetic radiation Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000016273 neuron death Effects 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The method for identifying the deep learning model on the local server based on GPU space-time resource consumption comprises the following steps: s1, building an experiment platform for collecting utilization rate and power consumption data of a local server GPU; (S2) running various common deep learning models on a local server by using an open source deep learning framework such as tensorflow, pytorch; (S3): capturing the utilization rate and power consumption data of the GPU in real time in the process of running the deep learning model by the local server; (S4) processing the acquired data; and (S5) constructing a convolutional neural network to classify the acquired data, and presenting a test result in a confusion matrix mode. The final result shows that when the deep learning model runs, the GPU utilization rate and the power consumption of the local server have correlation with the model in the deep learning model, and through analysis of the correlation, a lot of effective information about the model can be obtained.
Description
Technical Field
The invention relates to the field of deep learning model identification, and is mainly applied to the field of safety of deep learning models, in particular to a method for identifying a deep learning model on a local server based on GPU space-time resource consumption.
Background
Side channel attacks are an attack technique based on side channel information. Side channel information means other information in the encryption device than explicit information directly related to the ciphertext, such as GPU usage of the device, memory occupancy of the GPU, power consumption, electromagnetic radiation, time consumption, and the like. With the continuous penetration of artificial intelligence technology in industries such as military, civil and the like, the safety problem and attack and defense technology of the artificial intelligence technology are more and more concerned. Traditional artificial intelligence technology safety considerations only stay at the software level, and model output is misclassified by adding disturbance to the input of the deep learning model, namely generating an countermeasure sample. Such attacks are generally classified into black box attacks, which are known about model information, and white box attacks, which are known about model architecture, training data, model weights, and the like, according to the degree of knowledge about the model. Clearly, the more known the model, the more threatening the attack, white-box attacks are generally superior to black-box attacks in terms of their effectiveness. It would be further advantageous for an attacker's attack if some model information could be known more. In fact, at model operation, an attacker may analyze the space-time resource consumption of the device at the hardware level to obtain part of model information, and such analyzed information is collectively referred to as side channel information. The implementation of the model cracking through the side channel information is called a side channel attack.
It is very common to run deep learning model applications based on servers, such as image recognition, signal recognition, network classification, etc. Because the deep learning model operation itself requires extremely high computational power, the model is deployed on a server, and the input and output results of the model are transmitted through a local network or the internet, which is a common application scenario. Thus, there is a large portion of deep learning models that provide artificial intelligence services to the marketplace in a server-deployed manner. For this type of model, we cannot directly obtain the information of the running models from the server, but can obtain the use condition of the hardware resources of the server by the models, and then determine the running model by using the hardware resources. As for the local server, the side channel information available to the attacker is side channel information such as CPU cache, data transmission time, etc. In the prior art, the information of the deep learning model is stolen by utilizing the Cache (Cache) of the CPU. The invention provides a method for identifying a deep learning model in a server by acquiring the utilization rate and the power consumption of a GPU, which realizes the identification of different deep learning models by simple actual operation.
Disclosure of Invention
The invention provides a method for directly identifying a deep learning model in a local server through the utilization rate and power consumption information of a GPU, which aims to overcome the defect that the class of the deep learning model in the local server is difficult to identify by using a traditional attack method in the prior art.
The technical conception of the invention is as follows: experiments show that when the deep learning model is operated, the GPU utilization rate and the power consumption of the local server have correlation with the internal model, and the correlation is shown in that when the input is the same, the more complex parameters of the model are, the higher the GPU utilization rate and the power consumption are, and otherwise, the lower the GPU utilization rate and the power consumption are. Through analysis of the correlation, the information of the deep learning model in the local server can be deduced according to the utilization rate and the power consumption information of the GPU, and an attacker can change the local server from black box attack to gray box attack, even white box attack, so that the success accuracy of the attack is improved.
The technical scheme adopted by the invention for achieving the aim of the invention is as follows:
1. a method for identifying a deep learning model on a local server based on GPU space-time resource consumption, comprising the steps of:
s1, building an experiment platform for collecting utilization rate and power consumption data of a local server GPU;
s2, running various common deep learning models on a local server by using an open-source deep learning framework such as tensorflow, pytorch;
s3: capturing the utilization rate and power consumption data of the GPU in real time in the process of running the deep learning model by the local server;
s4, processing the acquired data;
and S5, constructing a convolutional neural network to classify the acquired data, and presenting a test result in a confusion matrix mode.
Further, the step S3 specifically includes: the utilization rate and the power consumption data of the GPU when different deep learning models run are acquired in real time, and input signals of the different deep learning models can be one-dimensional time sequence data, two-dimensional image data and complex network data.
Further, the step S4 specifically includes:
s4.1, carrying out linear normalization processing on the acquired data, wherein the purpose of normalization is to limit the acquired data to 0 to 1, so that the speed of gradient descent to solve the optimal solution is increased, and the recognition precision is improved, and the formula is as follows:
wherein x is the minimum value in the original data acquired by min (x), the maximum value in the original data acquired by max (x), and the normalized value of the original data.
And S4.2, converting the normalized data into 512-512 two-dimensional gray-scale pictures, fully playing the advantages of the convolutional neural network in terms of image feature extraction, reducing the computational complexity, and then marking each picture with a corresponding label, so that the data can be directly input into the convolutional neural network for training and testing.
Further, the step S5 specifically includes: the convolutional neural network comprises four convolutional layers, the number of model parameters is compressed by using 3 multiplied by 3 and 1 multiplied by 1 convolution kernels, the number of characteristic channels is doubled after each pooling operation to keep the integrity of the characteristics as much as possible, the probability of neuronal death and overfitting in training are reduced by using a relu nonlinear activation function, and the output of the neural network becomes probability distribution by using a softmax activation function, so that classification is more accurate.
The beneficial effects of the invention are as follows:
(1) And the local server platform is reasonably utilized, the deployment is simple, the data acquisition is convenient, and the analysis is easy.
(2) The invention shows that when the deep learning model runs, the GPU utilization rate and the power consumption of the local server have correlation with the internal model, and a lot of effective information about the model can be obtained through analysis of the correlation.
(3) The method for identifying the deep learning model in the local server can change the traditional challenge sample attack from black box attack to gray box attack and even white box attack, and remarkably improves the attack accuracy.
(4) The method for converting the one-dimensional time sequence data into the two-dimensional image can fully play the advantages of the convolutional neural network in the aspect of image feature extraction, reduce the computational complexity, and provide a solution idea for the difficulty in processing the one-dimensional time sequence data.
(5) The convolutional neural network algorithm has good classification effect on one-dimensional time sequence data transfer.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
fig. 2 (a) is a GPU usage data visualization, and fig. 2 (b) is a visualization of GPU usage data normalized;
fig. 3 (a) is a graphic view of the GPU power consumption data, and fig. 3 (b) is a graphic view of the normalized GPU power consumption data;
FIG. 4 is a graph of normalized GPU usage data;
FIG. 5 is a graph of normalized GPU power consumption data;
FIG. 6 is a graph of the classification result of an image model input as two-dimensional image data;
FIG. 7 is a graph of the classification result of a model of a one-dimensional time series data signal;
fig. 8 is a diagram of the classification result of the data node model input as a complex network.
Detailed Description
The following describes the embodiments of the present invention in further detail with reference to the drawings.
Referring to fig. 1 to 8, a method for identifying a deep learning model on a local server based on GPU space-time resource consumption includes the steps of:
s1, building an experimental platform for collecting utilization rate and power consumption data of a local server GPU, wherein the experimental platform specifically comprises the following steps:
the local server side is configured to use I7-6700 of Intel for CPU and GTX TITAN X PASCAL of Injeida for GPU;
s2, running various common deep learning models on a local server by using an open source deep learning framework such as tensorflow, pytorch and the like, wherein the method specifically comprises the following steps:
running image recognition models alexnet, mobilenetv, mobiletv 2, internv 3, resnetv1 and resnetv2 on the server; the signal recognition models are fx_ crmr, nin, alexnet, fx _resnet and lstm respectively; the node classification models are demo_ net, gat, gcn, graphsage, h-gcn and mixhop respectively;
s3: capturing the utilization rate and power consumption data of the GPU in real time in the process of running the deep learning model by the local server; the method specifically comprises the following steps:
the python program package pynvml and psutil can be combined to collect the utilization rate and power consumption data of the GPU when different deep learning models run in real time, wherein the input of the signal recognition model is one-dimensional time sequence data, the input of the image recognition model is two-dimensional image data and the input of the node classification model is complex network data;
s4, processing the acquired data; the method specifically comprises the following steps:
s4.1, carrying out linear normalization processing on the acquired data, as shown in fig. 2 and 3, wherein the purpose of normalization is to limit the acquired data between 0 and 1, so that the speed of gradient descent to obtain the optimal solution is increased, and the recognition accuracy is improved, wherein the formula is as follows:
wherein x is the minimum value in the original data acquired by min (x), the maximum value in the original data acquired by max (x), and the value after normalization of the x' original data;
s4.2, converting normalized data into 512 grayscale images shown in figures 4 and 5, fully playing the advantages of the convolutional neural network in terms of image feature extraction, reducing the computational complexity, and then marking each image with a corresponding label, so that the images can be directly input into the convolutional neural network for training and testing;
s5, constructing a convolutional neural network to classify the acquired data, and presenting a test result in a confusion matrix mode; the method specifically comprises the following steps:
s5.1, the convolutional neural network comprises four convolutional layers, the number of model parameters is compressed by using 3 multiplied by 3 and 1 multiplied by 1 convolution kernels, the number of characteristic channels is doubled after each maximum pooling operation to keep the integrity of the characteristics as much as possible, a relu nonlinear activation function is used to reduce the probability of death and overfitting of neurons in training, and the softmax activation function enables the output of the neural network to become probability distribution, so that classification is more accurate;
s5.2, inputting the processed data into a convolutional neural network, training the training set, and outputting a classification result by a test set through a confusion matrix, wherein the number represents the classification precision such as 0.96 represents 96% accuracy, FIG. 6 is the classification result of the image recognition model, FIG. 7 is the classification result of the signal recognition model, and FIG. 8 is the classification result of the node classification model.
S6, combining the model classification result with the image model attack method, the signal model attack method and the node model attack method, the special attack can be carried out on the deep learning model operated by the local server, so that the attack accuracy is improved.
The embodiments described in the present specification are merely examples of implementation forms of the inventive concept, and the scope of protection of the present invention should not be construed as being limited to the specific forms set forth in the embodiments, and the scope of protection of the present invention and equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.
Claims (1)
1. A method for identifying a deep learning model on a local server based on GPU space-time resource consumption, comprising the steps of:
s1, building an experiment platform for collecting utilization rate and power consumption data of a local server GPU;
s2, running a deep learning model on a local server by using a tensorflow, pytorch open source deep learning framework;
s3: capturing the utilization rate and power consumption data of the GPU in real time in the process of running the deep learning model by the local server; the method specifically comprises the following steps:
the method comprises the steps that the utilization rate and the power consumption data of the GPU when different deep learning models run are collected in real time, and input signals of the different deep learning models are one-dimensional time sequence data, two-dimensional image data and complex network data; s4, processing the acquired data; the method specifically comprises the following steps:
s4.1, carrying out linear normalization processing on the acquired data, wherein the formula is as follows:
wherein x is the minimum value in the original data acquired by min (x), the maximum value in the original data acquired by max (x), and the value after normalization of the x' original data;
s4.2, converting the normalized data into 5125×12 gray-scale pictures, and marking each picture with a corresponding label;
s5, constructing a convolutional neural network to classify the acquired data, and presenting the test result in a confusion matrix mode specifically comprises the following steps:
the convolutional neural network comprises four convolutional layers, the convolutional kernels are 3 multiplied by 3 and 1 multiplied by 1, a maximum pooling layer is added behind each convolutional layer, two fully-connected layers are used behind a first fully-connected layer, a relu activation function is used behind a second fully-connected layer, a softmax activation function is used behind a second fully-connected layer, and a prediction result is output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427759.XA CN112463387B (en) | 2020-12-07 | 2020-12-07 | Method for identifying deep learning model on local server based on GPU space-time resource consumption |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011427759.XA CN112463387B (en) | 2020-12-07 | 2020-12-07 | Method for identifying deep learning model on local server based on GPU space-time resource consumption |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112463387A CN112463387A (en) | 2021-03-09 |
| CN112463387B true CN112463387B (en) | 2024-03-29 |
Family
ID=74800370
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011427759.XA Active CN112463387B (en) | 2020-12-07 | 2020-12-07 | Method for identifying deep learning model on local server based on GPU space-time resource consumption |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112463387B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113676311A (en) * | 2021-07-05 | 2021-11-19 | 浙江工业大学 | Method and system for obtaining deep learning model structure based on side channel information |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110048827A (en) * | 2019-04-15 | 2019-07-23 | 电子科技大学 | A kind of class template attack method based on deep learning convolutional neural networks |
| CN111291860A (en) * | 2020-01-13 | 2020-06-16 | 哈尔滨工程大学 | Anomaly detection method based on convolutional neural network feature compression |
| CN111401567A (en) * | 2020-03-20 | 2020-07-10 | 厦门渊亭信息科技有限公司 | Universal deep learning hyper-parameter optimization method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11055139B2 (en) * | 2018-06-12 | 2021-07-06 | International Business Machines Corporation | Smart accelerator allocation and reclamation for deep learning jobs in a computing cluster |
-
2020
- 2020-12-07 CN CN202011427759.XA patent/CN112463387B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110048827A (en) * | 2019-04-15 | 2019-07-23 | 电子科技大学 | A kind of class template attack method based on deep learning convolutional neural networks |
| CN111291860A (en) * | 2020-01-13 | 2020-06-16 | 哈尔滨工程大学 | Anomaly detection method based on convolutional neural network feature compression |
| CN111401567A (en) * | 2020-03-20 | 2020-07-10 | 厦门渊亭信息科技有限公司 | Universal deep learning hyper-parameter optimization method and device |
Non-Patent Citations (1)
| Title |
|---|
| TensorFlow平台下基于深度学习的数字识别;靳涛;张永爱;;信息技术与网络安全;20180410(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112463387A (en) | 2021-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bayar et al. | Constrained convolutional neural networks: A new approach towards general purpose image manipulation detection | |
| Kim et al. | Deep convolutional neural models for picture-quality prediction: Challenges and solutions to data-driven image quality assessment | |
| CN108388927A (en) | Small sample polarization SAR terrain classification method based on the twin network of depth convolution | |
| Jin et al. | Multi-focus image fusion method using S-PCNN optimized by particle swarm optimization | |
| Tang et al. | Improving cost learning for JPEG steganography by exploiting JPEG domain knowledge | |
| CN113033276A (en) | Behavior recognition method based on conversion module | |
| Thakur et al. | Machine learning based saliency algorithm for image forgery classification and localization | |
| CN112463387B (en) | Method for identifying deep learning model on local server based on GPU space-time resource consumption | |
| Ali et al. | Sscnets: Robustifying dnns using secure selective convolutional filters | |
| CN115393698A (en) | Digital image tampering detection method based on improved DPN network | |
| Hussain et al. | Image denoising to enhance character recognition using deep learning | |
| CN113111731A (en) | Deep neural network black box countermeasure sample generation method and system based on channel measurement information | |
| Olisah et al. | Understanding unconventional preprocessors in deep convolutional neural networks for face identification | |
| Zanddizari et al. | Generating black-box adversarial examples in sparse domain | |
| CN114638356A (en) | Static weight guided deep neural network back door detection method and system | |
| CN113676311A (en) | Method and system for obtaining deep learning model structure based on side channel information | |
| Sun et al. | Instance-level Trojan Attacks on Visual Question Answering via Adversarial Learning in Neuron Activation Space | |
| Vo et al. | Adversarial attacks on Deepfake detectors: a practical analysis | |
| Xu et al. | Steganography algorithms recognition based on match image and deep features verification | |
| Singh et al. | Multi-contextual design of convolutional neural network for steganalysis | |
| Shelke et al. | Multiple forgery detection in digital video with VGG-16-based deep neural network and KPCA | |
| Mahmood | Defocus Blur Segmentation Using Genetic Programming and Adaptive Threshold. | |
| Carrara et al. | Defending neural ODE image classifiers from adversarial attacks with tolerance randomization | |
| Huang et al. | Bark classification based on gabor filter features using rbpnn neural network | |
| Sun et al. | AANet: adaptive attention network for rolling bearing fault diagnosis under varying loads |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |