CN109587372B - Invisible image steganography based on generation of countermeasure network - Google Patents
Invisible image steganography based on generation of countermeasure network Download PDFInfo
- Publication number
- CN109587372B CN109587372B CN201811509096.9A CN201811509096A CN109587372B CN 109587372 B CN109587372 B CN 109587372B CN 201811509096 A CN201811509096 A CN 201811509096A CN 109587372 B CN109587372 B CN 109587372B
- Authority
- CN
- China
- Prior art keywords
- image
- network
- secret
- training
- carrying
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/32—Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
- H04N1/32101—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title
- H04N1/32144—Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
- H04N1/32149—Methods relating to embedding, encoding, decoding, detection or retrieval operations
- H04N1/32309—Methods relating to embedding, encoding, decoding, detection or retrieval operations in colour image data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Editing Of Facsimile Originals (AREA)
Abstract
The invention discloses invisible image steganography based on a generation countermeasure network, which can realize that a gray secret image is embedded in a color carrier image to obtain a secret image and can successfully recover the secret image from the secret image. The method comprises the following steps: the encoder network is responsible for embedding the secret image into the carrier image to generate a carrier image; the decoder network is responsible for recovering the secret image from the secret-carrying image; and the discriminator network is responsible for performing steganalysis on the natural image and the secret-carrying image so as to adjust the security of the encoder network and the decoder network. The invention provides a new design idea for hiding image information by constructing the image steganography based on the generation countermeasure network.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to image steganography based on a generation countermeasure network.
Background
In recent years, with the rapid development and popularization of the internet, communication has become more and more convenient, but new challenges are also presented to information security. On one hand, people always have some secret information in communication which is not hoped to be known by a third party; on the other hand, the emerging digital multimedia works require copyright protection, and a large amount of e-commerce data and the like require integrity confirmation and the like. Conventional cryptography has not solved these emerging problems well. Steganography (Steganography) is used to embed secret information into a normal carrier without changing the perceptual properties of the carrier, and the transmission of the secret information is achieved by the transmission of the carrier in a channel. Steganography hides not only the content of the communication but also the behavior of the communication, and therefore can be well applied to the above scenarios. Digital images have become one of the most frequently used data types on the internet, and have the characteristics of large redundancy and large capacity for embedding secret information, so the Image Steganography (Image Steganography) which takes images as carriers is the most mainstream at present.
There are three main indicators for measuring image steganography: capacity, invisibility, security. Capacity refers to the length of secret information that can be embedded in a digital image; invisibility refers to whether the carrier image can be indistinguishable from the original image after the secret information is embedded; the security requires that the secret-loaded image can get rid of the detection of the attack algorithm in the transmission process. Existing image steganography suffers from more or less of these three problems.
On the other hand, with the development of Image steganography, an Image steganography analysis (Image Steganalysis) algorithm, which is a detection algorithm for Image steganography, is also developing continuously. The purpose of the image steganalysis is to judge whether the image contains secret information or not by analyzing the image, and even estimate the information embedding amount, acquire the secret information, destroy the secret information and the like. Aiming at the existing image steganography, new image steganography analysis algorithms are continuously proposed, and the performance is better and better. Therefore, a new image steganography is provided, so that the requirements of more and more application scenes are met, the detection of a steganography analysis algorithm is effectively avoided, and the method has great significance on network information safety.
Existing image steganography generally relies on a well-designed algorithm to embed secret information in the spatial or transform domain of an image. This type of steganography requires a large amount of a priori knowledge and once the algorithm is designed, it cannot be automatically adjusted according to the emerging steganography analysis algorithm. Designing a new image steganography is therefore a great challenge.
With the great improvement of computing power and the arrival of the big data era in recent years, deep learning methods represented by convolutional neural networks have achieved excellent effects in tasks such as image recognition, object detection, and image generation. The deep learning carries out feature extraction and fusion on image information through a convolutional neural network, and parameter updating of the convolutional neural network is realized through a supervised or semi-supervised mode, and a specific task is finally completed. The generation of countermeasure networks is the latest result in the field of deep learning, and comprises two parts, namely a generator and an arbiter. Through the countermeasure training of the generator and the discriminator, the generation countermeasure network can produce samples with similar data distribution with real samples. The generation of the countermeasure network achieves excellent effects on tasks such as image generation, style migration, speech synthesis, and the like.
Disclosure of Invention
The invention provides a digital image steganography algorithm based on a generation countermeasure network, which finally realizes invisible steganography by embedding a gray level secret image into a color carrier image through automatic learning of a convolutional neural network, and the safety is continuously improved in the learning process of the neural network.
The invention provides an image steganography method based on a generation countermeasure network, which comprises the following steps:
1) under the framework of an encoder-decoder network, the secret image is hidden into the carrier image through an encoder network to generate a secret-carrying image, and the secret image is recovered from the secret-carrying image through a decoder network;
2) under the framework of generating a countermeasure network, an encoder network and a decoder network form an end-to-end model as a generator network, a secret image and a recovered secret image are generated through the generator network, and the authenticity of the generated secret image is judged through a discriminator network;
3) through the countertraining process of the generator network and the discriminator network, the secret image generated by the generator network is consistent with the corresponding carrier image and is as close to the real image as possible, and the secret image restored by the generator network is as similar as possible to the original secret image.
4) The model is trained using images of various sizes, so that the model has better generalization capability.
Further, the countermeasure training process of the generator network and the arbiter network includes:
a) for a generator network, the encoder network is to make the generated secret image and the generated carrier image as similar as possible, and the decoder network is to make the recovered secret image and the original secret image as similar as possible;
b) the discriminator network plays a role of steganalysis, discriminates the input image as a natural image or a secret-carrying image generated by the generator network, and changes the gradient change direction according to the steganalysis result;
c) the generator network obtains the gradient sent back by the discriminator network to update the parameters of the encoder network and the decoder network, and generates a better secret-carrying image.
Further, the encoder network in step 1) embeds only the grayscale secret image into the Y-channel under the YCrCb color space of the carrier image, and the decoder network also recovers only the secret image from the Y-channel under the YCrCb color space of the carrier image.
Further, the encoder network in the step 1) is composed of an inclusion module, so that different features can be fused together to realize steganography of the image; the decoder network is a full convolutional structure. Both can handle images of arbitrary size.
Further, the discriminator in the step 2) is a steganography analysis model realized by a convolutional neural network, wherein a space pyramid pooling module is used for mapping the feature map with any size to the feature vector with fixed length, so that the size limitation of the input image is broken, and the steganography analysis performance is better.
Further, a complex loss function is used in step 3) to guide the training of the generator network.
The method of the invention can well embed the secret image into the carrier image, and has the following advantages compared with the prior art:
1. the invention improves the methods proposed by Baluja and the like and Atique and the like, and solves the problem that the color of a secret-carrying image is different from that of a carrier image;
2. the method uses the generation countermeasure network for training, so that the model can adjust parameters according to the analysis condition of the steganography analyzer in the process of realizing image steganography, and the generated secret-carrying image is not easy to detect by the steganography analyzer;
3. the invention uses a composite loss function to guide the training of the image steganography process, accelerates the training speed and improves the quality of the secret-carrying image and the recovered secret image.
Drawings
FIG. 1 is a flow chart of image steganography performed by the method of the present invention. Wherein, the encoder network and the decoder network form a generator network, and the steganalyser is used as the discriminator network of the invention.
FIG. 2 is a diagram of an example of image steganography using the method of the present invention.
FIG. 3 is a table showing the detection rate comparison of the steganalysis model with different training times.
Detailed Description
In order to make the aforementioned and other features and advantages of the present invention more comprehensible, embodiments accompanying figures are described in further detail below.
The image steganography method designed by the invention is based on a generation countermeasure network and is suitable for embedding the gray secret image into the color carrier image. The method obtains optimal model parameters by training a model by using a data set, and the specific training process is shown in figure 1, and the method mainly comprises the following steps:
step 101, converting the color carrier image from an RGB color space to a YCrCb color space.
And 102, splicing the Y-channel of the color carrier image and the gray secret image together and then outputting the spliced image to an encoder network.
And 103, outputting a single-channel image by the encoder network after feature extraction and fusion, combining the Y channel of the image as a secret-carrying image and the Cr channel and the Cb channel of the original carrier image together, and converting the combined image into an RGB color space to obtain the color secret-carrying image.
Step 201, the secret-loaded image is converted into YCrCb color space, and only Y channel is input into the decoder network.
Step 202, the decoder network outputs a single-channel image through feature extraction and fusion, namely the recovered secret image.
And step 203, calculating the difference between the secret image and the original secret image and the difference between the recovered secret image and the original secret image by using the designed composite loss function, and taking the difference as a part of the loss function value.
Step 301, performing steganalysis on a natural carrier image sample by using a discriminator network, and judging whether steganalysis results are consistent with real categories or not;
step 302, performing steganalysis on the generated secret-carrying image by using a discriminator network, and judging whether the steganalysis result is consistent with the real category;
step 401, calculating a difference value between a secret image sample and a natural carrier image sample and a difference value between a recovered secret image and an original secret image by using a composite loss function, obtaining a discriminant loss value according to a discriminant result of the discriminator network on the secret image, adding the three values into a block to be used as a loss of a generator network (an encoder network and a decoder network), calculating a gradient, and updating parameters of the generator network (the encoder network and the decoder network);
and step 402, calculating a loss value of the discriminator according to the steganalysis result obtained in the step 3, calculating a gradient, and updating parameters of the discriminator network.
Step 501, after one round of training on the training set is completed, performing a verification test on the test set, and calculating the average similarity between the secret image and the carrier image and the average similarity between the recovered secret image and the original secret image;
step 502, checking whether the result of the step 501 reaches an expected index, if so, performing the next operation, and if not, returning to the step A to start the next round of training on the training set;
and 503, performing fine tuning training on the model by using image samples with various sizes, so as to improve the generalization capability of the model.
The invention is trained and verified on three data sets of LFW, Pascal VOC2012 and ImageNet, and fig. 2 is a sample illustration of image steganography performed by the method of the invention, so that the secret image is seen to be highly consistent with the original carrier image, and the recovered secret image is highly consistent with the original secret image. FIG. 3 is a graph showing the detection rate of steganalysis algorithm for secret-carrying images compared with the detection rate of the method of the present invention at different training times. This comparison shows that the security of the method of the present invention is increasing with the training of the generation of the countermeasure network.
The invention realizes the image steganography task of embedding a gray secret image in a color image and recovering the secret image under the framework of an encoder-decoder, allows the steganography model to consider the attack of a steganography analysis algorithm in the training process under the framework of a generated countermeasure network, and continuously adjusts parameters to improve the safety of the model.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (4)
1. An invisible image steganography based on generation of a countermeasure network, comprising:
A. generating a secret-carrying image: the encoder network is realized by a convolutional neural network and is responsible for splicing the gray secret image with a Y channel of a color carrier image YCrCb color space, performing feature extraction and fusion to generate a single-channel image, and combining the single-channel image with two channels of Cr and Cb of an original carrier image to generate a color secret-carrying image;
B. and (3) recovering the secret image: the decoder network is realized by a full convolution neural network and is responsible for carrying out feature extraction and fusion on a Y channel of a color space of the color secret-carrying image YCrCb and recovering a gray secret image;
C. steganalysis performed using a network of discriminators: the discriminator network carries out steganalysis on the input natural image or the secret-carrying image generated by the encoder network;
D. updating parameters and performing iterative training: obtaining a discrimination loss value according to a discrimination result of the discriminator network, calculating a difference value between a secret image sample and a natural carrier image sample and a difference value between a recovered secret image and an original secret image by using a composite loss function, adding the three values into a block to be used as the loss of a generator network (an encoder network and a decoder network), calculating gradient, and updating parameters;
E. verifying the performance of the model and generalizing training: and verifying the performance of the model through the structural similarity index, and performing generalized training by using a multi-scale sample.
2. The invisibility image steganography based on generation of countermeasure network of claim 1, wherein step C further comprises the steps of:
c1, inputting natural carrier image samples into the discriminator network to obtain analysis categories, and judging whether the analysis categories are consistent with the real categories;
and C2, inputting the secret-carrying image sample generated in the step A into the discriminator network to obtain an analysis type, and judging whether the analysis type is consistent with the real type.
3. The invisibility image steganography based on generation of countermeasure network of claim 1, wherein step D further comprises the steps of:
d1, calculating gradient according to the loss of the generator network (the encoder network and the decoder network), and updating the parameters of the generator network (the encoder network and the decoder network);
d2, calculating the loss value of the discriminator network according to the result of the step C, calculating the gradient and updating the parameters of the discriminator network.
4. The invisible image steganography based on generation of a countermeasure network according to claim 1, wherein step E further comprises the steps of:
e1, after completing one round of training on the training set, performing a verification test on the test set, and calculating the average similarity between the secret image and the carrier image and the average similarity between the restored secret image and the original secret image;
e2, checking whether the result of the step E1 reaches the expected index, if so, carrying out the next operation, and if not, returning to the step A to start the next round of training on the training set;
e3, performing fine tuning training on the model by using image samples with various sizes, and improving the generalization capability of the model.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811509096.9A CN109587372B (en) | 2018-12-11 | 2018-12-11 | Invisible image steganography based on generation of countermeasure network |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811509096.9A CN109587372B (en) | 2018-12-11 | 2018-12-11 | Invisible image steganography based on generation of countermeasure network |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109587372A CN109587372A (en) | 2019-04-05 |
CN109587372B true CN109587372B (en) | 2020-06-16 |
Family
ID=65928897
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811509096.9A Active CN109587372B (en) | 2018-12-11 | 2018-12-11 | Invisible image steganography based on generation of countermeasure network |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109587372B (en) |
Families Citing this family (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110334805B (en) * | 2019-05-05 | 2022-10-25 | 中山大学 | JPEG domain image steganography method and system based on generation countermeasure network |
WO2020251976A1 (en) * | 2019-06-12 | 2020-12-17 | Rutgers, The State University Of New Jersey | Light-field messaging |
CN110502646A (en) * | 2019-08-02 | 2019-11-26 | 广州多益网络股份有限公司 | A kind of multimedia file purification method, device, storage medium and terminal device |
CN110503157B (en) * | 2019-08-27 | 2021-11-19 | 安徽大学 | Image steganalysis method of multitask convolution neural network based on fine-grained image |
CN110889797B (en) * | 2019-10-15 | 2021-06-08 | 浙江大学 | Depth self-adaptive image hiding method based on confrontation sample generation |
CN113099066B (en) * | 2019-12-23 | 2022-09-30 | 浙江工商大学 | Large-capacity image steganography method based on multi-scale fusion cavity convolution residual error network |
CN111768325B (en) * | 2020-04-03 | 2023-07-25 | 南京信息工程大学 | Security improvement method based on generation of countermeasure sample in big data privacy protection |
CN111598762B (en) * | 2020-04-21 | 2023-09-19 | 中山大学 | Generating type robust image steganography method |
CN111681154B (en) * | 2020-06-09 | 2022-06-21 | 湖南大学 | Color image steganography distortion function design method based on generation countermeasure network |
CN112243005B (en) * | 2020-10-14 | 2022-03-15 | 合肥工业大学 | Secure non-embedded steganography method based on generation of countermeasure network |
CN112529758B (en) * | 2020-12-18 | 2022-08-09 | 海南大学 | Color image steganography method based on convolutional neural network |
CN112561773B (en) * | 2020-12-28 | 2024-03-29 | 中国科学技术大学 | Deep disturbance-based countersteganography method |
CN112884630B (en) * | 2021-02-04 | 2022-06-28 | 南开大学 | Reversible neural network-based high-capacity image steganography and recovery method |
CN113077377B (en) * | 2021-05-13 | 2022-06-10 | 海南大学 | Color image steganography method based on generation countermeasure network |
CN113254891B (en) * | 2021-05-17 | 2022-08-16 | 山东大学 | Information hiding method, device and system based on void space pyramid |
CN113326531B (en) * | 2021-06-29 | 2022-07-26 | 湖南汇视威智能科技有限公司 | Robust efficient distributed face image steganography method |
CN113538202A (en) * | 2021-08-05 | 2021-10-22 | 齐鲁工业大学 | Image steganography method and system based on generative steganography confrontation |
CN114025053B (en) * | 2021-09-28 | 2023-12-01 | 上海电力大学 | Image steganography method combined with high-color similarity preprocessing |
CN114220443B (en) * | 2021-11-04 | 2023-04-07 | 合肥工业大学 | BN optimization SNGAN-based training method and system for adaptive audio steganography model |
CN113989097B (en) * | 2021-12-30 | 2022-05-06 | 南京中孚信息技术有限公司 | Information steganography model training method, information steganography device and storage medium |
CN115086674B (en) * | 2022-06-16 | 2024-04-02 | 西安电子科技大学 | Image steganography method based on generation of countermeasure network |
CN117292696B (en) * | 2023-10-08 | 2024-03-12 | 合肥工业大学 | End-to-end audio steganography method, system, storage medium and electronic equipment |
CN117132671B (en) * | 2023-10-27 | 2024-02-23 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Multi-task steganography method, system and medium based on depth self-adaptive steganography network |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108171266A (en) * | 2017-12-25 | 2018-06-15 | 中国矿业大学 | A kind of learning method of multiple target depth convolution production confrontation network model |
CN108346125A (en) * | 2018-03-15 | 2018-07-31 | 中山大学 | A kind of spatial domain picture steganography method and system based on generation confrontation network |
CN108921764A (en) * | 2018-03-15 | 2018-11-30 | 中山大学 | A kind of image latent writing method and system based on generation confrontation network |
CN108923922A (en) * | 2018-07-26 | 2018-11-30 | 北京工商大学 | A kind of text steganography method based on generation confrontation network |
-
2018
- 2018-12-11 CN CN201811509096.9A patent/CN109587372B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108171266A (en) * | 2017-12-25 | 2018-06-15 | 中国矿业大学 | A kind of learning method of multiple target depth convolution production confrontation network model |
CN108346125A (en) * | 2018-03-15 | 2018-07-31 | 中山大学 | A kind of spatial domain picture steganography method and system based on generation confrontation network |
CN108921764A (en) * | 2018-03-15 | 2018-11-30 | 中山大学 | A kind of image latent writing method and system based on generation confrontation network |
CN108923922A (en) * | 2018-07-26 | 2018-11-30 | 北京工商大学 | A kind of text steganography method based on generation confrontation network |
Non-Patent Citations (3)
Title |
---|
END-TO-END TRAINED CNN ENCODER-DECODER NETWORKS FOR IMAGE STEGANOGRAPHY;Atique ur Rehman,et al;《arXiv.org》;20171120;全文 * |
Hiding Images in Plain Sight: Deep Steganography;Shumeet Baluja;《31st Conference on Neural Information Processing Systems (NIPS 2017)》;20171231;正文第1-4节、图1-9 * |
深度学习在图像隐写术与隐写分析领域中的研究进展;翟黎明 等;《信息安全学报》;20181130;第3卷(第6期);全文 * |
Also Published As
Publication number | Publication date |
---|---|
CN109587372A (en) | 2019-04-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109587372B (en) | Invisible image steganography based on generation of countermeasure network | |
Guo et al. | Fake face detection via adaptive manipulation traces extraction network | |
CN111709408B (en) | Image authenticity detection method and device | |
CN110084734B (en) | Big data ownership protection method based on object local generation countermeasure network | |
CN110276708B (en) | Image digital watermark generation and identification system and method based on GAN network | |
CN113344826B (en) | Image processing method, device, electronic equipment and storage medium | |
Wang et al. | HidingGAN: High capacity information hiding with generative adversarial network | |
Bi et al. | High‐Capacity Image Steganography Algorithm Based on Image Style Transfer | |
CN109413068B (en) | Wireless signal encryption method based on dual GAN | |
Li et al. | Image manipulation localization using multi-scale feature fusion and adaptive edge supervision | |
Mazumdar et al. | Two-stream encoder–decoder network for localizing image forgeries | |
Wei et al. | Generative steganographic flow | |
CN108648135B (en) | Hidden model training and using method, device and computer readable storage medium | |
CN114494387A (en) | Data set network generation model and fog map generation method | |
Sultan et al. | A new framework for analyzing color models with generative adversarial networks for improved steganography | |
CN115879516B (en) | Data evidence obtaining method | |
CN115936961B (en) | Steganalysis method, equipment and medium based on few-sample comparison learning network | |
Yao et al. | A secure image evidence management framework using multi-bits watermark and blockchain in IoT environments | |
CN114596609A (en) | Audio-visual counterfeit detection method and device | |
CN114241493A (en) | Training method and training device for training data of amplification document analysis model | |
Jabbar et al. | Digital watermarking by utilizing the properties of self-organization map based on least significant bit and most significant bit | |
Zeng et al. | Towards Secure and Robust Steganography for Black-box Generated Images | |
Abdollahi et al. | Image steganography based on smooth cycle-consistent adversarial learning | |
CN113378723A (en) | Automatic safety identification system for hidden danger of power transmission and transformation line based on depth residual error network | |
CN112149631A (en) | Image processing method, apparatus, device and medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |