CN114360090B - Patrol instrument with real-time display function - Google Patents
Patrol instrument with real-time display function Download PDFInfo
- Publication number
- CN114360090B CN114360090B CN202111622855.4A CN202111622855A CN114360090B CN 114360090 B CN114360090 B CN 114360090B CN 202111622855 A CN202111622855 A CN 202111622855A CN 114360090 B CN114360090 B CN 114360090B
- Authority
- CN
- China
- Prior art keywords
- video
- storage module
- data
- module
- static parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000003068 static effect Effects 0.000 claims abstract description 74
- 238000007689 inspection Methods 0.000 claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 12
- 230000000007 visual effect Effects 0.000 claims abstract description 7
- 238000005259 measurement Methods 0.000 claims abstract description 6
- 230000001360 synchronised effect Effects 0.000 claims abstract description 6
- 230000001960 triggered effect Effects 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 8
- 238000001514 detection method Methods 0.000 abstract description 13
- 238000012856 packing Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 230000007547 defect Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000001351 cycling effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000003137 locomotive effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
Landscapes
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a patrol instrument with a real-time display function, which comprises a data acquisition module, a data processing module and a display module; the data acquisition module comprises a measurement camera, a video camera and an odometer which can be synchronized and continuously triggered at a fixed frequency; the measuring camera can acquire and store the static parameters of the overhead contact system; the field of view of the video camera is larger than that of the measuring camera, and the video camera is used for collecting and storing video pictures; the odometer is used for collecting and storing mileage data; when the video data and the static parameters of the overhead contact system in the storage module reach the preset quantity, packing and transmitting the video data and the static parameters of the overhead contact system to a fourth storage module; the display module comprises at least two player modules which respectively acquire video packets from the fourth storage module, load the video packets and play the video packets on the display screen in parallel and alternately according to mileage or time, and each frame is marked with mileage data and contact net static parameters. The inspection instrument can display the detection result in real time, the display result is more visual, and the fault position is easy to locate.
Description
Technical Field
The invention relates to the field of visual detection, in particular to a patrol instrument with a real-time display function.
Background
The overhead contact system is used as the most main power supply network of the electrified railway in China, and whether the suspended state is normal or not is directly related to whether the electrified railway locomotive can normally supply power or not. Therefore, the development of the inspection work of the overhead contact system has important practical significance and urgent need for guaranteeing the normal operation of electrified railways in China.
Most of the conventional detection tools are handheld inspection equipment, and the requirements of detection efficiency and precision cannot be met because only handheld equipment and single-point measurement are available; in addition, the operation and maintenance department of the overhead line system needs to perform static recheck and confirm on the position of the dynamic detection overrun, and the fault point cannot be accurately positioned because of no continuous detection result curve, so that recheck and recheck become difficult.
Along with the development of image recognition, vision measurement and data fusion technologies, railway operation and maintenance intellectualization becomes a development trend. CN112002031a discloses a full-automatic intelligent inspection system and inspection method for overhead line systems based on unmanned aerial vehicles, which provides a system and method for automatically detecting the status of overhead line systems by unmanned aerial vehicles, but because the unmanned aerial vehicles are adopted to collect data, the data are analyzed and processed by a working machine after return, the real-time performance is not provided, and the inspection efficiency is lower; meanwhile, the detection result adopts a text mode, the intuitiveness is poor, no other track related information is used as auxiliary reference, and the fault point and the fault cause are difficult to accurately position. CN111309845A discloses a method and a system for displaying macroscopic situation data of a track traffic network, which propose a method for reconstructing and modeling multisource data of the macroscopic situation of the track traffic network and finally displaying the multisource data in the same road network diagram, which provides visual and comprehensive information display for the regional road network, and enables railway operators to stand at a global angle to control the macroscopic situation of the road network in the whole region, but the field of inspection of the overhead line system is limited to detection of a single data source and output of a single type of result at present.
Disclosure of Invention
In order to solve the technical problems, the invention provides the inspection instrument with the real-time display function, which can display the detection result in real time, linearly display the static data of the overhead contact system, has more visual display result and is easy to locate the fault position.
For this purpose, the technical scheme of the invention is as follows:
the inspection instrument with the real-time display function comprises a data acquisition module, a data processing module and a display module;
The data acquisition module comprises a measurement camera, a video camera and an odometer which can be synchronized and continuously triggered at a fixed frequency; the measuring camera is used for acquiring picture data required by resolving the static parameters of the overhead contact system, calculating the static parameters of the overhead contact system based on the picture data and storing a calculation result in the first storage module; the visual field of the video camera is larger than that of the measuring camera, and the video camera is used for collecting video pictures and storing the video pictures in the second storage module; the odometer is used for acquiring mileage data corresponding to the process of simultaneously acquiring pictures by the measuring camera and the video camera and storing the mileage data in the third storage module;
When the number of pictures stored in the second storage module reaches N, N mileage data are stored in the third storage module, N contact net static parameters are stored in the first storage module, at the moment, the data processing module acquires the contact net static parameters from the first storage module, acquires video pictures from the second storage module, acquires mileage data from the third storage module, and marks the contact net static parameters and mileage data corresponding to the same mileage on the video pictures at the same time to obtain video picture I; compressing N video pictures I into video packets and transmitting the video packets to a fourth storage module;
the display module comprises at least two player modules which respectively acquire video packets from the fourth storage module, load the video packets and play the video packets on a display screen in parallel and alternately according to mileage or time, and each frame of the video packets is displayed with a mileage number and a contact net static parameter corresponding to the same mileage number.
Further, the at least two players sequentially load different video packages respectively, and when the video packages loaded by the previous player are played on the display screen in sequence, the loaded video packages are played by the current player so as to ensure the fluency of playing the video on the display screen.
Further, the system also comprises an alarm module; the measuring camera transmits the calculated static parameter data of the overhead line system to the alarm module; the alarm module respectively judges whether the static parameters of the overhead line system meet preset conditions according to a preset threshold value; if the contact network static parameters are met, marking the contact network static parameters as normal states; if the contact network static parameters do not meet the alarm conditions, the contact network static parameters containing the alarm conditions are marked with the corresponding alarm conditions and sent to a fourth storage module.
Further, the fourth storage module is dynamically updated according to mileage data; judging whether the fourth storage module has newly received mileage data, if not, directly storing the newly received mileage data and the corresponding overhead line system static parameters containing the alarm state in the fourth storage module; if the contact network static parameters containing the alarm state are stored in the fourth storage module in a mode of covering original data of the same mileage.
And further, after the N video pictures I are compressed into a video packet, the video packet is packed together with N mileage data and N groups of overhead line system static parameters containing alarm states, and the video packet is transmitted to a fourth storage module.
Further, the display module further comprises an alarm popup module;
The alarm popup module acquires mileage data and corresponding overhead line system static parameters containing alarm states from the fourth storage module at fixed frequency, and each frame popup content comprises mileage, overhead line system static parameters corresponding to the same mileage and corresponding alarm states.
Further, after N video pictures I are compressed into video packets, the video packets are packed together with N mileage data and N contact network static parameters, and the video packets are transmitted to a fourth storage module.
Further, the display module further comprises a graph drawing module;
And the graph drawing module acquires mileage data and corresponding contact net static parameters from the fourth storage module at fixed frequency, draws the mileage data and the corresponding contact net static parameters into a trend graph, and displays the trend graph in a corresponding display frame of the display screen.
Furthermore, the display frames are arranged according to the quantity requirement of the contact net static parameters for simultaneous display.
The inspection instrument with the real-time display function provided by the invention has the following characteristics:
1. Real-time performance of data processing and display: by adopting a working mode of timing triggering and adopting a mode of whole serial and partial parallel in a single flow, the efficiency of a data processing end is greatly improved, meanwhile, the video playing adopts multi-player switching, the display fourth storage module of a trend chart is dynamically updated and regularly refreshed, the real-time performance of the data display end is also ensured, and the maximum delay time from triggering to display is not more than 1.5s.
2. Richness and intuitiveness of data: through synchronous display of the terminal interface, the multi-source data comprises: mileage data, detection results (namely, pull-out values, contact line heights, whether locating points, hanger point key features and the like) and video images are displayed in windows of interfaces according to preset conditions. And the display information of each window takes mileage as a keyword, so that data correspondence based on mileage is realized, and rich detection information is provided for quasi-position fault points and fault reasons. In addition, the detection result is displayed in a two-dimensional curve mode, and compared with a text form, the method has the following advantages: (1) The continuity of the data in the detection process is more obviously highlighted; (2) Intuitively, the overrun position and the fault point are easier to accurately position.
3. The system of the inspection instrument has expansibility and reusability: the multi-source data processing adopts a parallel framework, if the requirement of increasing the data source exists in the practical application, the corresponding acquisition module is only required to be increased, a new thread expansion process is only required to be created, other processes in the original system are not affected, and the system has good expansibility and reusability.
Drawings
FIG. 1 is a schematic diagram of a system structure of a patrol instrument with a real-time display function provided by the invention;
FIG. 2 is a schematic diagram of another system structure of the inspection apparatus with real-time display function according to the present invention;
FIG. 3 is a schematic diagram of a third system structure of the inspection apparatus with real-time display function according to the present invention;
FIG. 4 is a schematic diagram of a workflow of the inspection apparatus with real-time display function according to the present invention;
Fig. 5 is a schematic diagram of a video playing process given by taking two players as examples.
Detailed Description
The technical scheme of the present invention is described in detail below with reference to the accompanying drawings and examples.
As shown in fig. 1, the inspection instrument with the real-time display function comprises a data acquisition module, a data processing module and a display module;
The data acquisition module comprises a measurement camera, a video camera and an odometer which can be synchronized and continuously triggered at a fixed frequency; the measuring camera is used for acquiring picture data required by resolving the static parameters of the overhead contact system, calculating the static parameters of the overhead contact system based on the picture data, and storing a calculation result in the first storage module; the visual field of the video camera is larger than that of the measuring camera, and the video camera is used for collecting video pictures and storing the video pictures in the second storage module; the odometer is used for acquiring mileage data corresponding to the process of simultaneously acquiring pictures by the measuring camera and the video camera and storing the mileage data in the third storage module;
When the number of pictures stored in the second storage module reaches N, N mileage data are stored in the third storage module, N contact net static parameters are stored in the first storage module, at the moment, the data processing module acquires the contact net static parameters from the first storage module, acquires video pictures from the second storage module, acquires mileage data from the third storage module, and marks the contact net static parameters and mileage data corresponding to the same mileage on the video pictures at the same time to obtain a video picture I; compressing N video pictures I into video packets and transmitting the video packets to a fourth storage module; (N is a preset value, which is comprehensively set by a person skilled in the art according to the performance of the video player)
The display module comprises at least two player modules which respectively acquire video packets from the fourth storage module, load the video packets and play the video packets on the display screen in parallel and alternately according to mileage or time, and each frame of the video packets is displayed with a mileage number and a contact net static parameter corresponding to the same mileage number.
The first storage module, the second storage module, the third storage module and the fourth storage module are memory areas capable of storing different data types, and the memory areas of the modules are distinguished through memory addresses.
In order to ensure the fluency of playing the video on the display screen, at least two players sequentially load different video packets respectively, and the loaded video packets are played by the current player when the video packets loaded by the previous player are played on the display screen in sequence. In the implementation, as shown in fig. 4, two players are selected, when the first player finishes playing the video packet i, the second player loads the video packet i+1, and when the first player finishes playing the video i, the second player plays the video packet i+1; at the same time the first player loads video packet i +2, in this way cycling the play-out process. And each player finishes playing on the same display screen, so that the fluency of video playing on the display screen is realized.
In order to facilitate observers to quickly acquire the abnormal position of the static parameters of the overhead line system, as one embodiment of the invention, the inspection instrument with the real-time display function further comprises an alarm module; the measuring camera transmits the calculated static parameter data of the overhead line system to the alarm module; the alarm module respectively judges whether the static parameters of the overhead contact system meet preset conditions according to a preset threshold value; if the contact network static parameters are met, marking the contact network static parameters as normal states; if the contact network static parameters do not meet the alarm conditions, the contact network static parameters containing the alarm conditions are marked with the corresponding alarm conditions and sent to a fourth storage module. Correspondingly, the data transmitted to the fourth storage module is: and compressing the N video pictures I into a video packet, and then packaging the video packet with N mileage data and N groups of overhead line system static parameters containing alarm states. In order to complete the display of the alarm data, the display module further comprises an alarm popup module; the alarm popup module acquires mileage data and corresponding overhead line system static parameters containing alarm states from the fourth storage module at fixed frequency, and each frame popup content comprises mileage, overhead line system static parameters corresponding to the same mileage and corresponding alarm states.
The fourth storage module is dynamically updated according to mileage data; judging whether the fourth storage module has newly received mileage data, if not, directly storing the newly received mileage data and the corresponding overhead line system static parameters containing the alarm state in the fourth storage module; if the contact network static parameters containing the alarm state are stored in the fourth storage module in a mode of covering original data of the same mileage.
In order to observe the trend of the static parameters of the overhead contact system along with the mileage data in real time while observing the video, as another embodiment of the present invention, a fourth storage module stores: and the video packet compressed by the N video pictures I is packed together with N mileage data and N contact network static parameters.
The display module further comprises a graph drawing module; the graph drawing module acquires mileage data and corresponding contact net static parameters from the fourth storage module at fixed frequency, draws the mileage data and the corresponding contact net static parameters into a trend graph, and displays the trend graph in a corresponding display frame of the display screen. The display frames are arranged according to the quantity requirements of the contact net static parameters for simultaneous display. According to the user's viewing requirement, the display frame can be provided with one, and the change trend lines of all the contact net static parameters compared with mileage are distinguished by different colors, lines or other marks; if two display frames are arranged, each display frame can respectively display a preselected change trend chart of the static parameters of the contact network compared with mileage.
The inspection instrument with the real-time display function provided by the invention has the following characteristics:
1. Real-time performance of data processing and display: the maximum delay time from triggering to display does not exceed 1.5s;
2. richness and intuitiveness of data: through synchronous display of the terminal interface, the multi-source data comprises: mileage data, overhead line system static parameters and video images; the defect position can be conveniently positioned, and the situation near the defect position can be acquired according to the video image, so that the fault/defect occurrence cause can be conveniently and manually judged;
3. The inspection instrument adopts a parallel framework for multi-source data processing, can meet the requirement of increasing data sources, and improves the expansibility and reusability of the system.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application to thereby enable others skilled in the art to make and utilize the invention in various exemplary embodiments and with various alternatives and modifications. It is intended that the scope of the invention be defined by the following claims and their equivalents.
Claims (9)
1. An inspection instrument with real-time display function, which is characterized in that: the system comprises a data acquisition module, a data processing module and a display module;
The data acquisition module comprises a measurement camera, a video camera and an odometer which can be synchronized and continuously triggered at a fixed frequency; the measuring camera is used for acquiring picture data required by resolving the static parameters of the overhead contact system, calculating the static parameters of the overhead contact system based on the picture data and storing a calculation result in the first storage module; the visual field of the video camera is larger than that of the measuring camera, and the video camera is used for collecting video pictures and storing the video pictures in the second storage module; the odometer is used for acquiring mileage data corresponding to the process of simultaneously acquiring pictures by the measuring camera and the video camera and storing the mileage data in the third storage module;
When the number of pictures stored in the second storage module reaches N, N mileage data are stored in the third storage module, N contact net static parameters are stored in the first storage module, at the moment, the data processing module acquires the contact net static parameters from the first storage module, acquires video pictures from the second storage module, acquires mileage data from the third storage module, and marks the contact net static parameters and mileage data corresponding to the same mileage on the video pictures at the same time to obtain video picture I; compressing N video pictures I into video packets and transmitting the video packets to a fourth storage module;
the display module comprises at least two player modules which respectively acquire video packets from the fourth storage module, load the video packets and play the video packets on a display screen in parallel and alternately according to mileage or time, and each frame of the video packets is displayed with a mileage number and a contact net static parameter corresponding to the same mileage number.
2. The inspection instrument with real-time display function as claimed in claim 1, wherein: and the at least two players sequentially load different video packages respectively, and when the video packages loaded by the previous player are played on the display screen in sequence, the loaded video packages are played by the current player so as to ensure the fluency of playing the video on the display screen.
3. The inspection instrument with real-time display function as claimed in claim 1, wherein: the system also comprises an alarm module; the measuring camera transmits the calculated static parameter data of the overhead line system to the alarm module; the alarm module respectively judges whether the static parameters of the overhead line system meet preset conditions according to a preset threshold value; if the contact network static parameters are met, marking the contact network static parameters as normal states; if the contact network static parameters do not meet the alarm conditions, the contact network static parameters containing the alarm conditions are marked with the corresponding alarm conditions and sent to a fourth storage module.
4. A patrol instrument having a real-time display function according to claim 3, wherein: the fourth storage module is dynamically updated according to mileage data; judging whether the fourth storage module has newly received mileage data, if not, directly storing the newly received mileage data and the corresponding overhead line system static parameters containing the alarm state in the fourth storage module; if the contact network static parameters containing the alarm state are stored in the fourth storage module in a mode of covering original data of the same mileage.
5. The inspection instrument with real-time display function as claimed in claim 1, wherein: and after the N video pictures I are compressed into video packets, the video packets are packed together with N mileage data and N groups of overhead line system static parameters containing alarm states, and the video packets are transmitted to a fourth storage module.
6. The inspection instrument with real-time display function as claimed in claim 1, wherein: the display module further comprises an alarm popup module;
The alarm popup module acquires mileage data and corresponding overhead line system static parameters containing alarm states from the fourth storage module at fixed frequency, and each frame popup content comprises mileage, overhead line system static parameters corresponding to the same mileage and corresponding alarm states.
7. The inspection instrument with real-time display function as claimed in claim 1, wherein: and after the N video pictures I are compressed into video packets, the video packets are packed together with N mileage data and N contact network static parameters, and are transmitted to a fourth storage module.
8. The inspection instrument with real-time display function according to claim 7, wherein: the display module further comprises a graph drawing module;
And the graph drawing module acquires mileage data and corresponding contact net static parameters from the fourth storage module at fixed frequency, draws the mileage data and the corresponding contact net static parameters into a trend graph, and displays the trend graph in a corresponding display frame of the display screen.
9. The inspection instrument with real-time display function according to claim 8, wherein: the display frames are arranged according to the quantity requirements of the contact net static parameters for simultaneous display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111622855.4A CN114360090B (en) | 2021-12-28 | 2021-12-28 | Patrol instrument with real-time display function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111622855.4A CN114360090B (en) | 2021-12-28 | 2021-12-28 | Patrol instrument with real-time display function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114360090A CN114360090A (en) | 2022-04-15 |
| CN114360090B true CN114360090B (en) | 2024-06-14 |
Family
ID=81103530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111622855.4A Active CN114360090B (en) | 2021-12-28 | 2021-12-28 | Patrol instrument with real-time display function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114360090B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102538762A (en) * | 2012-01-10 | 2012-07-04 | 广州科易光电技术有限公司 | Online inspection device of high-speed railway contact network and inspection method of online inspection device as well as high-speed rail contact network detection system formed by online inspection device |
| CN106327875A (en) * | 2016-08-29 | 2017-01-11 | 苏州金螳螂怡和科技有限公司 | Traffic video monitoring management control system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9721350B2 (en) * | 2015-06-26 | 2017-08-01 | Getalert Ltd. | Methods circuits devices systems and associated computer executable code for video feed processing |
| CN106056691A (en) * | 2016-06-02 | 2016-10-26 | 深圳朝伟达科技有限公司 | Instant patrol system for security guard |
| CN108415447A (en) * | 2018-03-09 | 2018-08-17 | 广东电网有限责任公司机巡作业中心 | A kind of machine patrols visual remote fortune pipe platform |
| CN110400387A (en) * | 2019-06-26 | 2019-11-01 | 广东康云科技有限公司 | A kind of joint method for inspecting, system and storage medium based on substation |
-
2021
- 2021-12-28 CN CN202111622855.4A patent/CN114360090B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102538762A (en) * | 2012-01-10 | 2012-07-04 | 广州科易光电技术有限公司 | Online inspection device of high-speed railway contact network and inspection method of online inspection device as well as high-speed rail contact network detection system formed by online inspection device |
| CN106327875A (en) * | 2016-08-29 | 2017-01-11 | 苏州金螳螂怡和科技有限公司 | Traffic video monitoring management control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114360090A (en) | 2022-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101834989B (en) | Real-time data acquisition and storage system of helicopter in electric inspection process | |
| CN1020356C (en) | Simulative interference video signals generator | |
| AU2014277666B2 (en) | Event recorder playback with integrated GPS mapping | |
| US7936372B2 (en) | Image processing method and system for generating and analyzing metadata and server for such system | |
| CN108872796A (en) | Ground control platform based on vehicle-mounted contact net condition monitoring | |
| CN112153267B (en) | Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses | |
| CN1409180A (en) | Data preparing method and device | |
| CN211741554U (en) | Millimeter wave radar test system and test vehicle | |
| CN114360090B (en) | Patrol instrument with real-time display function | |
| CN102158633B (en) | Synchronous image signal processing method | |
| CN112104086B (en) | Transformer substation remote live-action holographic exploration system and method | |
| US20010022863A1 (en) | Image recording apparatus | |
| US7027082B2 (en) | Image recording apparatus and image recording system | |
| CN105573707A (en) | Debug method and system for virtual reality head-mounted visual device | |
| CN103678559A (en) | Method and device for displaying monitoring data | |
| CN108416870A (en) | Running information recording method and system | |
| CN116151557A (en) | Visual management method and device for inspection equipment and data cockpit system | |
| CN115345771A (en) | Automatic driving simulation test image processing method and device | |
| CN109510980B (en) | Live broadcast delay measurement method and system | |
| CN114360087A (en) | Mining AR system of patrolling and examining | |
| CN111141742B (en) | Intelligent image detection system for leaky coaxial cable and working method thereof | |
| JP3201509B2 (en) | Train diagram display | |
| CN104935830A (en) | Splicing display apparatus video information rendering and displaying methods and systems | |
| CN113296607B (en) | VR-based multi-user virtual experiment teaching system | |
| CN219107135U (en) | Inspection system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |