CN108736972A - LED vision-based detections based on ITS-VLC and tracking and its system - Google Patents
LED vision-based detections based on ITS-VLC and tracking and its system Download PDFInfo
- Publication number
- CN108736972A CN108736972A CN201810578665.9A CN201810578665A CN108736972A CN 108736972 A CN108736972 A CN 108736972A CN 201810578665 A CN201810578665 A CN 201810578665A CN 108736972 A CN108736972 A CN 108736972A
- Authority
- CN
- China
- Prior art keywords
- traffic lights
- led
- image
- led traffic
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 33
- 230000003287 optical effect Effects 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims abstract description 7
- 238000005516 engineering process Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 230000005611 electricity Effects 0.000 claims 1
- 230000000149 penetrating effect Effects 0.000 claims 1
- 238000007796 conventional method Methods 0.000 abstract 1
- 238000003672 processing method Methods 0.000 abstract 1
- 230000009466 transformation Effects 0.000 abstract 1
- 238000004891 communication Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/11—Arrangements specific to free-space transmission, i.e. transmission through air or vacuum
- H04B10/114—Indoor or close-range type systems
- H04B10/116—Visible light communication
Landscapes
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The LED vision-based detections that the invention discloses a kind of based on ITS-VLC and tracking and its system, this method include：Light emitting diode in LED traffic lights is captured using high-speed camera and synchronizes the light emitted at random, and the preliminary vision-based detection of LED traffic lights is realized by image processing techniques；The rectangle candidate area divided according to edge obtained using preliminary vision-based detection tracks the LED traffic lights in successive frame by differential density defined in calculating and mean intensity；When can not search in the candidate regions divided based on marginal information LED traffic lights by optical flow method, new LED traffic lights candidate regions are re-searched in original image, until the image processing method based on marginal information can smoothly detect LED traffic lights.The present invention using existing system equipment transformation realize, algorithm simple possible, solve the disadvantage that conventional method can only tracking section light-emitting diode pole pipe, have a vast market value.
Description
Technical field
The present invention relates to vision-based detections and tracking technique field, and in particular to a kind of LED vision-based detections based on ITS-VLC
With tracking and its system.
Background technology
Visible light communication (VLC) is a kind of communication using visible light, by shining come transmission data.It can
By emitting light source come transmission data, and them can be received by optical sensor.With traditional infrared and wireless communication
It compares, it is seen that optic communication has high, confidentiality without electromagnetic interference, without applying for frequency spectrum resource and information of transmission power etc. excellent
Point.
With the development of information technology, intelligent transportation system (ITS) has been introduced into control traffic congestion and traffic accident etc.
The solution of traffic problems.ITS is by advanced information technology, data communication transmission technology, Electronic transducer technology, electron controls technology
And computer processing technology etc. is effectively integrated into entire traffic management system, and the one kind set up is in big model
In enclosing, it is comprehensive play a role, in real time, accurately and efficiently multi-transportation and management system.
Communication in LED vision-based detections based on ITS-VLC and tracking and its system between vehicle and LED traffic lights
Using LED traffic lights as transmitter, vehicle-mounted high-speed camera as receiver.Traditional uses marginal information in successive frame
In the detected transmitter algorithm of tracking, part LED be used to detect and part LED be used to detect, so not all
LED may be used to continuously communicate.
Invention content
The purpose of the present invention is to solve drawbacks described above in the prior art, a kind of LED based on ITS-VLC is provided and is regarded
Feel detection and tracking and its system, it is simple, convenient that this method is realized, real by LED traffic lights and vehicle-mounted high-speed camera
Existing vehicle and traffic system continuous communiction.
According to disclosed embodiment, the first aspect of the present invention disclose a kind of LED vision-based detections based on ITS-VLC with
Tracking, the method include the following steps：
Light emitting diode synchronizes after LED drive circuit is modulated in S1, LED traffic lights sends out certain frequency at random
Light；
S2, vehicle-mounted high-speed camera shoot road image with the rate of twice transmitting light frequency, and capture contains LED traffic
The image of lamp determines that LED traffic lights candidate regions split from original image it, after singulation using image processing techniques
Judge in region and detects LED traffic lights；
S3, using the LED traffic lights marginal informations that have detected of previous frame in successive frame, search for upper one in the current frame
The LED traffic lights candidate region being partitioned into frame is detected in candidate regions whether contain LED traffic lights, is detected in next frame
LED traffic lights；
If S4, the method based on marginal information cannot detect LED traffic lights, tracked using optical flow method, again again
New candidate regions are found in search in original image, until LED traffic lights is detected by the method based on marginal information.
Further, the step S2 processes are as follows：
S21, receiver capture road image, obtain the image information containing transmitter, and collect all pictures in image
Prime number evidence generates difference image by calculating continuous frame difference by frame difference method；
S22, with suitable threshold process binary image, the morphology operations for being expanded, being corroded, selection with it is acquired
The sizable rectangle of white area as transmitter candidate regions；
S23, pass through search differential density D in the corresponding candidate region in difference imagedenWith mean intensity LavgJudge
Whether candidate regions object is transmitter, calculates D as followsdenAnd Lavg：
DpixAnd TpixRespectively represent total number of pixels of pixel and candidate regions of the Pixel gray difference value more than 20, LavgTable
Show the mean difference degree of pixel in candidate regions, LpixThe gray value for indicating each pixel in candidate regions, when meeting the following conditions,
Candidate regions are confirmed to be transmitter：
Dden×Larg>threshold
Threshold is preset judgment threshold in formula.
Further, optical flow method tracking process is as follows in the step S4：
The previous frame of present frame is pre-processed using gauss low frequency filter, the previous frame of wherein present frame is known as base
Frame calculates deformation matrix D at each pixel with directional derivative, as follows：
dxxRepresent the second dervative on the directions x, dxyRepresent the inclined y derivatives of the inclined x of d, dyyRepresent the second dervative on the directions y, dyx
The inclined y derivatives of the inclined x of d are represented, the characteristic value of deformation matrix D is calculated, and carry out non-maximum value inhibition, it is assumed that local minimum is present in
5 × 5 neighborhood IcutIt is interior, there is no transmitter, n optimum value of characteristic point is selected using the characteristic value of priority,
This feature point image-region I selected to usefeaIt is as follows：
Ifea=Icut-(Itra+Ioffset)
Wherein ItraIt is that former frame detects transmitter region, IoffsetIt is newly-established region, selects the feature of previous frame
After point, corresponding feature point set is obtained in the current frame.
According to disclosed embodiment, the second aspect of the present invention disclose a kind of LED vision-based detections based on ITS-VLC with
Tracking system, the system include：As the LED traffic lights of transmitter, as the vehicle-mounted high-speed camera of receiver；
Wherein, the LED traffic lights includes several light emitting diodes and for driving the LED of light emitting diode to drive
Circuit, light emitting diode synchronize the random light for sending out certain frequency after LED drive circuit is modulated；
The vehicle-mounted high-speed camera includes sequentially connected camera lens, sensor, A/D converter, DSP, and camera lens is with two
The rate of times glow frequency shoots road image, and sensor captures the image containing LED traffic lights, after reading in video data,
The analog signal of image is converted into digital signal by A/D converter, DSP is again processed digital signal, uses
Image processing techniques detects transmitter.
Further, the light emitting diode synchronizes after LED drive circuit is modulated sends out 500Hz frequencies at random
Light, the camera lens shoot road image with the rate of 1000Hz frequencies.
The present invention has the following advantages and effects with respect to the prior art：
(1) present invention proposes a kind of effective ways of new high-speed camera detection and track transmitter.Traditional
Vision-based detection is with tracking, and the light emitting diode in a part of LED be used to detect and tracking, and another part part is for leading to
Letter, so not every light emitting diode may be used to continuously communicate, proposed by the present invention based on ITS-VLC's
LED vision-based detections can be used in continuous communiction with all light emitting diodes in tracking, improve data rate.
(2) present invention uses optical flow methods, and in the detection algorithm failure based on marginal information, tracking transmitter can be fast
Fast effective locking transmitter, overcome the detection based on marginal information cannot again light emitting diode random light emission when can not detect
With the difficulty of tracking, the scope of application of ITS-VLC systems is expanded.
Description of the drawings
Fig. 1 is the schematic diagram of LED vision-based detections and tracking system disclosed by the invention based on ITS-VLC；
Fig. 2 is the algorithm flow schematic diagram that the present invention realizes vision-based detection and tracking；
Fig. 3 (a) is transmitting sequence schematic diagram when LED section is used to track for detection part in the prior art；
Fig. 3 (b) is the transmitting sequence schematic diagram that the present invention realizes.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
The every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Embodiment
As shown in Figure 1, a kind of LED vision-based detections and tracking system based on ITS-VLC.Including：LED as transmitter
Traffic lights, the vehicle-mounted high-speed camera as receiver.LED traffic lights includes several light emitting diodes, LED drive circuit.Through
LED drive circuit modulation is crossed, light emitting diode synchronizes the random light for sending out 500Hz.Vehicle-mounted high-speed camera includes being sequentially connected
Camera lens, sensor, A/D converter, DSP.In receiver one end, vehicle-mounted high-speed camera shoots road image with 1000fps,
To which sensor captures the image containing LED traffic lights, after reading in video data, the simulation of image is believed by A/D converter
Number it is converted into digital signal, DSP is again processed digital signal, using image processing techniques, detects transmitter.
As shown in Fig. 2, vehicle-mounted high-speed camera shoots road video, it can be seen as a width RGB image per frame video, by RGB
Gray level image is converted to, the difference image of successive frame is found out, and then is bianry image using suitable threshold transition, using swollen
The morphology operations such as swollen, corrosion determine LED traffic lights candidate regions and by it from original to image preprocessing, using image processing techniques
It is split in image, judge in candidate region after singulation and detects LED traffic lights.If the method based on marginal information
It cannot detect LED traffic lights, then utilize optical flow method to track, new candidate regions are found in search in original image again again, until
LED traffic lights is detected by the method based on marginal information.
It is described divide again after candidate regions be judged and detect LED traffic lights and be specially：It is corresponding in difference image
Differential density D is searched in candidate regiondenWith mean intensity LavgJudge whether candidate regions object is transmitter, is counted as follows
Calculate DdenAnd Lavg：
DpixAnd TpixTotal pixel of discrepant pixel (Pixel gray difference value is more than 20) and candidate regions are respectively represented
Number, LavgIndicate the mean difference degree of pixel in candidate regions, LpixThe gray value for indicating each pixel in candidate regions, meet with
When lower condition, candidate regions are confirmed to be transmitter：
Dden×Lavg>threshold
Threshold is preset suitable threshold value in formula.
The optical flow method tracks：Using gauss low frequency filter to the previous frame of present frame (being known as base frame) into
Row pretreatment, deformation matrix D is calculated at each pixel with directional derivative.
dxxRepresent the second dervative on the directions x, dxyRepresent the inclined y derivatives of the inclined x of d, similarly, dyyThe second order represented on the directions y is led
Number, dyxThe inclined y derivatives of the inclined x of d are represented, the characteristic value of deformation matrix D is calculated and carry out non-maximum value inhibition, it is assumed that local minimum is deposited
It is I in 5 × 5 neighborhoodscut, there is no transmitter, select the n of characteristic point a most using the characteristic value of priority
Good value, this feature point image-region I selected to usefeaIt is as follows：
Ifea=Icut-(Itra+Ioffset)
Wherein ItraIt is that former frame detects transmitter region, IoffsetIt is newly-established region, selects the feature of previous frame
After point, corresponding feature point set is obtained in the current frame.
The above embodiment is a preferred embodiment of the present invention, but embodiments of the present invention are not by above-described embodiment
Limitation, it is other it is any without departing from the spirit and principles of the present invention made by changes, modifications, substitutions, combinations, simplifications,
Equivalent substitute mode is should be, is included within protection scope of the present invention.
Claims (5)
1. a kind of LED vision-based detections and tracking based on ITS-VLC, which is characterized in that the method includes following step
Suddenly：
Light emitting diode synchronizes the random light for sending out certain frequency after LED drive circuit is modulated in S1, LED traffic lights；
S2, vehicle-mounted high-speed camera shoot road image with the rate of twice transmitting light frequency, and capture contains LED traffic lights
Image determines that LED traffic lights candidate regions split from original image it, region after singulation using image processing techniques
Interior judgement simultaneously detects LED traffic lights；
S3, using the LED traffic lights marginal informations that have detected of previous frame in successive frame, search in previous frame in the current frame
The LED traffic lights candidate region being partitioned into, detect candidate regions in whether contain LED traffic lights, detect the LED in next frame
Traffic lights；
If S4, the method based on marginal information cannot detect LED traffic lights, tracked using optical flow method, it is again original again
New candidate regions are found in search in image, until LED traffic lights is detected by the method based on marginal information.
2. LED vision-based detections and tracking according to claim 1 based on ITS-VLC, which is characterized in that described
Step S2 processes are as follows：
S21, receiver capture road image, obtain the image information containing transmitter, and collect all pixels number in image
According to by calculating continuous frame difference generation difference image by frame difference method；
S22, with suitable threshold process binary image, the morphology operations for being expanded, being corroded, selection with it is obtained white
The comparable rectangle of color area size is as transmitter candidate regions；
S23, pass through search differential density D in the corresponding candidate region in difference imagedenWith mean intensity LavgJudge candidate
Whether area's object is transmitter, calculates D as followsdenAnd Lavg：
DpixAnd TpixRespectively represent total number of pixels of pixel and candidate regions of the Pixel gray difference value more than 20, LavgIt indicates to wait
The mean difference degree of pixel, L in constituencypixThe gray value for indicating each pixel in candidate regions, it is candidate when meeting the following conditions
Area is confirmed to be transmitter：
Dden×Larg>threshold
Threshold is preset judgment threshold in formula.
3. LED vision-based detections and tracking according to claim 1 based on ITS-VLC, which is characterized in that described
Optical flow method tracking process is as follows in step S4：
The previous frame of present frame is pre-processed using gauss low frequency filter, the previous frame of wherein present frame is known as base frame,
Deformation matrix D is calculated with directional derivative at each pixel, it is as follows：
dxxRepresent the second dervative on the directions x, dxyRepresent the inclined y derivatives of the inclined x of d, dyyRepresent the second dervative on the directions y, dyxIt represents
The inclined y derivatives of the inclined x of d, calculate the characteristic value of deformation matrix D, and carry out non-maximum value inhibition, it is assumed that local minimum is present in 5 × 5
Neighborhood IcutIt is interior, there is no transmitter, n optimum value of characteristic point, the spy are selected using the characteristic value of priority
Sign point image-region I selected to usefeaIt is as follows：
Ifea=Icut-(Itra+Ioffset)
Wherein ItraIt is that former frame detects transmitter region, IoffsetIt is newly-established region, after the characteristic point for selecting previous frame,
Corresponding feature point set is obtained in the current frame.
4. a kind of LED vision-based detections and tracking system based on ITS-VLC, which is characterized in that the system includes：As hair
LED traffic lights, the vehicle-mounted high-speed camera as receiver for penetrating machine；
Wherein, the LED traffic lights includes several light emitting diodes and the LED driving electricity for driving light emitting diode
Road, light emitting diode synchronize the random light for sending out certain frequency after LED drive circuit is modulated；
The vehicle-mounted high-speed camera includes sequentially connected camera lens, sensor, A/D converter, DSP, and camera lens is with twice
The rate of glow frequency shoots road image, and sensor captures the image containing LED traffic lights, after reading in video data, passes through
The analog signal of image is converted into digital signal by A/D converter, and DSP is again processed digital signal, uses image
Treatment technology detects transmitter.
5. LED vision-based detections and tracking system according to claim 4 based on ITS-VLC, which is characterized in that described
Light emitting diode synchronizes the random light for sending out 500Hz frequencies after LED drive circuit is modulated, and the camera lens is with 1000Hz frequencies
The rate of rate shoots road image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578665.9A CN108736972A (en) | 2018-06-07 | 2018-06-07 | LED vision-based detections based on ITS-VLC and tracking and its system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810578665.9A CN108736972A (en) | 2018-06-07 | 2018-06-07 | LED vision-based detections based on ITS-VLC and tracking and its system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108736972A true CN108736972A (en) | 2018-11-02 |
Family
ID=63932694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810578665.9A Pending CN108736972A (en) | 2018-06-07 | 2018-06-07 | LED vision-based detections based on ITS-VLC and tracking and its system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108736972A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110289909A (en) * | 2019-06-28 | 2019-09-27 | 华南理工大学 | Outdoor visible light communication target signal source based on optical flow method tracks extracting method |
| CN112164221A (en) * | 2020-09-23 | 2021-01-01 | 北京百度网讯科技有限公司 | Image data mining method, device and equipment and road side equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101435732A (en) * | 2008-12-09 | 2009-05-20 | 华中科技大学 | Space target rotating axle based on binocular light flow and mass center estimation method |
| CN103077539A (en) * | 2013-01-23 | 2013-05-01 | 上海交通大学 | Moving object tracking method under complicated background and sheltering condition |
| CN103325258A (en) * | 2013-06-24 | 2013-09-25 | 武汉烽火众智数字技术有限责任公司 | Red light running detecting device and method based on video processing |
| US20150332097A1 (en) * | 2014-05-15 | 2015-11-19 | Xerox Corporation | Short-time stopping detection from red light camera videos |
| CN107764271A (en) * | 2017-11-15 | 2018-03-06 | 华南理工大学 | A kind of photopic vision dynamic positioning method and system based on light stream |
| CN208386554U (en) * | 2018-06-07 | 2019-01-15 | 华南理工大学 | LED vision-based detection and tracking system based on ITS-VLC |
-
2018
- 2018-06-07 CN CN201810578665.9A patent/CN108736972A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101435732A (en) * | 2008-12-09 | 2009-05-20 | 华中科技大学 | Space target rotating axle based on binocular light flow and mass center estimation method |
| CN103077539A (en) * | 2013-01-23 | 2013-05-01 | 上海交通大学 | Moving object tracking method under complicated background and sheltering condition |
| CN103325258A (en) * | 2013-06-24 | 2013-09-25 | 武汉烽火众智数字技术有限责任公司 | Red light running detecting device and method based on video processing |
| US20150332097A1 (en) * | 2014-05-15 | 2015-11-19 | Xerox Corporation | Short-time stopping detection from red light camera videos |
| CN107764271A (en) * | 2017-11-15 | 2018-03-06 | 华南理工大学 | A kind of photopic vision dynamic positioning method and system based on light stream |
| CN208386554U (en) * | 2018-06-07 | 2019-01-15 | 华南理工大学 | LED vision-based detection and tracking system based on ITS-VLC |
Non-Patent Citations (1)
| Title |
|---|
| H. CHINTHAKA N. PREMACHANDRA: "《High-speed-camera image processing based LED traffic light detection for road-to-vehicle visible light communication》" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110289909A (en) * | 2019-06-28 | 2019-09-27 | 华南理工大学 | Outdoor visible light communication target signal source based on optical flow method tracks extracting method |
| CN112164221A (en) * | 2020-09-23 | 2021-01-01 | 北京百度网讯科技有限公司 | Image data mining method, device and equipment and road side equipment |
| CN112164221B (en) * | 2020-09-23 | 2022-01-25 | 阿波罗智联(北京)科技有限公司 | Image data mining method, device and equipment and road side equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1962226B1 (en) | Image recognition device for vehicle and vehicle head lamp controller and method of controlling head lamps | |
| CN103795467B (en) | Method and apparatus for identifying visible light communication signal received by camera | |
| US11700457B2 (en) | Flicker mitigation via image signal processing | |
| CN107506760B (en) | Traffic signal detection method and system based on GPS positioning and visual image processing | |
| CN107421506B (en) | Visual tracking and positioning system and method based on indoor visible light communication | |
| CN101872546B (en) | Video-based method for rapidly detecting transit vehicles | |
| WO2018041136A1 (en) | Optical communication device and system and corresponding information transferring and receiving method | |
| US20060215882A1 (en) | Image processing apparatus and method, recording medium, and program | |
| JP5147369B2 (en) | Day and night image identification method and apparatus | |
| CN101739827B (en) | Vehicle detecting and tracking method and device | |
| CN101375312A (en) | Multi-mode region-of-interest video object segmentation | |
| CN102722707A (en) | License plate character segmentation method based on connected region and gap model | |
| CN108288289B (en) | LED visual detection method and system for visible light positioning | |
| CN107255524B (en) | Method for detecting frequency of LED light source based on mobile equipment camera | |
| CN111190141B (en) | Method for determining light source emission sequence based on image superposition in visible light indoor positioning | |
| CN108736972A (en) | LED vision-based detections based on ITS-VLC and tracking and its system | |
| CN111931726B (en) | Traffic light detection method, device, computer storage medium and road side equipment | |
| KR100948382B1 (en) | Security service method and system | |
| CN208386554U (en) | LED vision-based detection and tracking system based on ITS-VLC | |
| Choi et al. | Deep learning technique for improving data reception in optical camera communication-based V2I | |
| JP3857129B2 (en) | Moving object detection method and moving object detection apparatus | |
| CN109472767B (en) | Stage lamp missing state analysis system | |
| CN105006156A (en) | Automatic adjusting and control system used for traffic signal lamp | |
| CN112034423B (en) | High-precision mobile vehicle positioning method based on LED visible light communication | |
| Fregin et al. | Feature detectors for traffic light recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |
|
| RJ01 | Rejection of invention patent application after publication |