CN108021875A - A kind of vehicle driver's personalization fatigue monitoring and method for early warning - Google Patents

A kind of vehicle driver's personalization fatigue monitoring and method for early warning Download PDF

Info

Publication number
CN108021875A
CN108021875A CN201711206462.9A CN201711206462A CN108021875A CN 108021875 A CN108021875 A CN 108021875A CN 201711206462 A CN201711206462 A CN 201711206462A CN 108021875 A CN108021875 A CN 108021875A
Authority
CN
China
Prior art keywords
driver
fatigue
early warning
vehicle
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711206462.9A
Other languages
Chinese (zh)
Inventor
夏凌翔
项祯桢
张克军
苏剑波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Ling Technology Co Ltd
Original Assignee
Shanghai Ling Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Ling Technology Co Ltd filed Critical Shanghai Ling Technology Co Ltd
Priority to CN201711206462.9A priority Critical patent/CN108021875A/en
Publication of CN108021875A publication Critical patent/CN108021875A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • G06V40/165Detection; Localisation; Normalisation using facial parts and geometric relationships

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Emergency Alarm Devices (AREA)

Abstract

The present invention relates to vehicle driver's personalization fatigue monitoring and method for early warning, belongs to active safety systems of vehicles field.A kind of vehicle driver's personalization fatigue monitoring and method for early warning, it is characterised in that:Gather driver personalityization feature, it is recorded in vehicle database, driver's facial characteristics is gathered in vehicle operation, is compared with driver personalityization feature in database, the integrated judgment of driver fatigue is carried out, and to more than default value when carries out early warning.The present invention can carry out visual information quantization analysis to driver under the conditions of a variety of driving environments, different accessories wearing conditions and different Characteristics of Drivers ' Behavior differences, the fatigue strength for preferably obtaining driver judges result, and fatigue driving early warning can be carried out, ensure that driver carries out safe driving, compared with prior art with universal adaptability and more preferable monitoring and warning effect.

Description

A kind of vehicle driver's personalization fatigue monitoring and method for early warning
Technical field
The present invention relates to a kind of fatigue monitoring and method for early warning, more particularly to a kind of vehicle driver's personalization fatigue monitoring And method for early warning.
Background technology
With the high speed development of social economy, automobile quantity and driver's quantity increase sharply, and automobile gives people's life While work brings convenient, many traffic accidents are also triggered.Traffic safety has become the hot spot that society pays close attention to for a long time. Fatigue driving is to trigger the major reason of major traffic accidents, its major traffic accidents triggered account for the 40% of accident total amount with On.If it is possible to detect the fatigue state of driver and give warning in advance, then it can to a certain extent reduce and be driven by fatigue The accident rate of initiation is sailed, guarantee is provided for driver safety.
Fatigue driving monitoring is that the specificity such as a kind of psychological to physiological driver during traveling and vehicle running state refer to Mark is detected, and determine driver whether the means of fatigue driving.Current generally accepted detection method is divided into following 3 kinds:
1) using the physiological change index of biosensor detection driver, such as brain electricity, electrocardio, heart rate, breathing, myoelectricity Deng;
2) using the vehicle running state variation characteristic of onboard sensor detection driver's driving behavior and its generation, such as turn To, throttle, gear, brake, and the position of speed, acceleration, vehicle in track etc.;
3) using machine vision technique or the external change feature of other sensors technology for detection driver, as eyelid blinks Move, nod, yawn.
In above-mentioned three kinds of fatigue detection methods, physical signs most can accurately detect driver fatigue, but the party Method needs to install physiological signal sensor with driver, the restriction normal driver behavior of driver and posture, nothing Method widespread adoption in actual scene.
The method detected using vehicle traveling information as fatigue strength, it is only necessary to utilize existing sensor on car.But this method is only In degree of fatigue seriously to can just play forewarning function when accident will occurring, delay degree is high, can not be in fatigue driving Early warning is given during in early stage state, therefore it is very narrow to leave the time window that driver makes a response for, driver hear it is pre- The generation of accident can not be probably avoided when alert.
The method detected using driver's facial characteristics as fatigue strength takes into account the advantage of above two means.On the one hand drive Member's fatigue state more can intuitively show facial characteristics and the head poses such as eyes (blink), face (yawning) On (nodding), another aspect onboard sensor can effectively avoid the accident that fatigue driving triggers under certain environment.1994, Repetition test and demonstration are passed through by Carnegie Mellon University, it is proposed that measure fatigue physical quantity " PERCLOS ", define for unit when Time shared by interior (generally taking 1 minute or 30 seconds) eyes closed certain proportion (70% or 80%).The United States Federal's highway Management board (FHWA) and National Highway Traffic safety management bureau (NHTSA) drive simulating in the lab, complete nine kinds The comparison of fatigue detecting index, the results show that PERCLOS (percentage of eyelid closure over the Pupil over time) it is best with the correlation of driving fatigue.
There are many inventions using PERCLOS as fatigue strength evaluation index at present, but PERCLOS also has its shortcoming, The facial characteristics and motor habit of different people are all different, not only drive every time and are required for the more wasteful time of resampling, If someone natively looks smug or conceited, little trick also be easy to cause erroneous judgement.
The content of the invention
The purpose of the application is to propose a kind of personalization fatigue prison based on driver's facial characteristics, head pose information Survey method for early warning.System can be according to the custom of different drivers and the individualized feature of driver itself, at driver Automatic early-warning is carried out when fatigue state, so as to reduce the traffic accident incidence triggered by driver tired driving.
The present invention is achieved through the following technical solutions:
A kind of vehicle driver's personalization fatigue monitoring and method for early warning, it is characterised in that:To driver personalityization feature It is acquired, is recorded in vehicle database, driver's facial characteristics is gathered in vehicle operation, with being driven in database Member's individualized feature is compared, and carries out the integrated judgment of driver fatigue, and to more than default value when carries out early warning.
The step of using personalized fatigue monitoring and method for early warning, is as follows:
Step 1, carry out driver personalityization collection apparatus, input database;
Step 2, driver prepare drive before, loading of databases;
Step 3, carry out face monitoring, and vehicle launch is not allowed if not detecting successfully;Allow vehicle if detecting successfully Start, and driver's facial characteristics was gathered every 30-60 seconds, at the same time;
Step 4, will the driver's facial characteristics point location and image normalization that collect processing after, carry out recognition of face, Head pose is calculated towards calculating, the differentiation of eye state and PERCLOS indexs;
Step 5, according to recognition of face as a result, if in database there are driver personalityization feature if use the number According to, then use default default data if there is no driver personalityization feature, comprehensive head pose, eye state and PERCLOS indexs compare and analyze driver's fatigue degree, and the fatigue strength for obtaining driver judges result;
If step 6, driver fatigue return to step 3, if driver fatigue not less than the early warning value of default Degree then reminds driver to pay attention to more than the early warning value of default by the alarm mode of sound, vibrations.
Further, the step of driver personalityization feature is gathered described in step 1 is as follows:
Step 1a, Face datection is carried out to driver;
Step 1b, positioning feature point is carried out to the facial characteristics detected to handle with image normalization;
Step 1c, eye opening image, eye closing image and normal driving pose presentation are gathered, eyelid when opening eyes and closing one's eyes is calculated and puts down Equal distance, head pose and direction when calculating normal driving;
Step 1d, the individualized feature of collection is stored in database.
Further, the particular content of the image normalization processing described in step 4 is included under varying environment illumination Driver's image has driver's image under different accessories states to carry out intelligent characteristic through row brightness normalized and to wearing Extraction.
Further, the head pose described in step 5, eye state and PERCLOS indexs include:Driver's single regards Line deviates the duration, the number that sight deviates in driver 30 seconds, the upper palpebra inferior of driver are yawned behavior apart from, driver Number, driver head is regular swings up and down the time.
Beneficial effect:
1) driver personalityization fatigue strength monitoring and pre-alarming method of the invention, is added by gathering driver personalityization feature Database, and comprehensive individualized feature, head pose, eye state and PERCLOS indexs carry out driver's fatigue degree pair Than analysis, the fatigue strength that can preferably obtain driver is judged as a result, compared with prior art with universal adaptability and more Good monitoring and warning effect.
2) present invention, can be to driver in a variety of driving environments, different accessories using means such as graphics process, quantitative analysis Visual information quantization analysis is carried out under the conditions of wearing condition and different Characteristics of Drivers ' Behavior differences, and it is pre- to carry out fatigue driving It is alert, it is ensured that driver carries out safe driving.
Brief description of the drawings
Fig. 1 present invention carries out fatigue monitoring and the flow chart of early warning;
Fig. 2 gathers the flow chart of driver personalityization feature;
Embodiment
The present invention is described in detail with specific embodiment below in conjunction with the accompanying drawings:
As shown in Figure 1 a kind of vehicle driver's personalization fatigue monitoring and method for early warning, to driver personalityization feature into Row collection, is recorded in vehicle database, driver's facial characteristics is gathered in vehicle operation, with driver in database Individualized feature is compared, and carries out the integrated judgment of driver fatigue, and to more than default value when carries out early warning.
The step of using personalized fatigue monitoring and method for early warning, is as follows:
Step 1, carry out driver personalityization collection apparatus, input database;
Step 2, driver prepare drive before, loading of databases;
Step 3, carry out face monitoring, and vehicle launch is not allowed if not detecting successfully;Allow vehicle if detecting successfully Start, and driver's facial characteristics was gathered every 30-60 seconds, at the same time;
Step 4, will the driver's facial characteristics point location and image normalization that collect processing after, carry out recognition of face, Head pose is calculated towards calculating, the differentiation of eye state and PERCLOS indexs;
Step 5, according to recognition of face as a result, if in database there are driver personalityization feature if use the number According to, then use default default data if there is no driver personalityization feature, comprehensive head pose, eye state and PERCLOS indexs compare and analyze driver's fatigue degree, and the fatigue strength for obtaining driver judges result;
If step 6, driver fatigue return to step 3, if driver fatigue not less than the early warning value of default Degree then reminds driver to pay attention to more than the early warning value of default by the alarm mode of sound, vibrations.
As shown in Figure 1, the step of collection driver personalityization feature, is as follows:
Step 1a, Face datection is carried out to driver;
Step 1b, positioning feature point is carried out to the facial characteristics detected to handle with image normalization;
Step 1c, eye opening image, eye closing image and normal driving pose presentation are gathered, eyelid when opening eyes and closing one's eyes is calculated and puts down Equal distance, head pose and direction when calculating normal driving;
Step 1d, the individualized feature of collection is stored in database.
The flow of driver's facial image acquisition is:Original image is analyzed and is handled, obtains the position of human face region Put, carry out the positioning of face key feature points, mark out the crucial geometric position such as eyes, eyebrow, nose, face, face mask, And the rotation and scaling correction that face is normalized, obtain head pose information;At the same time the state of eyes open closing The statistic of classification of eye, and by head pose information and facial characteristics with identity information it is corresponding after be entered into database.
The particular content of image normalization processing is included to driver's image under varying environment illumination through row brightness normalizing Change processing and there is driver's image under different accessories states to carry out intelligent characteristic extraction wearing.
Head pose, eye state and PERCLOS indexs include:Driver's single sight deviates duration, driver Sight deviates in 30 seconds number, the upper palpebra inferior of driver are regular apart from yawn behavior number, driver head of, driver Swing up and down the time.
In vehicle travel process, classification that system opens and closes eyes the state of driver's eyes in the image that collects Differentiate, count closed-eye time and PERCLOS fatigue indexes and common fatigue driving situation.Recognition of face and individualized feature mould Block is matched with registered user in database, obtained and active user couple by carrying out feature extraction to human face region The personalized fatigue characteristic answered, and as the discriminant criterion of the user's fatigue strength, the comprehensive distinguishing for fatigue strength.
At the beginning of vehicle launch, system at once differentiates driver identity information.In driver's driving procedure, this Fatigue monitoring and method for early warning will monitor in real time with driving procedure, and give early warning when fatigue state occurs in driver.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all originals in the present invention Then with all any modification, equivalent and improvement made within spirit etc., it should all be included in the protection scope of the present invention.

Claims (5)

1. a kind of vehicle driver's personalization fatigue monitoring and method for early warning, it is characterised in that:Driver personalityization feature is gathered, It is recorded in vehicle database, driver's facial characteristics is gathered in vehicle operation, with driver personalityization in database Feature is compared, and carries out the integrated judgment of driver fatigue, and to more than default value when carries out early warning.
2. vehicle driver's personalization fatigue monitoring as claimed in claim 1 and method for early warning, it is characterised in that:
The personalization fatigue monitoring and method for early warning use the following steps:
Step 1, collection driver personalityization feature, input database;
Step 2, driver prepare drive before, loading of databases;
Step 3, carry out Face datection, and vehicle launch is not allowed if not detecting successfully;Allow vehicle launch if detecting successfully, And gathered driver's facial characteristics every 30-60 seconds;
Step 4, after handling the driver's facial characteristics point location and image normalization that collect, carry out recognition of face, head Posture is calculated towards calculating, the differentiation of eye state and PERCLOS indexs;
Step 5, according to recognition of face as a result, if in database there are driver personalityization feature if use the data, such as Fruit then to use default default data there is no driver personalityization feature, and comprehensive head pose, eye state and PERCLOS refer to Mark compares and analyzes driver's fatigue degree, and the fatigue strength for obtaining driver judges result;
If step 6, driver fatigue return to step 3, if driver fatigue surpasses not less than the early warning value of default The early warning value of default is crossed, then reminds driver to pay attention to by the alarm mode of sound, vibrations.
3. vehicle driver's personalization fatigue monitoring as claimed in claim 2 and method for early warning, it is characterised in that:
The step of driver personalityization feature is gathered described in step 1 is as follows:
Step 1a, Face datection is carried out to driver;
Step 1b, positioning feature point is carried out to the facial characteristics detected to handle with image normalization;
Step 1c, eye opening image, eye closing image and normal driving pose presentation are gathered, calculates eyelid average departure when opening eyes and closing one's eyes From head pose and direction when calculating normal driving;
Step 1d, the individualized feature of collection is stored in database.
4. vehicle driver's personalization fatigue monitoring as claimed in claim 2 and method for early warning, it is characterised in that:
The particular content of image normalization processing described in step 4 is included to driver's image under varying environment illumination through row Brightness normalized and there is driver's image under different accessories states to carry out intelligent characteristic extraction wearing.
5. vehicle driver's personalization fatigue monitoring as claimed in claim 2 and method for early warning, it is characterised in that:
Head pose, eye state and PERCLOS indexs described in step 5 include:When driver's single sight deviates lasting Between, the number that sight deviates in driver 30 seconds, the upper palpebra inferior of driver yawn behavior number, driver apart from, driver Head is regular to swing up and down the time.
CN201711206462.9A 2017-11-27 2017-11-27 A kind of vehicle driver's personalization fatigue monitoring and method for early warning Pending CN108021875A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711206462.9A CN108021875A (en) 2017-11-27 2017-11-27 A kind of vehicle driver's personalization fatigue monitoring and method for early warning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711206462.9A CN108021875A (en) 2017-11-27 2017-11-27 A kind of vehicle driver's personalization fatigue monitoring and method for early warning

Publications (1)

Publication Number Publication Date
CN108021875A true CN108021875A (en) 2018-05-11

Family

ID=62077293

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711206462.9A Pending CN108021875A (en) 2017-11-27 2017-11-27 A kind of vehicle driver's personalization fatigue monitoring and method for early warning

Country Status (1)

Country Link
CN (1) CN108021875A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446600A (en) * 2018-02-27 2018-08-24 上海汽车集团股份有限公司 A kind of vehicle driver's fatigue monitoring early warning system and method
CN109460780A (en) * 2018-10-17 2019-03-12 深兰科技(上海)有限公司 Safe driving of vehicle detection method, device and the storage medium of artificial neural network
CN111079476A (en) * 2018-10-19 2020-04-28 上海商汤智能科技有限公司 Driving state analysis method and device, driver monitoring system and vehicle
CN111160126A (en) * 2019-12-11 2020-05-15 深圳市锐明技术股份有限公司 Driving state determination method and device, vehicle and storage medium
CN111476977A (en) * 2019-01-23 2020-07-31 上海博泰悦臻电子设备制造有限公司 Safe driving early warning system
CN111563456A (en) * 2020-05-07 2020-08-21 安徽江淮汽车集团股份有限公司 Driving behavior early warning method and system
CN114360210A (en) * 2021-12-13 2022-04-15 广西中卫北斗科技有限公司 Vehicle fatigue driving early warning system

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908152A (en) * 2010-06-11 2010-12-08 电子科技大学 Customization classifier-based eye state identification method
CN102610057A (en) * 2011-01-25 2012-07-25 深圳市高斯贝尔家居智能电子有限公司 Vehicle-mounted system and method for intelligently processing information
CN105069976A (en) * 2015-07-28 2015-11-18 南京工程学院 Integrated fatigue detection and driving record system and fatigue detection method
CN105096528A (en) * 2015-08-05 2015-11-25 广州云从信息科技有限公司 Fatigue driving detection method and system
CN105185038A (en) * 2015-10-20 2015-12-23 华东交通大学 Safety driving system based on Android smart phone
WO2017149045A1 (en) * 2016-03-01 2017-09-08 Valeo Comfort And Driving Assistance Personalized device and method for monitoring a motor vehicle driver
WO2017193272A1 (en) * 2016-05-10 2017-11-16 深圳市赛亿科技开发有限公司 Vehicle-mounted fatigue pre-warning system based on human face recognition and pre-warning method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101908152A (en) * 2010-06-11 2010-12-08 电子科技大学 Customization classifier-based eye state identification method
CN102610057A (en) * 2011-01-25 2012-07-25 深圳市高斯贝尔家居智能电子有限公司 Vehicle-mounted system and method for intelligently processing information
CN105069976A (en) * 2015-07-28 2015-11-18 南京工程学院 Integrated fatigue detection and driving record system and fatigue detection method
CN105096528A (en) * 2015-08-05 2015-11-25 广州云从信息科技有限公司 Fatigue driving detection method and system
CN105185038A (en) * 2015-10-20 2015-12-23 华东交通大学 Safety driving system based on Android smart phone
WO2017149045A1 (en) * 2016-03-01 2017-09-08 Valeo Comfort And Driving Assistance Personalized device and method for monitoring a motor vehicle driver
WO2017193272A1 (en) * 2016-05-10 2017-11-16 深圳市赛亿科技开发有限公司 Vehicle-mounted fatigue pre-warning system based on human face recognition and pre-warning method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
徐翠: "基于计算机视觉的汽车安全辅助驾驶若干关键问题研究", 《中国博士学位论文全文数据库信息科技辑》 *
范薇: "基于人脸识别的疲劳驾驶检测***的设计与实现", 《中国优秀硕士学位论文全文数据库工程科技Ⅱ辑》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446600A (en) * 2018-02-27 2018-08-24 上海汽车集团股份有限公司 A kind of vehicle driver's fatigue monitoring early warning system and method
CN109460780A (en) * 2018-10-17 2019-03-12 深兰科技(上海)有限公司 Safe driving of vehicle detection method, device and the storage medium of artificial neural network
CN111079476A (en) * 2018-10-19 2020-04-28 上海商汤智能科技有限公司 Driving state analysis method and device, driver monitoring system and vehicle
CN111079476B (en) * 2018-10-19 2024-03-26 上海商汤智能科技有限公司 Driving state analysis method and device, driver monitoring system and vehicle
CN111476977A (en) * 2019-01-23 2020-07-31 上海博泰悦臻电子设备制造有限公司 Safe driving early warning system
CN111160126A (en) * 2019-12-11 2020-05-15 深圳市锐明技术股份有限公司 Driving state determination method and device, vehicle and storage medium
CN111160126B (en) * 2019-12-11 2023-12-19 深圳市锐明技术股份有限公司 Driving state determining method, driving state determining device, vehicle and storage medium
CN111563456A (en) * 2020-05-07 2020-08-21 安徽江淮汽车集团股份有限公司 Driving behavior early warning method and system
CN114360210A (en) * 2021-12-13 2022-04-15 广西中卫北斗科技有限公司 Vehicle fatigue driving early warning system

Similar Documents

Publication Publication Date Title
CN108021875A (en) A kind of vehicle driver's personalization fatigue monitoring and method for early warning
CN108446600A (en) A kind of vehicle driver's fatigue monitoring early warning system and method
CN104183091B (en) System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode
CN105336105B (en) A kind of method of preventing fatigue driving, smart machine and system
Wang et al. Driver fatigue detection: a survey
CN101639894B (en) Method for detecting train driver behavior and fatigue state on line and detection system thereof
CN102310771B (en) Motor vehicle safety control method and system based on driver face identification
CN107972671A (en) A kind of driving behavior analysis system
CN107697069A (en) Fatigue of automobile driver driving intelligent control method
CN106571015A (en) Driving behavior data collection method based on Internet
US20100090839A1 (en) Driver management apparatus and travel management system
CN105719431A (en) Fatigue driving detection system
CN108230619A (en) Method for detecting fatigue driving based on multi-feature fusion
CN208498370U (en) Fatigue driving based on steering wheel detects prior-warning device
CN102098955A (en) Method and device for the detection of microsleep events
CN106080194A (en) The method for early warning of anti-fatigue-driving and system
CN111753674A (en) Fatigue driving detection and identification method based on deep learning
CN104068868A (en) Method and device for monitoring driver fatigue on basis of machine vision
CN111645694B (en) Driver driving state monitoring system and method based on attitude estimation
CN107563346A (en) One kind realizes that driver fatigue sentences method for distinguishing based on eye image processing
CN105931430A (en) Alarm sensitivity detection method and apparatus for driver state early warning system
CN109543577A (en) A kind of fatigue driving detection method for early warning based on facial expression feature
Charniya et al. Drunk driving and drowsiness detection
CN112528843A (en) Motor vehicle driver fatigue detection method fusing facial features
CN104299363A (en) Fatigue driving pre-warning system based on multi-feature fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180511

RJ01 Rejection of invention patent application after publication