CN112418081A - Method and system for air-ground joint rapid investigation of traffic accidents - Google Patents

Method and system for air-ground joint rapid investigation of traffic accidents Download PDF

Info

Publication number
CN112418081A
CN112418081A CN202011314262.7A CN202011314262A CN112418081A CN 112418081 A CN112418081 A CN 112418081A CN 202011314262 A CN202011314262 A CN 202011314262A CN 112418081 A CN112418081 A CN 112418081A
Authority
CN
China
Prior art keywords
road
module
traffic
image
accident
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011314262.7A
Other languages
Chinese (zh)
Other versions
CN112418081B (en
Inventor
何诚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Forest Police College
Original Assignee
Nanjing Forest Police College
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Forest Police College filed Critical Nanjing Forest Police College
Priority to CN202011314262.7A priority Critical patent/CN112418081B/en
Publication of CN112418081A publication Critical patent/CN112418081A/en
Application granted granted Critical
Publication of CN112418081B publication Critical patent/CN112418081B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/176—Urban or other man-made structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/10—Image enhancement or restoration using non-spatial domain filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
    • G08G1/096805—Systems involving transmission of navigation instructions to the vehicle where the transmitted instructions are used to compute a route
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A30/00—Adapting or protecting infrastructure or their operation
    • Y02A30/60—Planning or developing urban green infrastructure

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention belongs to the technical field of traffic guidance, and discloses a method and a system for air-ground combined rapid investigation of traffic accidents, wherein traffic accident occurrence alarm information is obtained, and the position of a traffic accident is determined according to the alarm information; the unmanned aerial vehicle flies to the scene according to the position information, takes a picture right above the traffic accident and sends the picture of the traffic accident to a command center; fixing known parameters according to the road width of a road or the length of a vehicle, and giving photo measurement information; setting and drawing a reference line on the photo picture; recognizing traffic accident vehicles and automatically generating the positions of the vehicles and the reference line; filling fixed vehicle traffic accident symbols on the photo graph; unmanned aerial vehicle carries the microphone to call the suggestion when taking and obtaining the photo information function. According to the invention, the rescue route is planned through the established model after the congestion interval is determined, the planned route is short, the time consumption is short, the rescue is more timely, and the rescue effect is better.

Description

Method and system for air-ground joint rapid investigation of traffic accidents
Technical Field
The invention belongs to the technical field of traffic guidance, and particularly relates to a method and a system for air-ground combined rapid investigation of traffic accidents.
Background
At present: the traffic accident refers to an event that the vehicle causes personal injury or property loss on the road due to mistake or accident. At present, a system for managing traffic accidents is a traffic management system. Traffic management systems are used in many areas to detect and respond to changing traffic conditions, and one key area in such traffic management systems is efficient accident management, which can help reduce delays caused by traffic accidents and improve the efficiency of traffic networks. Effective management of traffic accidents may include rapid removal of accident vehicles and other debris. However, the existing traffic management system has no scheme of combining the open space, so that the accurate investigation of traffic accidents cannot be realized, and accident rescue is influenced.
Through the above analysis, the problems and defects of the prior art are as follows: the existing traffic management system has no scheme of combining the open space for a while, so that the accurate investigation of traffic accidents cannot be realized, and accident rescue is influenced.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a system for air-ground joint rapid investigation of traffic accidents.
The invention is realized in such a way that a method for rapidly surveying traffic accidents by combining air and ground, which comprises the following steps:
acquiring alarm information of a traffic accident, and determining the position of the traffic accident according to the alarm information;
the unmanned aerial vehicle flies to the scene according to the position information, takes a picture right above the traffic accident and sends the picture of the traffic accident to a command center;
fixing known parameters according to the road width of a road or the length of a vehicle, and giving photo measurement information;
setting and drawing a reference line on the photo picture;
recognizing traffic accident vehicles by utilizing the function of automatically recognizing the vehicles, and automatically generating the positions of the vehicles and the reference line;
filling fixed vehicle traffic accident symbols including brake tire marks and blood trace marks on the photo graph;
the man-machine carries the microphone to call out the suggestion when taking and acquireing the photo information function.
The identifying the traffic accident vehicle by using the function of automatically identifying the vehicle and automatically generating the position of the vehicle and the reference line further comprises:
acquiring urban road planning information by using a planning information acquisition program through a planning information acquisition module; analyzing the acquired urban road planning information by using a planning information analysis program through a planning information analysis module to obtain a planning information analysis result; acquiring a road traffic map in the acquired urban road planning information by using a traffic model construction program through a traffic model construction module; the method comprises the steps that a traffic model building module is used for associating road trunks in a road traffic map by a traffic model building program to build a road network structure;
acquiring map information corresponding to a plurality of target intersections in a road traffic map; constructing a plurality of target road sections according to the map information corresponding to each target intersection; associating the traffic information of the target road section with the road network structure, and simultaneously constructing an urban traffic model by combining the planning information analysis result obtained in the step one; the method comprises the following steps of (1) acquiring urban road images by using an unmanned aerial vehicle carrying a camera through an image acquisition module; acquiring a normalized difference water body index NDWI and a soil brightness index SBI of the acquired urban road image by using an image processing program and a preset enhancement algorithm through an image analysis processing module;
step three, according to the difference value of the NDWI and the SBI, enhancing the road area in the urban road image to obtain an enhanced image; filtering the enhanced image to obtain a filtered image, and calculating the similarity between the filtered road area and other non-road areas by adopting a similarity calculation method; according to the similarity, separating a road area from a non-road area in the filtered image to obtain a separated image; extracting and acquiring a road area image from the separated image by adopting object-oriented feature extraction;
analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result; determining a crowded road section by a crowded road determining module according to an image analysis result by using a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section; positioning a crowded road section in the urban traffic model by using a positioning program through a positioning module;
step five, calculating the congestion degree of the section according to the length of the congested road section and the vehicle density by using a congestion degree calculation program through a congestion degree calculation module; acquiring the surrounding information of the crowded road section by using a surrounding information acquisition program through a surrounding information acquisition module; forming road combinations from any different roads in the urban road network by a traffic condition judging program according to the acquired peripheral information through a traffic condition judging module, and acquiring attribute values of different attributes of the road combinations;
evaluating the criticality score of each road in the urban road network, arranging the roads from high to low according to the criticality score, and marking the roads which account for 10 percent of the total number of the roads in the sequence as critical roads; acquiring a key road combination in an urban road network, and marking road sections contained in the key road combination as key road sections; selecting a key road section to perform structured analysis on the intersection image to obtain a structured analysis result; judging the traffic condition based on the structural analysis result and the congestion degree of the interval, and determining the traffic condition as common congestion or congestion caused by traffic accidents;
step seven, searching accident information reported by the vehicle owner by using a reported accident searching program through a reported accident searching module to obtain accident information of a crowded road section; the rescue route planning module is used for planning a rescue route by using a rescue route planning program in combination with the urban traffic model; and performing accident rescue by using an accident rescue program through an accident rescue module according to the determined rescue route.
Further, in the second step, the constructing the urban traffic model includes: determining the acquisition equipment of the road and the intersection corresponding to the target road section; and acquiring traffic information acquired by acquisition equipment of the road and the intersection corresponding to the target road section, and associating the traffic information with the road network structure to construct an urban traffic model.
Further, in the fourth step, analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result includes:
firstly, acquiring an acquired image and calculating the gray value of a pixel in the image; calculating the gray value gradient of the pixel according to the obtained gray value of the pixel, and obtaining a special point on the image;
secondly, segmenting the preprocessed road image by adopting a Canny edge detection algorithm based on a maximum inter-class variance method to obtain a relatively complete lane line edge image;
and finally, judging the road type by using an identification algorithm, and combining special points on the image to obtain an image analysis result.
Further, in step four, the determining the congested road section by the congested road determining module using a congested road determining program according to the image analysis result includes: and acquiring an image analysis result, calculating a graph and a graph parameter formed by the special points according to the special points on the acquired image, and outputting the graph and the graph parameter, wherein the graph area is a crowded road section.
Further, in the sixth step, the judging of the traffic condition based on the structured analysis result and the degree of congestion of the section, and the determining of the common congestion or the congestion caused by the traffic accident includes:
acquiring a road structural analysis result and the section crowding degree; comparing the road structural analysis results obtained by adjacent intersections with the difference value of the interval crowding degrees within a preset time interval, wherein the greater the difference value is, the higher the possibility of congestion caused by traffic accidents is; otherwise, normal congestion is obtained.
Further, in the seventh step, the planning of the rescue route by the rescue route planning module by using a rescue route planning program in combination with the urban traffic model further includes determining a key road segment, including:
(1) planning a rescue route based on the acquired urban traffic model;
(2) extracting a surface object point cloud set in the model;
(3) denoising the surface object point cloud set and then establishing a bounding box of the surface object;
(4) and establishing a Thiessen polygon map according to the bounding box, searching a path in the Thiessen polygon map by adopting a single-source shortest path algorithm, and acquiring an optimal path by a cubic spline interpolation algorithm.
Further, in step (2), the extracting the surface object point cloud set in the model includes: and traversing all the point pairs in the point pair set.
Another object of the present invention is to provide an air-ground combined rapid reconnaissance system for a traffic accident implementing a method for air-ground combined rapid reconnaissance of a traffic accident, the air-ground combined rapid reconnaissance system comprising:
the system comprises a planning information acquisition module, a planning information analysis module, a traffic model construction module, a central control module, an image acquisition module, an image analysis processing module, a crowded road determination module, a positioning module, a congestion degree calculation module, a peripheral information acquisition module, a traffic condition judgment module, a reported accident search module, a rescue route planning module and an accident rescue module;
the planning information acquisition module is connected with the central control module and is used for acquiring the urban road planning information through a planning information acquisition program;
the planning information analysis module is connected with the central control module and used for analyzing the acquired planning information through a planning information analysis program to obtain a planning information analysis result;
the traffic model building module is connected with the central control module and used for building an urban traffic model according to the planning information analysis result through a traffic model building program;
the central control module is connected with the planning information acquisition module, the planning information analysis module, the traffic model construction module, the image acquisition module, the image analysis processing module, the crowded road determination module, the positioning module, the congestion degree calculation module, the peripheral information acquisition module, the traffic condition judgment module, the reported accident search module, the rescue route planning module and the accident rescue module and is used for controlling the normal operation of each module through the main control computer;
the image acquisition module is connected with the central control module and is used for acquiring urban road images by the unmanned aerial vehicle carrying the camera;
the image analysis processing module is connected with the central control module and is used for preprocessing the acquired urban road image by using an image processing program and analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result;
the crowded road determining module is connected with the central control module and used for determining a crowded road section according to the image analysis result through a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section;
the positioning module is connected with the central control module and used for positioning the crowded road section in the urban traffic model through a positioning program;
the congestion degree calculating module is connected with the central control module and used for calculating the congestion degree of the section according to the length of the congested road section and the vehicle density through a congestion degree calculating program;
the peripheral information acquisition module is connected with the central control module and is used for acquiring the peripheral information of the crowded road section through a peripheral information acquisition program;
the traffic condition judging module is connected with the central control module and used for judging the traffic condition according to the acquired peripheral information and the congestion degree of the section through a traffic condition judging program and determining the traffic condition is common congestion or congestion caused by a traffic accident;
the reported accident searching module is connected with the central control module and used for searching accident information reported by a vehicle owner through a reported accident searching program to obtain accident information of a crowded road section;
the rescue route planning module is connected with the central control module and used for planning a rescue route by combining a rescue route planning program with an urban traffic model;
and the accident rescue module is connected with the central control module and is used for carrying out accident rescue according to the determined rescue route through an accident rescue program.
It is another object of the present invention to provide a computer program product stored on a computer readable medium, comprising a computer readable program for providing a user input interface to implement the method for air-ground joint rapid reconnaissance of traffic accidents when executed on an electronic device.
It is another object of the present invention to provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the air-ground joint rapid reconnaissance method for a traffic accident.
By combining all the technical schemes, the invention has the advantages and positive effects that: the invention realizes the construction of a traffic model by acquiring and analyzing the urban road planning information; the constructed model is used for displaying the distribution information of the road, so that the display effect is more visual and accurate; and after the congestion interval is determined, the rescue route is planned through the established model, the planned route is short, the time consumption is short, the rescue is more timely, and the rescue effect is better.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained from the drawings without creative efforts.
Fig. 1 is a flowchart of a method for automatically identifying a vehicle in a traffic accident and automatically generating a position of the vehicle and a reference line according to a function of automatically identifying the vehicle according to an embodiment of the present invention.
Fig. 2 is a block diagram of a system for air-ground joint rapid investigation of traffic accidents according to an embodiment of the present invention.
Fig. 3 is a flowchart of constructing an urban traffic model by a traffic model construction module according to a planning information analysis result by using a traffic model construction program according to an embodiment of the present invention.
Fig. 4 is a flowchart of a key road segment provided by an embodiment of the present invention.
Fig. 5 is a flowchart for planning a rescue route by a rescue route planning module using a rescue route planning program in combination with an urban traffic model according to an embodiment of the present invention.
In fig. 2: 1. a planning information acquisition module; 2. a planning information analysis module; 3. a traffic model construction module; 4. a central control module; 5. an image acquisition module; 6. an image analysis processing module; 7. a congested road determination module; 8. a positioning module; 9. a congestion degree calculation module; 10. a peripheral information acquisition module; 11. a traffic condition determination module; 12. reporting an accident searching module; 13. a rescue route planning module; 14. and an accident rescue module.
Fig. 6 is a first diagram illustrating an effect of an application example of the method for air-ground combined rapid investigation of traffic accidents according to the embodiment of the present invention.
Fig. 7 is a second diagram illustrating the effect of the application example of the method for air-ground combined rapid investigation of traffic accidents according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Aiming at the problems in the prior art, the invention provides a method and a system for air-ground combined rapid investigation of traffic accidents, and the invention is described in detail below with reference to the accompanying drawings.
Acquiring alarm information of a traffic accident, and determining the position of the traffic accident according to the alarm information;
the unmanned aerial vehicle flies to the scene according to the position information, takes a picture right above the traffic accident and sends the picture of the traffic accident to a command center;
fixing known parameters according to the road width of a road or the length of a vehicle, and giving photo measurement information;
setting and drawing a reference line on the photo picture;
recognizing traffic accident vehicles by utilizing the function of automatically recognizing the vehicles, and automatically generating the positions of the vehicles and the reference line;
filling fixed vehicle traffic accident symbols including brake tire marks and blood trace marks on the photo graph;
the man-machine carries the microphone to call out the suggestion when taking and acquireing the photo information function.
The identifying the traffic accident vehicle by using the function of automatically identifying the vehicle and automatically generating the position of the vehicle and the reference line further comprises:
as shown in fig. 1, the embodiment of the present invention provides a method for identifying a vehicle in a traffic accident and automatically generating the positions of the vehicle and a reference line by using a function of automatically identifying the vehicle, including:
the method for quickly surveying the traffic accident by combining the air and the ground comprises the following steps:
s101, acquiring urban road planning information by using a planning information acquisition program through a planning information acquisition module; analyzing the acquired planning information by using a planning information analysis program through a planning information analysis module to obtain a planning information analysis result; constructing an urban traffic model by using a traffic model construction program through a traffic model construction module according to the planning information analysis result;
s102, acquiring urban road images by using an unmanned aerial vehicle carrying a camera through an image acquisition module; preprocessing the acquired urban road image by using an image processing program through an image analysis processing module, and analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result;
s103, determining a crowded road section by a crowded road determining module according to an image analysis result by using a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section;
s104, positioning the crowded road section in the urban traffic model by using a positioning program through a positioning module; calculating the congestion degree of the section according to the length of the congested road section and the vehicle density by using a congestion degree calculation program through a congestion degree calculation module;
s105, acquiring the peripheral information of the crowded road section by utilizing a peripheral information acquisition program through a peripheral information acquisition module; judging the traffic condition according to the acquired peripheral information and the congestion degree of the section by a traffic condition judging module by utilizing a traffic condition judging program, and determining the traffic condition as common congestion or congestion caused by a traffic accident;
s106, searching accident information reported by the vehicle owner through a reported accident searching module by utilizing a reported accident searching program to obtain accident information of a crowded road section;
s107, a rescue route planning module plans a rescue route by using a rescue route planning program in combination with the urban traffic model; and performing accident rescue by using an accident rescue program through an accident rescue module according to the determined rescue route.
As shown in fig. 2, the system for air-ground joint rapid investigation of traffic accidents provided by the embodiment of the invention comprises:
the system comprises a planning information acquisition module 1, a planning information analysis module 2, a traffic model construction module 3, a central control module 4, an image acquisition module 5, an image analysis processing module 6, a crowded road determination module 7, a positioning module 8, a congestion degree calculation module 9, a peripheral information acquisition module 10, a traffic condition judgment module 11, a reported accident search module 12, a rescue route planning module 13 and an accident rescue module 14;
the planning information acquisition module 1 is connected with the central control module 4 and is used for acquiring urban road planning information through a planning information acquisition program;
the planning information analysis module 2 is connected with the central control module 4 and is used for analyzing the acquired planning information through a planning information analysis program to obtain a planning information analysis result;
the traffic model building module 3 is connected with the central control module 4 and used for building an urban traffic model according to the planning information analysis result through a traffic model building program;
the central control module 4 is connected with the planning information acquisition module 1, the planning information analysis module 2, the traffic model construction module 3, the image acquisition module 5, the image analysis processing module 6, the crowded road determination module 7, the positioning module 8, the congestion degree calculation module 9, the peripheral information acquisition module 10, the traffic condition judgment module 11, the reported accident search module 12, the rescue route planning module 13 and the accident rescue module 14, and is used for controlling each module to normally operate through a main control computer;
the image acquisition module 5 is connected with the central control module 4 and is used for acquiring urban road images by carrying a camera by the unmanned aerial vehicle;
the image analysis processing module 6 is connected with the central control module 4 and is used for preprocessing the acquired urban road image through an image processing program and analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result;
a crowded road determining module 7 connected to the central control module 4 for determining a crowded road section according to the image analysis result by a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section;
the positioning module 8 is connected with the central control module 4 and used for positioning the crowded road section in the urban traffic model through a positioning program;
a congestion degree calculation module 9 connected to the central control module 4 for calculating a congestion degree of a congested road section according to a length of the section and a vehicle density;
a peripheral information acquisition module 10 connected to the central control module 4 for acquiring peripheral information of the congested road section by a peripheral information acquisition program;
a traffic condition determination module 11, connected to the central control module 4, for determining the traffic condition according to the acquired peripheral information and the degree of congestion in the section by a traffic condition determination program, and determining that the traffic condition is common congestion or congestion caused by a traffic accident;
the reported accident searching module 12 is connected with the central control module 4 and used for searching accident information reported by the vehicle owner through a reported accident searching program to obtain accident information of a crowded road section;
the rescue route planning module 13 is connected with the central control module 4 and used for planning a rescue route by combining a rescue route planning program with an urban traffic model;
and the accident rescue module 14 is connected with the central control module 4 and is used for carrying out accident rescue according to the determined rescue route through an accident rescue program.
The technical solution of the present invention is further illustrated by the following specific examples.
Example 1
The method for air-ground joint rapid investigation of traffic accidents provided by the embodiment of the invention is shown in fig. 1, and as a preferred embodiment, as shown in fig. 3, the method for constructing an urban traffic model by using a traffic model construction program through a traffic model construction module according to a planning information analysis result, which is provided by the embodiment of the invention, comprises the following steps:
s201, acquiring a road traffic map in the acquired urban road planning information;
s202, associating road trunks in the road traffic map to construct a road network structure;
s203, obtaining map information corresponding to a plurality of target intersections in the road traffic map;
s204, constructing a plurality of target road sections according to the map information corresponding to each target intersection;
and S205, associating the traffic information of the target road section with the road network structure to construct an urban traffic model.
In step S205, the constructing an urban traffic model according to the embodiment of the present invention includes: determining the acquisition equipment of the road and the intersection corresponding to the target road section; and acquiring traffic information acquired by acquisition equipment of the road and the intersection corresponding to the target road section, and associating the traffic information with the road network structure to construct an urban traffic model.
Example 2:
as shown in fig. 1, the method for air-ground joint rapid investigation of traffic accidents according to the embodiment of the present invention, as a preferred embodiment, the preprocessing of the acquired urban road image by the image processing program through the image analysis processing module according to the embodiment of the present invention includes:
acquiring a normalized difference water body index NDWI and a soil brightness index SBI of the acquired urban road image by using an image processing program and a preset enhancement algorithm; according to the difference value of the NDWI and the SBI, enhancing the road area in the urban road image to obtain an enhanced image; filtering the enhanced image to obtain a filtered image, and calculating the similarity between the filtered road area and other non-road areas by adopting a similarity calculation method; according to the similarity, separating a road area from a non-road area in the filtered image to obtain a separated image; and extracting and acquiring a road area image from the separated image by adopting object-oriented feature extraction.
The image analysis processing module provided by the embodiment of the invention utilizes an image analysis program to analyze the preprocessed urban road image, and the obtained image analysis result comprises the following steps:
acquiring an acquired image and calculating the gray value of pixels in the image; calculating the gray value gradient of the pixel according to the obtained gray value of the pixel, and obtaining a special point on the image; segmenting the preprocessed road image by adopting a Canny edge detection algorithm based on a maximum inter-class variance method to obtain a relatively complete lane line edge image; and judging the road type by using an identification algorithm, and obtaining an image analysis result by combining special points on the image.
Example 3
As shown in fig. 1, the method for air-ground joint rapid investigation of traffic accidents according to the embodiment of the present invention is implemented by a congested road determination module, which determines a congested road section according to an image analysis result by using a congested road determination program according to a preferred embodiment of the present invention, and includes: and acquiring an image analysis result, calculating a graph and a graph parameter formed by the special points according to the special points on the acquired image, and outputting the graph and the graph parameter, wherein the graph area is a crowded road section.
Example 4
As shown in fig. 1, the method for air-ground joint rapid reconnaissance of a traffic accident according to an embodiment of the present invention is a preferred embodiment, where a traffic condition determination module determines a traffic condition according to acquired peripheral information and a degree of congestion in an interval by using a traffic condition determination program, and determining that the traffic condition is a common congestion or a congestion caused by a traffic accident includes:
carrying out image structural analysis on the real-time image of the intersection to obtain a road structural analysis result and the section crowding degree; comparing the road structural analysis results obtained by adjacent intersections with the difference value of the interval crowding degrees within a preset time interval, wherein the greater the difference value is, the higher the possibility of congestion caused by traffic accidents is; otherwise, normal congestion is obtained.
As shown in fig. 4, before performing image structural analysis on a real-time image of an intersection, the method further includes:
s301, forming a road combination by any different roads in an urban road network, and acquiring attribute values of different attributes of the road combination;
s302, evaluating the criticality score of each road in the urban road network, arranging the roads from high to low according to the criticality score, and marking the roads which account for 10 percent of the total number of the roads in the sequence as critical roads;
s303, acquiring a key road combination in the urban road network, and marking the road sections contained in the key road combination as key road sections;
and S304, selecting a key road section to perform structured analysis on the road junction image.
Example 5
The method for air-ground joint rapid investigation of traffic accidents provided by the embodiment of the invention is shown in fig. 1, as a preferred embodiment, as shown in fig. 5, the method for planning the rescue route by using a rescue route planning program through a rescue route planning module and combining with an urban traffic model provided by the embodiment of the invention further comprises determining a key road section, and comprises the following steps:
s401, planning a rescue route based on the acquired urban traffic model;
s402, extracting a surface object point cloud set in the model;
s403, denoising the surface object point cloud set and then establishing a bounding box of the surface object;
s404, establishing a Thiessen polygon map according to the bounding box, searching a path in the Thiessen polygon map by adopting a single-source shortest path algorithm, and obtaining an optimal path through a cubic spline interpolation algorithm.
In step S402, the extracting a surface object point cloud set in the model provided by the embodiment of the present invention includes: and traversing all the point pairs in the point pair set.
Example 6
The invention provides a method for quickly surveying traffic accidents by combining air and ground, which comprises the following steps:
after a traffic accident occurs, alarming by someone, and determining the position of the traffic accident according to the alarming information;
the unmanned aerial vehicle flies to the scene according to the position information, takes a picture right above the traffic accident and sends the picture of the traffic accident to a command center;
the method for processing the traffic accident comprises the steps of fixing known parameters according to the road width of a road or the length of a vehicle and the like, and giving photo measurement information;
setting up and drawing a reference line on the graph;
developing the function of automatically identifying vehicles, identifying vehicles in traffic accidents, and automatically generating the positions of the vehicles and the reference line;
the map can be filled with some commonly used fixed vehicle traffic accident symbols, such as brake tyre marks, blood trace marks and the like;
unmanned aerial vehicle carries the microphone function of shouting when shooting and obtaining photo information function, lets the vehicle drive away.
Fig. 6 is a first diagram illustrating an effect of an application example of the method for air-ground combined rapid investigation of traffic accidents according to the embodiment of the present invention.
Fig. 7 is a second diagram illustrating the effect of the application example of the method for air-ground combined rapid investigation of traffic accidents according to the embodiment of the present invention.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, and any modification, equivalent replacement, and improvement made by those skilled in the art within the technical scope of the present invention disclosed herein, which is within the spirit and principle of the present invention, should be covered by the present invention.

Claims (10)

1. A method for air-ground combined rapid traffic accident reconnaissance, comprising the following steps:
acquiring alarm information of a traffic accident, and determining the position of the traffic accident according to the alarm information;
the unmanned aerial vehicle flies to the scene according to the position information, takes a picture right above the traffic accident and sends the picture of the traffic accident to a command center;
fixing known parameters according to the road width of a road or the length of a vehicle, and giving photo measurement information;
setting and drawing a reference line on the photo picture;
recognizing traffic accident vehicles by utilizing the function of automatically recognizing the vehicles, and automatically generating the positions of the vehicles and the reference line;
filling fixed vehicle traffic accident symbols including brake tire marks and blood trace marks on the photo graph;
unmanned aerial vehicle carries the microphone to call the suggestion when taking and obtaining the photo information function.
2. The air-ground combined rapid traffic accident reconnaissance method of claim 1, wherein the automatic vehicle recognition function is used to identify a traffic accident vehicle and automatically generate the positions of the vehicle and the reference line; the method comprises the following steps:
acquiring urban road planning information by using a planning information acquisition program; analyzing the acquired urban road planning information by using a planning information analysis program through a planning information analysis module to obtain a planning information analysis result; acquiring a road traffic map in the acquired urban road planning information by using a traffic model construction program through a traffic model construction module; the method comprises the steps that a traffic model building module is used for associating road trunks in a road traffic map by a traffic model building program to build a road network structure;
acquiring map information corresponding to a plurality of target intersections in a road traffic map; constructing a plurality of target road sections according to the map information corresponding to each target intersection; associating the traffic information of the target road section with the road network structure, and simultaneously constructing an urban traffic model by combining the planning information analysis result obtained in the step one; the method comprises the following steps of (1) acquiring urban road images by using an unmanned aerial vehicle carrying a camera through an image acquisition module; acquiring a normalized difference water body index NDWI and a soil brightness index SBI of the acquired urban road image by using an image processing program and a preset enhancement algorithm through an image analysis processing module;
step three, according to the difference value of the NDWI and the SBI, enhancing the road area in the urban road image to obtain an enhanced image; filtering the enhanced image to obtain a filtered image, and calculating the similarity between the filtered road area and other non-road areas by adopting a similarity calculation method; according to the similarity, separating a road area from a non-road area in the filtered image to obtain a separated image; extracting and acquiring a road area image from the separated image by adopting object-oriented feature extraction;
analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result; determining a crowded road section by a crowded road determining module according to an image analysis result by using a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section; positioning a crowded road section in the urban traffic model by using a positioning program through a positioning module;
step five, calculating the congestion degree of the section according to the length of the congested road section and the vehicle density by using a congestion degree calculation program through a congestion degree calculation module; acquiring the surrounding information of the crowded road section by using a surrounding information acquisition program through a surrounding information acquisition module; forming road combinations from any different roads in the urban road network by a traffic condition judging program according to the acquired peripheral information through a traffic condition judging module, and acquiring attribute values of different attributes of the road combinations;
evaluating the criticality score of each road in the urban road network, arranging the roads from high to low according to the criticality score, and marking the roads which account for 10 percent of the total number of the roads in the sequence as critical roads; acquiring a key road combination in an urban road network, and marking road sections contained in the key road combination as key road sections; selecting a key road section to perform structured analysis on the intersection image to obtain a structured analysis result; judging the traffic condition based on the structural analysis result and the congestion degree of the interval, and determining the traffic condition as common congestion or congestion caused by traffic accidents;
step seven, searching accident information reported by the vehicle owner by using a reported accident searching program through a reported accident searching module to obtain accident information of a crowded road section; the rescue route planning module is used for planning a rescue route by using a rescue route planning program in combination with the urban traffic model; and performing accident rescue by using an accident rescue program through an accident rescue module according to the determined rescue route.
3. The air-ground combined rapid traffic accident reconnaissance method of claim 2, wherein in the second step, the constructing of the urban traffic model comprises: determining the acquisition equipment of the road and the intersection corresponding to the target road section; acquiring traffic information acquired by acquisition equipment of a road and an intersection corresponding to the target road section, associating the traffic information with the road network structure, and constructing an urban traffic model;
in the fourth step, the analyzing the preprocessed urban road image by using the image analysis program to obtain an image analysis result comprises:
firstly, acquiring an acquired image and calculating the gray value of a pixel in the image; calculating the gray value gradient of the pixel according to the obtained gray value of the pixel, and obtaining a special point on the image;
secondly, segmenting the preprocessed road image by adopting a Canny edge detection algorithm based on a maximum inter-class variance method to obtain a relatively complete lane line edge image;
and finally, judging the road type by using an identification algorithm, and combining special points on the image to obtain an image analysis result.
4. The air-ground combined rapid traffic accident investigation method according to claim 2, wherein in step four, the determination of the congested road section by the congested road determination module using a congested road determination procedure according to the image analysis result comprises: and acquiring an image analysis result, calculating a graph and a graph parameter formed by the special points according to the special points on the acquired image, and outputting the graph and the graph parameter, wherein the graph area is a crowded road section.
5. The air-ground combined rapid traffic accident reconnaissance method of claim 2, wherein in step six, the determining of the traffic condition based on the structural analysis result and the congestion degree of the section comprises:
acquiring a road structural analysis result and the section crowding degree; comparing the road structural analysis results obtained by adjacent intersections with the difference value of the interval crowding degrees within a preset time interval, wherein the greater the difference value is, the higher the possibility of congestion caused by traffic accidents is; otherwise, normal congestion is obtained.
6. The air-ground combined rapid traffic accident reconnaissance method of claim 2, wherein in step seven, the rescue route planning module is used for planning the rescue route by using a rescue route planning program in combination with the urban traffic model, and the method further comprises determining a key road segment, comprising:
(1) planning a rescue route based on the acquired urban traffic model;
(2) extracting a surface object point cloud set in the model;
(3) denoising the surface object point cloud set and then establishing a bounding box of the surface object;
(4) and establishing a Thiessen polygon map according to the bounding box, searching a path in the Thiessen polygon map by adopting a single-source shortest path algorithm, and acquiring an optimal path by a cubic spline interpolation algorithm.
7. The air-ground joint rapid traffic accident reconnaissance method of claim 6, wherein in the step (2), the extracting the surface object point cloud set in the model comprises: and traversing all the point pairs in the point pair set.
8. A space-land combined rapid traffic accident reconnaissance system for implementing the method of claims 1-7, comprising:
the system comprises a planning information acquisition module, a planning information analysis module, a traffic model construction module, a central control module, an image acquisition module, an image analysis processing module, a crowded road determination module, a positioning module, a congestion degree calculation module, a peripheral information acquisition module, a traffic condition judgment module, a reported accident search module, a rescue route planning module and an accident rescue module;
the planning information acquisition module is connected with the central control module and is used for acquiring the urban road planning information through a planning information acquisition program;
the planning information analysis module is connected with the central control module and used for analyzing the acquired planning information through a planning information analysis program to obtain a planning information analysis result;
the traffic model building module is connected with the central control module and used for building an urban traffic model according to the planning information analysis result through a traffic model building program;
the central control module is connected with the planning information acquisition module, the planning information analysis module, the traffic model construction module, the image acquisition module, the image analysis processing module, the crowded road determination module, the positioning module, the congestion degree calculation module, the peripheral information acquisition module, the traffic condition judgment module, the reported accident search module, the rescue route planning module and the accident rescue module and is used for controlling the normal operation of each module through the main control computer;
the image acquisition module is connected with the central control module and is used for acquiring urban road images by the unmanned aerial vehicle carrying the camera;
the image analysis processing module is connected with the central control module and is used for preprocessing the acquired urban road image by using an image processing program and analyzing the preprocessed urban road image by using an image analysis program to obtain an image analysis result;
the crowded road determining module is connected with the central control module and used for determining a crowded road section according to the image analysis result through a crowded road determining program to obtain the length of the crowded road section and the vehicle density information of the section;
the positioning module is connected with the central control module and used for positioning the crowded road section in the urban traffic model through a positioning program;
the congestion degree calculating module is connected with the central control module and used for calculating the congestion degree of the section according to the length of the congested road section and the vehicle density through a congestion degree calculating program;
the peripheral information acquisition module is connected with the central control module and is used for acquiring the peripheral information of the crowded road section through a peripheral information acquisition program;
the traffic condition judging module is connected with the central control module and used for judging the traffic condition according to the acquired peripheral information and the congestion degree of the section through a traffic condition judging program and determining the traffic condition is common congestion or congestion caused by a traffic accident;
the reported accident searching module is connected with the central control module and used for searching accident information reported by a vehicle owner through a reported accident searching program to obtain accident information of a crowded road section;
the rescue route planning module is connected with the central control module and used for planning a rescue route by combining a rescue route planning program with an urban traffic model;
and the accident rescue module is connected with the central control module and is used for carrying out accident rescue according to the determined rescue route through an accident rescue program.
9. A computer program product stored on a computer readable medium, comprising a computer readable program for providing a user input interface for implementing a method for combined air and ground rapid reconnaissance of a traffic accident as claimed in any one of claims 1 to 7 when executed on an electronic device.
10. A computer-readable storage medium storing instructions which, when executed on a computer, cause the computer to perform a method of combined air and ground rapid reconnaissance of a traffic accident according to any one of claims 1 to 7.
CN202011314262.7A 2020-11-20 2020-11-20 Method and system for quickly surveying traffic accidents by air-ground combination Active CN112418081B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011314262.7A CN112418081B (en) 2020-11-20 2020-11-20 Method and system for quickly surveying traffic accidents by air-ground combination

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011314262.7A CN112418081B (en) 2020-11-20 2020-11-20 Method and system for quickly surveying traffic accidents by air-ground combination

Publications (2)

Publication Number Publication Date
CN112418081A true CN112418081A (en) 2021-02-26
CN112418081B CN112418081B (en) 2023-09-29

Family

ID=74777801

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011314262.7A Active CN112418081B (en) 2020-11-20 2020-11-20 Method and system for quickly surveying traffic accidents by air-ground combination

Country Status (1)

Country Link
CN (1) CN112418081B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113706932A (en) * 2021-10-29 2021-11-26 天津云圣智能科技有限责任公司 Road section safety warning method and device and unmanned aerial vehicle
CN116110228A (en) * 2023-04-06 2023-05-12 齐鲁云商数字科技股份有限公司 Urban traffic rapid guiding system based on block chain

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104332053A (en) * 2014-11-13 2015-02-04 深圳大学 Road traffic inspection system and method based on small unmanned aerial vehicle
CN106781593A (en) * 2017-03-06 2017-05-31 辉泰（太仓）汽配有限公司 A kind of intelligent transportation system based on intelligent signal lamp
CN108389354A (en) * 2018-04-16 2018-08-10 南京森林警察学院 A kind of method of unmanned plane joint ground micro robot detection forest ground fire
CN109035117A (en) * 2018-09-01 2018-12-18 李善伯 A kind of automatically road traffic network system realization
US20180376305A1 (en) * 2017-06-23 2018-12-27 Veniam, Inc. Methods and systems for detecting anomalies and forecasting optimizations to improve smart city or region infrastructure management using networks of autonomous vehicles
US20190051046A1 (en) * 2017-08-11 2019-02-14 Jing Jin Incident Site Investigation and Management Support System Based on Unmanned Aerial Vehicles
CN109993970A (en) * 2019-03-15 2019-07-09 西南交通大学 A kind of city area-traffic accident risk prediction method
CN110119839A (en) * 2019-04-24 2019-08-13 华南理工大学 A kind of Urban Road Traffic Accidents emergency management and rescue paths planning method
CN111553539A (en) * 2020-05-09 2020-08-18 上海大学 Driving path planning method based on probabilistic model inspection
CN111815989A (en) * 2020-06-19 2020-10-23 勇鸿(重庆)信息科技有限公司 C-V2X technology-based road accident rescue method and system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104332053A (en) * 2014-11-13 2015-02-04 深圳大学 Road traffic inspection system and method based on small unmanned aerial vehicle
CN106781593A (en) * 2017-03-06 2017-05-31 辉泰（太仓）汽配有限公司 A kind of intelligent transportation system based on intelligent signal lamp
US20180376305A1 (en) * 2017-06-23 2018-12-27 Veniam, Inc. Methods and systems for detecting anomalies and forecasting optimizations to improve smart city or region infrastructure management using networks of autonomous vehicles
US20190051046A1 (en) * 2017-08-11 2019-02-14 Jing Jin Incident Site Investigation and Management Support System Based on Unmanned Aerial Vehicles
CN108389354A (en) * 2018-04-16 2018-08-10 南京森林警察学院 A kind of method of unmanned plane joint ground micro robot detection forest ground fire
CN109035117A (en) * 2018-09-01 2018-12-18 李善伯 A kind of automatically road traffic network system realization
CN109993970A (en) * 2019-03-15 2019-07-09 西南交通大学 A kind of city area-traffic accident risk prediction method
CN110119839A (en) * 2019-04-24 2019-08-13 华南理工大学 A kind of Urban Road Traffic Accidents emergency management and rescue paths planning method
CN111553539A (en) * 2020-05-09 2020-08-18 上海大学 Driving path planning method based on probabilistic model inspection
CN111815989A (en) * 2020-06-19 2020-10-23 勇鸿(重庆)信息科技有限公司 C-V2X technology-based road accident rescue method and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HUIYING WEN等: "Co-Evolutionary Optimization Algorithm Based on the Future Traffic Environment for Emergency Rescue Path Planning", 《IEEE》, pages 148125 - 148135 *
王俊: "面向智能交通的路径规划相关技术研究", 《中国优秀硕士论文电子期刊网 工程科技Ⅱ辑》, pages 034 - 78 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113706932A (en) * 2021-10-29 2021-11-26 天津云圣智能科技有限责任公司 Road section safety warning method and device and unmanned aerial vehicle
CN116110228A (en) * 2023-04-06 2023-05-12 齐鲁云商数字科技股份有限公司 Urban traffic rapid guiding system based on block chain
CN116110228B (en) * 2023-04-06 2023-07-07 齐鲁云商数字科技股份有限公司 Urban traffic rapid guiding system based on block chain

Also Published As

Publication number Publication date
CN112418081B (en) 2023-09-29

Similar Documents

Publication Publication Date Title
US10846874B2 (en) Method and apparatus for processing point cloud data and storage medium
KR102266830B1 (en) Lane determination method, device and storage medium
CN108345822B (en) Point cloud data processing method and device
US10133941B2 (en) Method, apparatus and device for detecting lane boundary
DE102012207620B4 (en) System and method for light signal detection
CN111818313B (en) Vehicle real-time tracking method and device based on monitoring video
CN110458050B (en) Vehicle cut-in detection method and device based on vehicle-mounted video
CN107909012B (en) Real-time vehicle tracking detection method and device based on disparity map
CN109903574B (en) Method and device for acquiring intersection traffic information
CN111295666A (en) Lane line detection method, device, control equipment and storage medium
CN113421432A (en) Traffic restriction information detection method and device, electronic equipment and storage medium
CN112418081B (en) Method and system for quickly surveying traffic accidents by air-ground combination
CN113887418A (en) Method and device for detecting illegal driving of vehicle, electronic equipment and storage medium
CN110610153A (en) Lane recognition method and system for automatic driving
CN113989784A (en) Road scene type identification method and system based on vehicle-mounted laser point cloud
CN114037966A (en) High-precision map feature extraction method, device, medium and electronic equipment
CN112633722A (en) Vehicle-mounted road safety risk assessment system and method
CN111931683A (en) Image recognition method, image recognition device and computer-readable storage medium
CN114299457A (en) Accumulated water depth detection method and device
CN114926791A (en) Method and device for detecting abnormal lane change of vehicles at intersection, storage medium and electronic equipment
US20240078815A1 (en) Device and method for recognizing obstacles for a vehicle
CN117011413B (en) Road image reconstruction method, device, computer equipment and storage medium
CN114998863B (en) Target road identification method, device, electronic equipment and storage medium
CN116206326A (en) Training method of missing detection model, missing detection method and device of diversion area
CN115973190A (en) Decision-making method and device for automatically driving vehicle and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant