CN114862961B - Position detection method and device for calibration plate, electronic equipment and readable storage medium - Google Patents

Position detection method and device for calibration plate, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN114862961B
CN114862961B CN202210393201.7A CN202210393201A CN114862961B CN 114862961 B CN114862961 B CN 114862961B CN 202210393201 A CN202210393201 A CN 202210393201A CN 114862961 B CN114862961 B CN 114862961B
Authority
CN
China
Prior art keywords
calibration plate
point cloud
cloud data
frame
plate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210393201.7A
Other languages
Chinese (zh)
Other versions
CN114862961A (en
Inventor
李怡康
闫国行
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai AI Innovation Center
Original Assignee
Shanghai AI Innovation Center
Filing date
Publication date
Application filed by Shanghai AI Innovation Center filed Critical Shanghai AI Innovation Center
Priority to CN202210393201.7A priority Critical patent/CN114862961B/en
Publication of CN114862961A publication Critical patent/CN114862961A/en
Application granted granted Critical
Publication of CN114862961B publication Critical patent/CN114862961B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

The application discloses a position detection method and device of a calibration plate, electronic equipment and a readable storage medium, and belongs to the technical field of sensing. The method comprises the following steps: acquiring target point cloud data, wherein the target point cloud data are obtained by detecting a calibration plate through a radar to be calibrated; detecting the frame position of the calibration plate from the target point cloud data; and detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the appointed position relation, so as to finish the position detection of the calibration plate, wherein the appointed position relation is the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate. According to the embodiment of the application, the frame position of the calibration plate is detected from the target point cloud data, and the position of the circular hole is determined according to the position relation between the frame position and the circle center of the circular hole in the calibration plate, so that the position detection of the calibration plate can be completed without using the wire harness id of the radar, and the applicability of the position detection is improved.

Description

Position detection method and device for calibration plate, electronic equipment and readable storage medium
Technical Field
The present application relates to the field of sensing technologies, and in particular, to a method and an apparatus for detecting a position of a calibration board, an electronic device, and a readable storage medium.
Background
With the development of radar technology, the application range of various radars is wider and wider, for example, the radars can be applied to the field of automobiles. In the automotive field, vehicles can generally perform environmental sensing through radars such as laser radars and millimeter wave radars to identify, detect and track static and dynamic objects, so as to improve the intelligentization and the driving safety of the vehicles. In order to ensure the accuracy of the radar to the environmental perception, the internal and external parameters of the radar are usually required to be calibrated to determine the conversion relation between the coordinate system of the laser transmitter inside the radar and the coordinate system of the radar and the conversion relation between the coordinate system of the radar and the coordinate system of the vehicle body. In the radar calibration process, a calibration plate is usually arranged, a plurality of circular holes are usually arranged on the calibration plate, and the radar realizes pose calibration by detecting the positions of the plurality of circular holes. That is, in order to calibrate the radar, it is generally necessary to detect the position of the calibration plate.
At present, the radar can transmit a plurality of radar wire harnesses to the position where the calibration plate is located, each radar wire harness is provided with a corresponding ring id (the id of the wire harness), therefore, different radar wire harnesses can be distinguished according to the ring ids, the distance detected by each radar wire harness is determined, the edge point of a circular hole is extracted through the distance difference detected by the plurality of radar wire harnesses, then a circle is fitted through the edge point, and the circle center of the circle is calculated, and therefore the position detection of the calibration plate is achieved.
However, some radars do not have ring id, for example, solid radars do not have ring id, so that detection of the position of the calibration plate cannot be performed in the above manner, and thus calibration of the radar cannot be performed.
Disclosure of Invention
The embodiment of the application provides a position detection method and device of a calibration plate, electronic equipment and a readable storage medium, which can solve the problem that in the related art, some radars cannot perform position detection of the calibration plate, so that the radars cannot perform calibration. The technical scheme is as follows:
in a first aspect, there is provided a position detection method of a calibration plate, the method comprising:
Acquiring target point cloud data, wherein the target point cloud data are obtained by detecting a calibration plate through a radar to be calibrated;
detecting the frame position of the calibration plate from the target point cloud data;
And detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the appointed position relation, so as to finish the position detection of the calibration plate, wherein the appointed position relation is the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate.
As an example of the present application, the detecting the frame position of the calibration board from the target point cloud data includes:
extracting plane point cloud data corresponding to the calibration plate from the target point cloud data;
And detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
As an example of the present application, the detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate includes:
drawing a reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate, wherein the frame size of the reference frame is the same as that of the calibration plate, and the shape of the reference frame is the same as that of the calibration plate;
Moving the reference frame in the planar point cloud data;
detecting the quantity of point cloud data falling into the reference frame in the process of moving the reference frame;
and after the movement of the reference frame is finished, determining the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
As an example of the present application, the detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate includes:
Performing point cloud edge detection in the plane point cloud data based on the shape information of the calibration plate to obtain an edge detection result;
performing line segment fitting according to the edge detection result to obtain a plurality of line segments;
Based on the frame size of the calibration plate, the size of at least one line segment in the plurality of line segments is adjusted, so that the shape of a polygon formed by the adjusted line segments is the same as the shape of the calibration plate, and the size of the polygon is the same as the frame size of the calibration plate;
acquiring the position information of the polygon;
and determining the position information of the polygon as the frame position of the calibration plate.
As an example of the present application, the number of circular holes in the calibration plate is a plurality; based on the frame position of the calibration plate and the specified position relation, detecting the position of the circular hole in the calibration plate comprises the following steps:
Obtaining a mask plate corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask plate are the same as those of a plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask plate is the same as that among the plurality of circular holes in the calibration plate;
Determining an initial movement position of the mask plate in the frame range of the calibration plate, wherein the initial movement position refers to a position at which the circle center of each circular hole in the mask plate overlaps with the circle center of each corresponding circular hole in the calibration plate;
Starting from the initial moving position, moving the mask plate in the frame range of the calibration plate;
detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate;
And after the movement is finished, determining the positions of the plurality of circular holes in the mask plate at a target position as the positions of the plurality of circular holes in the calibration plate, wherein the target position is the position of the mask plate when the total number of point cloud data falling into the plurality of circular holes of the mask plate is minimum.
As an example of the present application, the detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the specified positional relationship includes:
Determining the initial position of the center of a circle of a circular hole in the calibration plate according to the frame position of the calibration plate and the appointed position relation;
acquiring size information of a circular hole in the calibration plate;
Performing curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center to obtain a fitting circle;
and determining the position of the fitting circle as the position of the circular hole in the calibration plate.
In a second aspect, there is provided a position detection device for a calibration plate, the device comprising:
the acquisition module is used for acquiring target point cloud data, wherein the target point cloud data are obtained by detecting the calibration plate through a radar to be calibrated;
the first detection module is used for detecting the frame position of the calibration plate from the target point cloud data;
The second detection module is used for detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the appointed position relation, so as to finish the position detection of the calibration plate, wherein the appointed position relation is the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate.
As an example of the present application, the first detection module includes:
the extraction sub-module is used for extracting plane point cloud data corresponding to the calibration plate from the target point cloud data;
and the first detection submodule is used for detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
As an example of the present application, the first detection submodule is configured to:
drawing a reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate, wherein the frame size of the reference frame is the same as that of the calibration plate, and the shape of the reference frame is the same as that of the calibration plate;
Moving the reference frame in the planar point cloud data;
detecting the quantity of point cloud data falling into the reference frame in the process of moving the reference frame;
and after the movement of the reference frame is finished, determining the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
As an example of the present application, the first detection submodule is configured to:
Performing point cloud edge detection in the plane point cloud data based on the shape information of the calibration plate to obtain an edge detection result;
performing line segment fitting according to the edge detection result to obtain a plurality of line segments;
Based on the frame size of the calibration plate, the size of at least one line segment in the plurality of line segments is adjusted, so that the shape of a polygon formed by the adjusted line segments is the same as the shape of the calibration plate, and the size of the polygon is the same as the frame size of the calibration plate;
acquiring the position information of the polygon;
and determining the position information of the polygon as the frame position of the calibration plate.
As an example of the present application, the number of circular holes in the calibration plate is a plurality; the second detection module includes:
The first acquisition submodule is used for acquiring a mask plate corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask plate are the same as those of a plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask plate is the same as that among the plurality of circular holes in the calibration plate;
The first determining submodule is used for determining an initial moving position of the mask plate in the frame range of the calibration plate, wherein the initial moving position refers to a position where the circle center of each circular hole in the mask plate is overlapped with the circle center of each corresponding circular hole in the calibration plate;
The moving submodule is used for moving the mask plate in the frame range of the calibration plate from the initial moving position;
The second detection sub-module is used for detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate;
And the second determining submodule is used for determining the positions of the plurality of circular holes in the mask plate at the target position as the positions of the plurality of circular holes in the calibration plate after the movement is finished, wherein the target position is the position of the mask plate when the total number of point cloud data falling into the plurality of circular holes of the mask plate is minimum.
As an example of the present application, the second detection module includes:
The third determining submodule is used for determining the circle center initial position of the circular hole in the calibration plate according to the frame position of the calibration plate and the appointed position relation;
the second acquisition sub-module is used for acquiring the size information of the circular hole in the calibration plate;
the fitting sub-module is used for performing curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center to obtain a fitting circle;
and the fourth determination submodule is used for determining the position of the fitting circle as the position of the circular hole in the calibration plate.
In a third aspect, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the method for position detection of a calibration plate according to the first aspect described above when executing the computer program.
In a fourth aspect, a computer readable storage medium is provided, on which instructions are stored, which instructions, when executed by a processor, implement the method for position detection of a calibration plate according to the first aspect.
In a fifth aspect, there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method for position detection of a calibration plate as described in the first aspect above.
It will be appreciated that the advantages of the second to fifth aspects may be found in the relevant description of the first aspect, and are not described here again.
The technical scheme provided by the embodiment of the application has the beneficial effects that:
According to the embodiment of the application, the frame position of the calibration plate can be detected from the cloud data of the target point, and then the position of the circular hole of the calibration plate can be determined through the position relation between the frame position and the circle center of the circular hole in the calibration plate, so that the position detection of the calibration plate can be completed without using the wire harness id of the radar, and the applicability of the position detection is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart illustrating a method for detecting the position of a calibration plate according to an exemplary embodiment.
FIG. 2 is a schematic diagram illustrating drawing of a reference frame in planar point cloud data of a calibration plate, according to an example embodiment.
FIG. 3 is a schematic diagram of a reticle corresponding to a calibration plate and the reticle in an initial movement position according to an exemplary embodiment.
Fig. 4 is a schematic structural view of a position detecting device of a calibration plate according to an exemplary embodiment.
Fig. 5 is a schematic diagram of an electronic device according to an exemplary embodiment.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.
It should be understood that references to "a plurality" in this disclosure refer to two or more. In the description of the present application, "/" means or, unless otherwise indicated, for example, A/B may represent A or B; "and/or" herein is merely an association relationship describing an association object, and means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of the present application, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and function. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "in some other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more, but not all, embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
Before explaining a position detection method of a calibration plate provided by the embodiment of the application in detail, an application scenario provided by the embodiment of the application is explained.
With the development of automobile technology, the functions of vehicles are increasing, and the vehicles can realize various intelligent driving operations through ADAS (ADVANCED DRIVER ASSISTANCE SYSTEM, intelligent driving assistance system), for example, the vehicles can realize blind zone detection, automatic emergency braking, pedestrian collision early warning, lane departure early warning and the like through the ADAS. The ADAS performs the above-described intelligent driving operation by sensing the environment using a sensor such as a radar mounted on the vehicle body. In order to ensure the accuracy of environmental awareness, before the vehicle is put into use, the sensor such as a radar is usually required to be calibrated with internal parameters and external parameters, wherein the internal parameter calibration of the radar refers to determining the conversion relation between the coordinate system of a laser transmitter in the radar and the coordinate system of the radar. The external parameter calibration of the radar refers to determining the conversion relation between the self coordinate system of the laser radar and the vehicle body coordinate system.
At present, the production line calibration is usually carried out on the radar in batches on an automobile production line, the production line calibration usually depends on a calibration plate, when the calibration plate is a calibration plate provided with round holes, the radar can send radar wiring harnesses to the calibration plate, the positions of the round holes in the calibration plate can be detected through ring ids (wiring harness ids) of the radar wiring harnesses, and then the positions of the round holes in the calibration plate can be utilized to calibrate the pose of the radar. And the positions of the circular holes in the calibration plate cannot be detected in the mode because some radars do not have ring ids. In addition, because edge point detection is performed through ring id, and a circle extracted through the edge point is not necessarily a standard circle, sometimes may be an ellipse, thus the subsequent radar calibration is inaccurate. And after the radar emits the ray bundle, the reflected data of the corresponding ray bundle may be received, and the corresponding reflected data may not be received, so that the reflected data received by the radar has sparsity, and the electronic equipment is easy to extract an incorrect circle center at a sparse place, so that the position detection of the calibration plate is inaccurate, and the subsequent calibration of the radar is inaccurate.
Based on the application scene, the embodiment of the application provides a position detection method of a calibration plate, which can improve the applicability and the accuracy of the position detection of the calibration plate.
Fig. 1 is a flowchart of a method for detecting the position of a calibration plate according to an exemplary embodiment, where the method for detecting the position of the calibration plate is applied to an electronic device for illustration, the method for detecting the position of the calibration plate may include the following steps:
step 101: the electronic equipment acquires target point cloud data, wherein the target point cloud data is obtained by detecting a calibration plate through a radar to be calibrated.
In one embodiment, after the radar detects the calibration board, the detected point cloud data is generally sent to the electronic device, so that the electronic device can obtain the target point cloud data according to the received point cloud data.
In order to increase the density of the point cloud data, the radar usually detects for multiple times when detecting the calibration plate to obtain multi-frame point cloud data, and the multi-frame point cloud data are overlapped and then sent to the electronic equipment, so that the electronic equipment can acquire the target point cloud data according to the received point cloud data.
Because the range of coverage of radar is wider, for example, the radius of range of coverage of radar is 150 meters, therefore, when the radar detects the calibration plate, the radar can also obtain the point cloud data of other objects except the calibration plate, and the point cloud data can possibly increase the complexity and accuracy of the position detection of the calibration plate. Therefore, in order to reduce the operation complexity and improve the accuracy of position detection, after receiving the point cloud data obtained by radar detection, the electronic device can preprocess the obtained point cloud data to obtain target point cloud data.
In one embodiment, after the electronic device obtains the point cloud data in the radar look-around coverage area, the electronic device may filter the received point cloud data through the ROI (region of interest, the region of interest) to obtain the target point cloud data.
The ROI is set in advance according to the requirement.
Since the position of the calibration plate is generally fixed and the position of the calibration plate relative to the radar is also relatively fixed in the automobile production line, the ROI may be set according to the position of the calibration plate relative to the radar, the installation error of the calibration plate, the installation error of the radar, and the redundant area.
It should be noted that, the operation of setting the ROI by the electronic device according to the position of the calibration board relative to the radar, the installation error of the calibration board, the installation error of the radar, the redundant area, and the like may refer to the related art, which is not described in detail in the embodiment of the present application.
Step 102: and the electronic equipment detects the frame position of the calibration plate from the cloud data of the target point.
In order to be able to accurately determine the position of the calibration plate, the electronic device may detect the frame position of the calibration plate.
As an example, the electronic device detects the frame position of the calibration board from the target point cloud data at least includes: extracting plane point cloud data corresponding to the calibration plate from the target point cloud data; and detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
Since the point cloud data is usually three-dimensional data, the target point cloud data is also three-dimensional data, and the position detection is performed through plane data, so that the electronic device can extract plane point cloud data corresponding to the calibration plate from the target point cloud data.
In one embodiment, the electronic device may extract the plane point cloud data corresponding to the calibration board from the target point cloud data using a RANSAC (Random Sample Consensus ) algorithm, an SVD (Singular Value Decomposition, singular value decomposition) algorithm, or the like.
It should be noted that, the operation of extracting the planar point cloud data corresponding to the calibration board from the target point cloud data by the electronic device using the RANSAC algorithm or the SVD algorithm may refer to the related art. For example, in the process of extracting plane point cloud data corresponding to the calibration plate from the target point cloud data by adopting a RANSAC algorithm, the electronic device can randomly select three point cloud data from the target point cloud data, and perform plane fitting through the three selected point cloud data, so as to obtain a fitting plane; then calculating the distance between other point cloud data in the target point cloud data and the fitting plane, if the distance between any point cloud data and the fitting plane is smaller than a first distance threshold value, determining that the point cloud data is positioned in the fitting plane, otherwise, determining that the point cloud data is not a point in the fitting plane; and after determining the distances between all the rest point cloud data and the fitting plane, repeating the steps, and determining the fitting plane with the largest included point cloud data as the plane where the plane point cloud data of the calibration plate are located.
Because the angle of the calibration plate relative to the radar on the production line is fixed, the electronic equipment can be preset with a normal direction, the electronic equipment can acquire a fitting plane with the most point cloud data from fitting planes perpendicular to the set normal direction, and the acquired fitting plane is determined to be the plane where the plane point cloud data of the calibration plate are located.
And because the calibration plate and the radar have certain installation errors, the electronic equipment can be preset with an angle threshold value, the electronic equipment can acquire a fitting platform with the greatest point cloud data from a fitting plane with an angle between the electronic equipment and the normal direction within the angle threshold value, and the acquired fitting plane is determined to be the plane where the plane point cloud data of the calibration plate are located.
It should be noted that, the first distance threshold value and the angle threshold value may be set in advance according to requirements, for example, the angle threshold value may be 3 degrees, 5 degrees, or the like. The first distance threshold is related to the calibration plate size.
In one embodiment, the electronic device detecting the position of the border of the calibration plate in the planar point cloud data based on the border size and shape information of the calibration plate comprises: drawing a reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate, wherein the frame size of the reference frame is the same as that of the calibration plate, and the shape of the reference frame is the same as that of the calibration plate; moving a reference frame in the planar point cloud data; detecting the quantity of point cloud data falling into the reference frame in the process of moving the reference frame; and after the movement of the reference frame is finished, determining the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
Because the frame size of the calibration plate is fixed, and the shape of the calibration plate is also fixed, the electronic equipment can draw the reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate. The frame size of the reference frame is the same as the frame size of the calibration plate, and the shape of the reference frame is the same as the shape of the calibration plate, so that the electronic device can determine the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
In one implementation environment, if the frame shape of the calibration board is rectangular, the electronic device may draw, according to the frame size and shape information of the calibration board, a reference frame as shown in fig. 2 in the planar point cloud data of the calibration board, where the reference frame is shown in a thickened form in fig. 2.
As an example, in the process of moving the reference frame, the electronic device may move the reference frame in the plane point cloud data of the calibration board at random, or may move the reference frame according to a first movement rule, and stop moving after the number of movements reaches a first time threshold, or stop moving after the movement duration reaches a first time threshold.
It should be noted that, the first time number threshold and the first time length threshold are set in advance according to the requirement, for example, the first time number threshold may be 200 times, 100 times, and so on, and the first time length threshold may be 5 seconds, 10 seconds, and so on.
In an embodiment, the first movement rule is set in advance according to the requirement, for example, the first movement rule includes moving the reference frame along the horizontal direction and/or the vertical direction, and moving the reference frame by a second distance threshold each time, rotating the reference frame by a first angle after moving the reference frame by a first distance along the horizontal direction or moving the reference frame by a second distance along the vertical direction, and continuing to move the reference frame along the horizontal direction and/or the vertical direction after rotating.
Because the mounting position of the calibration plate and the mounting position of the radar may have errors, the reference frame can be moved in the horizontal direction and the vertical direction in the process of moving the reference frame, and the reference frame can be rotated for a certain angle.
It should be noted that, the second distance threshold may be dynamically changed, for example, the second distance threshold may be a distance between two adjacent point cloud data, that is, in a process of moving the reference frame, the electronic device may move according to a distance between two adjacent point cloud data in a moving direction by using any vertex or center point of the reference frame as a moving reference. For example, when the position of the center point of the reference frame coincides with the position of the first reference point cloud data, after the reference frame is horizontally moved according to the second distance threshold, the position of the center point of the reference frame coincides with the position of the second reference point cloud data, and the first reference point cloud data and the second reference point cloud data are adjacent in the horizontal direction.
As an example, in the process of moving the reference frame, the electronic device may detect the amount of the point cloud data falling into the reference frame in a traversal manner, or may detect the amount of the point cloud data in other manners, which is not limited in particular by the embodiment of the present application.
In one embodiment, the manner in which the electronic device detects the frame position of the calibration plate in the planar point cloud data based on the frame size and shape information of the calibration plate may include not only the above manner but also other manners. For example, the electronic device may perform point cloud edge detection in the planar point cloud data based on the shape information of the calibration board, to obtain an edge detection result; performing line segment fitting according to the edge detection result to obtain a plurality of line segments; based on the frame size of the calibration plate, the size of at least one line segment in the plurality of line segments is adjusted, so that the shape of a polygon formed by the adjusted line segments is the same as the shape of the calibration plate, and the size of the polygon is the same as the frame size of the calibration plate; acquiring position information of the polygon; and determining the position information of the polygon as the frame position of the calibration plate.
Because the plane point cloud data corresponding to the calibration plate has a clear outline under the normal condition, the electronic equipment performs edge detection based on the shape information of the calibration plate, and after performing line segment fitting according to the edge detection result, a plurality of line segments can be obtained, the shape of a polygon formed by the plurality of line segments is matched with the shape of the calibration plate, and the number of the line segments is the same as the number of frames of the calibration plate.
The shape of the polygon formed by the plurality of line segments matches the shape of the calibration plate, which means that the shape of the polygon formed by the plurality of line segments is the same as or similar to the shape of the calibration plate.
Because the point cloud data has sparsity, after edge detection and line segment fitting operations are sequentially performed in the planar point cloud data, the lengths of the line segments obtained by fitting may be different from the frame size of the calibration plate, and therefore, the electronic device needs to adjust the size of at least one line segment in the line segments based on the frame size of the calibration plate.
As an example, the electronic device may compare the size of the target line segment with the size of the frame of the calibration board corresponding to the target line segment, and adjust the size of the target line segment to the size of the frame of the corresponding calibration board when the size of the target line segment is different from the size of the corresponding frame, where the target line segment is any one line segment of the plurality of line segments.
Step 103: the electronic equipment detects the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the appointed position relation so as to finish the position detection of the calibration plate.
The specified positional relationship refers to a positional relationship between the frame of the index plate and the center of the circular hole in the index plate.
Because the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate is fixed, the electronic equipment can detect the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the appointed position relation under the condition of determining the frame position of the calibration plate.
Because the number of the circular holes in the calibration plate can be one or a plurality of, and in most cases, the number of the circular holes in the calibration plate is a plurality of, at this time, the electronic device detects the positions of the circular holes in the calibration plate based on the frame position and the specified position relation of the calibration plate, and the operations of detecting the positions of the circular holes in the calibration plate include: obtaining a mask corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask are the same as those of the plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask is the same as that among the plurality of circular holes in the calibration plate; determining an initial moving position of the mask plate in the frame range of the calibration plate, wherein the initial moving position refers to a position where the circle center of each circular hole in the mask plate is overlapped with the circle center of each corresponding circular hole in the calibration plate; starting from the initial moving position, moving the mask plate in the frame range of the calibration plate; detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate; after the movement is finished, determining the positions of the plurality of circular holes in the mask plate at the target position as the positions of the plurality of circular holes in the calibration plate, wherein the target position is the position of the mask plate when the total number of point cloud data falling into the plurality of circular holes of the mask plate is minimum.
Under the condition that the number of the circular holes in the calibration plate is multiple, the position relation among the circular holes of the calibration plate is fixed, and the sizes of the circular holes of the calibration plate are fixed, so that the electronic equipment can set and store the mask corresponding to the calibration plate in advance according to the sizes of the circular holes in the calibration plate and the position relation among the circular holes, and acquire the mask from the storage space under the condition that the mask is required to be used. Of course, the electronic device may set the corresponding mask according to the sizes of the plurality of circular holes in the calibration plate and the positional relationship between the plurality of circular holes under the condition that the mask is required to be used. The embodiment of the present application is not particularly limited thereto.
It should be noted that the mask may be a mask describing a positional relationship between a plurality of circular holes and a size of the plurality of circular holes, or may be a mask describing a positional relationship between a plurality of circular holes, a size of the plurality of circular holes, and a positional relationship between each circular hole of the plurality of circular holes and a frame. That is, the shape of the outer contour of the mask plate and the size of the mask plate are not particularly limited, and the mask plate can be the mask plate which has the same shape as the calibration plate and the same frame size, and can also be the mask plate which has different shapes and/or different frame sizes.
In an implementation environment, in the case that the calibration plate includes 4 circular holes, a schematic diagram of the mask plate corresponding to the calibration plate and the mask plate at an initial moving position may be shown in fig. 3.
In the process of detecting the calibration plate, due to the existence of the circular holes in the calibration plate, after the radar harness of the radar passes through the circular holes of the calibration plate, the radar may not acquire reflection data at the positions of the circular holes, namely, almost no point cloud data is acquired at the positions of the circular holes of the calibration plate, so that under the condition that the total number of the point cloud data falling into the plurality of circular holes of the mask plate is minimum, the positions of the plurality of circular holes of the mask plate are the positions of the plurality of circular holes closest to the positions of the plurality of circular holes of the calibration plate, and the electronic equipment can determine the positions of the plurality of circular holes of the mask plate under the condition as the positions of the plurality of circular holes in the calibration plate.
As an example, in the process of moving the mask, the electronic device may detect the total number of point cloud data falling into a plurality of circular holes of the mask in a traversal manner, or may detect the total number of point cloud data in other manners, which is not limited in particular by the embodiment of the present application.
In one embodiment, the electronic device may detect the position of the circular hole in the calibration plate not only by detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and the specified positional relationship in the above manner, but also by detecting the position of the circular hole in the calibration plate in other manners. For example, the electronic device can determine the initial position of the center of the circle of the circular hole in the calibration plate according to the frame position of the calibration plate and the designated position relationship; acquiring size information of a circular hole in the calibration plate; performing curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center to obtain a fitting circle; and determining the position of the fitting circle as the position of the circular hole in the calibration plate.
The size information of the circular hole includes at least one of a radius, a diameter, an area, a perimeter, and the like of the circular hole.
Because the size information of the circular hole in the calibration plate is fixed, under the condition of determining the initial position of the circle center, the electronic equipment can perform curve fitting within the frame range of the calibration plate by taking the initial position of the circle center as the circle center of the fitting circle and taking the radius of the circular hole of the calibration plate as the radius of the fitting circle to obtain the fitting circle. Of course, the electronic device may also perform curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center in other manners, so as to obtain a fitting circle.
In the embodiment of the application, the electronic equipment can detect the frame position of the calibration plate from the cloud data of the target point, and then the position of the circular hole of the calibration plate can be determined through the frame position and the position relation between the frame position and the circle center of the circular hole in the calibration plate, so that the position detection of the calibration plate can be completed without using the wire harness id of the radar, and the applicability of the position detection is improved. Because the position detection is carried out according to the geometric structure of the circular hole, the influence of radar noise and the sparsity of the point cloud data on the position detection is reduced, and the accuracy of the position detection of the circular hole is improved.
It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and internal logic, and should not be construed as limiting the implementation process of the embodiments of the present application.
Fig. 4 is a schematic structural diagram of a position detecting device of a calibration plate according to an exemplary embodiment, which may be implemented by software, hardware or a combination of both. The position detection device of the calibration plate may include: an acquisition module 401, a first detection module 402 and a second detection module 403.
The acquisition module 401 is configured to acquire target point cloud data, where the target point cloud data is obtained by detecting the calibration board through a radar to be calibrated;
the first detection module 402 is configured to detect a frame position of the calibration plate from the target point cloud data;
the second detection module 403 is configured to detect a position of a circular hole in the calibration plate based on a frame position of the calibration plate and a specified position relationship, where the specified position relationship is a position relationship between a frame of the calibration plate and a center of the circular hole in the calibration plate, so as to complete position detection of the calibration plate.
As an example of the present application, the first detection module includes:
The extraction sub-module is used for extracting plane point cloud data corresponding to the calibration plate from the target point cloud data;
The first detection submodule is used for detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
As an example of the application, the first detection submodule is configured to:
Drawing a reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate, wherein the frame size of the reference frame is the same as that of the calibration plate, and the shape of the reference frame is the same as that of the calibration plate;
moving the reference frame in the planar point cloud data;
Detecting the quantity of point cloud data falling into the reference frame in the process of moving the reference frame;
And after the movement of the reference frame is finished, determining the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
As an example of the application, the first detection submodule is configured to:
performing point cloud edge detection in the plane point cloud data based on the shape information of the calibration plate to obtain an edge detection result;
Performing line segment fitting according to the edge detection result to obtain a plurality of line segments;
Based on the frame size of the calibration plate, the size of at least one line segment in the plurality of line segments is adjusted, so that the shape of a polygon formed by the adjusted line segments is the same as the shape of the calibration plate, and the size of the polygon is the same as the frame size of the calibration plate;
Acquiring position information of the polygon;
And determining the position information of the polygon as the frame position of the calibration plate.
As an example of the present application, the number of circular holes in the calibration plate is plural; the second detection module includes:
the first acquisition submodule is used for acquiring a mask plate corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask plate are the same as those of a plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask plate is the same as that among the plurality of circular holes in the calibration plate;
The first determining submodule is used for determining an initial moving position of the mask plate in the frame range of the calibration plate, wherein the initial moving position refers to a position where the circle center of each circular hole in the mask plate is overlapped with the circle center of each corresponding circular hole in the calibration plate;
The moving submodule is used for moving the mask plate in the frame range of the calibration plate from the initial moving position;
The second detection submodule is used for detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate;
And the second determining submodule is used for determining the positions of the plurality of circular holes in the mask plate at the target position as the positions of the plurality of circular holes in the calibration plate after the movement is finished, wherein the target position is the position of the mask plate when the total number of the point cloud data falling into the plurality of circular holes of the mask plate is minimum.
As an example of the present application, the second detection module includes:
The third determining submodule is used for determining the circle center initial position of the circular hole in the calibration plate according to the frame position of the calibration plate and the appointed position relation;
the second acquisition sub-module is used for acquiring the size information of the circular hole in the calibration plate;
the fitting sub-module is used for performing curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center to obtain a fitting circle;
and the fourth determination submodule is used for determining the position of the fitting circle as the position of the circular hole in the calibration plate.
In the embodiment of the application, the electronic equipment can detect the frame position of the calibration plate from the cloud data of the target point, and then the position of the circular hole of the calibration plate can be determined through the frame position and the position relation between the frame position and the circle center of the circular hole in the calibration plate, so that the position detection of the calibration plate can be completed without using the wire harness id of the radar, and the applicability of the position detection is improved. Because the position detection is carried out according to the geometric structure of the circular hole, the influence of radar noise and the sparsity of the point cloud data on the position detection is reduced, and the accuracy of the position detection of the circular hole is improved.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 5, the electronic apparatus 5 of this embodiment includes: at least one processor 50 (only one is shown in fig. 5), a memory 51 and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, the processor 50 implementing the steps in any of the various method embodiments described above when executing the computer program 52.
The electronic device 5 may be a control device in an automobile, a desktop computer, a notebook computer, a palm computer, a cloud server, and other computing devices. The electronic device may include, but is not limited to, a processor 50, a memory 51. It will be appreciated by those skilled in the art that fig. 5 is merely an example of the electronic device 5 and is not meant to be limiting of the electronic device 5, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The Processor 50 may be a CPU (Central Processing Unit ), the Processor 50 may also be other general purpose processors, DSP (DIGITAL SIGNAL Processor), ASIC (Application SPECIFIC INTEGRATED Circuit), FPGA (Field-Programmable GATE ARRAY) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 51 may in one embodiment be an internal storage unit of the electronic device 5, such as a hard disk or a memory of the electronic device 5. The memory 51 may in other embodiments also be an external storage device of the electronic device 5, such as a plug-in hard disk, SMC (SMART MEDIA CARD, smart card), SD (Secure Digital) card, flash memory card (FLASH CARD) or the like, which is provided on the electronic device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the electronic device 5. The memory 51 is used for storing an operating system, application programs, boot loader (BootLoader), data, other programs, etc., such as program codes of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.
It should be noted that, because the content of information interaction and execution process between the above devices/units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims (9)

1. A method for detecting the position of a calibration plate, the method comprising:
Acquiring target point cloud data, wherein the target point cloud data are obtained by detecting a calibration plate through a radar to be calibrated;
detecting the frame position of the calibration plate from the target point cloud data;
detecting the position of a circular hole in the calibration plate based on the frame position of the calibration plate and a specified position relation, so as to finish the position detection of the calibration plate, wherein the specified position relation is the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate;
Under the condition that the number of the circular holes in the calibration plate is a plurality of, detecting the positions of the circular holes in the calibration plate based on the frame positions of the calibration plate and the appointed position relation comprises the following steps:
Obtaining a mask plate corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask plate are the same as those of a plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask plate is the same as that among the plurality of circular holes in the calibration plate;
Determining an initial movement position of the mask plate in the frame range of the calibration plate, wherein the initial movement position refers to a position at which the circle center of each circular hole in the mask plate overlaps with the circle center of each corresponding circular hole in the calibration plate;
Starting from the initial moving position, moving the mask plate in the frame range of the calibration plate;
detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate;
And after the movement is finished, determining the positions of the plurality of circular holes in the mask plate at a target position as the positions of the plurality of circular holes in the calibration plate, wherein the target position is the position of the mask plate when the total number of point cloud data falling into the plurality of circular holes of the mask plate is minimum.
2. The method of claim 1, wherein detecting the bezel position of the calibration plate from the target point cloud data comprises:
extracting plane point cloud data corresponding to the calibration plate from the target point cloud data;
And detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
3. The method of claim 2, wherein detecting the bezel position of the calibration plate in the planar point cloud data based on the bezel size and shape information of the calibration plate comprises:
drawing a reference frame corresponding to the calibration plate according to the frame size and shape information of the calibration plate, wherein the frame size of the reference frame is the same as that of the calibration plate, and the shape of the reference frame is the same as that of the calibration plate;
Moving the reference frame in the planar point cloud data;
detecting the quantity of point cloud data falling into the reference frame in the process of moving the reference frame;
and after the movement of the reference frame is finished, determining the position of the reference frame with the most point cloud data as the frame position of the calibration plate.
4. The method of claim 2, wherein detecting the bezel position of the calibration plate in the planar point cloud data based on the bezel size and shape information of the calibration plate comprises:
Performing point cloud edge detection in the plane point cloud data based on the shape information of the calibration plate to obtain an edge detection result;
performing line segment fitting according to the edge detection result to obtain a plurality of line segments;
Based on the frame size of the calibration plate, the size of at least one line segment in the plurality of line segments is adjusted, so that the shape of a polygon formed by the adjusted line segments is the same as the shape of the calibration plate, and the size of the polygon is the same as the frame size of the calibration plate;
acquiring the position information of the polygon;
and determining the position information of the polygon as the frame position of the calibration plate.
5. The method of any one of claims 1-4, wherein the method further comprises:
Determining the initial position of the center of a circle of a circular hole in the calibration plate according to the frame position of the calibration plate and the appointed position relation;
acquiring size information of a circular hole in the calibration plate;
Performing curve fitting in the frame range of the calibration plate based on the size information of the circular hole in the calibration plate and the initial position of the circle center to obtain a fitting circle;
and determining the position of the fitting circle as the position of the circular hole in the calibration plate.
6. A position detection device for a calibration plate, the device comprising:
the acquisition module is used for acquiring target point cloud data, wherein the target point cloud data are obtained by detecting the calibration plate through a radar to be calibrated;
the first detection module is used for detecting the frame position of the calibration plate from the target point cloud data;
The second detection module is used for detecting the position of the circular hole in the calibration plate based on the frame position of the calibration plate and a specified position relation, so as to finish the position detection of the calibration plate, wherein the specified position relation is the position relation between the frame of the calibration plate and the circle center of the circular hole in the calibration plate;
in the case that the number of the circular holes in the calibration plate is plural, the second detection module includes:
The first acquisition submodule is used for acquiring a mask plate corresponding to the calibration plate, wherein the sizes of a plurality of circular holes in the mask plate are the same as those of a plurality of circular holes in the calibration plate, and the position relationship among the plurality of circular holes in the mask plate is the same as that among the plurality of circular holes in the calibration plate;
The first determining submodule is used for determining an initial moving position of the mask plate in the frame range of the calibration plate, wherein the initial moving position refers to a position where the circle center of each circular hole in the mask plate is overlapped with the circle center of each corresponding circular hole in the calibration plate;
The moving submodule is used for moving the mask plate in the frame range of the calibration plate from the initial moving position;
The second detection sub-module is used for detecting the total quantity of point cloud data falling into a plurality of circular holes of the mask plate in the process of moving the mask plate;
And the second determining submodule is used for determining the positions of the plurality of circular holes in the mask plate at the target position as the positions of the plurality of circular holes in the calibration plate after the movement is finished, wherein the target position is the position of the mask plate when the total number of point cloud data falling into the plurality of circular holes of the mask plate is minimum.
7. The apparatus of claim 6, wherein the first detection module comprises:
the extraction sub-module is used for extracting plane point cloud data corresponding to the calibration plate from the target point cloud data;
and the first detection submodule is used for detecting the frame position of the calibration plate in the plane point cloud data based on the frame size and shape information of the calibration plate.
8. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 5 when executing the computer program.
9. A computer readable storage medium having instructions stored thereon, which when executed by a processor, implement the steps of the method of any of claims 1 to 5.
CN202210393201.7A 2022-04-13 Position detection method and device for calibration plate, electronic equipment and readable storage medium Active CN114862961B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210393201.7A CN114862961B (en) 2022-04-13 Position detection method and device for calibration plate, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210393201.7A CN114862961B (en) 2022-04-13 Position detection method and device for calibration plate, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN114862961A CN114862961A (en) 2022-08-05
CN114862961B true CN114862961B (en) 2024-06-07

Family

ID=

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111965624A (en) * 2020-08-06 2020-11-20 北京百度网讯科技有限公司 Calibration method, device and equipment for laser radar and camera and readable storage medium
WO2021115071A1 (en) * 2019-12-12 2021-06-17 中国科学院深圳先进技术研究院 Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device
CN113138375A (en) * 2021-04-27 2021-07-20 北京理工大学 Combined calibration method, system and calibration plate
CN113506350A (en) * 2021-07-23 2021-10-15 北京轻舟智航科技有限公司 Camera external parameter calibration method for automatic driving

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021115071A1 (en) * 2019-12-12 2021-06-17 中国科学院深圳先进技术研究院 Three-dimensional reconstruction method and apparatus for monocular endoscope image, and terminal device
CN111965624A (en) * 2020-08-06 2020-11-20 北京百度网讯科技有限公司 Calibration method, device and equipment for laser radar and camera and readable storage medium
CN113138375A (en) * 2021-04-27 2021-07-20 北京理工大学 Combined calibration method, system and calibration plate
CN113506350A (en) * 2021-07-23 2021-10-15 北京轻舟智航科技有限公司 Camera external parameter calibration method for automatic driving

Similar Documents

Publication Publication Date Title
EP3650884B1 (en) Method and apparatus for determining relative pose, device and medium
US10267907B2 (en) Radar apparatus and radar state estimation method
EP3879307B1 (en) Method and apparatus for detecting obstacle
US9794519B2 (en) Positioning apparatus and positioning method regarding a position of mobile object
CN110867132B (en) Environment sensing method, device, electronic equipment and computer readable storage medium
US11561295B2 (en) Position detecting method, device and storage medium for vehicle ladar
CN109859260B (en) Method and device for determining parking position and computer readable storage medium
CN109828250B (en) Radar calibration method, calibration device and terminal equipment
CN109884639B (en) Obstacle detection method and device for mobile robot
US20200309899A1 (en) Antenna array design and processing to eliminate false detections in a radar system
EP4119977A1 (en) Method and apparatus for calibrating a vehicle-mounted lidar, vehicle and storage medium
CN106605153B (en) Shaft offset diagnostic device
CN108693517B (en) Vehicle positioning method and device and radar
CN112781893B (en) Spatial synchronization method and device for vehicle-mounted sensor performance test data and storage medium
CN114862961B (en) Position detection method and device for calibration plate, electronic equipment and readable storage medium
EP4141483A1 (en) Target detection method and apparatus
CN113034539B (en) Method and device for determining bounding box of point cloud
CN112689842B (en) Target detection method and device
JP6555132B2 (en) Moving object detection device
CN111699407A (en) Method for detecting stationary object near fence by microwave radar and millimeter wave radar
CN114862961A (en) Position detection method and device for calibration plate, electronic equipment and readable storage medium
CN115457506A (en) Target detection method, device and storage medium
WO2021164514A1 (en) Target recognition method and apparatus
CN114926551A (en) Parameter calibration method and device, vehicle and storage medium
CN116027288A (en) Method and device for generating data, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant