CN114859328B - Method and device for detecting stop swing of MEMS scanning mirror and laser radar - Google Patents
Method and device for detecting stop swing of MEMS scanning mirror and laser radar Download PDFInfo
- Publication number
- CN114859328B CN114859328B CN202210423543.9A CN202210423543A CN114859328B CN 114859328 B CN114859328 B CN 114859328B CN 202210423543 A CN202210423543 A CN 202210423543A CN 114859328 B CN114859328 B CN 114859328B
- Authority
- CN
- China
- Prior art keywords
- scanning
- auxiliary
- point
- original
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims description 19
- 230000010355 oscillation Effects 0.000 claims description 14
- 230000005856 abnormality Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 description 16
- 230000006870 function Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Electromagnetism (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The application discloses a method and a device for detecting the stop swing of an MEMS scanning mirror and a laser radar. The method comprises the following steps: acquiring an original point cloud generated by a laser radar within a preset time length; generating auxiliary point clouds corresponding to preset ranging values, wherein each auxiliary scanning point in the auxiliary point clouds corresponds to each original scanning point in the original point clouds one by one, and the corresponding auxiliary scanning points and the corresponding original scanning points have the same emission angles; clustering the auxiliary scanning points to obtain clustering blocks; and when the number of the clustering blocks is larger than a preset threshold value, determining that the MEMS scanning mirror stops scanning swing within the preset duration. In this application. Clustering is carried out on the original point cloud from the dimension of the emission angle, and whether the MEMS scanning mirror stops scanning swing within a preset duration is further judged, so that the detection of the MEMS scanning mirror stopping swing is realized.
Description
Technical Field
The present disclosure relates to the field of lidar technologies, and in particular, to a method and an apparatus for detecting a stall of a MEMS (micro electro mechanical systems) scanning mirror, and a lidar.
Background
Lidar is an object detection technique. The laser is used as a signal light source, and the reflected signal of the target object is collected by emitting the laser to the target object, so that information such as the azimuth and the speed of the target object is obtained. The laser radar has the advantages of high measurement accuracy, strong anti-interference capability and the like, and is widely applied to the fields of remote sensing, measurement, intelligent driving, robots and the like.
At present, in the working process of the laser radar, damage to the MEMS scanning mirror or damage to the MEMS scanning mirror driver can occur, so that the MEMS scanning mirror stops scanning swing (which can also be described as MEMS scanning mirror stop swing), thereby affecting the ranging performance of the laser radar. In addition, since the MEMS scanning mirror stops swinging, the outgoing light always irradiates the same angle, which may threaten the eye safety.
Therefore, how to detect MEMS scanning mirror stalling is a problem to be solved.
Disclosure of Invention
The application provides a method and a device for detecting the stop swing of an MEMS scanning mirror and a laser radar, so as to realize the detection of the stop swing of the MEMS scanning mirror and improve the ranging reliability of the laser radar.
In a first aspect, the present application provides a method of detecting a stall of a MEMS scanning mirror, which may be applied to a lidar, such as a microelectromechanical system (micro electro mechanical systems, MEMS) lidar. The method may include: acquiring an original point cloud generated by a laser radar within a preset time length; generating auxiliary point clouds corresponding to preset ranging values, wherein each auxiliary scanning point in the auxiliary point clouds corresponds to each original scanning point in the original point clouds one by one, and the corresponding auxiliary scanning point and the original scanning point have the same emission angle; clustering the auxiliary scanning points to obtain clustering blocks; and when the number of the clustering blocks is larger than a preset threshold value, determining that the MEMS scanning mirror stops scanning swing within a preset time length.
In some possible embodiments, after determining that the MEMS scanning mirror stops scanning oscillation for a preset period of time, the method further comprises: judging whether the MEMS scanning mirror stops scanning swing within a plurality of continuous preset time periods; if yes, outputting alarm information, and controlling the laser of the laser radar to be closed, wherein the alarm information is used for prompting the abnormality of the MEMS scanning mirror; if not, returning to the step of obtaining the original point cloud generated by the laser radar within the preset time.
In some possible embodiments, determining that the MEMS scanning mirror stops scanning oscillation for a preset period of time includes: and determining the original point cloud as a stop-swing point cloud, wherein the stop-swing point cloud is used for indicating that the MEMS scanning mirror stops scanning swing within a preset time length.
In some possible embodiments, generating an auxiliary point cloud corresponding to the preset ranging value includes: generating corresponding auxiliary scanning points according to the preset ranging value and the emission angle of the original scanning points; and converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, and generating auxiliary point clouds.
In some possible embodiments, the coordinates of each auxiliary scanning point in the auxiliary point cloud in the rectangular coordinate system are obtained by the following formula:
x i ＝D×cos(ele i )×sin(azi i )； (1)
y i ＝D×cos(ele i )×cos(azi i )； (2)
z i ＝D×sin(ele i )； (3)
wherein x is i Is the x-axis coordinate, y of the ith auxiliary scanning point i Z as the y-axis coordinate of the ith auxiliary scanning point i For the z-axis coordinate of the ith auxiliary scanning point, D is the preset ranging value, ele i For the height angle of the ith auxiliary scanning point, azi i And the azimuth angle of the ith auxiliary scanning point.
In some possible embodiments, clustering the auxiliary scanning points includes: and clustering the auxiliary scanning points by adopting an European clustering algorithm.
In a second aspect, the present application provides a device for detecting a stall of a MEMS scanning mirror, where the device may be a laser radar or a chip or a system on a chip in the laser radar, and may also be a functional module in the laser radar for implementing the method described in the first aspect and possible embodiments thereof. The detection means may implement the functions performed by the lidar in the first aspect and any possible implementation manner thereof, and these functions may be implemented by hardware executing corresponding software. Such hardware or software includes one or more modules corresponding to the functions described above. The detection device comprises: the acquisition module is used for acquiring an original point cloud generated by the laser radar within a preset time length; the generating module is used for generating auxiliary point clouds corresponding to the preset ranging values, each auxiliary scanning point in the auxiliary point clouds corresponds to each original scanning point in the original point clouds one by one, and the corresponding auxiliary scanning point and the corresponding original scanning point have the same emission angle; the clustering module is used for carrying out clustering processing on the auxiliary scanning points to obtain clustering blocks; and the determining module is used for determining that the MEMS scanning mirror stops scanning swing within a preset time length when the number of the clustering blocks is larger than a preset threshold value.
In some possible embodiments, the apparatus further comprises: an alarm module; the determining module is also used for judging whether the MEMS scanning mirror stops scanning swing in a plurality of continuous preset time periods after determining that the MEMS scanning mirror stops scanning swing in the preset time periods; if yes, triggering an alarm module to output alarm information, controlling a laser of the laser radar to be closed, and prompting the abnormality of the MEMS scanning mirror by the alarm information; if not, triggering the acquisition module.
In some possible embodiments, the determining module is further configured to determine the original point cloud as a stall point cloud, where the stall point cloud is configured to indicate that the MEMS scanning mirror stops scanning oscillation within a preset time period.
In some possible embodiments, the generating module is configured to generate a corresponding auxiliary scanning point according to a preset ranging value and an emission angle of an original scanning point; and converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system to generate auxiliary point clouds.
In some possible embodiments, the clustering module is configured to perform clustering on the auxiliary scanning points by using an european clustering algorithm.
In a third aspect, the present application provides a lidar comprising: a memory storing computer executable instructions; a processor, coupled to the memory, for executing computer-executable instructions to implement the method according to the first aspect and any possible implementation thereof.
In a fourth aspect, the present application provides a computer storage medium having stored thereon computer executable instructions which, when executed by a processor, are capable of carrying out the method according to the first aspect and any one of its possible embodiments.
Compared with the prior art, the technical scheme provided by the application has the beneficial effects that:
in the method, after the laser radar obtains the original point cloud generated in the preset time period, the auxiliary point cloud with the same emission angle as the original point cloud is generated, clustering processing is carried out on the auxiliary point cloud pairs (namely, the laser radar can carry out clustering processing on the original point cloud from the dimension of the emission angle), when the number of clustering blocks obtained through the clustering processing is larger than a preset threshold value, the MEMS scanning mirror can be determined to stop scanning swing in the preset time period, so that the detection of the MEMS scanning mirror stop swing is realized, the ranging reliability of the laser radar is improved, and the damage to human eyes is avoided. In addition, because the auxiliary point cloud corresponding to the original point cloud is generated on the given preset ranging value, the ranging values of all scanning points in the auxiliary point cloud are the same, and when the MEMS scanning mirror stalling is detected according to the auxiliary point cloud, the influence of the ranging values on point cloud clustering can be removed, so that the distribution condition of the emission angle can be accurately reflected, whether the MEMS scanning mirror stalls or not can be judged, and the accuracy of detecting the MEMS scanning mirror stalling is improved.
Further, the original point cloud is clustered from the dimension of the emission angle, so that the MEMS scanning mirror stop-swing is detected without adding an additional detection device, and the MEMS scanning mirror stop-swing is detected on the premise of not changing the hardware structure of the laser radar.
Further, through European clustering, whether a plurality of faults appear in the emission angle distribution of the original point cloud is judged to judge whether the original point cloud is the stopping point cloud of the MEMS scanning mirror, namely whether the MEMS scanning mirror stops scanning swing within a preset duration is detected, so that the detection of the stopping swing of the MEMS scanning mirror is realized.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the scope of the application.
Drawings
Fig. 1 is a schematic structural diagram of a laser radar in the related art;
FIG. 2 is a schematic flow chart of a first implementation of a method for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a second implementation of a method for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a structure of a device for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a lidar according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to illustrate the technical solutions described in the present application, the following description is made by specific examples.
Lidar is an object detection technique. The laser radar emits laser beams through the laser, the laser beams are diffusely reflected after encountering a target object, the reflected beams are received through the detector, and the characteristic quantities such as the distance, the azimuth, the height, the speed, the gesture and the shape of the target object are determined according to the emitted beams and the reflected beams.
The application field of laser radar is very wide. In addition to its use in the military field, it is now widely used in the life field, including but not limited to: intelligent piloting vehicles, intelligent piloting airplanes, three-dimensional (3D) printing, virtual reality, augmented reality, service robots, and the like. Taking intelligent driving technology as an example, a laser radar is arranged in an intelligent driving vehicle, and the laser radar can scan the surrounding environment by rapidly emitting laser beams so as to acquire point cloud data reflecting the morphology, the position, the movement and the like of one or more target objects in the surrounding environment.
The intelligent driving technique may refer to unmanned, automatic, auxiliary, and the like.
Fig. 1 is a schematic structural diagram of a lidar in the related art, and referring to fig. 1, a lidar 10 may include: a light emitting device 101, a light receiving device 102, and a processor 103. Wherein the light emitting device 101 and the light receiving device 102 are connected with the processor 103.
The connection relationship between the devices may be electrical connection or optical fiber connection. More specifically, in the light emitting device 101 and the light receiving device 102, a plurality of optical devices may be included, respectively, and the connection relationship between these optical devices may also be a spatial light transmission connection.
The processor 103 is used to realize control of the transmitting device 101 and the light receiving device 102 so that the light transmitting device 101 and the light receiving device 102 can operate normally. Illustratively, the processor 103 may provide driving voltages for the light emitting device 101 and the light receiving device 102, respectively, and the processor 103 may also provide control signals for the light emitting device 101 and the light receiving device 102.
By way of example, the processor 103 may be a general-purpose processor such as a central processing unit (central processing unit, CPU), a network processor (network processor, NP), etc.; the processor 103 may also be a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.
The light emitting device 101 further comprises a light source (not shown in fig. 1). It will be appreciated that the light source may refer to a laser and the number of lasers may be one or more. Alternatively, the laser may be embodied as a pulsed laser diode (pulsed laser diode, PLD), semiconductor laser, fiber laser, or the like. The light source is used for emitting a laser beam. In particular, the processor 103 may send an emission control signal to the light source, triggering the light source to emit a laser beam.
It will be appreciated that the laser beam may also be referred to as a laser pulse, laser, emitted beam, etc.
The following describes the detection process of the target object 104 by the lidar in brief in connection with the structure of the lidar shown in fig. 1.
Referring to fig. 1, the laser beam propagates in the emission direction, and when the laser beam encounters the target object 104, reflection occurs on the surface of the target object 104, and the reflected beam is received by the light receiving device 102 of the laser radar. Here, the beam of the laser beam reflected back by the target object 104 may be referred to as an echo beam (the laser beam and the echo beam are indicated by solid lines in fig. 1).
After the light receiving device 102 receives the echo beam, photoelectric conversion is performed on the echo beam, that is, the echo beam is converted into an electrical signal, the light receiving device 102 outputs the electrical signal corresponding to the echo beam to the processor 103, and the processor 103 may obtain the morphology, the position, the moving point cloud data, and the like of the target object 104 according to the electrical signal of the echo beam.
In practical application, with the continuous development of MEMS technology, a scanning mirror based on MEMS technology has been widely used in lidar as a micro-actuator. Scanning mirrors based on MEMS technology may be referred to as MEMS scanning mirrors, which include in particular mirror surfaces or other structures capable of reflecting light. When a laser radar (which may be called a MEMS laser radar) with a MEMS scanning mirror works, a laser beam is emitted to the MEMS scanning mirror, and the MEMS scanning mirror can reflect the laser beam to different outgoing directions and angles by changing the rotation angle of its mirror surface.
To achieve rotation of the mirror itself, the MEMS scanning mirror may be rotationally driven by the mirror in a number of different ways. The driving modes of the MEMS scanning mirror at present are mainly electrostatic driving, electromagnetic driving, piezoelectric driving, electrothermal driving and the like.
In practical application, the MEMS laser radar can be provided with a plurality of lasers to adopt field of view concatenation to realize great detection scope, when the MEMS laser radar works, the laser continuously emits light, relies on the scanning swing of MEMS scanning mirror to change the emergent light path.
However, in the operation process of the MEMS laser radar, damage to the MEMS scanning mirror or damage to the driver may occur, which may cause the MEMS scanning mirror to stop swinging, i.e., the MEMS scanning mirror stops swinging, thereby affecting the ranging reliability of the laser radar. In addition, since the MEMS scanning mirror stops swinging, the outgoing light always irradiates the same angle, which may threaten the eye safety. Therefore, how to detect MEMS scanning mirror stalling is a problem to be solved.
In order to solve the above problems, an embodiment of the present application provides a method for detecting a stall of a MEMS scanning mirror, which can be applied to the MEMS laser radar described above.
First, it should be noted that "scan point" in the embodiments of the present application may be understood as an effective scan point (may also be described as an effective point) in the point cloud. That is, the effective scan point is a scan point having an effective ranging value, the ranging value (distance) is not 0 or non-number (NaN), and does not exceed the theoretical maximum ranging value.
FIG. 2 is a schematic flow chart of a first implementation of a method for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application, and referring to FIG. 2, the method may include:
s201, obtaining an original point cloud generated in a preset time period.
It should be appreciated that lidar scans in a scanning cycle, and that lidar scans a full field of view of a point cloud may be generally referred to as a frame of point cloud. Since a frame of point cloud is obtained in the scanning period, a frame of point cloud can also be described as a point cloud (i.e., an original point cloud) generated in a preset period of time. Here, the preset time period may be understood as a scanning period of the lidar.
In some possible embodiments, after obtaining the original point cloud, the lidar also needs to obtain the emission angles of each scanning point (which may be the original scanning point) in the original point cloud.
In one embodiment, the lidar may calculate the firing angle of each scan point from the three-dimensional coordinates (x, y, z) of each scan point. Here, the emission angle of one scan point may include an azimuth angle (azi) and an altitude angle (ele).
Exemplary, the emission angle (azi) of the ith original scan point in the original point cloud i ,ele i ) Can be determined according to the following formulas (4) and (5):
wherein, azi i For the azimuth of the ith original scan point, ele i For the height angle of the ith original scan point, x' i Is the x-axis coordinate, y 'of the ith original scanning point' i Z 'as the y-axis coordinate of the ith original scan point' i Is the z-axis coordinate of the ith original scan point.
In another embodiment, the laser radar can also obtain the emission angles of the original scanning points in the original point cloud by inquiring the calibration file.
Of course, the laser radar may also determine the emission angles of each original scanning point in the original point cloud by other methods, which is not specifically limited in the embodiments of the present application.
S202, generating auxiliary point clouds corresponding to preset ranging values according to the emission angles of all original scanning points in the original point clouds.
Here, each scanning point (which may be referred to as an auxiliary scanning point) in the auxiliary point cloud corresponds to each original scanning point in the original point cloud one by one, and the corresponding auxiliary scanning point and the original scanning point have the same emission angle.
It should be understood that, in order to more intuitively and conveniently extract the distribution characteristics of the emission angles of the scanning points, the laser radar may use the emission angles of each original scanning point in the original point cloud as the emission angles of the auxiliary scanning points, so as to convert the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, thereby generating the auxiliary point cloud. In addition, in the process of coordinate transformation, in order to shield the influence of the ranging value on the subsequent clustering process, a preset ranging value (e.g. denoted as D, d=2m) may be set. Then, through coordinate conversion, auxiliary point clouds corresponding to the preset ranging values can be obtained.
In practical application, the preset ranging value may be determined according to parameters such as power, gain, maximum ranging value of the laser radar, which is not specifically limited in the embodiment of the present application.
In some possible embodiments, the step S202 may include: generating corresponding auxiliary scanning points according to the preset ranging value and the emission angle of the original scanning points; and converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, and generating auxiliary point clouds.
It should be understood that when the laser radar generates the auxiliary point cloud, the auxiliary scanning points corresponding to the original scanning points one by one may be generated according to the preset ranging value and the emission angle of the original scanning points, and at this time, the coordinates of the auxiliary scanning points in the spherical coordinate system may be (D, ele, azi). Then, the laser radar performs coordinate conversion on each auxiliary scanning point, and converts the coordinates in the spherical coordinate system into the coordinates in the rectangular coordinate system, namely the coordinates of the auxiliary scanning points in the rectangular coordinate system can be (x, y, z), so as to generate an auxiliary point cloud.
Note that, ele represents the altitude angle of the original scan point (or the auxiliary scan point), azi represents the azimuth angle of the original scan point (or the auxiliary scan point), x represents the x-axis coordinate of the auxiliary scan point, y represents the y-axis coordinate of the auxiliary scan point, and z represents the z-axis coordinate of the auxiliary scan point.
For example, the coordinates of the ith auxiliary scanning point in the auxiliary point cloud in the rectangular coordinate system may be expressed as formulas (1) to (3).
x i ＝D×cos(ele i )×sin(azi i )； (1)
y i ＝D×cos(ele i )×cos(azi i )； (2)
z i ＝D×sin(ele i )； (3)
Wherein x is i Is the x-axis coordinate, y of the ith auxiliary scanning point i Z as the y-axis coordinate of the ith auxiliary scanning point i For the z-axis coordinate of the ith auxiliary scanning point, D is the preset ranging value, ele i For the height angle of the ith auxiliary scanning point (or original scanning point) i An azimuth angle for the i-th auxiliary scanning point (or original scanning point).
It should be noted that, the original scanning points in the original point cloud correspond to the auxiliary scanning points in the auxiliary point cloud one by one, and the corresponding emission angles of the original scanning points and the auxiliary scanning points are the same.
S203, clustering the auxiliary scanning points to obtain a clustering block.
It should be understood that, after the auxiliary point cloud is obtained in S202, the laser radar may perform clustering processing on the auxiliary scanning points in the auxiliary point cloud to obtain a distribution situation of the emission angles of each auxiliary scanning point, so as to obtain a cluster block (i.e., the emission angles are clustered).
In practical applications, an euclidean clustering algorithm, a density-based noise application spatial clustering (density based spatial clustering of applications with noise, DBSCAN) algorithm, a range image-based clustering algorithm, and the like may be employed for the clustering process of the auxiliary scanning points.
For example, clustering is performed on the auxiliary scanning points by using an European clustering algorithm. In the first step, for a certain auxiliary scanning point p in the space, using KD-Tree to find n auxiliary scanning points nearest to p (i.e. n nearest neighbors) in the neighborhood, and placing the point from p to p in the n nearest neighbors in the set Q with a distance smaller than a distance threshold r (e.g. r=0.05m). If the elements in the set Q are not increasing, ending the clustering process; otherwise, selecting points except p points from the set Q, and repeating the steps.
In the embodiment of the application, through European clustering, whether a plurality of faults appear in the emission angle distribution is judged to judge whether an original point cloud is a stop-swing point cloud of the MEMS scanning mirror, namely whether the MEMS scanning mirror stops scanning swing within a preset duration is detected, so that the detection of the stop-swing of the MEMS scanning mirror is realized.
And S204, when the number of the clustering blocks is larger than a preset threshold value, determining that the MEMS scanning mirror stops scanning swing within a preset time period.
It should be understood that, after the laser radar performs the clustering processing on the auxiliary scanning points in S203, K clustering blocks may be obtained. Then, the laser radar judges whether K is larger than a preset threshold value, if so, the laser radar can determine an original point cloud, namely a frame of point cloud generated within a preset time period as a stop-swing point cloud, that is, the laser radar can determine that the MEMS scanning mirror stops scanning swing within the preset time period. Otherwise, the laser radar determines that the original point cloud is not the stop swing point cloud, that is, the MEMS scanning mirror does not stop scanning swing within the preset time period, and then the laser radar returns to S201 to continue to detect the next frame point cloud.
In practical applications, the preset threshold may be at least determined according to the number of subfields of the lidar. The number of subfields of the lidar is related to the number of outgoing light paths. For example, 8 lasers in the laser radar form an emergent light path, and at this time, the laser radar has 8 sub-fields of view; and 2 lasers are arranged in the laser radar, one of the lasers is divided into two paths of emergent light through a beam splitter, so that a total of 3 paths of emergent light paths exist, and at the moment, the laser radar correspondingly has 3 sub-fields of view.
Therefore, the process of detecting the stop swing of the MEMS scanning mirror based on the single-frame point cloud is completed.
In some possible implementations, the embodiments of the present application further provide a method for detecting a stall of a MEMS scanning mirror, which can be applied to the MEMS lidar described in the above embodiments.
FIG. 3 is a schematic flow chart of a second implementation of a method for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application, and referring to FIG. 3, the method may include:
s301, original point clouds generated in a plurality of continuous preset time periods are obtained.
It should be appreciated that a lidar may obtain a multi-frame origin cloud that is continuous in time.
S302, generating auxiliary point clouds corresponding to preset ranging values according to the original point clouds generated in each preset time period. Here, the emission angle of the auxiliary point cloud is the same as that of the original point cloud.
It should be understood that the lidar performs S202 described above for each frame original point cloud, generating a corresponding auxiliary point cloud. At this time, the lidar obtains a multi-frame auxiliary point cloud.
It should be noted that, the scanning points in the original point cloud correspond to the scanning points in the auxiliary point cloud one by one.
S303, clustering is carried out on the auxiliary point cloud corresponding to each original point cloud to obtain a corresponding clustering block.
It should be understood that, for each auxiliary point cloud corresponding to each original point cloud, the lidar may execute S203 separately, and perform clustering processing on auxiliary scanning points in each frame of auxiliary point cloud, so as to obtain a corresponding clustering block.
S304, judging whether the MEMS scanning mirror stops scanning swing within a plurality of continuous preset time periods according to the number of the clustering blocks; if yes, jump to S305; if not, jumping to S301;
it should be understood that, the laser radar may determine whether the original point cloud of each frame is a stop-swing point cloud according to the number of the clustering blocks of the auxiliary point cloud of each frame by executing S203 multiple times, so as to determine whether the MEMS scanning mirror stops scanning swing within a plurality of continuous preset durations. If it is determined that the continuous multi-frame original point clouds are the stop-swing point clouds according to the judgment result, it can be determined that the MEMS scanning mirror stops scanning swing within a plurality of continuous preset durations, at this time, the laser radar can determine that the MEMS scanning mirror is abnormal, and execute S305; otherwise, the process returns to S301.
For example, the plurality of preset durations may be 5, that is, 5 consecutive frames of original point clouds are stop-and-swing point clouds. Of course, the number of frames of the original point cloud may be other numbers, which is not specifically limited in the embodiment of the present application.
In some possible embodiments, the lidar may acquire the original point cloud multiple times, such as one frame of the original point cloud at a time, while performing S301. Then, S202 to S203 are performed for each frame origin cloud to determine whether the frame origin cloud is a stall point cloud, and thus whether the MEMS scanning mirror stops scanning oscillation within a preset period of time. After determining that a certain frame of original point cloud is a stop-and-swing point cloud, the laser radar can also judge whether a plurality of frames of original point clouds before the frame of original point cloud are the stop-and-swing point clouds. If the multi-frame original point cloud before the frame original point cloud is the stop-swing point cloud, the laser radar can determine that the MEMS scanning mirror stops scanning swing within a plurality of continuous preset time periods, and then execute S305; otherwise, the process returns to S301.
S305: and outputting alarm information.
The warning information is used for prompting the abnormality of the MEMS scanning mirror.
It should be appreciated that after determining that the MEMS scanning mirror stops scanning oscillation within a plurality of consecutive preset durations, the lidar may consider that the MEMS scanning mirror is damaged or that the MEMS scanning mirror driver is damaged, at which time the lidar may output alarm information to prompt that the MEMS scanning mirror is abnormal, so that a serviceman or a user may repair the MEMS scanning mirror in time, thereby avoiding accidents caused by inaccurate ranging.
Further, the lidar may also control the laser to be turned off when S305 is performed.
In the embodiment of fig. 3, the implementation process of S202 to S203 may be referred to the description in the embodiment of fig. 2, and will not be described in detail herein.
Thus, the process of detecting the stop swing of the MEMS scanning mirror is completed.
In the embodiment of the application, after the laser radar obtains the original point cloud generated within the preset time period, the auxiliary point cloud with the same emission angle as the original point cloud is generated, clustering processing is carried out on the auxiliary point cloud (namely, the laser radar can carry out clustering processing on the original point cloud from the dimension of the emission angle), when the number of clustering blocks obtained by the clustering processing is larger than a preset threshold value, the MEMS scanning mirror can be determined to stop scanning swing within the preset time period, so that the detection of the MEMS scanning mirror stop swing is realized, the ranging reliability of the laser radar is improved, and the damage to human eyes is avoided. In addition, because the auxiliary point cloud corresponding to the original point cloud is generated on the given preset ranging value, the ranging values of all scanning points in the auxiliary point cloud are the same, and when the MEMS scanning mirror stalling is detected according to the auxiliary point cloud, the influence of the ranging values on point cloud clustering can be removed, so that the distribution condition of the emission angle can be accurately reflected, whether the MEMS scanning mirror stalls or not can be judged, and the accuracy of detecting the MEMS scanning mirror stalling is improved.
Further, the original point cloud is clustered from the dimension of the emission angle, so that the MEMS scanning mirror stop-swing is detected without adding an additional detection device, and the MEMS scanning mirror stop-swing is detected on the premise of not changing the hardware structure of the laser radar.
Based on the same inventive concept, the embodiments of the present application provide a device for detecting a stall of a MEMS scanning mirror, which may be a laser radar or a chip or a system on a chip in the laser radar, and may also be a functional module in the laser radar for implementing the method described in the first aspect and possible embodiments thereof. The detection means may implement the functions performed by the lidar in the first aspect and any possible implementation manner thereof, and these functions may be implemented by hardware executing corresponding software. Such hardware or software includes one or more modules corresponding to the functions described above. Fig. 4 is a schematic structural diagram of a device for detecting a stall of a MEMS scanning mirror according to an embodiment of the present application, and referring to fig. 4, the device 400 includes: an obtaining module 401, configured to obtain an original point cloud generated by the laser radar within a preset duration; the generating module 402 is configured to generate an auxiliary point cloud corresponding to a preset ranging value, where each auxiliary scanning point in the auxiliary point cloud corresponds to each original scanning point in the original point cloud one by one, and emission angles of the corresponding auxiliary scanning point and the original scanning point are the same; the clustering module 403 is configured to perform clustering processing on the auxiliary scanning points to obtain a clustering block; a determining module 404, configured to determine that the MEMS scanning mirror stops scanning swing within a preset time period when the number of cluster blocks is greater than a preset threshold.
In some possible embodiments, referring to the dashed line in fig. 4, the apparatus 400 further includes: an alarm module 405; the determining module 404 is further configured to determine whether the MEMS scanning mirror stops scanning oscillation for a plurality of consecutive preset durations after determining that the MEMS scanning mirror stops scanning oscillation for the preset duration; if yes, the alarm module 405 is triggered to output alarm information, and the laser of the laser radar is controlled to be closed, wherein the alarm information is used for prompting the abnormality of the MEMS scanning mirror; if not, the acquisition module 401 is triggered.
In some possible embodiments, the determining module 404 is further configured to determine the original point cloud as a stall point cloud, where the stall point cloud is configured to indicate that the MEMS scanning mirror stops scanning oscillation within a preset time period.
In some possible embodiments, the generating module 402 is configured to generate the auxiliary scanning point according to a preset ranging value and an emission angle of the original point cloud; and converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, and generating auxiliary point clouds.
In some possible embodiments, the clustering module 403 is configured to perform clustering on the auxiliary scanning points by using an european clustering algorithm.
It should be noted that, the specific implementation processes of the obtaining module 401, the generating module 402, the clustering module 403, the determining module 404, and the alarm module 405 may refer to the detailed descriptions of the embodiments of fig. 2 to 3, and are not repeated herein for brevity of the description.
The obtaining module 401, the generating module 402, the clustering module 403, and the determining module 404 mentioned in the embodiments of the present application may be one or more processors; the alarm module 405 may be a display unit, an audio output unit, etc.
Based on the same inventive concept, embodiments of the present application provide a lidar, which may be the lidar described in one or more of the embodiments above. Fig. 5 is a schematic structural diagram of a lidar according to an embodiment of the present application, and referring to fig. 5, a lidar 500 may employ general-purpose computer hardware, including a processor 501 and a memory 502.
In the alternative, processor 501 and memory 502 may communicate via a bus 503.
In some possible implementations, the at least one processor 501 may constitute any physical device having circuitry to perform logical operations on one or more inputs. For example, the at least one processor may include one or more integrated circuits (integrated circuit, ICs) including application specific integrated circuits (application specific integrated circuit, ASIC), microchips, microcontrollers, microprocessors, all or part of a central processing unit (central processing unit, CPU), a graphics processing unit (graphics processing unit, GPU), a digital signal processor (digital signal process, DSP), a field programmable gate array (field programmable gate array, FPGA), or other circuit suitable for executing instructions or performing logic operations. The instructions executed by the at least one processor may, for example, be preloaded into a memory integrated with or embedded in the controller, or may be stored in a separate memory. The memory may include random access memory (random access memory, RAM), read-only memory (ROM), hard disk, optical disk, magnetic medium, flash memory, other permanent, fixed, or volatile memory, or any other mechanism capable of storing instructions. In some embodiments, at least one processor may comprise more than one processor. Each processor may have a similar structure, or the processors may have different configurations electrically connected or disconnected from each other. For example, the processors may be separate circuits or integrated in a single circuit. When more than one processor is used, the processors may be configured to operate independently or cooperatively. The processors may be coupled in electrical, magnetic, optical, acoustical, mechanical, or by other means that allow them to interact. According to one embodiment of the present application, there is also provided a computer readable storage medium having stored thereon computer instructions for execution by a processor of the steps of the calibration method described above. Memory 502 may include computer storage media in the form of volatile and/or nonvolatile memory such as read only memory and/or random access memory. Memory 502 may store an operating system, application programs, other program modules, executable code, program data, user data, and the like.
In addition, the above-mentioned memory 502 stores computer-executable instructions for implementing the functions of the obtaining module 401, the generating module 402, the clustering module 403, the determining module 404, and the alerting module 405 in fig. 4. The functions/implementation of the obtaining module 401, the generating module 402, the clustering module 403, the determining module 404 and the alarm module 405 in fig. 4 may be implemented by the processor 501 in fig. 5 calling the computer-executable instructions stored in the memory 502, and the specific implementation and functions refer to the above-mentioned related embodiments.
Based on the same inventive concept, embodiments of the present application provide a laser radar including: a memory storing computer executable instructions; and a processor, coupled to the memory, for executing computer-executable instructions and capable of implementing the methods described in one or more embodiments above.
Based on the same inventive concept, the present application provides a computer storage medium storing computer executable instructions that, when executed by a processor, are capable of implementing a method as described in one or more embodiments above.
It will be understood by those skilled in the art that the sequence number of each step in the above embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; while the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those skilled in the art that variations may be made in the techniques described in the foregoing embodiments, or equivalents may be substituted for elements thereof; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (12)
1. A method for detecting a stall of a MEMS scanning mirror of a microelectromechanical system, comprising:
acquiring an original point cloud generated by a laser radar within a preset time length;
generating auxiliary point clouds corresponding to preset ranging values, wherein each auxiliary scanning point in the auxiliary point clouds corresponds to each original scanning point in the original point clouds one by one, and the corresponding auxiliary scanning points and the corresponding original scanning points have the same emission angles;
clustering the auxiliary scanning points to obtain clustering blocks;
when the number of the clustering blocks is larger than a preset threshold value, determining that the MEMS scanning mirror stops scanning swing within the preset duration;
the generating the auxiliary point cloud corresponding to the preset ranging value comprises the following steps:
generating the auxiliary scanning point according to the preset ranging value and the emission angle of the original scanning point;
converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, and generating the auxiliary point cloud;
the coordinates of each auxiliary scanning point in the auxiliary point cloud in the rectangular coordinate system are obtained by the following formula:
x i ＝D×cos(ele i )×sin(azi i )；
y i ＝D×cos(ele i )×cos(azi i )；
z i ＝D×sin(ele i )；
wherein x is i Is the x-axis coordinate, y of the ith auxiliary scanning point i Z as the y-axis coordinate of the ith auxiliary scanning point i For the z-axis coordinate of the ith auxiliary scanning point, D is the preset ranging value, ele i For the height angle of the ith auxiliary scanning point, azi i And the azimuth angle of the ith auxiliary scanning point.
2. The method of claim 1, wherein after said determining that the MEMS scanning mirror ceases scanning oscillation for the predetermined period of time, the method further comprises:
determining that the MEMS scanning mirror stops scanning oscillation in a plurality of continuous preset time periods;
and outputting alarm information and controlling the laser of the laser radar to be closed, wherein the alarm information is used for prompting the abnormality of the MEMS scanning mirror.
3. The method of claim 1, wherein said determining that the MEMS scanning mirror ceases scanning oscillation for the predetermined period of time comprises:
and determining the original point cloud as a stop-swing point cloud, wherein the stop-swing point cloud is used for indicating that the MEMS scanning mirror stops scanning swing within the preset duration.
4. A method according to any one of claims 1 to 3, wherein said clustering said auxiliary scanning points comprises:
and clustering the auxiliary scanning points by adopting an European clustering algorithm.
5. A method according to any one of claims 1 to 3, wherein the preset threshold is determined at least from the number of subfields of the lidar.
6. A stall detection apparatus for a MEMS scanning mirror of a microelectromechanical system, comprising:
the acquisition module is used for acquiring an original point cloud generated by the laser radar within a preset time length;
the generating module is used for generating auxiliary point clouds corresponding to preset ranging values, each auxiliary scanning point in the auxiliary point clouds corresponds to each original scanning point in the original point clouds one by one, and the corresponding auxiliary scanning points and the corresponding original scanning points have the same emission angles;
the clustering module is used for carrying out clustering processing on the auxiliary scanning points to obtain clustering blocks;
the determining module is used for determining that the MEMS scanning mirror stops scanning swing within the preset duration when the number of the clustering blocks is larger than a preset threshold value;
the generating module is further configured to generate the auxiliary scanning point according to the preset ranging value and the emission angle of the original scanning point; converting the coordinates of the auxiliary scanning points from a spherical coordinate system to a rectangular coordinate system, and generating the auxiliary point cloud;
the coordinates of each auxiliary scanning point in the auxiliary point cloud in the rectangular coordinate system are obtained by the following formula:
x i ＝D×cos(ele i )×sin(azi i )；
y i ＝D×cos(ele i )×cos(azi i )；
z i ＝D×sin(ele i )；
wherein x is i Is the x-axis coordinate, y of the ith auxiliary scanning point i Z as the y-axis coordinate of the ith auxiliary scanning point i For the z-axis coordinate of the ith auxiliary scanning point, D is the preset ranging value, ele i For the height angle of the ith auxiliary scanning point, azi i And the azimuth angle of the ith auxiliary scanning point.
7. The apparatus of claim 6, wherein the apparatus further comprises: an alarm module;
the determining module is further configured to determine whether the MEMS scanning mirror stops scanning oscillation for a plurality of consecutive preset durations after determining that the MEMS scanning mirror stops scanning oscillation for the preset durations; triggering the alarm module to output alarm information and controlling the laser of the laser radar to be closed, wherein the alarm information is used for prompting the abnormality of the MEMS scanning mirror.
8. The apparatus of claim 6, wherein the means for determining is further configured to determine the original point cloud as a sweep point cloud, the sweep point cloud being configured to indicate that the MEMS scanning mirror is to cease scanning oscillation for the predetermined period of time.
9. The apparatus according to any one of claims 6 to 8, wherein the clustering module is configured to perform clustering on the auxiliary scanning points by using an european clustering algorithm.
10. The apparatus according to any one of claims 6 to 8, wherein the preset threshold is determined at least from the number of subfields of the lidar.
11. A lidar, comprising:
a memory storing computer executable instructions;
a processor, coupled to the memory, for implementing the method of any one of claims 1 to 5 by executing the computer-executable instructions.
12. A computer storage medium having stored thereon computer executable instructions which, when executed by a processor, are capable of carrying out the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210423543.9A CN114859328B (en) | 2022-04-21 | 2022-04-21 | Method and device for detecting stop swing of MEMS scanning mirror and laser radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210423543.9A CN114859328B (en) | 2022-04-21 | 2022-04-21 | Method and device for detecting stop swing of MEMS scanning mirror and laser radar |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114859328A CN114859328A (en) | 2022-08-05 |
| CN114859328B true CN114859328B (en) | 2023-05-05 |
Family
ID=82631454
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210423543.9A Active CN114859328B (en) | 2022-04-21 | 2022-04-21 | Method and device for detecting stop swing of MEMS scanning mirror and laser radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114859328B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001215275A (en) * | 2000-02-03 | 2001-08-10 | Mitsubishi Electric Corp | Laser radar device |
| CN114325642A (en) * | 2021-11-30 | 2022-04-12 | 深圳元戎启行科技有限公司 | Laser radar scanning method, scanning apparatus, and computer-readable storage medium |
| CN114322828A (en) * | 2021-12-15 | 2022-04-12 | 云南昆钢电子信息科技有限公司 | Point cloud sampling system and method based on laser radar scanning angle switching |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010102121A (en) * | 2008-10-23 | 2010-05-06 | Olympus Corp | Bar code reader |
| JP6634667B2 (en) * | 2014-07-18 | 2020-01-22 | 船井電機株式会社 | Laser scanning device |
| US11016178B2 (en) * | 2017-03-13 | 2021-05-25 | OPSYS Tech Ltd. | Eye-safe scanning LIDAR system |
| US10613200B2 (en) * | 2017-09-19 | 2020-04-07 | Veoneer, Inc. | Scanning lidar system and method |
| CN208314328U (en) * | 2018-06-29 | 2019-01-01 | 上海禾赛光电科技有限公司 | scanning mirror |
| CN111257856B (en) * | 2020-02-21 | 2022-05-27 | 奥比中光科技集团股份有限公司 | Scanning mirror monitoring system and method |
| CN111323205B (en) * | 2020-02-21 | 2021-08-03 | 奥比中光科技集团股份有限公司 | Micro-mirror array-based MEMS micro-galvanometer monitoring device and method |
| CN211979325U (en) * | 2020-02-21 | 2020-11-20 | 深圳奥比中光科技有限公司 | Scanning mirror monitoring system and scanning device |
-
2022
- 2022-04-21 CN CN202210423543.9A patent/CN114859328B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001215275A (en) * | 2000-02-03 | 2001-08-10 | Mitsubishi Electric Corp | Laser radar device |
| CN114325642A (en) * | 2021-11-30 | 2022-04-12 | 深圳元戎启行科技有限公司 | Laser radar scanning method, scanning apparatus, and computer-readable storage medium |
| CN114322828A (en) * | 2021-12-15 | 2022-04-12 | 云南昆钢电子信息科技有限公司 | Point cloud sampling system and method based on laser radar scanning angle switching |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114859328A (en) | 2022-08-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210278540A1 (en) | Noise Filtering System and Method for Solid-State LiDAR | |
| CN110914705A (en) | Integrated LIDAR lighting power control | |
| JP2021510417A (en) | LIDAR-based distance measurement with layered power control | |
| JP7214888B2 (en) | Radar power control method and apparatus | |
| JP2022539706A (en) | Adaptive multi-pulse LIDAR system | |
| US20240159879A1 (en) | Detection control method and apparatus | |
| US12019188B2 (en) | Eye-safe scanning lidar with virtual protective housing | |
| CN112034486B (en) | Laser radar and control method of laser radar | |
| WO2023083198A1 (en) | Echo signal processing method and apparatus, device, and storage medium | |
| CN114008477A (en) | Techniques for supporting coexistence of multiple independent LIDAR sensors | |
| CN116964485A (en) | Detection method and device and laser radar | |
| CN115047471B (en) | Method, device, equipment and storage medium for determining laser radar point cloud layering | |
| CN114859328B (en) | Method and device for detecting stop swing of MEMS scanning mirror and laser radar | |
| US20200292667A1 (en) | Object detector | |
| CN114755666B (en) | Point cloud expansion evaluation method, device and equipment | |
| CN114578316B (en) | Method, device and equipment for determining ghost points in point cloud and storage medium | |
| US20210341580A1 (en) | Ranging device and mobile platform | |
| CN113574410A (en) | Dynamic control and configuration of autonomous navigation systems | |
| CN114935747B (en) | Laser radar calibration method, device, equipment and storage medium | |
| CN111670527B (en) | Discharge circuit for distance measuring device, distributed radar system and movable platform | |
| JP6804619B1 (en) | Laser distance measuring device | |
| CN117761708A (en) | Multi-echo signal processing method, device, equipment and storage medium | |
| JP2024075331A (en) | Control device, optical detection system, control method, and control program | |
| CN114265076A (en) | Control method and control device of detection device, laser radar system and terminal | |
| JP2021135157A (en) | Object detection device, moving body, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |