CN115372933A - Stray light filtering method and device and laser radar - Google Patents
Stray light filtering method and device and laser radar Download PDFInfo
- Publication number
- CN115372933A CN115372933A CN202211057733.XA CN202211057733A CN115372933A CN 115372933 A CN115372933 A CN 115372933A CN 202211057733 A CN202211057733 A CN 202211057733A CN 115372933 A CN115372933 A CN 115372933A
- Authority
- CN
- China
- Prior art keywords
- light
- light spot
- spot
- abnormal
- environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000001914 filtration Methods 0.000 title claims abstract description 59
- 230000002159 abnormal effect Effects 0.000 claims description 94
- 230000004313 glare Effects 0.000 claims description 25
- 238000005286 illumination Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 15
- 238000004590 computer program Methods 0.000 description 8
- 230000009191 jumping Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 229910052736 halogen Inorganic materials 0.000 description 3
- 150000002367 halogens Chemical class 0.000 description 3
- 230000001678 irradiating effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003071 parasitic effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The embodiment of the invention relates to the technical field of data processing of laser radars, and discloses a stray light filtering method, a device and a laser radar.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing of laser radars, in particular to a stray light filtering method and device and a laser radar.
Background
Laser Radar (Laser Radar) is a Radar system which emits Laser beams to detect characteristic quantities such as position, speed and the like of a target, and has the working principle that detection Laser is emitted to the target, then the received Laser reflected from the target is compared with the emitted Laser signals, and after appropriate processing is carried out, the related information of the target can be obtained.
Ranging is a current common application scenario of laser radars, and one working principle of laser radar ranging is that laser radars output laser beams, the laser beams strike a laser spot formed on the surface of an object and are reflected to an area array camera, and after image processing, the centroid of the spot is extracted, so that the distance of the object can be calculated.
In implementing the embodiments of the present invention, the inventors found that at least the following problems exist in the above related art: in an actual working scene, when strong light also irradiates the surface of a sensor of a laser radar, interference is easily caused to laser spots, and the appearance of a point cloud is ray-shaped or scattered miscellaneous points, so that a centroid extraction error is caused, and misjudgment on the scene can be caused.
Disclosure of Invention
The embodiment of the application provides a parasitic light filtering method and device and a laser radar.
The purpose of the embodiment of the invention is realized by the following technical scheme:
in order to solve the above technical problem, in a first aspect, an embodiment of the present invention provides a stray light filtering method applied to a laser radar, where the method includes: acquiring light spot data received by the laser radar in the current environment, and extracting light spots from the light spot data; judging whether the current environment is a strong light environment or not according to the light spots and the light spot data; if the light spot is judged to be in a strong light environment, judging whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform; and if the abnormal light spot is judged, filtering the abnormal light spot as an invalid light spot.
In some embodiments, the spot data comprises at least: the judging whether the current environment is a strong light environment according to the light spots and the light spot data comprises the following steps: calculating the background brightness of the light spots and the ratio of the peak value of the light spots to the background brightness according to the light spot data; judging whether the ratio of the spot peak value to the background brightness is lower than a preset ratio threshold value and/or whether the background brightness is higher than a preset brightness threshold value; if yes, determining that the current environment is a strong light environment.
In some embodiments, the method further comprises: constructing two scenes of normal and strong light with different illumination and collecting experimental data; and setting the preset ratio threshold and/or the preset brightness threshold for judging whether the environment is a strong light environment according to the experimental data.
In some embodiments, said extracting a spot from said spot data comprises: drawing a light spot oscillogram according to the light spot data, and determining a plurality of candidate light spots from the light spot oscillogram; determining the spot mass center and the spot width of each candidate spot; determining an upper limit value and a lower limit value of the light spot width corresponding to each candidate light spot mass center position according to the light spot mass center; taking the spot with the largest peak value in the candidate spots with the spot width within the range limited by the upper limit value and the lower limit value of the spot width as a spot to be extracted; and extracting the light spots to be extracted.
In some embodiments, the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform includes: calculating the average variation of the gray value of the background brightness of the light spot within a preset range around the light spot; judging whether the average variation of the gray value is higher than a preset average variation threshold of the gray value or not; and if so, determining the light spot as an abnormal light spot.
In some embodiments, the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform further includes: determining a brightness standard value based on the peak value of the light spot; obtaining the distance from the light spot center of the light spot to the nearest peak boundary; and determining whether the light spot is an abnormal light spot or not based on the brightness standard value and the distance.
In some embodiments, the determining whether the spot is an anomalous spot based on the brightness criterion value and the distance comprises: determining the total width of the waveform of which the brightness value is greater than the brightness standard value; and if the total width of the waveform is greater than the width of the light spot, and the distance is lower than a preset distance threshold, determining that the light spot is an abnormal light spot.
In some embodiments, the method further comprises: and outputting light spot information when the current environment is a conventional environment or the current environment is a highlight environment and the light spots are judged to be normal light spots, wherein the light spot information at least comprises a light spot mass center and light spot brightness.
In order to solve the above technical problem, in a second aspect, an embodiment of the present invention provides a stray light filtering apparatus applied to a laser radar, where the apparatus includes: the acquisition unit is used for acquiring light spot data received by the laser radar in the current environment and extracting light spots from the light spot data; the first judgment unit is used for judging whether the current environment is a strong light environment or not according to the light spots and the light spot data; the second judging unit is used for judging whether the light spots are abnormal light spots with abnormal energy distribution and/or abnormal waveform when the current environment is a strong light environment; and the filtering unit is used for filtering the abnormal light spots as invalid light spots.
In order to solve the above technical problem, in a third aspect, an embodiment of the present invention provides a laser radar, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect as described above.
In order to solve the above technical problem, in a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method according to the first aspect.
In order to solve the above technical problem, in a fifth aspect, the embodiments of the present invention further provide a computer program product, the computer program product comprising a computer program stored on a computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to execute the method according to the first aspect.
Compared with the prior art, the invention has the beneficial effects that: the method comprises the steps of firstly acquiring light spot data received by the laser radar in the current environment, extracting light spots from the light spot data, then judging whether the current environment is a highlight environment according to the light spots and the light spot data, determining whether the light spots are abnormal light spots or not by judging whether the light spots are abnormal in energy distribution and/or waveform when the light spots are in the highlight environment, and filtering the abnormal light spots as invalid light spots when the abnormal light spots are determined, so that the stray light is filtered.
Drawings
One or more embodiments are illustrated by the accompanying figures in the drawings that correspond thereto and are not to be construed as limiting the embodiments, wherein elements/modules and steps having the same reference numerals are represented by like elements/modules and steps, unless otherwise specified, and the drawings are not to scale.
FIG. 1 (a) is a light spot image and a light spot waveform image thereof collected under a conventional (non-intense light) environment;
FIG. 1 (b) is a waveform of a light spot in a strong light environment where only a strong light spot exists;
FIG. 1 (c) is a light spot waveform diagram of annihilation laser light spots with too high brightness of strong light in a strong light environment;
FIG. 1 (d) is a waveform diagram of a light spot in a strong light environment, wherein the strong light amplifies background noise;
fig. 1 (e) is a light spot waveform diagram of strong light reflected light generated after strong light irradiates the surface of a high-reflective material object in a strong light environment;
FIG. 2 is a schematic diagram of an application environment of the stray light filtering method according to the embodiment of the present invention;
FIG. 3 is a schematic flow chart of a veiling glare filtering method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of a sub-process of step S10 in the stray light filtering method shown in FIG. 3;
fig. 5 is an exemplary diagram of a light spot waveform provided by an embodiment of the invention;
FIG. 6 is a schematic flow chart of step S20 of the stray light filtering method shown in FIG. 3;
FIG. 7 is a distribution diagram of spot data for spot background brightness and ratio of spot peak to spot background brightness collected in both bright and normal (non-bright) scenes;
FIG. 8 is a schematic view of another sub-flowchart of step S20 in the stray light filtering method shown in FIG. 3;
FIG. 9 is a schematic flow chart of a sub-process of step S30 in the stray light filtering method shown in FIG. 3;
FIG. 10 is a schematic view of another sub-flowchart of step S30 in the stray light filtering method shown in FIG. 3;
FIG. 11 is a comparison graph of point clouds obtained without and with the veiling glare filtering method provided by the embodiment of the invention;
FIG. 12 is a schematic flow chart of a veiling glare filtering method according to an embodiment of the present invention;
fig. 13 is a schematic structural diagram of a veiling glare filter according to a second embodiment of the present invention;
FIG. 14 is a schematic structural diagram of another veiling glare filter apparatus according to a second embodiment of the present disclosure;
fig. 15 is a schematic diagram of a hardware structure of a laser radar according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It should be noted that, if not conflicting, various features of the embodiments of the present invention may be combined with each other within the scope of protection of the present application. Additionally, while functional block divisions are performed in device schematics, where a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed out of order within the device, or within the flowcharts. Further, the terms "first," "second," "third," and the like, as used herein do not limit the order of data and execution, but merely distinguish between identical or similar items that have substantially the same function or effect.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Currently, unlike the light spot pattern and the light spot waveform pattern thereof collected in the conventional environment as shown in fig. 1 (a), the light spot waveform pattern collected in the strong light environment has four situations as shown in fig. 1 (b), fig. 1 (c), fig. 1 (d) and fig. 1 (e): as shown in fig. 1 (b), when there is no laser spot and only strong light spot, the waveform shows an irregular random waveform; as shown in fig. 1 (c), in the case of annihilation laser spot with too high brightness of strong light, such as the case of halogen lamp interference, the waveform diagram shows a very wide spot; as shown in fig. 1 (d), when the strong light amplifies the background noise, the waveform shows that an interference light spot similar to the laser light spot is formed; as shown in fig. 1 (e), the strong light irradiates the surface of the high-reflective material object and is reflected to the laser radar to generate an interference light spot, and the waveform diagram shows that the peak brightness of the interference light spot is higher than that of the target light spot, that is, the peak brightness of the laser light spot.
In order to solve the problem that effective spot particles are difficult to extract in a spot image due to interference caused by strong light, embodiments of the present invention provide a stray light filtering method, an apparatus, and a laser radar.
Fig. 2 is a schematic diagram of an application environment of the stray light filtering method according to the embodiment of the present invention, where the application environment includes: the laser radar 10, an object A and strong light B.
The laser radar 10 can emit a laser beam, and receive optical information through a sensor (sensor) in a camera to form a light spot pattern, where the light spot pattern includes not only a laser spot formed by the laser beam reflected by an object a, but also an interference/null spot formed by directly irradiating a strong light B onto the sensor (as shown in fig. 1 (B) and fig. 1 (c)), or an interference/null spot formed by irradiating a strong light B onto the sensor (as shown in fig. 1 (e)) after the strong light B is reflected by the object a, or an interference/null spot formed by irradiating a strong light B onto the sensor (as shown in fig. 1 (d)) through other indirect illumination methods.
The object a may be a surface object with high reflectivity, or may be another object, and in practical application, may be any object that the laser radar 10 can measure in a real scene.
The strong light B can be natural light, such as sunlight in summer, or artificial light, such as light emitted by a halogen lamp, and can be any strong light source in an actual ranging scene in practical application.
Specifically, the embodiments of the present invention will be further explained below with reference to the drawings.
Example one
An embodiment of the present invention provides a stray light filtering method, which is applied to a laser radar, where the laser radar may be the laser radar 10 described in the above application scenario, please refer to fig. 3, which shows a flow of the stray light filtering method provided in the embodiment of the present invention, where the method includes, but is not limited to, the following steps:
step S10: acquiring light spot data received by the laser radar in the current environment, and extracting light spots from the light spot data;
the light spot data is the pixel value of each pixel point on the photosensitive sheet collected by the camera sensor, and if the camera sensor adopts a camera with 30 ten thousand pixels, 480 × 640 pixel points are arranged on the photosensitive sheet, in order to improve the data processing efficiency, only 4 × 640 pixel points in the area where the light spot is located and the corresponding pixel values are collected to be used as the light spot data, namely, only the pixel values of 4 rows of pixel points are collected, and specifically, which 4 rows are collected is calibrated in advance (the brightest row is found first, then 2 rows are taken upwards, 1 row is taken downwards, and 4 rows are determined). After the light spot data is acquired, the light spot can be extracted according to the pixel value of each pixel point, for example, if the camera sensor is a black-and-white camera, the pixel value is expressed by a gray value, the pixel value is 0-255, 0 is black, 255 is white, and the pixel point with the gray value larger than a certain value and close to 255 is taken as the pixel point of the light spot, so that the light spot is extracted.
Specifically, referring to fig. 4, it shows a sub-flow of step S10 in the stray light filtering method shown in fig. 3, where the extracting the light spot from the light spot data includes:
step S11: drawing a light spot oscillogram according to the light spot data, and determining a plurality of candidate light spots from the light spot oscillogram;
step S12: determining the spot mass center and the spot width of each candidate spot;
step S13: determining an upper limit value and a lower limit value of the light spot width corresponding to each candidate light spot centroid position according to the light spot centroid;
step S14: taking the spot with the largest peak value in the candidate spots with the spot width within the range limited by the upper limit value and the lower limit value of the spot width as a spot to be extracted;
step S15: and extracting the light spots to be extracted.
In the embodiment of the invention, the light spot to be extracted needs to be extracted first, and then whether the light spot is an abnormal light spot is judged. In particular, from triangulation
Wherein d represents the distance from the light source/object to the lidarFrom, n 1 、n 2 As calibration parameters (as fixed values), c x Indicating the spot centre, i.e. spot centre c x The larger the distance, the smaller the spot width should be, i.e. the order of proximity of the spot with respect to the lidar is inversely related to the spot width. Based on the method, the centroid and the width of the light spot of each light spot can be extracted, the centroid position of each light spot and the upper limit value and the lower limit value of the corresponding light spot width are determined according to the centroid of the light spot, and whether the light spot is to be extracted is determined according to the relation between the width of the light spot and the upper limit value and the lower limit value.
For example, please refer to fig. 5, which shows an example of a light spot waveform, according to the above summarized rule, in fig. 5, after determining the centroid position of the light spot, the width of the light spot on the left side is w1, the width of the light spot on the right side is w2, assuming that m is the lower limit value of the light spot width and n is the upper limit value of the light spot width in fig. 5, the light spot width w1 on the left side is obviously within the range defined by the upper limit value n and the lower limit value m of the light spot width, and at this time, the light spot width w2 on the right side is obviously not within the range defined by the upper limit value n and the lower limit value m of the light spot width, so that the light spot on the left side can be determined as the light spot to be extracted, and further the determination of whether the strong light environment and the light spot is abnormal can be performed on the light spot to be extracted.
Step S20: judging whether the current environment is a strong light environment or not according to the light spots and the light spot data; if yes, jumping to the step S30; if not, jumping to the step S50;
in the embodiment of the invention, after the light spot and the light spot data are obtained, the light spot data which is collected in a strong light scene or not is analyzed according to the light spot and the light spot data; if yes, further judgment and processing are needed to be carried out on the light spots; if not, namely when the spot data is not acquired in a strong light scene, namely in a normal scene, the step S50 is skipped, and the spot information can be directly output. The light spot may be a light spot to be extracted, which is extracted after the processing of the steps S11 to S15. Specifically, please refer to fig. 6, which shows a sub-flow of step S20 in the stray light filtering method shown in fig. 3, wherein the determining whether the current environment is a strong light environment according to the light spots and the light spot data includes:
step S21: calculating the background brightness of the light spots and the ratio of the peak value of the light spots to the background brightness according to the light spot data;
step S22: judging whether the ratio of the spot peak value to the background brightness is lower than a preset ratio threshold value and/or whether the background brightness is higher than a preset brightness threshold value; if yes, jumping to the step S23; if not, jumping to the step S24;
step S23: determining that the current environment is a strong light environment;
step S24: determining that the current environment is a normal environment.
In the embodiment of the present invention, after the light spot data is obtained, a relationship diagram between pixel coordinates and light intensity, that is, a light spot oscillogram, may be further drawn according to the light spot data, that is, the pixel values of the respective pixels, and the ratio of the light spot width, the light spot background brightness, the light spot peak value, and the light spot background brightness may be determined according to the relationship diagram/the light spot oscillogram.
Please refer to fig. 7, which shows distribution diagrams of two spot data, i.e., spot background brightness and ratio of spot peak value to spot background brightness, acquired in two scenes of strong light and conventional light, where it is easy to see that the background brightness of the spots is significantly different in the two scenes of strong light and conventional light, and the ratio of the spot peak value to spot background brightness of the spots is also significantly different in the two scenes of strong light and conventional light, so that the strong light environment can be distinguished and judged according to the ratio of the spot peak value to the spot background brightness and/or the background brightness, and the judgment can be specifically made by the following formula:
wherein, scene _ type represents a type of an environment, and represents that the current environment is a strong light environment when scene _ type =1, represents that the current environment is a non-strong light environment/a normal environment when scene _ type =0, bg represents the background luminance, bg _ thr represents the preset luminance threshold value, peak _ bg _ ratio represents a ratio of the light spot peak value to the light spot background luminance, and pb _ ratio _ thr represents the preset ratio threshold value.
It should be noted that, in the above formula for determining the type of the environment, the determination of the strong light environment takes two conditions that the ratio of the spot peak value to the spot background brightness is lower than the preset ratio threshold and the background brightness is higher than the preset brightness threshold as an example, in some other embodiments, a condition may be selected as the determination standard of the strong light environment, or other spot data is adopted as the determination standard, which may be specifically set according to actual needs.
Further, please refer to fig. 8, which illustrates another sub-flow of step S20 in the veiling glare filtering method shown in fig. 3, the method further includes:
step S25: constructing two scenes of normal illumination and strong light illumination and collecting experimental data;
step S26: and setting the preset ratio threshold and/or the preset brightness threshold for judging whether the environment is a strong light environment according to the experimental data.
In some embodiments, before determining whether the current environment is a strong light environment, it is also possible to construct two scenes with different illumination intensities, namely, a normal scene and a strong light scene, and collect a large amount of experimental data to statistically obtain and socialize the preset ratio threshold and the preset brightness threshold. For example, a halogen lamp with an illumination of 60klux is used for simulating an illumination of 200klux of summer sunlight, and after experimental data under both normal and strong light conditions are collected, the preset ratio threshold value can be set to 10, and the preset brightness threshold value can be set to 36. In other embodiments, an experimental environment may be constructed according to actual needs to obtain the preset ratio threshold and the preset brightness threshold through calculation, which is not limited by the embodiments of the present invention.
Step S30: judging whether the light spots are abnormal light spots with abnormal energy distribution and/or abnormal waveforms; if yes, jumping to the step S40; if not, jumping to the step S50;
in the embodiment of the invention, after the current environment is determined to be a strong light environment, the form of the light spot needs to be analyzed, invalid light spots which do not conform to Gaussian distribution and abnormal light spots with abnormal waveforms are screened out, and then the abnormal light spots are filtered. When the light spots with abnormal energy distribution and/or abnormal waveform are not screened out, the step S50 can be directly skipped to output the light spot information.
Specifically, on the one hand, please refer to fig. 9, which shows a sub-flow of step S30 in the stray light filtering method shown in fig. 3, where the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform includes:
step S31: calculating the average variation of gray values of the background brightness of the light spots in a preset range around the light spots;
step S32: judging whether the average variation of the gray value is higher than a preset average variation threshold of the gray value or not; if yes, jumping to step S33;
step S33: and determining the light spot as an abnormal light spot.
In the embodiment of the invention, firstly, whether the energy distribution of the light spots accords with Gaussian distribution needs to be judged, the light spots which do not accord with the Gaussian distribution are screened out as abnormal light spots, and then the abnormal light spots are filtered. The method for screening the light spots which do not conform to the gaussian distribution may be determined by counting the average variation of the gray value of the area around the light spots, and specifically, the determination may be implemented by calculating the average variation of the gray value, where a calculation formula of the average variation of the gray value is as follows:
wherein avg _ var represents the average variation of the gray values, and img (i) represents the gray value of the pixel point. And the gray value of the pixel point of the background around the light spot can be extracted through the gray value image of the light spot, and the background and the light spot can be distinguished according to the gradient change of the gray value.
Furthermore, after calculating the average variation of the gray value of the background brightness around the light spot, an average variation threshold, that is, the preset average variation threshold avg _ var _ thr of the gray value, may be set to determine whether the gaussian distribution is satisfied. And marking the currently calculated spot as an invalid spot when avg _ var > avg _ var _ thr is satisfied. The specific numerical value of the preset gray value average variation threshold can be adaptively set according to objective conditions such as the brightness conditions of actual light spots and backgrounds and the accuracy of the sensor.
On the other hand, please refer to fig. 10, which shows another sub-flow of step S30 in the stray light filtering method shown in fig. 3, where the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform further includes:
step S34: determining a brightness standard value based on the peak value of the light spot;
step S35: obtaining the distance from the light spot center of the light spot to the nearest peak boundary;
step S36: and determining whether the light spot is an abnormal light spot or not based on the brightness standard value and the distance.
In the embodiment of the present invention, in addition to the method for determining whether the light spot is abnormal according to the energy distribution of the light spot described in the above steps S31 to S33, it may also be determined whether the waveform of the light spot is abnormal according to the pixel distance between the light spot and the nearest peak. The brightness standard value may be a certain percentage of the peak value of the light spot, for example, the brightness standard value is 75% of the peak value of the light spot, and may be specifically set according to an actual situation.
Specifically, the determining whether the light spot is an abnormal light spot based on the brightness standard value and the distance includes: determining the total width w of the waveform of which the brightness value is greater than the brightness standard value; if the total width w of the waveform is greater than the spot width spot _ w of the light spot, and the distance d is lower than a preset distance threshold d _ thr, it is determined that the light spot is an abnormal light spot, that is, the light spot can be determined by the following determination formula:
wherein, scene _ type =1 indicates that the current environment is a strong light environment, and when filter =1 indicates that the total width w of each light spot in the strong light environment is greater than the width spot _ w of the light spot or the distance d from the center of the light spot to the nearest peak boundary of the light spot is lower than a preset distance threshold d _ thr, then filter =1 is output and the corresponding light spot is marked as an abnormal light spot.
The preset distance threshold d _ thr is a minimum distance threshold which can be set, and can be specifically set according to actual conditions; in the example shown in fig. 5, the total width w is the sum of the widths of the left-side light spot w1 and the right-side light spot w2, i.e. the total width w = w 1 +w 2 And judging whether the total width w is greater than the spot width spot _ w to determine whether the spot is an abnormal spot, wherein the calculation of the total width w can be obtained by calculation according to actually acquired spot data and a drawn spot oscillogram.
It should be noted that, in the above formula for determining whether the abnormal light spot is an abnormal light spot, the determination of the abnormal light spot is based on the premise that the current environment is determined to be a strong light environment, and it may be determined that the light spot is an invalid light spot by selecting two determination conditions, that is, whether the total width w of the light spot is greater than the width spot _ w of the light spot and whether the distance d from the center of the light spot to the nearest peak boundary is less than the preset distance threshold d _ thr.
Further, in a normal state, the closer the distance, the larger the spot should be (the larger the spot width), and the higher the luminance, the farther the distance, the smaller the spot (the smaller the spot width), and the lower the luminance. In fig. 5, the center of the light spot of the right wave is larger and farther from the laser radar, the width w2 of the light spot of the right wave should be smaller than the width w1 of the light spot of the left wave, and it is obvious that the width w2 of the light spot of the right wave in fig. 5 is larger, which does not conform to the rule that the distance sequence should be inversely related to the width of the light spot, so that the right wave can be judged as an abnormal light spot.
Step S40: and filtering the abnormal light spots as invalid light spots.
In the embodiment of the present invention, after the abnormal light spots are screened out in step S30, the abnormal light spots are further required to be filtered as invalid light spots. As shown in fig. 11, the left graph is a point cloud graph which is not processed by the veiling glare filtering method provided by the embodiment of the present invention, the left graph is close to the center of the cross-shaped marking line, and the light spot point which does not form a straight line is the invalid light spot/veiling glare, and the right graph is a point cloud graph which is processed by the veiling glare filtering method provided by the embodiment of the present invention.
Further, please refer to fig. 12, which illustrates another flow of the veiling glare filtering method provided in the embodiment of the present invention, wherein the method further includes:
step S50: outputting spot information, wherein the spot information comprises at least a spot centroid and a spot brightness.
In the embodiment of the present invention, when it is determined that the current environment is not a strong light environment, that is, is determined as a current normal environment, and/or it is determined that there is no light spot with abnormal energy distribution and/or abnormal waveform in step S20, the light spot information may be directly output, where the light spot information includes at least a light spot centroid and light spot brightness.
After the current environment is determined to be the strong light environment through the step S20, the step S30 is skipped to determine whether there is a light spot with abnormal energy distribution and/or abnormal waveform, when it is determined that there is a light spot with abnormal energy distribution and/or abnormal waveform, the abnormal light spot is screened and filtered through the step S40, and finally, the retained light spot information is output after the abnormal light spot is filtered. The remaining light spots are the remaining light spots in the data of the filtered light spots at the beginning except the filtered invalid light spots. The filtered invalid spots are discarded, and the corresponding points in the point cloud image are also discarded, so that the point cloud image without the invalid spots can be obtained as shown in the right image of fig. 11.
Example two
An embodiment of the present invention provides a stray light filtering apparatus, which can be applied to a laser radar, and please refer to fig. 13, which shows a structure of the stray light filtering apparatus provided by the embodiment of the present invention, where the stray light filtering apparatus 100 includes: the device comprises an acquisition unit 110, a first judgment unit 120, a second judgment unit 130 and a filtering unit 140.
The acquisition unit 110 is configured to acquire light spot data received by the laser radar in a current environment, and extract a light spot from the light spot data; the first judging unit 120 is configured to judge whether the current environment is a strong light environment according to the light spot and the light spot data; the second determining unit 130 is configured to determine whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform when the current environment is a strong light environment; the filtering unit 140 is configured to filter the abnormal light spot as an invalid light spot when the light spot is an abnormal light spot.
In some embodiments, the first determining unit 120 is further configured to calculate a ratio of the background brightness of the light spot, a peak value of the light spot, and the background brightness according to the light spot data; judging whether the ratio of the spot peak value to the background brightness is lower than a preset ratio threshold value and/or whether the background brightness is higher than a preset brightness threshold value; and if so, determining that the current environment is a strong light environment.
In some embodiments, please refer to fig. 14, which illustrates a structure of another veiling glare filter apparatus provided in the embodiments of the present invention, where the veiling glare filter apparatus 100 further includes: the experiment unit 150 is used for constructing two scenes with different illumination intensities, namely normal and strong light, and collecting experiment data; and setting the preset ratio threshold and/or the preset brightness threshold for judging whether the environment is a strong light environment according to the experimental data.
In some embodiments, the acquisition unit 110 is further configured to draw a light spot waveform map according to the light spot data, and determine a plurality of candidate light spots from the light spot waveform map; determining the spot mass center and the spot width of each candidate spot; determining an upper limit value and a lower limit value of the light spot width corresponding to each candidate light spot mass center position according to the light spot mass center; taking the spot with the largest peak value in the candidate spots with the spot width within the range defined by the upper limit value and the lower limit value of the spot width as a spot to be extracted; and extracting the light spots to be extracted.
In some embodiments, the second determining unit 130 is further configured to calculate an average variation of gray-level values of background brightness of the light spot within a preset range around the light spot; judging whether the average variation of the gray value is higher than a preset average variation threshold of the gray value or not; and if so, determining the light spot as an abnormal light spot.
In some embodiments, the second determining unit 130 is further configured to determine a brightness criterion value based on the peak value of the light spot; obtaining the distance from the light spot center of the light spot to the nearest peak boundary; and determining whether the light spot is an abnormal light spot or not based on the brightness standard value and the distance.
In some embodiments, the second determining unit 130 is further configured to determine a total width of the waveform with a brightness value greater than the brightness standard value; and if the total width of the waveform is greater than the width of the light spot, and the distance is lower than a preset distance threshold, determining that the light spot is an abnormal light spot.
In some embodiments, with continued reference to fig. 14, the veiling glare filter apparatus 100 further comprises: and the output unit 160 is configured to output the light spot information when the current environment is a normal environment or the current environment is a bright light environment and the light spot is determined to be a normal light spot, where the light spot information at least includes a light spot center of mass and light spot brightness.
EXAMPLE III
An embodiment of the present invention further provides a laser radar, please refer to fig. 15, which shows a hardware structure of a laser radar capable of executing the stray light filtering method described in fig. 3 to 12. The lidar 10 may be the lidar 10 shown in fig. 2.
The laser radar 10 includes: at least one processor 11; and a memory 12 communicatively connected to the at least one processor 11, with one processor 11 being taken as an example in fig. 15. The memory 12 stores instructions executable by the at least one processor 11, the instructions being executable by the at least one processor 11 to enable the at least one processor 11 to perform the veiling glare filtering method described above with reference to fig. 3 to 12. The processor 11 and the memory 12 may be connected by a bus or other means, and fig. 15 illustrates the connection by a bus as an example.
The memory 12, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the veiling glare filtering method in the embodiments of the present application, for example, the respective modules shown in fig. 13 to 14. The processor 11 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 12, so as to implement the stray light filtering method of the above method embodiment.
The memory 12 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the flare filtering means, and the like. Further, the memory 12 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 12 optionally includes memory located remotely from processor 11, which may be connected to the veiling glare filter apparatus via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 12, and when executed by the one or more processors 11, perform the veiling glare filtering method in any of the method embodiments described above, for example, perform the method steps of fig. 3 to 12 described above, and implement the functions of the modules and units in fig. 13 to 14.
The product can execute the method provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the methods provided in the embodiments of the present application.
Embodiments of the present application further provide a non-transitory computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the method steps of fig. 3 to 12 described above, and implement the functions of the modules in fig. 13 to 14.
Embodiments of the present application further provide a computer program product, including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the veiling glare filtering method in any of the method embodiments described above, for example, to execute the method steps of fig. 3 to 12 described above, and implement the functions of the modules in fig. 13 to 14.
The embodiment of the invention provides a stray light filtering method, a device and a laser radar, the method comprises the steps of firstly obtaining light spot data received by the laser radar in the current environment, extracting light spots from the light spot data, then judging whether the current environment is a highlight environment or not according to the light spots and the light spot data, determining whether the light spots are abnormal light spots or not by judging whether the light spots are abnormal in energy distribution and/or waveform when the highlight environment is the highlight environment, and filtering the abnormal light spots as invalid light spots when the light spots are determined to be abnormal light spots so as to filter stray light.
It should be noted that the above-described device embodiments are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A stray light filtering method is characterized by being applied to a laser radar, and the method comprises the following steps:
acquiring light spot data received by the laser radar in the current environment, and extracting light spots from the light spot data;
judging whether the current environment is a strong light environment or not according to the light spots and the light spot data;
if the light spot is judged to be in a strong light environment, judging whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform;
and if the abnormal light spots are judged, filtering the abnormal light spots as invalid light spots.
2. A veiling glare filtering method according to claim 1, wherein the determining whether the current environment is a strong light environment according to the light spot and the light spot data comprises:
calculating the background brightness of the light spot, the ratio of the peak value of the light spot to the background brightness according to the light spot data;
judging whether the ratio of the spot peak value to the background brightness is lower than a preset ratio threshold value and/or whether the background brightness is higher than a preset brightness threshold value;
and if so, determining that the current environment is a strong light environment.
3. A veiling glare filtering method according to claim 2, characterized in that the method further comprises:
constructing two scenes of normal and strong light with different illumination and collecting experimental data;
and setting the preset ratio threshold and/or the preset brightness threshold for judging whether the environment is a strong light environment according to the experimental data.
4. A veiling glare filtering method according to claim 1, wherein the extracting of the light spot from the light spot data comprises:
drawing a light spot oscillogram according to the light spot data, and determining a plurality of candidate light spots from the light spot oscillogram;
determining the spot mass center and the spot width of each candidate spot;
determining an upper limit value and a lower limit value of the light spot width corresponding to each candidate light spot mass center position according to the light spot mass center;
taking the spot with the largest peak value in the candidate spots with the spot width within the range defined by the upper limit value and the lower limit value of the spot width as a spot to be extracted;
and extracting the light spots to be extracted.
5. A veiling glare filtering method according to claim 1, wherein the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform comprises:
calculating the average variation of the gray value of the background brightness of the light spot within a preset range around the light spot;
judging whether the average variation of the gray value is higher than a preset average variation threshold of the gray value or not;
and if so, determining the light spot as an abnormal light spot.
6. A veiling glare filtering method according to claim 5, wherein the determining whether the light spot is an abnormal light spot with abnormal energy distribution and/or abnormal waveform further comprises:
determining a brightness standard value based on the peak value of the light spot;
obtaining the distance from the light spot center of the light spot to the nearest peak boundary;
and determining whether the light spot is an abnormal light spot or not based on the brightness standard value and the distance.
7. A veiling glare filtering method according to claim 6, wherein the determining whether the light spot is an abnormal light spot based on the brightness criterion value and the distance comprises:
determining the total width of the waveform of which the brightness value is greater than the brightness standard value;
and if the total width of the waveform is greater than the width of the light spot and the distance is lower than a preset distance threshold, determining that the light spot is an abnormal light spot.
8. A veiling glare filtering method according to any one of claims 1 to 7, wherein the method further comprises:
and outputting light spot information when the current environment is a conventional environment or the current environment is a strong light environment and the light spots are judged to be normal light spots, wherein the light spot information at least comprises a light spot mass center and light spot brightness.
9. A stray light filtering apparatus, applied to a laser radar, the apparatus comprising:
the acquisition unit is used for acquiring light spot data received by the laser radar in the current environment and extracting light spots from the light spot data;
the first judgment unit is used for judging whether the current environment is a strong light environment or not according to the light spots and the light spot data;
the second judging unit is used for judging whether the light spots are abnormal light spots with abnormal energy distribution and/or abnormal waveforms when the current environment is a strong light environment;
and the filtering unit is used for filtering the abnormal light spots as invalid light spots.
10. A lidar, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211057733.XA CN115372933A (en) | 2022-08-31 | 2022-08-31 | Stray light filtering method and device and laser radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211057733.XA CN115372933A (en) | 2022-08-31 | 2022-08-31 | Stray light filtering method and device and laser radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115372933A true CN115372933A (en) | 2022-11-22 |
Family
ID=84070030
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211057733.XA Pending CN115372933A (en) | 2022-08-31 | 2022-08-31 | Stray light filtering method and device and laser radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115372933A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116299497A (en) * | 2023-05-12 | 2023-06-23 | 深圳深浦电气有限公司 | Method, apparatus and computer readable storage medium for optical detection |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09178440A (en) * | 1995-12-27 | 1997-07-11 | Technol Res Assoc Of Medical & Welfare Apparatus | Three-dimensional shape measuring device |
| JPH1172382A (en) * | 1997-08-28 | 1999-03-16 | Sharp Corp | Apparatus for measuring light intensity and recording medium |
| US20090140665A1 (en) * | 2007-12-04 | 2009-06-04 | Mun-Soo Park | Light source module, method for driving the light source module, display device having the light source module |
| US20110221919A1 (en) * | 2010-03-11 | 2011-09-15 | Wenbo Zhang | Apparatus, method, and system for identifying laser spot |
| CN106651801A (en) * | 2016-12-23 | 2017-05-10 | Tcl集团股份有限公司 | Method and system for removing noises during light spot locating |
| CN111402178A (en) * | 2020-03-24 | 2020-07-10 | 成都国科微电子有限公司 | Non-mean filtering method and non-mean filtering device |
| US20210278540A1 (en) * | 2020-03-05 | 2021-09-09 | OPSYS Tech Ltd. | Noise Filtering System and Method for Solid-State LiDAR |
| CN113808193A (en) * | 2021-08-30 | 2021-12-17 | 西安理工大学 | Light spot centroid positioning method based on block threshold |
| CN114371483A (en) * | 2022-03-21 | 2022-04-19 | 深圳市欢创科技有限公司 | Laser radar ranging method and device, laser radar and robot |
| CN114690158A (en) * | 2020-12-30 | 2022-07-01 | 北京万集科技股份有限公司 | Laser radar abnormal signal processing method and device and computer equipment |
| CN114708232A (en) * | 2022-04-11 | 2022-07-05 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
-
2022
- 2022-08-31 CN CN202211057733.XA patent/CN115372933A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09178440A (en) * | 1995-12-27 | 1997-07-11 | Technol Res Assoc Of Medical & Welfare Apparatus | Three-dimensional shape measuring device |
| JPH1172382A (en) * | 1997-08-28 | 1999-03-16 | Sharp Corp | Apparatus for measuring light intensity and recording medium |
| US20090140665A1 (en) * | 2007-12-04 | 2009-06-04 | Mun-Soo Park | Light source module, method for driving the light source module, display device having the light source module |
| US20110221919A1 (en) * | 2010-03-11 | 2011-09-15 | Wenbo Zhang | Apparatus, method, and system for identifying laser spot |
| CN106651801A (en) * | 2016-12-23 | 2017-05-10 | Tcl集团股份有限公司 | Method and system for removing noises during light spot locating |
| US20210278540A1 (en) * | 2020-03-05 | 2021-09-09 | OPSYS Tech Ltd. | Noise Filtering System and Method for Solid-State LiDAR |
| CN111402178A (en) * | 2020-03-24 | 2020-07-10 | 成都国科微电子有限公司 | Non-mean filtering method and non-mean filtering device |
| CN114690158A (en) * | 2020-12-30 | 2022-07-01 | 北京万集科技股份有限公司 | Laser radar abnormal signal processing method and device and computer equipment |
| CN113808193A (en) * | 2021-08-30 | 2021-12-17 | 西安理工大学 | Light spot centroid positioning method based on block threshold |
| CN114371483A (en) * | 2022-03-21 | 2022-04-19 | 深圳市欢创科技有限公司 | Laser radar ranging method and device, laser radar and robot |
| CN114708232A (en) * | 2022-04-11 | 2022-07-05 | Oppo广东移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 王凤鹏 等: "基于激光光斑检测的数码相机测距技术", 赣南师范学院学报, no. 3, 20 June 2011 (2011-06-20), pages 45 - 47 * |
| 许丹丹 等: "基于自适应ROI算法的光斑中心检测", 组合机床与自动化加工技术, no. 3, 20 March 2021 (2021-03-20), pages 83 - 86 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116299497A (en) * | 2023-05-12 | 2023-06-23 | 深圳深浦电气有限公司 | Method, apparatus and computer readable storage medium for optical detection |
| CN116299497B (en) * | 2023-05-12 | 2023-08-11 | 深圳深浦电气有限公司 | Method, apparatus and computer readable storage medium for optical detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6259928B2 (en) | Lane data processing method, apparatus, storage medium and equipment | |
| CN106204547B (en) | The method that rod-shaped atural object spatial position is automatically extracted from Vehicle-borne Laser Scanning point cloud | |
| JP5867596B2 (en) | Three-dimensional object detection apparatus and three-dimensional object detection method | |
| JP6667065B2 (en) | Position estimation device and position estimation method | |
| CN111208533A (en) | Real-time ground detection method based on laser radar | |
| CN116503388B (en) | Defect detection method, device and storage medium | |
| US10380743B2 (en) | Object identifying apparatus | |
| CN109712158A (en) | A kind of infrared small target catching method based on target background pixel statistical restraint | |
| CN106504289B (en) | indoor target detection method and device | |
| CN111951322B (en) | Image pickup module quality detection method, device and computer storage medium | |
| CN115524683A (en) | Light spot extraction method, distance measurement method, laser radar and robot | |
| CN112749643A (en) | Obstacle detection method, device and system | |
| CN115372933A (en) | Stray light filtering method and device and laser radar | |
| CN109190452B (en) | Crop row identification method and device | |
| CN112789521B (en) | Method and device for determining perception area, storage medium and vehicle | |
| JP2011165170A (en) | Object detection device and program | |
| CN108520255B (en) | Infrared weak and small target detection method and device | |
| CN105205485B (en) | Large scale image partitioning algorithm based on maximum variance algorithm between multiclass class | |
| CN113192094A (en) | Ghost outline extraction method, electronic device and storage medium | |
| JP7501398B2 (en) | Object detection device | |
| CN116310889A (en) | Unmanned aerial vehicle environment perception data processing method, control terminal and storage medium | |
| GB2579634A (en) | Method and apparatus for determining temporal behaviour of an object | |
| CN113221910A (en) | Structured light image processing method, obstacle detection method, module and equipment | |
| CN114241054A (en) | Image processing method, device and system based on mesh light spots and storage medium | |
| CN114845041B (en) | Focusing method and device for nanoparticle imaging and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 518000, Floor 1801, Block C, Minzhi Stock Commercial Center, North Station Community, Minzhi Street, Longhua District, Shenzhen City, Guangdong Province Applicant after: Shenzhen Huanchuang Technology Co.,Ltd. Address before: 518000 2407-2409, building 4, phase II, Tian'an Yungu Industrial Park, Gangtou community, Bantian street, Longgang District, Shenzhen, Guangdong Applicant before: SHENZHEN CAMSENSE TECHNOLOGIES Co.,Ltd. |