CN118259237A

CN118259237A - Method, device, equipment and storage medium for calibrating millimeter wave radar

Info

Publication number: CN118259237A
Application number: CN202211677576.2A
Authority: CN
Inventors: 郭文举; 刘蒙蒙
Original assignee: Beijing Voyager Technology Co Ltd
Current assignee: Beijing Voyager Technology Co Ltd
Priority date: 2022-12-26
Filing date: 2022-12-26
Publication date: 2024-06-28

Abstract

Embodiments of the present disclosure provide a method, apparatus, device, and storage medium for calibrating millimeter wave radar. The method comprises the steps of acquiring a first set of points collected by a millimeter wave radar to be calibrated and related to a calibration tool; acquiring a second set of points acquired by the calibrated lidar with respect to the calibration tool; determining a plurality of matching point pairs from the first set of points and the second set of points based on the acquisition times of the first set of points and the second set of points, the first point of the plurality of matching point pairs being from the first set of points and the second point matching the first point being from the second set of points; and calibrating a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar. According to the embodiment of the disclosure, the external parameters of the millimeter wave radar can be calibrated by using the laser radar directly by using the open road, the road sealing is not needed, the cost is low, and the calibration efficiency is high.

Description

Method, device, equipment and storage medium for calibrating millimeter wave radar

Technical Field

Embodiments of the present disclosure relate to the field of autopilot, and more particularly, to a method, apparatus, device, and storage medium for calibrating millimeter wave radar.

Background

Autopilot is becoming a key technology affecting the future industry, and sensors are the key to the perception of the outside world in autopilot systems, the cooperative capabilities of which directly determine the safety of an autopilot vehicle. For the need of automatic driving, for example, for acquiring data required for a high-definition map, it is sometimes necessary to arrange various sensors such as cameras, radars, and the like on the road side to acquire road surface information. Various sensors disposed on the road side are generally disposed at higher positions like a overpass to acquire related data of the road surface below.

Disclosure of Invention

According to a first aspect of the present disclosure, a method for calibrating a millimeter wave radar is provided. The method comprises the following steps: acquiring a first set of points acquired by a millimeter wave radar to be calibrated and related to a calibration tool; acquiring a second set of points acquired by the calibrated lidar with respect to the calibration tool; determining a plurality of matching point pairs from the first set of points and the second set of points based on acquisition times of the first set of points and the second set of points, a first point of the plurality of matching point pairs from the first set of points and a second point of the plurality of matching point pairs from the second set of points; and calibrating a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

According to a second aspect of the present disclosure, an apparatus for calibrating millimeter wave radar is provided. The device comprises: a first acquisition module configured to acquire a first set of points acquired by the millimeter wave radar to be calibrated with respect to the calibration tool; a second acquisition module configured to acquire a second set of points acquired by the calibrated lidar with respect to the calibration tool; a determining module configured to determine a plurality of matching point pairs from the first set of points and the second set of points based on acquisition times of the first set of points and the second set of points, a first point of the plurality of matching point pairs from the first set of points and a second point of the plurality of matching point pairs from the second set of points; and a calibration module configured to calibrate a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the apparatus to perform the method according to the first aspect of the preceding.

According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program which, when executed by a processor, implements a method according to the first aspect hereinbefore.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following more particular descriptions of exemplary embodiments of the disclosure as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the disclosure.

FIG. 1 illustrates a simplified schematic diagram of an example environment of a roadside awareness system;

FIG. 2 shows a simplified schematic diagram of a conventional approach for calibrating millimeter wave radar in a roadside awareness system;

FIG. 3 illustrates a schematic diagram of a calibration tool for calibrating millimeter wave radar in a roadside awareness system, according to an embodiment of the disclosure;

FIGS. 4A-4C illustrate views of a calibration tool from different angles according to embodiments of the present disclosure;

FIG. 5 illustrates a flow chart for calibrating a millimeter wave radar using a calibration tool implemented in accordance with the present disclosure;

fig. 6 shows a flow diagram of a method for calibrating millimeter wave radar in accordance with an embodiment of the present disclosure;

fig. 7 illustrates a block diagram of an apparatus for calibrating millimeter wave radar in accordance with some embodiments of the present disclosure; and

Fig. 8 illustrates an electronic device in which one or more embodiments of the present disclosure may be implemented.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure have been shown in the accompanying drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.

In describing embodiments of the present disclosure, the term "comprising" and its like should be taken to be open-ended, i.e., including, but not limited to. The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first," "second," and the like, may refer to different or the same object. Other explicit and implicit definitions are also possible below.

The principles of the present disclosure will be described below with reference to several example embodiments shown in the drawings. While the preferred embodiments of the present disclosure are illustrated in the drawings, it should be understood that these embodiments are merely provided to enable those skilled in the art to better understand and practice the present disclosure and are not intended to limit the scope of the present disclosure in any way.

Further, the term "responsive to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be appreciated that the execution timing of a subsequent action that is executed in response to the event or condition is not necessarily strongly correlated with the time at which the event occurs or the condition is established. For example, in some cases, the follow-up actions may be performed immediately upon occurrence of an event or establishment of a condition; in other cases, the subsequent action may be performed after a period of time has elapsed after the event occurred or the condition was established.

The vehicle-road cooperation is an important development direction in the current automatic driving field and is also a key component of intelligent traffic development. The vehicle-road cooperation mainly comprises a plurality of core modules such as intelligent vehicles, intelligent road sides, data communication, cloud control platforms and the like, and safe driving of the vehicles and effective configuration of road resources are realized through real-time interaction of dynamic information among the vehicles and the roads. The road side perception system is one of important means for improving the intelligent level of the road. Based on advanced technologies such as vehicle cloud computing, the intelligent driving of different grades of vehicles is realized by applying the perception fusion and decision planning capabilities of road ends and cloud ends.

The road side perception system is generally arranged at the intersection and road section with complex traffic flow, on one hand, the cost of the vehicle end can be shared to the road side, meanwhile, the defect of intelligent perception of a single vehicle can be made up, the safety range of single vehicle intelligence is effectively enlarged, and the traffic accident rate of the vehicle is reduced. On the other hand, the intelligent road promotes the management level of a traffic road network system, can effectively solve traffic jam and improves the traffic capacity and efficiency of the road network. Fig. 1 shows a simplified example in which a roadside perception system is arranged on a red light pole 202 of an intersection.

As shown in fig. 1, lidar 2011 and millimeter wave radar 2012 in the roadside awareness system 201 are co-deployed at a higher location of the intersection (e.g., on traffic light posts or on an overhead bridge) to scan model the environment of the intersection. The lidar 2011 and the millimeter wave radar 2012 may have different field of view (FOV) and have at least partially overlapping fields of view, which may be fixed such that the normals of both are parallel to each other, or the projections of the normals of both on the road surface are parallel to each other, or the angle between the normals of both is less than a threshold angle. The threshold angle represents the maximum value of the angle between the normals of the lidar 2011 and the millimeter wave radar 2012. That is, the normal of the lidar 2011 may be deflected by at most the threshold angle in a clockwise direction or a counterclockwise direction with respect to the normal of the millimeter-wave radar 2012. In some embodiments, the appropriate threshold angle may be set according to the particular application environment, e.g., may be set to 5 ° or any other appropriate angle value. It should be appreciated that the greater the threshold angle, the lower the installation accuracy requirements for the lidar 2011 and millimeter wave radar 2012; the smaller the threshold angle is, the higher the mounting accuracy requirements for the lidar 2011 and the millimeter wave radar 2012 are.

A laser radar 2011 (LiDAR), which is a short-term for laser detection and ranging systems, has become one of the important roadside sensors as a high-precision, high-sensitivity, short-delay sensor. The laser radar 2011 can accurately identify traffic information of road ends of complex intersections, provide high-precision tracks of various traffic participants, and support intelligent traffic data acquisition, cooperative sensing of roads, automatic driving test verification and the like. The lidar 2011 system includes a laser and a receiving system. The laser generates and emits a beam of light pulses that impinges on the object and reflects back to be received by the receiver. The receiver accurately measures the propagation time of the light pulse from the emission to the reflection back. Because the light pulse propagates at the speed of light, the receiver always receives the previous reflected pulse before the next pulse is sent out. In view of the fact that the speed of light is known, the travel time can be converted into a measure of distance. The coordinates of each ground spot can be accurately calculated by combining the height of the laser, the laser scanning angle, and the position and laser emission direction from a Global Positioning System (GPS) laser.

For some lidar 2011, due to the lower intensity of the emitted laser beams, for some distant objects there may be insufficient perceptibility such as a sparser acquired point cloud. To compensate for these disadvantages, the roadside sensor system may be equipped with a laser radar 2011 and a millimeter wave radar 2012. Millimeter wave radar 2012 is a radar sensor device that measures distance, angle, and speed using millimeter waves. The millimeter wave radar 2012 can calculate the relative distance between the millimeter wave radar 2012 and the monitoring target by combining the millimeter wave propagation speed, the carrier speed and the monitoring target speed according to the time difference between receiving and transmitting the millimeter wave. Millimeter wave radar 2012 generally has a high measurement accuracy.

In general, after the laser radar 2011 and the millimeter wave radar 2012 are disposed on a road, external parameters of the radar are usually required to be calibrated in order to improve the detection accuracy of the radar on objects and distances. For lidar 2011, its external parameters can generally be calibrated in a relatively easy manner. After calibration, the coordinate system of the laser radar is considered to have a fixed mapping relation or conversion relation with a universal transverse ink card grid (UTM) coordinate system. To some extent, the coordinate system of the lidar may correspond to the UTM coordinate system. For millimeter wave radar 2012, it is usually necessary to calibrate by means of external parameters calibrated by laser radar 2011. Through carrying out collaborative calibration to laser radar 2011 and millimeter wave radar 2012, can make both gather coordinate system unification, be convenient for subsequent data processing.

For millimeter wave radar 2012 and its cooperative calibration relative to lidar 2011, it is currently required to calibrate by combining an RTK mobile station and a triangular reflection cone, which has severe requirements for field environment.

Triangular reflecting cones, also known as radar reflectors or corner reflectors, are cone-shaped structures with multiple angles, such as triangular pyramid structures. The triangular reflecting cone generally comprises a plurality of reflecting planes perpendicular to each other, when the radar electromagnetic wave enters the triangular reflecting cone from the incident plane of the triangular reflecting cone, the electromagnetic wave is reflected by the plurality of reflecting planes and then is reflected back to the radar along the opposite direction of the incident electromagnetic wave, thereby facilitating the calibration of the millimeter wave radar 2012. However, since the triangular reflection cone has a smaller size and the laser radar 2011 has a sparse scanning point cloud, calibration of the millimeter wave radar 2012 and collaborative calibration of the two cannot be performed by directly using the triangular reflection cone through external parameters of the laser radar 2011.

In the conventional scheme, as shown in fig. 2, to perform calibration of the millimeter wave radar 2012 by using the laser radar 2011 and collaborative calibration of the two, 12-18 points are generally required to be selected after radar blind areas (typically about 25 meters). These points need to be tapped (precise locations determined) with the RTK mobile station. The calibration triangular reflecting cones may be placed at these points in sequence. Thus calibrating millimeter wave radar 2012. It is obvious that this measurement mode needs to be performed under the condition that the road is closed because a plurality of points are required to be arranged on the road which is not dead zone of the laser radar 2011 and the millimeter wave radar 2012, so that the data acquisition in the urban road scene in a mode of not closing the road is basically impossible. The measures such as road closure bring higher calibration cost. In addition, the current calibration method has low data acquisition efficiency and is easy to introduce errors in calibration.

Embodiments of the present disclosure provide a calibration tool 100 for a millimeter wave radar 2012 to address or at least partially address the above-mentioned problems, or other potential problems, presented in the foregoing approaches to calibrating millimeter wave radar 2012. Fig. 3 shows a schematic diagram of a calibration tool 100 implemented in accordance with the present disclosure calibrating a roadside deployed millimeter wave radar 2012. During the calibration process, the calibration tool 100 according to the embodiment of the present disclosure moves toward or away from the millimeter wave radar 2012 or the laser radar 2011 along the projection or the parallel lines of the projection of the normal direction of the millimeter wave radar 2012 or the laser radar 2011 on the ground under the load of the mobile device 203. The moving device 203 may be various vehicles driven to move by various power, unmanned vehicles, or the like. In some embodiments, the calibration tool 100 according to embodiments of the present disclosure may also be manually moved by a human hand to calibrate external parameters of the millimeter wave radar 2012.

According to the embodiment of the disclosure, in the case that the external parameters of the lidar 2011 have been determined or calibrated, the external parameters of the millimeter wave radar 2012 can be calibrated by using the lidar 2011 directly by using the open road, without sealing the road, and with low cost. In addition, the device according to the embodiment of the disclosure can complete calibration of the millimeter wave radar 2012 in a moving state, and has low environmental requirements, thereby bringing high calibration efficiency.

The calibration tool 100 according to an embodiment of the present disclosure will be described below with reference to fig. 4A to 4C. Fig. 4A shows a front view of the calibration tool 100, fig. 4B shows a perspective view of the calibration tool 100, and fig. 4C shows a view of the calibration tool 100 from the back side, according to an embodiment of the present disclosure. As shown in fig. 4A to 4C, generally, a calibration tool 100 according to an embodiment of the present disclosure includes a body 101, a first calibration member 102, and a second calibration member 103. The body 101 is mainly used for fixing at least one of the first calibration member 102 and the second calibration member 103. In some embodiments, the body 101 may be a support stand. To reduce the impact on calibration, in some embodiments, the body 101 may be made of a non-metallic material.

In some embodiments, the body 101 may include a frame portion and a support portion 1011. The frame portion is used to fix the first calibration member 102 and/or the second calibration member 103. The support portion 1011 is for supporting the frame portion, and may be telescopic, whereby the height of the frame portion may be adjusted. In some embodiments, alternatively or additionally, the support 1011 may also be bendable and may remain at each angle of bending, thereby facilitating adjustment of the angle of the frame portion and the first and second calibration members 102 and 103.

In some embodiments, the body 101 may also have no frame portion in case the strength of the first calibration member 102 is sufficient, and the supporting portion 1011 directly supports the first calibration member 102 and the second calibration member 103.

The first calibration part 102 is fixed to the body 101 and includes a calibration surface adapted to be scanned by the laser radar 2011 to thereby acquire point cloud data (hereinafter referred to as laser point cloud data). The calibration surface may have a larger size so that it can be easily scanned by the lidar 2011 with a lower resolution to obtain its corresponding point cloud data. In some embodiments, the first calibration component 102 may be a calibration plate having a calibration surface.

To facilitate resolution and identification of the lidar 2011, in some embodiments, the calibration surface may have at least two regions, and the at least two regions may have different reflectivities to the lidar 2011. For example, in some embodiments, at least two regions may be coated with or formed of different materials, respectively, that enable electromagnetic waves emitted by lidar 2011 to exhibit different reflectivities thereon. The difference in reflectivity results in different echo signals, and thus the lidar 2011 can distinguish the at least two areas, so that the point cloud data of the lidar 2011 can be further accurate and reliable.

In some embodiments, at least two regions may be arranged in a predetermined pattern. For example, as shown in fig. 4A and 4B, the at least two regions may include a plurality of regions arranged in a checkerboard. The regions of different reflectivity may be spaced apart from one another. In some embodiments, the regions of different reflectivity may have different colors. With the first calibration part 102 having such a calibration surface, it is advantageous not only to improve the accuracy and reliability of the point cloud data acquired by the lidar 2011, but also to calibrate visual sensors such as cameras, and the like.

The second calibration member 103 is also fixed to the body 101 and has a fixed positional relationship with the calibration surface. As mentioned before, the second calibration member 103 may also be directly fixed to the first calibration member 102 in a fixed positional relationship with the calibration surface of the first calibration member 102. The fixed positional relationship indicates that the distance and orientation between the calibration surfaces of the second calibration member 103 and the first calibration member 102 are fixed, or that the distance and orientation between the reference points (e.g., origin or any other suitable reference point) of the second calibration member 103 and the first calibration member 102 are fixed. For example, in some embodiments, the origin of the calibration surface of the first calibration component 102 may coincide with the origin of the second calibration surface or the distance and orientation between the two may be fixed. For example, in some embodiments, the second calibration member 103 may be fixed into the calibration surface of the first calibration member 102.

It should be understood, of course, that the above-described embodiments with respect to the second calibration member 103 being fixed in the calibration surface of the first calibration member 102 are illustrative only and are not intended to limit the scope of the present disclosure. The relative position between the second calibration member 103 and the first calibration member 102 may be arbitrary as long as they have a fixed positional relationship. For example, in some alternative embodiments, the second calibration member 103 may also be fixed in any suitable position outside of the calibration surface of the first calibration member 102.

The second calibration component 103 is adapted to scan and obtain corresponding millimeter wave point cloud data by the millimeter wave radar 2012. It should be understood that. Since the position of the second calibration part 103 is fixed with respect to the calibration surface of the first calibration part 102, the millimeter wave point cloud data can determine a plurality of matching point pairs with the laser point cloud data. For example, the matching point pair may be at least a portion of millimeter wave point cloud data and laser point cloud data. One point (hereinafter referred to as a second point) in the matching point pair comes from the laser point cloud data, and the other point (hereinafter referred to as a first point) paired with the laser point cloud data comes from the millimeter wave point cloud data.

In this way, the external parameters of the millimeter wave radar 2012 can be calibrated based on the laser point cloud data and the millimeter wave point cloud data and the calibrated external parameters of the laser radar 2011. Specifically, the external parameters (hereinafter referred to as second external parameters) of the millimeter wave radar 2012 may be calibrated based on the plurality of matching point pairs mentioned above and the calibrated external parameters (hereinafter referred to as first external parameters) of the lidar 2011. The specific manner will be further described below.

In some embodiments, the second calibration component 103 may include a triangular reflecting cone. In embodiments where the second calibration component 103 is fixed in the calibration surface of the first calibration component 102, the incident surface of the triangular reflecting cone may be arranged flush with the calibration surface of the first calibration component 102, thereby facilitating the utilization of the first point cloud data of the lidar 2011 with respect to the calibration surface.

To achieve more accurate calibration, the calibration tool 100 according to an embodiment of the present disclosure may be moved away from or toward the millimeter wave radar 2012 along the projection of the normal of the millimeter wave radar 2012 or the lidar 2011 or a parallel line thereof on the ground, as shown in fig. 3. As mentioned previously, the calibration tool 100 may be coupled to a moving device 203, such as a vehicle, for movement. To facilitate coupling of the calibration tool 100 to the mobile device 203, in some embodiments, the calibration tool 100 may further comprise a coupling portion 104 adapted to be coupled to the mobile device 203, such that the mobile device 203 may be caused to carry the calibration tool 100 away from or towards the millimeter wave radar 2012 in a normal direction of the millimeter wave radar 2012 or the lidar 2011.

In some embodiments, the coupling portion 104 may be connected to the moving device 203 by a snap-fit connection or the like, thereby facilitating the assembly of the calibration tool 100 on the moving device 203. In some embodiments, the coupling portion 104 may also be coupled to the mobile device 203 by way of fasteners, adhesives, interference fits, and the like.

Fig. 5 shows a schematic diagram of a calibration process for calibrating external parameters of millimeter wave radar 2012 using a calibration tool implemented in accordance with the present disclosure. At block 510, during the course of the calibration tool 100 movement, the lidar 2011 and millimeter wave radar are caused to acquire point cloud data. During this time, the millimeter wave radar 2012 and the laser radar 2011 may collect the point cloud data within the field of view continuously or at predetermined intervals a plurality of times, including the point cloud data of the calibration tool 100, whereby a plurality of sets of point cloud data may be obtained. Each set of point cloud data has a corresponding acquisition time point. Multiple sets of point cloud data may be correlated for subsequent processing. Further, to ensure calibration accuracy, the calibration tool 100 may be moved a distance exceeding a predetermined length, such as 30 meters or 50 meters.

At block 520, lidar point cloud data may be processed. Specifically, the lidar 2011 data may be processed using a point cloud ground point filtering (CSF) algorithm. The basic principle of the CSF algorithm is that a piece of cloth is placed on an inverted point cloud, then the action of the cloth and the point cloud is simulated, and finally the shape of the cloth can approximate the trend of the terrain, and the original point cloud (ground points and non-ground points) can be classified by utilizing the distance between the points and the cloth. After the CSF algorithm is utilized to complete the ground point cloud segmentation, each point in the point cloud can be marked with a time point associated with the point cloud, wherein the time point is the acquisition time point, and multiple groups of laser point cloud data are combined.

For the processed laser point cloud data, clustering and segmentation can be performed, so that the laser point cloud data can be clustered into point cloud clusters through spatial relationships, and the laser point cloud data about the calibration tool 100 obtained by clustering and segmentation can be extracted. The laser point cloud data about the calibration surface of the calibration tool 100 is then segmented at time points and the segmented point clouds are weighted averaged according to the acquisition time points to determine the center point of the calibration surface. And connecting the center points corresponding to the time points to obtain the movement track of the calibration tool.

For millimeter wave point cloud data, each point in the millimeter wave point cloud data may be marked with a time point (i.e., an acquisition time point) associated with the point, and multiple sets of millimeter wave point cloud data may be combined. The millimeter wave point cloud data may then be processed according to parameters such as speed and reflection intensity to extract trajectory data for the second calibration component 103 of the calibration tool 100. The resulting trajectory data about the second calibration part 103 may then be divided in time points, resulting in a sequence of point coordinates of the second calibration part 103.

From the data obtained by the above processing, the determined center point of the calibration surface is obtained by the calibrated lidar 2011, and there is a fixed mapping relationship or conversion relationship between the coordinate system of the lidar 2011 and the UTM coordinate system, so that the X, Y axis component of the center point (also referred to as the second set of points) at these different time points in the coordinate system of the lidar 2011 can be converted into the X, Y axis component in the UTM coordinate system. Correspondingly, the obtained coordinates of the sequence points (also referred to as a first group of points) with respect to the second calibration part 103 are acquired by the millimeter wave radar 2012, that is, are coordinate points in the coordinate system of the millimeter wave radar 2012 to be calibrated. Since the positional relationship of the millimeter wave radar 2012 and the laser radar 2011 is fixed and the positions of the calibration surface of the calibration tool 100 and the second calibration member 103 are fixed, the external parameters of the millimeter wave radar 2012 can be calibrated using the center point coordinates of the calibration surface and the external parameters of the laser radar 2011.

Specifically, at block 530, the center point and sequence points processed above may be matched. In particular, the second set of points acquired by the above-described process with respect to the calibration surface and the first set of points with respect to the second calibration tool 100 may be paired in terms of acquisition time points. For example, in some embodiments, for a second point at a given point in time at a plurality of points in time may be determined with respect to a second set of points. A first point of the first set of points of the second calibration component 103 at the given point in time is then determined. A given matching point pair is then determined based on the two points. The above steps may be performed a plurality of times to determine a plurality of given point-to-point matching point pairs corresponding to a plurality of given time points.

After a plurality of given matching point pairs are determined, a relative relationship between the coordinate system of the millimeter wave radar 2012 and the UTM coordinate system is established. Since the mapping relationship between the coordinate system of the UTM and the coordinate system of the lidar 2011 is fixed, the establishment of the relative relationship between the coordinate system of the millimeter-wave radar 2012 and the coordinate system of the UTM is equivalent to the establishment of the relative relationship between the coordinate system of the millimeter-wave radar 2012 and the coordinate system of the lidar 2011. The relative relationship here may be expressed, for example, as a transformation matrix between coordinate systems. Next, at block 540, lidar point sets are each constructed from the correspondenceAnd millimeter wave radar point setThe set of points includes a plurality of sets of matched pairs of first points and second points. Thus, the following equations may be used to determine R and T and thereby calibrate the external parameters of millimeter wave radar 2012.

Wherein the method comprises the steps ofFor a corresponding point in the set of lidar points,For the corresponding points in the millimeter wave radar point set, R is a rotation matrix for converting from millimeter wave coordinates to UTM coordinates, T is a translation matrix, that is, R is a rotation matrix for calibrating external parameters of the millimeter wave radar 2012, T is a translation matrix, and e _i is an error term. The relative relationship between the rotation matrix R and the translation matrix T, i.e., the millimeter wave radar coordinate system and the UTM coordinate system (i.e., the lidar coordinate system).

In equation (1) above, only R and T are unknown, while the remaining parameters are known. By solving for the error term e _i to be the smallest, R, T is solved, and thus calibration of the millimeter wave radar 2012 is completed. As can be seen from the above description of the calibration process, by using the calibration tool 100, the requirement on the road environment is low, and the calibration tool can be used in cooperation with vehicles, and is suitable for the data acquisition and calibration of millimeter wave radar on roads with large urban traffic, so that the road is not required to be closed and a plurality of points are not required to be arranged, the calibration efficiency can be improved, and the cooperative deployment of the laser radar 2011 and the millimeter wave radar 2012 is facilitated.

Fig. 6 illustrates a flow chart of a process 600 for calibrating millimeter wave radar in accordance with some embodiments of the present disclosure. Process 600 may be implemented at a suitable electronic device.

At block 610, the electronic device obtains a first set of points collected by the millimeter wave radar to be calibrated with respect to the calibration tool. At block 620, the electronics acquire a second set of points acquired by the calibrated lidar with respect to the calibration tool.

At block 630, the electronic device determines a plurality of matching point pairs from the first set of points and the second set of points based on the acquisition times of the first set of points and the second set of points, the first point of the plurality of matching point pairs being from the first set of points and the second point of the plurality of matching point pairs being from the second set of points.

At block 640, the electronic device calibrates a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

In some embodiments, the calibration tool includes a calibration plate and a triangular reflecting cone, and the origin of the calibration plate and the origin of the triangular reflecting cone have a fixed positional relationship. The origin of the calibration plate may refer to the centre point of the calibration surface mentioned in the foregoing. The origin of the triangular reflecting cone may coincide therewith. Of course, it should be understood that the origin of the calibration plate may also be any suitable reference point having an associated relation with the calibration plate, e.g. it may be a centre point, a corner point, etc. of the calibration surface of the calibration plate. Similarly, the origin of the triangular reflecting cone may also be any suitable reference point having an associated relation to the triangular reflecting cone or its incident surface, which may coincide with the origin of the calibration plate or have a fixed distance and orientation.

In some embodiments, the calibration tool is adapted to move away from or towards the millimeter wave radar along a projection of a normal of the millimeter wave radar on the ground for the millimeter wave radar and the lidar to acquire the first set of points and the second set of points, respectively.

In some embodiments, the calibration tool is moved a distance equal to or greater than a predetermined length. For example, in some embodiments, the calibration tool is moved a distance exceeding 30 meters or 50 meters.

In some embodiments, the first set of points and the second set of points are acquired at a plurality of time points. Determining a plurality of matching point pairs from the first set of points and the second set of points includes: acquiring a first point acquired at a given time point of a plurality of time points from a first set of points; acquiring a second point acquired at a given point in time from a second set of points; and determining a given matching point pair of the plurality of matching point pairs based on the first point and the second point.

In some embodiments, determining the second external parameter of the millimeter wave radar includes: determining a relative relationship between the millimeter wave radar coordinate system and the laser radar coordinate system based on the first set of points and the second set of points; and calibrating a second external parameter of the millimeter wave radar based on the first external parameter of the laser wave radar and the relative relationship.

In some embodiments, millimeter wave radar and lidar are co-deployed in a roadside awareness system.

Fig. 7 illustrates a schematic block diagram of an apparatus 700 for content presentation according to some embodiments of the present disclosure. The apparatus 700 may be implemented as or included in an electronic device. The various modules/components in apparatus 700 may be implemented in hardware, software, firmware, or any combination thereof.

The apparatus 700 includes a first acquisition module configured to acquire a first set of points acquired by a millimeter wave radar to be calibrated with respect to a calibration tool. The apparatus 700 further comprises a second acquisition module configured to acquire a second set of points acquired by the calibrated lidar with respect to the calibration tool. The apparatus 700 further includes a determination module configured to determine a plurality of matching point pairs from the first set of points and the second set of points based on the acquisition times of the first set of points and the second set of points, a first point of the plurality of matching point pairs from the first set of points and a second point of the plurality of matching point pairs from the second set of points. The apparatus 700 further includes a calibration module configured to calibrate a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

In some embodiments, the first acquisition module and the second acquisition module are further configured to: the first and second sets are acquired at a plurality of time points, respectively. The determination module is further configured to: acquiring a first point acquired at a given time point of a plurality of time points from a first set of points; acquiring a second point acquired at a given point in time from a second set of points; and determining a given matching point pair of the plurality of matching point pairs based on the first point and the second point.

In some embodiments, the calibration module is further configured to: determining a relative relationship between the millimeter wave radar coordinate system and the laser radar coordinate system based on the first set of points and the second set of points; and calibrating a second external parameter of the millimeter wave radar based on the first external parameter of the laser wave radar and the relative relationship. This relative relationship is represented by the rotation matrix R and the translation matrix T mentioned in the foregoing.

Fig. 8 illustrates a block diagram of an electronic device 800 in which one or more embodiments of the disclosure may be implemented. It should be understood that the electronic device 800 illustrated in fig. 8 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 800 shown in fig. 5 may be used to implement the terminal device 110 of fig. 1.

As shown in fig. 8, the electronic device 800 is in the form of a general-purpose electronic device. Components of electronic device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 may be a real or virtual processor and is capable of performing various processes according to programs stored in the memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 800.

Electronic device 800 typically includes multiple computer storage media. Such a medium may be any available medium that is accessible by electronic device 800 including, but not limited to, volatile and non-volatile media, removable and non-removable media. The memory 820 may be volatile memory (e.g., registers, cache, random Access Memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 830 may be a removable or non-removable medium and may include a machine-readable medium such as a flash drive, a magnetic disk, or any other medium that may be capable of storing information and/or data (e.g., training data for training) and that may be accessed within electronic device 800.

The electronic device 800 may further include additional removable/non-removable, volatile/nonvolatile storage media. Although not shown in fig. 5, a magnetic disk drive for reading from or writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data medium interfaces. Memory 820 may include a computer program product 825 having one or more program modules configured to perform the various methods or acts of the various embodiments of the present disclosure.

The communication unit 840 enables communication with other electronic devices through a communication medium. Additionally, the functionality of the components of the electronic device 800 may be implemented in a single computing cluster or in multiple computing machines capable of communicating over a communications connection. Thus, the electronic device 800 may operate in a networked environment using logical connections to one or more other servers, a network Personal Computer (PC), or another network node.

The input device 850 may be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 860 may be one or more output devices such as a display, speakers, printer, etc. The electronic device 800 may also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., with one or more devices that enable a user to interact with the electronic device 800, or with any device (e.g., network card, modem, etc.) that enables the electronic device 800 to communicate with one or more other electronic devices, as desired, via the communication unit 840. Such communication may be performed via an input/output (I/O) interface (not shown).

According to an exemplary implementation of the present disclosure, a computer-readable storage medium having stored thereon computer-executable instructions, wherein the computer-executable instructions are executed by a processor to implement the method described above is provided. According to an exemplary implementation of the present disclosure, there is also provided a computer program product tangibly stored on a non-transitory computer-readable medium and comprising computer-executable instructions that are executed by a processor to implement the method described above.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, devices, and computer program products implemented according to the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The foregoing description of implementations of the present disclosure has been provided for illustrative purposes, is not exhaustive, and is not limited to the implementations disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various implementations described. The terminology used herein was chosen in order to best explain the principles of each implementation, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand each implementation disclosed herein.

Claims

1. A method for calibrating a millimeter wave radar, comprising:

Acquiring a first set of points acquired by a millimeter wave radar to be calibrated and related to a calibration tool;

Acquiring a second set of points acquired by the calibrated lidar with respect to the calibration tool;

Determining a plurality of matching point pairs from the first set of points and the second set of points based on acquisition times of the first set of points and the second set of points, a first point of the plurality of matching point pairs from the first set of points and a second point matching the first point from the second set of points; and

Calibrating a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

2. The method of claim 1, wherein the calibration tool comprises a calibration plate and a triangular reflecting cone, and an origin of the calibration plate and an origin of the triangular reflecting cone have a fixed positional relationship.

3. The method of claim 2, wherein the calibration tool is adapted to move away from or towards the millimeter wave radar along a projection of a normal of the millimeter wave radar on the ground or a parallel line of the projection for the millimeter wave radar and the lidar to acquire the first set of points and the second set of points, respectively.

4. A method according to claim 3, wherein the distance of movement of the calibration tool during acquisition is equal to or greater than a predetermined length.

5. The method of claim 1, wherein the first set of points and the second set of points are acquired at a plurality of points in time, and wherein determining a plurality of matching point pairs from the first set of points and the second set of points comprises:

acquiring a first point acquired at a given point in time of the plurality of points in time from the first set of points;

Acquiring a second point acquired at the given point in time from the second set of points; and

A given matching point pair of the plurality of matching point pairs is determined based on the first point and the second point.

6. The method of claim 5, wherein calibrating the second external parameter of the millimeter wave radar comprises:

determining a relative relationship between a coordinate system of the millimeter wave radar and a coordinate system of the lidar based on the first set of points and the second set of points; and

Calibrating a second external parameter of the millimeter wave radar based on the first external parameter of the laser wave radar and the relative relationship.

7. The method of claim 1, wherein the millimeter wave radar and the lidar are co-deployed in a roadside awareness system.

8. An apparatus for calibrating a millimeter wave radar, comprising:

A first acquisition module configured to acquire a first set of points acquired by the millimeter wave radar to be calibrated with respect to the calibration tool;

a second acquisition module configured to acquire a second set of points acquired by the calibrated lidar with respect to the calibration tool;

A determining module configured to determine a plurality of matching point pairs from the first set of points and the second set of points based on acquisition times of the first set of points and the second set of points, a first point of the plurality of matching point pairs from the first set of points and a second point matching the first point from the second set of points; and

And a calibration module configured to calibrate a second external parameter of the millimeter wave radar based on the plurality of matching point pairs and the calibrated first external parameter of the lidar.

9. The apparatus of claim 8, wherein the calibration tool comprises a calibration plate and a triangular reflecting cone, and an origin of the calibration plate and an origin of the triangular reflecting cone have a fixed positional relationship.

10. The apparatus of claim 9, wherein the calibration tool is adapted to move away from or towards the millimeter wave radar along a projection of a normal of the millimeter wave radar on the ground or a parallel line of the projection for the millimeter wave radar and the lidar to acquire the first set of points and the second set of points, respectively.

11. The apparatus of claim 10, wherein a distance of movement of the calibration tool during acquisition is equal to or greater than a predetermined length.

12. The apparatus of claim 8, wherein the first set of points and the second set of points are acquired at a plurality of points in time, and wherein the determination module is further configured to:

13. The apparatus of claim 12, wherein the calibration module is further configured to:

Determining a conversion relationship between the millimeter wave radar coordinate system and the lidar coordinate system based on the first set of points and the second set of points; and

Calibrating a second external parameter of the millimeter wave radar based on the first external parameter of the laser wave radar and the conversion relation.

14. The apparatus of claim 8, wherein the millimeter wave radar and the lidar are co-deployed roadside awareness systems.

15. An electronic device, comprising:

at least one processing unit; and

At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit cause the apparatus to perform the method of any one of claims 1 to 7.

16. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to any of claims 1 to 7.