WO2022242416A1 - 点云数据的生成方法和装置 - Google Patents
点云数据的生成方法和装置 Download PDFInfo
- Publication number
- WO2022242416A1 WO2022242416A1 PCT/CN2022/088312 CN2022088312W WO2022242416A1 WO 2022242416 A1 WO2022242416 A1 WO 2022242416A1 CN 2022088312 W CN2022088312 W CN 2022088312W WO 2022242416 A1 WO2022242416 A1 WO 2022242416A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- point cloud
- cloud set
- pseudo
- real
- coordinate information
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 37
- 238000001514 detection method Methods 0.000 claims description 23
- 230000014759 maintenance of location Effects 0.000 claims description 23
- 230000004927 fusion Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 abstract description 13
- 238000013473 artificial intelligence Methods 0.000 abstract description 8
- 238000013135 deep learning Methods 0.000 abstract description 8
- 238000012545 processing Methods 0.000 description 16
- 238000004364 calculation method Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 8
- 238000012544 monitoring process Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2016—Rotation, translation, scaling
Definitions
- the present disclosure relates to the field of artificial intelligence, in particular to computer vision and deep learning technology, which can be applied to automatic driving and intelligent traffic scenarios.
- Deep learning technology has achieved great success in the fields of computer vision and natural language processing in recent years.
- the point cloud 3D target detection task has also become a hot topic for deep learning researchers in recent years.
- the data collected by the radar is displayed and processed in the form of a point cloud.
- the disclosure provides a method, device, electronic equipment and storage medium for generating point cloud data.
- a method for generating point cloud data Collect the real point cloud collection of the target object based on lidar; collect the image of the target object, and generate a pseudo point cloud collection based on the collected image; fuse the real point cloud collection and pseudo point cloud collection to generate for model training
- the target point cloud collection of This application can make the far and near point clouds in the target point cloud set used for model training more balanced, which can better meet the training requirements, so as to provide the training accuracy of the model and facilitate the monitoring of far and near targets.
- a device for generating point cloud data is provided.
- an electronic device is provided.
- a non-transitory computer readable storage medium is provided.
- a computer program product is provided.
- the embodiment of the first aspect of the present disclosure proposes a method for generating point cloud data, including: collecting a real point cloud set of a target object based on lidar; collecting an image of the target object, and based on the collected images to generate a pseudo point cloud set; the real point cloud set and the pseudo point cloud set are fused to generate a target point cloud set for model training.
- the embodiment of the second aspect of the present disclosure proposes a device for generating point cloud data, including: a real point cloud set acquisition module, which is used to collect the real point cloud set of the target object based on lidar; the pseudo point cloud set The acquisition module is used to collect images of the target object, and based on the collected images, generate a pseudo point cloud set; the point cloud set fusion module is used to fuse the real point cloud set and the pseudo point cloud set , to generate a set of target point clouds for model training.
- the embodiment of the third aspect of the present disclosure provides an electronic device, including a memory and a processor.
- the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to implement the method for generating point cloud data according to the embodiment of the first aspect of the present disclosure.
- the embodiment of the fourth aspect of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to achieve the points described in the embodiment of the first aspect of the present disclosure. How to generate cloud data.
- the embodiment of the fifth aspect of the present disclosure proposes a computer program product, including a computer program, when the computer program is executed by a processor, it can realize the point cloud data as described in the embodiment of the first aspect of the present disclosure. generation method.
- FIG. 1 is a schematic diagram of a method for generating point cloud data according to an embodiment of the present disclosure
- Fig. 2 is an RGB map returned by a forward-looking camera of an automatic driving system according to an embodiment of the present disclosure
- FIG. 3 is lidar sparse point cloud data corresponding to an RGB image according to an embodiment of the present disclosure
- Fig. 4 is an RGB map returned by a forward-looking camera of an automatic driving system according to an embodiment of the present disclosure
- Fig. 5 is the pseudo lidar dense point cloud data corresponding to the RGB image according to an embodiment of the present disclosure
- FIG. 6 is a schematic diagram of a method for obtaining a first point cloud according to an embodiment of the present disclosure
- FIG. 7 is a schematic diagram of generating a target point cloud set according to an embodiment of the present disclosure.
- Fig. 8 is a schematic diagram of generating a target point cloud set according to an embodiment of the present disclosure
- FIG. 9 is a schematic diagram of obtaining the Euclidean distance from the first point cloud to the real point cloud set according to an embodiment of the present disclosure.
- FIG. 10 is a schematic diagram of a method for generating point cloud data according to an embodiment of the present disclosure
- Fig. 11 is a schematic diagram of an apparatus for generating point cloud data according to an embodiment of the present disclosure
- FIG. 12 is a schematic diagram of an electronic device according to an embodiment of the present disclosure.
- Image processing is a technology that uses a computer to analyze images to achieve the desired results. Also known as image processing. Image processing generally refers to digital image processing. A digital image refers to a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of this array are called pixels, and their values are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.
- Deep Learning is a new research direction in the field of Machine Learning (ML for short). It is introduced into machine learning to make it closer to the original goal-artificial intelligence. Deep learning is to learn the internal law and representation level of sample data. The information obtained in the learning process is of great help to the interpretation of data such as text, images and sounds. Its ultimate goal is to enable machines to have the ability to analyze and learn like humans, and to be able to recognize data such as text, images, and sounds. Deep learning is a complex machine learning algorithm that has achieved results in speech and image recognition that far exceed previous related techniques.
- Computer Vision is a science that studies how to make machines "see”. To put it further, it refers to the use of cameras and computers instead of human eyes to identify, track and measure targets, and further make graphics. Processing, so that the computer processing becomes an image that is more suitable for human observation or sent to the instrument for detection.
- computer vision studies related theories and technologies, trying to build artificial intelligence systems that can obtain 'information' from images or multidimensional data.
- the information referred to here refers to information that can be used to help make a "decision” as defined by Shannon. Because perception can be thought of as extracting information from sensory signals, computer vision can also be thought of as the science of how to make artificial systems "perceive" from images or multidimensional data.
- Artificial Intelligence is a subject that studies certain thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) technology.
- Artificial intelligence hardware technology generally includes computer vision technology, speech recognition technology, natural language processing technology and its learning/deep learning, big data processing technology, knowledge map technology and other major aspects.
- Fig. 1 is the flowchart of the generation method of point cloud data according to an embodiment of the present disclosure, as shown in Fig. 1, the generation method of this point cloud data comprises the following steps:
- Laser detection and ranging system also known as laser radar, consists of a transmitting system, a receiving system, information processing and other parts.
- LIDAR Light Detection and Ranging
- a point cloud is simply a number of points scattered in space. Each point contains three-dimensional coordinates (XYZ), laser reflection intensity (Intensity) or color information (Red Green Blue, RGB), and is the laser radar that emits laser light to objects or the ground. The signal is collected from the laser signal reflected by the object or the ground. Through joint calculation and deviation correction, the accurate spatial information of these points can be calculated.
- the point cloud data obtained by lidar can be used to make digital elevation models, 3D modeling, agricultural and forestry censuses, earthwork calculations, monitoring geological disasters or automatic driving systems.
- the lidar installed on the automatic driving vehicle can collect point cloud collections of objects and the ground in front of the automatic driving vehicle's field of view as real point cloud collections.
- an object in front may be used as a target object, such as a vehicle, a pedestrian, or a tree.
- Figure 2 is the RGB image returned by the forward-looking camera of the automatic driving system
- Figure 3 is the lidar sparse point cloud data corresponding to the RGB image.
- the forward-looking camera may include a forward-looking monocular RGB camera or a forward-looking binocular RGB camera.
- S102 Collect images of the target object, and generate a pseudo point cloud set based on the collected images.
- dense pseudo point cloud data can be obtained to assist the lidar to collect point cloud data of the target object.
- pseudo point cloud data can be acquired based on the depth image collected by the depth image acquisition device.
- the pixel depth of the acquired depth image is back-projected into a 3D point cloud to obtain Pseudo point cloud data.
- the image acquisition of the target object can be performed based on binocular vision, based on the principle of parallax and using imaging equipment to obtain two images of the object under test from different positions, and by calculating the position deviation between corresponding points of the image, a pseudo point cloud data.
- the image acquisition of the target object can be performed based on monocular vision, the relationship between the rotation and translation between the acquired images can be calculated, and the pseudo point cloud data can be obtained through the calculation based on the triangulation of matching points.
- a forward-looking monocular RGB camera or a forward-looking binocular RGB camera can be used to collect point clouds of objects and the ground in front of the field of view of the autonomous driving vehicle as a collection of pseudo point clouds .
- Figure 4 is the RGB image returned by the forward-looking camera of the automatic driving system
- Figure 5 is the pseudo-lidar dense point cloud data corresponding to the RGB image.
- the forward-looking camera may include a forward-looking monocular RGB camera or a forward-looking binocular RGB camera.
- the obtained real point cloud set and pseudo point cloud set are fused to obtain the target point cloud set. Since the pseudo point cloud set has a large amount of data, the real point cloud can be compared with the dense pseudo point cloud set. The set is added to the point cloud, so that the far and near point clouds in the target point cloud set used for model training are more balanced, which can better meet the training requirements, so as to provide the training accuracy of the model and facilitate the monitoring of far and near targets.
- the embodiment of the present application provides a method for generating point cloud data, which collects the real point cloud collection of the target object based on the laser radar; collects the image of the target object, and generates a pseudo point cloud collection based on the collected image; The cloud set and the pseudo point cloud set are fused to generate the target point cloud set for model training.
- This application can make the far and near point clouds in the target point cloud set used for model training more balanced, which can better meet the training requirements, so as to provide the training accuracy of the model and facilitate the monitoring of far and near targets.
- Fig. 6 is a flow chart of a method for generating point cloud data according to an embodiment of the present disclosure. As shown in Fig. 6 , before the real point cloud set and the pseudo point cloud set are fused to generate the target point cloud set for model training , including the following steps:
- the ground equation is calculated.
- the method for obtaining the surface equation may be a singular value decomposition (Singular Value Decomposition, SVD) method. After the ground equation is obtained, each point cloud in the pseudo point cloud set is used as the first point cloud, and the ground distance between each first point cloud and the ground equation is obtained according to the coordinate information of each first point cloud.
- SVD singular Value Decomposition
- a distance threshold is set, and if the ground distance between the first point cloud and the ground equation in the pseudo point cloud set is smaller than the set distance threshold, the first point cloud is removed from the pseudo point cloud set.
- the distance threshold as 10 as an example, the first point cloud whose ground distance between the first point cloud and the ground equation in the pseudo point cloud set is less than 10 is removed from the pseudo point cloud set.
- the ground point cloud is removed from the false point cloud set, reducing a large amount of invalid point cloud data, thereby reducing the calculation amount of the target detection model, and increasing the robustness and accuracy of the target detection model.
- Fig. 7 is a flowchart of a method for generating point cloud data according to an embodiment of the present disclosure.
- the real point cloud set and the pseudo point cloud set are fused to generate a target point cloud set for model training, which also includes the following steps:
- the splicing of point clouds can be understood as the process of obtaining a perfect coordinate transformation through calculation, and integrating the point cloud data under different viewing angles into the specified coordinate system through rigid transformations such as rotation and translation.
- the method based on local feature description can be used to stitch the real point cloud set and the pseudo point cloud set: by extracting the neighborhood geometric features of each point cloud in the real point cloud set and the pseudo point cloud set , quickly determine the corresponding relationship between the two points through the geometric features, and then calculate this relationship to obtain the transformation matrix.
- the geometric features of point clouds include many kinds, and the more common ones are Fast Point Feature Histgrams (FPFH).
- the accurate registration method can be used to stitch the real point cloud collection and the pseudo point cloud collection: accurate registration is to use the known initial transformation matrix, through the iterative closest point algorithm (Iterative Closest Point, ICP) and other calculations to obtain a more accurate solution.
- ICP iterative Closest Point
- the ICP algorithm calculates the distance between the corresponding points in the real point cloud set and the pseudo point cloud set, constructs a rotation and translation matrix, transforms the real point cloud set through the rotation and translation matrix, and calculates the mean square error of the transformed point cloud set.
- the algorithm ends. Otherwise, continue to repeat iterations until the error meets the threshold condition or the number of iterations is terminated.
- Each point cloud in the real point cloud collection collected by the lidar is used as the second point cloud, and the center point coordinates of the real point cloud collection can be determined according to the coordinate information of all the second point clouds. Calculate the Euclidean distance between the coordinate information of each first point cloud in the pseudo point cloud set and the determined center point coordinates of the real point cloud set, and obtain the distance between each first point cloud and the center point coordinates of the real point cloud set Euclidean distance.
- the real point cloud set and the pseudo point cloud set are spliced to generate the candidate point cloud set, there are many point cloud data, which will cause a large amount of calculation. In order to reduce the amount of calculation, it can be calculated according to each first point cloud
- the Euclidean distance of the central point coordinates of the set, part of the point cloud data in the candidate point cloud set is removed, and the point cloud set after part of the point cloud data is removed is used as the target point cloud set.
- a down-sampling method may be used to remove part of the point cloud data in the candidate point cloud set.
- the real point cloud set and the pseudo point cloud set are spliced to increase the accuracy of the target detection model, and the point cloud is selected from the candidate point cloud set as the target point cloud set instead of using all point cloud data. The amount of calculation is reduced.
- FIG. 8 is an exemplary schematic diagram of selecting a point cloud from a candidate point cloud set based on the Euclidean distance of the first point cloud to generate a target point cloud set, as shown in FIG. 8 , including the following steps :
- a retention probability can be configured for each first point cloud in the pseudo point cloud set according to the Euclidean distance from the first point cloud to the real point cloud set.
- the retention probability of the first point cloud configuration with the larger Euclidean distance of the point cloud set is greater, and the retention probability of the first point cloud configuration with smaller Euclidean distance from the real point cloud set is smaller.
- the retention probability of the first point cloud configuration with the largest Euclidean distance from the real point cloud set may be 0.98
- the retention probability of the first point cloud configuration with the smallest Euclidean distance from the real point cloud set may be 0.22.
- a retention probability may be pre-configured for each second point cloud in the real point cloud collection collected by the lidar.
- the second point cloud in the real point cloud set can be preconfigured to be close to 1 Or a retention probability equal to 1. For example, a retention probability of 0.95 may be uniformly pre-configured for the second point cloud in the set of real point clouds.
- each first point cloud and second point cloud can be Cloud retention probability, part of the point cloud data in the candidate point cloud set is removed, and the point cloud set after part of the point cloud data is removed is used as the target point cloud set.
- a random downsampling method may be used to remove part of the point cloud data in the candidate point cloud set generated by splicing the real point cloud set and the pseudo point cloud set, wherein the probability used for random downsampling is the reserved probability.
- the probability used for random downsampling is the reserved probability.
- random downsampling is performed on the candidate point cloud set according to the retention probability of each first point cloud and the second point cloud, which reduces the amount of calculation, and at the same time makes the far and near points in the target point cloud set used for model training
- the point cloud is more balanced, which can better meet the training requirements.
- FIG. 9 is a flowchart of a method for generating point cloud data according to an embodiment of the present disclosure. As shown in FIG. 9 , based on the coordinate information of each first point cloud in the pseudo point cloud set and the coordinate information of each second point cloud in the real point cloud set, and obtain the Euclidean distance from the first point cloud to the real point cloud set, including the following steps:
- the coordinate information of each second point cloud in the real point cloud set is obtained, and the center point coordinate information of the real point cloud set is determined according to the coordinate information of all the second point clouds.
- the coordinate information of all the second point clouds can be averaged to obtain an average coordinate information, and this average coordinate information can be used as the center of the real point cloud set Point coordinate information.
- the mass point coordinate information of the real point cloud set when acquiring the center point coordinates of the real point cloud set, can be calculated, and the mass point coordinate information is used as the center point coordinate information of the real point cloud set.
- the Euclidean distance from the first point cloud to the coordinates of the center point is determined, which lays the foundation for the retention probability configuration of the first point cloud and facilitates calculation and reduce the amount of computation.
- FIG. 10 is a flow chart of a method for generating point cloud data according to an embodiment of the present disclosure. As shown in FIG. 10 , the method for generating point cloud data includes the following steps:
- the embodiment of the present application provides a method for generating point cloud data, by collecting the real point cloud collection of the target object based on the laser radar; the image collection device collects the image of the target object, and generates a pseudo point cloud collection based on the collected image ; Fuse the real point cloud set and the pseudo point cloud set to generate the target point cloud set for model training.
- This application can make the far and near point clouds in the target point cloud set used for model training more balanced, which can better meet the training requirements, so as to provide the training accuracy of the model and facilitate the monitoring of far and near targets.
- Fig. 11 is a structural diagram of an apparatus 1100 for generating point cloud data according to an embodiment of the present disclosure.
- the generation device 1100 of point cloud data comprises:
- the real point cloud set acquisition module 1101 is used to collect the real point cloud set of the target object based on lidar;
- Pseudo-point cloud set acquisition module 1102 used for image acquisition of the target object, and based on the collected image, generate a pseudo-point cloud set;
- the point cloud set fusion module 1103 is configured to fuse the real point cloud set and the pseudo point cloud set to generate a target point cloud set for model training.
- the embodiment of the present application provides a point cloud data generation device, which collects the real point cloud collection of the target object based on the laser radar; the image collection device collects the image of the target object, and generates a pseudo point cloud collection based on the collected image ; Fuse the real point cloud set and the pseudo point cloud set to generate the target point cloud set for model training.
- This application can make the far and near point clouds in the target point cloud set used for model training more balanced, which can better meet the training requirements, so as to provide the training accuracy of the model and facilitate the monitoring of far and near targets.
- the point cloud set fusion module 1103 is specifically configured to: obtain the first point cloud and ground The ground distance of the equation; remove the first point cloud whose ground distance is less than the set distance threshold from the pseudo point cloud set.
- the point cloud set fusion module 1103 is also used to: splice the real point cloud set and the pseudo point cloud set to generate a candidate point cloud set; based on the pseudo point cloud The coordinate information of each first point cloud in the set and the coordinate information of each second point cloud in the real point cloud set are obtained to obtain the Euclidean distance from the first point cloud to the real point cloud set; based on the Euclidean distance of the first point cloud, Point clouds are selected from the candidate point cloud set to generate the target point cloud set.
- the point cloud set fusion module 1103 is also configured to: generate the retention probability of the first point cloud based on the Euclidean distance of the first point cloud; obtain the second point cloud Pre-configured retention probability; randomly downsampling the candidate point cloud set to obtain the target point cloud set, where the probability used for random downsampling is the retention probability.
- the point cloud set fusion module 1103 is also used to: obtain the coordinate information of the second point cloud, obtain the center point coordinate information of the real point cloud set; The coordinate information of the point cloud and the coordinate information of the center point determine the Euclidean distance.
- the device 1100 for generating point cloud data further includes: a model training module 1104, configured to use the set of target point clouds to train the constructed 3D target detection model to generate A trained 3D object detection model.
- the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
- FIG. 12 shows a schematic block diagram of an example electronic device 1200 that may be used to implement embodiments of the present disclosure.
- Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers.
- Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices.
- the components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
- the device 1200 includes a computing unit 1201 that can execute according to a computer program stored in a read-only memory (ROM) 1202 or loaded from a storage unit 1208 into a random-access memory (RAM) 1203. Various appropriate actions and treatments. In the RAM 1203, various programs and data necessary for the operation of the device 1200 can also be stored.
- the computing unit 1201, ROM 1202, and RAM 1203 are connected to each other through a bus 1204.
- An input/output (I/O) interface 1205 is also connected to the bus 1204 .
- the I/O interface 1205 includes: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a magnetic disk, an optical disk, etc. ; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, and the like.
- the communication unit 1209 allows the device 1200 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.
- the computing unit 1201 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 1201 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc.
- the computing unit 1201 executes various methods and processes described above, such as a method for generating point cloud data.
- the method for generating point cloud data may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1208.
- part or all of the computer program may be loaded and/or installed on the device 1200 via the ROM 1202 and/or the communication unit 1209.
- the computer program When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the method for generating point cloud data described above can be performed.
- the computing unit 1201 may be configured in any other appropriate way (for example, by means of firmware) to execute the method for generating point cloud data.
- Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips Implemented in a system of systems (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof.
- FPGAs field programmable gate arrays
- ASICs application specific integrated circuits
- ASSPs application specific standard products
- SOC system of systems
- CPLD load programmable logic device
- computer hardware firmware, software, and/or combinations thereof.
- programmable processor can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.
- Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special purpose computer, or other programmable data processing devices, so that the program codes, when executed by the processor or controller, make the functions/functions specified in the flow diagrams and/or block diagrams Action is implemented.
- the program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
- a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
- a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
- a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
- machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
- RAM random access memory
- ROM read only memory
- EPROM or flash memory erasable programmable read only memory
- CD-ROM compact disk read only memory
- magnetic storage or any suitable combination of the foregoing.
- the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which the user can provide input to the computer.
- a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
- a keyboard and pointing device eg, a mouse or a trackball
- Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.
- the systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system.
- the components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.
- a computer system may include clients and servers.
- Clients and servers are generally remote from each other and typically interact through a communication network.
- the relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.
- the server can be a cloud server, also known as a cloud computing server or a cloud host. ), there are defects such as high management difficulty and weak business scalability.
- the server can also be a server of a distributed system, or a server combined with a block chain.
- steps may be reordered, added or deleted using the various forms of flow shown above.
- each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Architecture (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
Description
Claims (15)
- 一种点云数据的生成方法,包括:基于激光雷达采集目标对象的真实点云集合;对所述目标对象进行图像采集,并基于采集的图像,生成伪点云集合;对所述真实点云集合和所述伪点云集合进行融合,生成用于模型训练的目标点云集合。
- 根据权利要求1所述的方法,其中,所述对所述真实点云集合和所述伪点云集合进行融合,生成用于模型训练的目标点云集合,还包括:基于所述伪点云集合中每个第一点云的坐标信息,获取所述第一点云与地面方程的地面距离;从所述伪点云集合中剔除所述地面距离小于设定距离阈值的第一点云。
- 根据权利要求1或2所述的方法,其中,所述对所述真实点云集合和所述伪点云集合进行融合,生成用于模型训练的目标点云集合,还包括:将所述真实点云集合和所述伪点云集合进行拼接,生成候选点云集合;基于所述伪点云集合中每个第一点云的坐标信息和所述真实点云集合中每个第二点云的坐标信息,获取所述第一点云到所述真实点云集合的欧式距离;基于所述第一点云的欧式距离,从所述候选点云集合中选取点云,以生成所述目标点云集合。
- 根据权利要求3所述的方法,其中,所述基于所述第一点云的欧式距离,从所述候选点云集合中选取点云,以生成所述目标点云集合,包括:基于所述第一点云的欧式距离,生成所述第一点云的保留概率;获取所述第二点云预先配置的保留概率;对所述候选点云集合进行随机降采样,得到所述目标点云集合,其中,所述随机降采样使用的概率为所述保留概率。
- 根据权利要求3所述的方法,其中,所述基于所述伪点云集合中每个第一点云的坐标信息和所述真实点云集合中每个第二点云的坐标信息,获取所述第一点云到所述真实点云集合的欧式距离,包括:获取所述第二点云的坐标信息,获取所述真实点云集合的中心点坐标信息;基于所述第一点云的坐标信息和所述中心点坐标信息,确定所述欧式距离。
- 根据权利要求1所述的方法,其中,所述生成用于模型训练的目标点云集合进一步包括:利用所述目标点云集合,训练构建的3D目标检测模型,以生成训练好的3D目标检测模型。
- 一种点云数据的生成装置,包括:真实点云集合获取模块,用于基于激光雷达采集目标对象的真实点云集合;伪点云集合获取模块,用于对所述目标对象进行图像采集,并基于采集的图像,生成伪点云集合;点云集合融合模块,用于对所述真实点云集合和所述伪点云集合进行融合,生成用于模型训练的目标点云集合。
- 根据权利要求7所述的装置,其中,所述点云集合融合模块,用于:基于所述伪点云集合中每个第一点云的坐标信息,获取所述第 一点云与地面方程的地面距离;从所述伪点云集合中剔除所述地面距离小于设定距离阈值的第一点云。
- 根据权利要求7或8任一项所述的装置,其中,所述点云集合融合模块,还用于:将所述真实点云集合和所述伪点云集合进行拼接,生成候选点云集合;基于所述伪点云集合中每个第一点云的坐标信息和所述真实点云集合中每个第二点云的坐标信息,获取所述第一点云到所述真实点云集合的欧式距离;基于所述第一点云的欧式距离,从所述候选点云集合中选取点云,以生成所述目标点云集合。
- 根据权利要求9所述的装置,其中,所述点云集合融合模块,还用于:基于所述第一点云的欧式距离,生成所述第一点云的保留概率;获取所述第二点云预先配置的保留概率;对所述候选点云集合进行随机降采样,得到所述目标点云集合,其中,所述随机降采样使用的概率为所述保留概率。
- 根据权利要求9所述的装置,其中,所述点云集合融合模块,还用于:获取所述第二点云的坐标信息,获取所述真实点云集合的中心点坐标信息;基于所述第一点云的坐标信息和所述中心点坐标信息,确定所述欧式距离。
- 根据权利要求7所述的装置,其中,所述装置还包括:模型训练模块,用于利用所述目标点云集合,训练构建的3D 目标检测模型,以生成训练好的3D目标检测模型。
- 一种电子设备,包括:至少一个处理器;以及与所述至少一个处理器通信连接的存储器;其中,所述存储器存储有可被所述至少一个处理器执行的指令,所述指令被所述至少一个处理器执行,以使所述至少一个处理器能够执行权利要求1-6中任一项所述的方法。
- 一种存储有计算机指令的非瞬时计算机可读存储介质,其中,所述计算机指令用于使所述计算机执行根据权利要求1-6中任一项所述的方法。
- 一种计算机程序产品,包括计算机程序,所述计算机程序在被处理器执行时实现根据权利要求1-6中任一项所述的步骤。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020237008339A KR20230042383A (ko) | 2021-05-21 | 2022-04-21 | 포인트 클라우드 데이터 생성 방법 및 장치 |
JP2022561443A JP2023529527A (ja) | 2021-05-21 | 2022-04-21 | 点群データの生成方法及び装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110556351.0A CN113362444B (zh) | 2021-05-21 | 2021-05-21 | 点云数据的生成方法、装置、电子设备及存储介质 |
CN202110556351.0 | 2021-05-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022242416A1 true WO2022242416A1 (zh) | 2022-11-24 |
Family
ID=77526597
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/088312 WO2022242416A1 (zh) | 2021-05-21 | 2022-04-21 | 点云数据的生成方法和装置 |
Country Status (4)
Country | Link |
---|---|
JP (1) | JP2023529527A (zh) |
KR (1) | KR20230042383A (zh) |
CN (1) | CN113362444B (zh) |
WO (1) | WO2022242416A1 (zh) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116168366A (zh) * | 2023-01-19 | 2023-05-26 | 北京百度网讯科技有限公司 | 点云数据生成方法、模型训练方法、目标检测方法和装置 |
CN116222577A (zh) * | 2023-04-27 | 2023-06-06 | 苏州浪潮智能科技有限公司 | 闭环检测方法、训练方法、***、电子设备及存储介质 |
CN116758006A (zh) * | 2023-05-18 | 2023-09-15 | 广州广检建设工程检测中心有限公司 | 脚手架质量检测方法及装置 |
CN117058464A (zh) * | 2023-08-31 | 2023-11-14 | 强联智创(北京)科技有限公司 | 对生成健康血管表面的生成模型进行训练的方法及设备 |
CN117115225A (zh) * | 2023-09-01 | 2023-11-24 | 安徽羽亿信息科技有限公司 | 一种自然资源智慧综合信息化管理平台 |
CN117173342A (zh) * | 2023-11-02 | 2023-12-05 | 中国海洋大学 | 基于水下单双目相机的自然光下移动三维重建装置及方法 |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113362444B (zh) * | 2021-05-21 | 2023-06-16 | 北京百度网讯科技有限公司 | 点云数据的生成方法、装置、电子设备及存储介质 |
CN115235482A (zh) * | 2021-09-28 | 2022-10-25 | 上海仙途智能科技有限公司 | 地图更新方法、装置、计算机设备及介质 |
CN115830262B (zh) * | 2023-02-14 | 2023-05-26 | 济南市勘察测绘研究院 | 一种基于对象分割的实景三维模型建立方法及装置 |
KR102573935B1 (ko) * | 2023-04-27 | 2023-09-04 | 주식회사 루트릭스 | 수목 데이터 처리 방법 및 장치 |
CN116577350A (zh) * | 2023-07-13 | 2023-08-11 | 北京航空航天大学杭州创新研究院 | 物料表面毛球点云采集装置和物料表面毛球数据采集方法 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111340797A (zh) * | 2020-03-10 | 2020-06-26 | 山东大学 | 一种激光雷达与双目相机数据融合检测方法及*** |
CN112001958A (zh) * | 2020-10-28 | 2020-11-27 | 浙江浙能技术研究院有限公司 | 基于有监督单目深度估计的虚拟点云三维目标检测方法 |
CN112419494A (zh) * | 2020-10-09 | 2021-02-26 | 腾讯科技(深圳)有限公司 | 用于自动驾驶的障碍物检测、标记方法、设备及存储介质 |
CN113362444A (zh) * | 2021-05-21 | 2021-09-07 | 北京百度网讯科技有限公司 | 点云数据的生成方法、装置、电子设备及存储介质 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180136332A1 (en) * | 2016-11-15 | 2018-05-17 | Wheego Electric Cars, Inc. | Method and system to annotate objects and determine distances to objects in an image |
CN108230379B (zh) * | 2017-12-29 | 2020-12-04 | 百度在线网络技术(北京)有限公司 | 用于融合点云数据的方法和装置 |
US11378718B2 (en) * | 2018-10-02 | 2022-07-05 | Robert S. Phelan | Unmanned aerial vehicle system and methods |
US11494930B2 (en) * | 2019-06-17 | 2022-11-08 | SafeAI, Inc. | Techniques for volumetric estimation |
US11602974B2 (en) * | 2019-08-29 | 2023-03-14 | Here Global B.V. | System and method for generating map data associated with road objects |
CN111161202A (zh) * | 2019-12-30 | 2020-05-15 | 上海眼控科技股份有限公司 | 车辆行为信息获取方法、装置、计算机设备和存储介质 |
CN111292369B (zh) * | 2020-03-10 | 2023-04-28 | 中车青岛四方车辆研究所有限公司 | 激光雷达的伪点云数据生成方法 |
CN111652855B (zh) * | 2020-05-19 | 2022-05-06 | 西安交通大学 | 一种基于存活概率的点云精简方法 |
CN111649752B (zh) * | 2020-05-29 | 2021-09-21 | 北京四维图新科技股份有限公司 | 拥堵路段的地图数据处理方法、装置以及设备 |
CN111739005B (zh) * | 2020-06-22 | 2023-08-08 | 北京百度网讯科技有限公司 | 图像检测方法、装置、电子设备及存储介质 |
CN111784659A (zh) * | 2020-06-29 | 2020-10-16 | 北京百度网讯科技有限公司 | 图像检测的方法、装置、电子设备以及存储介质 |
CN111915746B (zh) * | 2020-07-16 | 2022-09-13 | 北京理工大学 | 一种基于弱标注的三维点云目标检测方法及标注工具 |
-
2021
- 2021-05-21 CN CN202110556351.0A patent/CN113362444B/zh active Active
-
2022
- 2022-04-21 WO PCT/CN2022/088312 patent/WO2022242416A1/zh active Application Filing
- 2022-04-21 JP JP2022561443A patent/JP2023529527A/ja active Pending
- 2022-04-21 KR KR1020237008339A patent/KR20230042383A/ko not_active Application Discontinuation
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN111340797A (zh) * | 2020-03-10 | 2020-06-26 | 山东大学 | 一种激光雷达与双目相机数据融合检测方法及*** |
CN112419494A (zh) * | 2020-10-09 | 2021-02-26 | 腾讯科技(深圳)有限公司 | 用于自动驾驶的障碍物检测、标记方法、设备及存储介质 |
CN112001958A (zh) * | 2020-10-28 | 2020-11-27 | 浙江浙能技术研究院有限公司 | 基于有监督单目深度估计的虚拟点云三维目标检测方法 |
CN113362444A (zh) * | 2021-05-21 | 2021-09-07 | 北京百度网讯科技有限公司 | 点云数据的生成方法、装置、电子设备及存储介质 |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116168366A (zh) * | 2023-01-19 | 2023-05-26 | 北京百度网讯科技有限公司 | 点云数据生成方法、模型训练方法、目标检测方法和装置 |
CN116168366B (zh) * | 2023-01-19 | 2023-12-05 | 北京百度网讯科技有限公司 | 点云数据生成方法、模型训练方法、目标检测方法和装置 |
CN116222577A (zh) * | 2023-04-27 | 2023-06-06 | 苏州浪潮智能科技有限公司 | 闭环检测方法、训练方法、***、电子设备及存储介质 |
CN116758006A (zh) * | 2023-05-18 | 2023-09-15 | 广州广检建设工程检测中心有限公司 | 脚手架质量检测方法及装置 |
CN116758006B (zh) * | 2023-05-18 | 2024-02-06 | 广州广检建设工程检测中心有限公司 | 脚手架质量检测方法及装置 |
CN117058464A (zh) * | 2023-08-31 | 2023-11-14 | 强联智创(北京)科技有限公司 | 对生成健康血管表面的生成模型进行训练的方法及设备 |
CN117058464B (zh) * | 2023-08-31 | 2024-06-11 | 强联智创(北京)科技有限公司 | 对生成健康血管表面的生成模型进行训练的方法及设备 |
CN117115225A (zh) * | 2023-09-01 | 2023-11-24 | 安徽羽亿信息科技有限公司 | 一种自然资源智慧综合信息化管理平台 |
CN117115225B (zh) * | 2023-09-01 | 2024-04-30 | 安徽羽亿信息科技有限公司 | 一种自然资源智慧综合信息化管理平台 |
CN117173342A (zh) * | 2023-11-02 | 2023-12-05 | 中国海洋大学 | 基于水下单双目相机的自然光下移动三维重建装置及方法 |
Also Published As
Publication number | Publication date |
---|---|
CN113362444B (zh) | 2023-06-16 |
JP2023529527A (ja) | 2023-07-11 |
CN113362444A (zh) | 2021-09-07 |
KR20230042383A (ko) | 2023-03-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022242416A1 (zh) | 点云数据的生成方法和装置 | |
JP6745328B2 (ja) | 点群データを復旧するための方法及び装置 | |
JP7106665B2 (ja) | 単眼深度推定方法およびその装置、機器ならびに記憶媒体 | |
CN110427917B (zh) | 用于检测关键点的方法和装置 | |
Schulter et al. | Learning to look around objects for top-view representations of outdoor scenes | |
WO2019161813A1 (zh) | 动态场景的三维重建方法以及装置和***、服务器、介质 | |
JP2021114296A (ja) | 近赤外画像の生成方法、近赤外画像の生成装置、生成ネットワークの訓練方法、生成ネットワークの訓練装置、電子機器、記憶媒体及びコンピュータプログラム | |
JP6471448B2 (ja) | 視差深度画像のノイズ識別方法及びノイズ識別装置 | |
WO2022257487A1 (zh) | 深度估计模型的训练方法, 装置, 电子设备及存储介质 | |
CN108230384B (zh) | 图像深度计算方法、装置、存储介质和电子设备 | |
WO2019079766A1 (en) | SYSTEM, APPARATUS, METHOD FOR DATA PROCESSING, AND INFORMATION MEDIUM | |
WO2019169884A1 (zh) | 基于深度信息的图像显著性检测方法和装置 | |
US11676294B2 (en) | Passive and single-viewpoint 3D imaging system | |
CN115861601B (zh) | 一种多传感器融合感知方法及装置 | |
CN115330940B (zh) | 一种三维重建方法、装置、设备和介质 | |
WO2022237821A1 (zh) | 生成交通标志线地图的方法、设备和存储介质 | |
WO2023216460A1 (zh) | 基于鸟瞰图的多视角3d目标检测方法、存储器及*** | |
US20230401826A1 (en) | Perception network and data processing method | |
Liu et al. | Microscopic 3D reconstruction based on point cloud data generated using defocused images | |
CN108352061B (zh) | 用于产生表示像素光束的数据的装置和方法 | |
CN117745944A (zh) | 预训练模型确定方法、装置、设备以及存储介质 | |
EP4086853A2 (en) | Method and apparatus for generating object model, electronic device and storage medium | |
US20230115765A1 (en) | Method and apparatus of transferring image, and method and apparatus of training image transfer model | |
CN113656629B (zh) | 视觉定位方法、装置、电子设备及存储介质 | |
CN113129352B (zh) | 一种稀疏光场重建方法及装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2022561443 Country of ref document: JP Kind code of ref document: A |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22803740 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20237008339 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22803740 Country of ref document: EP Kind code of ref document: A1 |