CN113989445B - Three-dimensional scene reconstruction method, device and system and computer readable storage medium - Google Patents
Three-dimensional scene reconstruction method, device and system and computer readable storage medium Download PDFInfo
- Publication number
- CN113989445B CN113989445B CN202111624638.9A CN202111624638A CN113989445B CN 113989445 B CN113989445 B CN 113989445B CN 202111624638 A CN202111624638 A CN 202111624638A CN 113989445 B CN113989445 B CN 113989445B
- Authority
- CN
- China
- Prior art keywords
- data
- scene
- control
- point cloud
- scene reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention provides a three-dimensional scene reconstruction method, a device, a system and a computer readable storage medium, which relate to the technical field of surveying and comprise the following steps: providing an angle adjustment control, a survey control and a scene reconstruction control through a graphical user interface; controlling the data surveying equipment to adjust the surveying angle in response to the triggering operation for the angle adjustment control; controlling the data surveying equipment to acquire visual data of a scene to be reconstructed in response to a triggering operation for the surveying control; wherein the visual data comprises image data and/or point cloud data; and responding to the triggering operation aiming at the scene reconstruction control, and controlling the data surveying equipment to send the visual data to the server so as to determine a scene reconstruction result according to the visual data through the server. The method can effectively reduce the operation difficulty of the three-dimensional scene reconstruction and can also obviously improve the three-dimensional scene reconstruction effect.
Description
Technical Field
The invention relates to the technical field of surveying, in particular to a three-dimensional scene reconstruction method, a three-dimensional scene reconstruction device, a three-dimensional scene reconstruction system and a computer-readable storage medium.
Background
At present, the data source of three-dimensional scene reconstruction is mainly point cloud data. The three-dimensional reconstruction device provided by the related technology aims at the problem of cooperative work of a laser radar, a camera and an inertial navigation system, and combines and fixes a plurality of sensors based on a 3D (3-dimension) printing technology, and has high operation difficulty, so that data acquired by the three-dimensional reconstruction device cannot meet the user requirements easily; the related technology also provides a camera device for collecting land data, wherein an unmanned aerial vehicle is provided with a camera for collecting the ground data, the camera device for collecting the land data adopts a quadrotor unmanned aerial vehicle, but the quadrotor unmanned aerial vehicle has the defects of small load, short endurance, limited camera which can be carried and the like, and the camera device for collecting the land data cannot acquire point cloud data, so that the three-dimensional reconstruction effect is poor; the related technology also provides a distribution line operation scene three-dimensional reconstruction method based on point cloud, which comprises the steps of processing initial point cloud to obtain complete three-dimensional point cloud of parts, namely establishing an off-line model base, then collecting the point cloud in real time, and sequentially registering the point cloud of each part with the model base model to complete three-dimensional reconstruction. In summary, the existing three-dimensional scene reconstruction method has the problems of high operation difficulty, poor three-dimensional reconstruction effect and the like.
Disclosure of Invention
In view of this, the present invention provides a method, an apparatus, a system and a computer readable storage medium for reconstructing a three-dimensional scene, which can effectively reduce the operation difficulty of reconstructing the three-dimensional scene and significantly improve the reconstruction effect of the three-dimensional scene.
In a first aspect, an embodiment of the present invention provides a three-dimensional scene reconstruction method, where the method is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system further includes a data survey device in communication connection with the mobile control terminal, and the mobile control terminal provides a graphical user interface, where the method includes: providing an angle adjustment control, a survey control, and a scene reconstruction control through the graphical user interface; controlling the data survey equipment to adjust a survey angle in response to a triggering operation directed to the angle adjustment control; controlling the data survey equipment to acquire visual data of a scene to be reconstructed in response to a triggering operation directed to the survey control; wherein the visual data comprises image data and/or point cloud data; and responding to the triggering operation aiming at the scene reconstruction control, controlling the data surveying equipment to send the visual data to a server so as to determine a scene reconstruction result according to the visual data through the server.
In one embodiment, the data surveying equipment comprises a liftable supporting structure, a control host and a rotary table, wherein the control host is fixedly arranged on the liftable supporting structure, and the rotary table is connected with the control host; the step of controlling the data survey equipment to adjust a survey angle comprises: sending an angle adjustment instruction to the control host machine so as to drive the rotary table to rotate through the control host machine based on a survey angle carried by the angle adjustment instruction; wherein the survey angle comprises a pitch angle and/or a horizontal angle.
In one embodiment, the data surveying equipment further comprises a cradle head arranged on the rotary table, a sensor group is arranged in the cradle head and connected with the control host, and the sensor group comprises a binocular camera and a laser radar sensor; the step of controlling the data survey device to collect visual data of a scene to be reconstructed comprises: sending a surveying instruction to the control host to control the binocular camera to acquire image data of a scene to be reconstructed and control the laser radar sensor to acquire point cloud data of the scene to be reconstructed through the control host.
In one embodiment, prior to the step of responding to the triggering operation for the survey control, the method further comprises: providing a calibration control through the graphical user interface; and responding to the trigger operation aiming at the calibration control, and controlling the data surveying equipment to calibrate the binocular camera and the laser radar sensor according to the position information of the binocular camera and the position information of the laser radar sensor.
In one embodiment, prior to the step of responding to the triggering operation for the scene reconstruction control, the method further comprises: in response to a file management operation directed to the visual data, controlling the data surveying device to perform a file management operation on the visual data; wherein the file management operation at least comprises a viewing operation, a copying operation, a moving operation and a deleting operation; if the file management operation is the viewing operation, providing the visual data through the graphical user interface; wherein the graphical user interface comprises an image display area and/or a point cloud display area; and responding to the display operation aiming at the visual data, and displaying the visual data with a display effect corresponding to the display operation.
In a second aspect, an embodiment of the present invention further provides a three-dimensional scene reconstruction method, where the method is applied to a server, and the method includes: acquiring visual data of a scene to be reconstructed; wherein the visual data is obtained by surveying according to any one of the three-dimensional scene reconstruction methods provided by the first aspect, and the visual data comprises image data and/or point cloud data; fitting a first target object contained in the scene to be reconstructed according to the point cloud data; carrying out image identification processing on the image data to obtain a second target object contained in the scene to be reconstructed; and determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object.
In one embodiment, the step of fitting the point cloud data to a first target object included in the scene to be reconstructed includes: preprocessing the point cloud data to obtain target point cloud data; wherein the preprocessing at least comprises splicing operation, filtering processing and down-sampling processing; clustering and dividing the target point cloud data to obtain at least one point cloud cluster; and respectively performing fitting processing on each point cloud cluster to obtain a first target object contained in the scene to be reconstructed.
In one embodiment, the step of determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object includes: establishing a matching relation between the first target object and the second target object according to the position information of the first target object and the position information of the second target object; and determining a scene reconstruction result of the scene to be reconstructed according to the matching relation.
In a third aspect, an embodiment of the present invention further provides a three-dimensional scene reconstruction apparatus, where the apparatus is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system further includes a data survey device in communication connection with the mobile control terminal, the mobile control terminal provides a graphical user interface, and the apparatus includes: the control providing module is used for providing an angle adjusting control, a surveying control and a scene rebuilding control through the graphical user interface; the angle adjusting module is used for responding to the triggering operation aiming at the angle adjusting control and controlling the data surveying equipment to adjust the surveying angle; the data acquisition module is used for responding to the triggering operation aiming at the surveying control and controlling the data surveying equipment to acquire the visual data of the scene to be reconstructed; wherein the visual data comprises image data and/or point cloud data; and the data sending module is used for responding to the triggering operation aiming at the scene reconstruction control, controlling the data surveying equipment to send the visual data to a server so as to determine a scene reconstruction result according to the visual data through the server.
In a fourth aspect, an embodiment of the present invention further provides a three-dimensional scene reconstruction apparatus, where the apparatus is applied to a server, and the apparatus includes: the data acquisition module is used for acquiring visual data of a scene to be reconstructed; wherein the visual data is obtained by surveying according to any one of the three-dimensional scene reconstruction methods provided by the first aspect, and the visual data comprises image data and/or point cloud data; the point cloud fitting module is used for fitting a first target object contained in the scene to be reconstructed according to the point cloud data; the image identification module is used for carrying out image identification processing on the image data to obtain a second target object contained in the scene to be reconstructed; and the scene reconstruction module is used for determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object.
In a fifth aspect, embodiments of the present invention further provide a system, including a processor and a memory, where the memory stores computer-executable instructions capable of being executed by the processor, and the processor executes the computer-executable instructions to implement the method of any one of the first aspect, or executes the computer-executable instructions to implement the method of any one of the second aspect.
In a sixth aspect, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any one of the first aspect, or cause the processor to implement the method of any one of the second aspect.
The three-dimensional scene reconstruction method, the device, the system and the computer readable storage medium provided by the embodiment of the invention are applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system also comprises a data survey device in communication connection with the mobile control terminal, the mobile control terminal provides a graphical user interface, an angle adjustment control, a survey control and a scene reconstruction control are firstly provided through the graphical user interface, then the data survey device is controlled to adjust a survey angle in response to the triggering operation aiming at the angle adjustment control, then the data survey device is controlled to collect the visual data (including image data and/or point cloud data) of a scene to be reconstructed in response to the triggering operation aiming at the survey control, finally the data survey device is controlled to send the visual data to a server in response to the triggering operation aiming at the scene reconstruction control, to determine scene reconstruction results from the visual data by the server. The method establishes the communication connection between the mobile control terminal and the data surveying equipment, so that the data surveying equipment is visually controlled by the mobile control terminal, the data surveying equipment executes corresponding actions according to the triggering operation of a user on each virtual control, and the operation difficulty of three-dimensional scene reconstruction is obviously reduced.
The three-dimensional scene reconstruction method, the three-dimensional scene reconstruction device, the three-dimensional scene reconstruction system and the computer readable storage medium are applied to a server, firstly, visual data (including image data and/or point cloud data) of a scene to be reconstructed are obtained, then, a first target object included in the scene to be reconstructed is fitted according to the point cloud data, image recognition processing is carried out on the image data to obtain a second target object included in the scene to be reconstructed, and finally, a scene reconstruction result of the scene to be reconstructed is determined according to the first target object and the second target object. Wherein, the visual data is obtained by surveying by adopting the three-dimensional scene reconstruction method. Compared with the prior art in which a scene is reconstructed only according to an image or only according to a point cloud, the method for reconstructing a three-dimensional scene can remarkably improve the reconstruction effect of the three-dimensional scene.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flowchart of a three-dimensional scene reconstruction method according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a data survey apparatus according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a graphical user interface of a mobile control terminal according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a three-dimensional scene reconstruction system according to an embodiment of the present invention;
fig. 5 is a schematic flowchart of another three-dimensional scene reconstruction method according to an embodiment of the present invention;
fig. 6 is a schematic flowchart of another three-dimensional scene reconstruction method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a three-dimensional scene reconstruction apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another three-dimensional scene reconstruction apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a three-dimensional scene reconstruction system according to an embodiment of the present invention.
Icon: 1.1-tripod head; 1.2-a turntable; 1.3-host box; 1.4-liftable support structures; 2.1-status bar; 2.2-taskbar; 2.3-right split screen; 2.4-left split screen.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the embodiments, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
At present, the existing three-dimensional scene reconstruction method has the problems of high operation difficulty, poor three-dimensional reconstruction effect and the like, and based on the problems, the invention provides the three-dimensional scene reconstruction method, the three-dimensional scene reconstruction device, the three-dimensional scene reconstruction system and the computer readable storage medium, so that the operation difficulty of three-dimensional scene reconstruction can be effectively reduced, and the three-dimensional scene reconstruction effect can be remarkably improved.
To facilitate understanding of the embodiment, first, a detailed description is given to a three-dimensional scene reconstruction method disclosed in the embodiment of the present invention, where the method is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the mobile control terminal may adopt a smart phone or a tablet computer, the three-dimensional scene reconstruction system further includes a data surveying device in communication connection with the mobile control terminal, and the mobile control terminal provides a graphical user interface, and the graphical user interface has a display function and an operation instruction sending function, as shown in a flowchart of the three-dimensional scene reconstruction method shown in fig. 1, the method mainly includes the following steps S102 to S108:
and S102, providing an angle adjusting control, a surveying control and a scene rebuilding control through a graphical user interface. The angle adjustment control is used for triggering a surveying angle adjustment function of the data surveying equipment, the surveying control is used for triggering a surveying function of the data surveying equipment, and the scene rebuilding control is used for triggering a data sending function of the data surveying equipment.
And step S104, responding to the triggering operation aiming at the angle adjusting control, and controlling the data surveying equipment to adjust the surveying angle. The angle adjusting control can be a virtual rocker, a virtual key and the like, the triggering operation can include a point touch operation, a sliding operation, a dragging operation or a gesture operation and the like, and the surveying angle can include a pitching angle and/or a horizontal angle. In one embodiment, the data surveying equipment comprises a control host and a rotary table which is in communication connection with the control host, wherein the control host is used for controlling the rotary table to rotate, and the purpose of adjusting the surveying angle can be achieved by controlling the rotation angle of the rotary table. Taking the virtual rocker as an example, when the mobile control terminal monitors the dragging operation acting on the virtual rocker, the surveying angle is determined according to the dragging track of the dragging operation, the angle adjusting instruction carrying the surveying angle is sent to the control host, so as to trigger the surveying angle adjusting function of the control host, and the control host rotates the control turntable according to the surveying angle.
And S106, responding to the triggering operation aiming at the survey control, and controlling the data survey equipment to collect the visual data of the scene to be reconstructed. The visual data includes image data and/or point cloud data, and the scene to be reconstructed is also the scene where the data surveying equipment is located. In an implementation mode, the data survey equipment further comprises a cloud platform fixedly arranged on the rotary table, a sensor group is arranged in the cloud platform and is in communication connection with the control host, when the mobile control terminal monitors point touch operation acting on a survey control, a survey instruction is sent to the control host to trigger the survey function of the control host, and the control host acquires image data and point cloud data of a scene to be reconstructed by the control sensor group.
And S108, responding to the triggering operation aiming at the scene reconstruction control, controlling the data surveying equipment to send the visual data to the server, and determining a scene reconstruction result according to the visual data through the server. The scene reconstruction result may be a three-dimensional model of the scene to be reconstructed. In one embodiment, when the mobile control terminal monitors point touch operation acting on a scene reconstruction control, a scene reconstruction instruction is sent to the control host to trigger a data sending function of the control host, the control host sends acquired visual data to the server, and the server performs preprocessing, point cloud clustering processing, image recognition processing, scene reconstruction processing and the like on the visual data, so that a three-dimensional model of a scene to be reconstructed is obtained.
The three-dimensional scene reconstruction method provided by the embodiment of the invention establishes the communication connection between the mobile control terminal and the data surveying equipment, so that the mobile control terminal is utilized to perform visual control on the data surveying equipment, the data surveying equipment executes corresponding actions according to the triggering operation of a user acting on each virtual control, and the operation difficulty of three-dimensional scene reconstruction is obviously reduced.
In order to facilitate understanding of the data surveying equipment provided by the foregoing embodiments, the embodiment of the present invention provides a specific structure of the data surveying equipment, referring to a schematic structural diagram of the data surveying equipment shown in fig. 2, and fig. 2 illustrates that the data surveying equipment includes a pan/tilt head 1.1, a turntable 1.2, a main machine box 1.3 and a liftable support structure 1.4. The control host is arranged in the host box 1.3, the host box 1.3 is fixedly arranged on the liftable supporting structure 1.4, the rotary table 1.2 is connected with the control host, the cloud platform 1.1 is arranged on the rotary table 1.2, the cloud platform 1.1 is internally provided with a sensor group, and the sensor group is connected with the control host.
Optionally, the sensor group may include a binocular camera and a lidar sensor, the binocular camera is used to collect image data, the lidar sensor is used to collect point cloud data, and the image data and the point cloud data are transmitted to the control host through a rostoplac transmission protocol. Optionally, the turntable has pitching and rotating functions, and the acquisition angle of view (i.e., the above-mentioned survey angle) of the pan/tilt head can be changed without moving the liftable support structure, so as to obtain the optimal acquisition angle of view. Optionally, the control host is arranged in the host box 1.3, a power supply is further arranged in the host box, the power supply adopts a mobile charging type power supply and is used for providing electric energy required by operation for the control host, the rotary table and the sensor group, the control host can adopt a small industrial personal computer, a Linux system is installed, and the control host is provided with basic interfaces such as a USB (Universal Serial Bus), an HDMI (High Definition Multimedia Interface) and an indicator light. Optionally, liftable bearing structure 1.4 can adopt the tripod, and the tripod provides stable support and raising and lowering functions, and the tripod is syllogic and scalable, and the tripod top has elevation structure, can be used to manual lift.
In practical application, the mobile control terminal can be a handheld operation end tablet, exemplarily, an android system is installed on the tablet, and developed visual software is android software. The main functions of the software are visual operation and data management. To facilitate understanding of the mobile control terminal provided in the foregoing embodiment, the embodiment of the present invention further exemplarily provides a graphical user interface diagram of a mobile control terminal as shown in fig. 3, where fig. 3 illustrates that the graphical user interface provides a status bar 2.1, a task bar 2.2, a right split screen 2.3, and a left split screen 2.4. The status bar 2.1 can be used for displaying the electric quantity state of the mobile control terminal or the data surveying equipment, the storage capacity display of the host and the tablet, the communication state between the mobile control terminal and the data surveying equipment and the like, the taskbar 2.2 can be used for displaying the angle adjustment control, the surveying control, the scene reconstruction control and the like, the right split screen 2.3 can be used for displaying point cloud data, and the left split screen 2.4 can be used for displaying image data.
In one embodiment, the control host of the data surveying equipment communicates with the mobile control terminal through a wireless network, a dual-frequency wireless network card is arranged in the control host, visual data is compressed, and the data transmission between the control host and the mobile control terminal adopts a transmission rate of 5GHz, so that the visual data can be displayed in real time.
For convenience of understanding, an embodiment of the present invention further provides a schematic structural diagram of a three-dimensional scene reconstruction system as shown in fig. 4, where fig. 4 illustrates that visual data acquired by a binocular camera and a laser radar sensor is transmitted to a control host through a rossopic communication protocol, the control host transmits point cloud data to a mobile control terminal through WIFI (wireless fidelity), and the mobile control terminal performs data management on the point cloud data stored in the control host, and in addition, fig. 4 also illustrates that the mobile control terminal can control a turntable of a data surveying device.
On the basis of the foregoing embodiments, the embodiments of the present invention provide an implementation manner for controlling a data surveying device to adjust a surveying angle, and an angle adjustment command may be sent to a control host, so that the control host drives the rotation of the rotary table based on the surveying angle carried by the angle adjustment command. Illustratively, the angle adjustment control virtual rocker controls the virtual rocker to move along with the dragging operation in response to the dragging operation directed at the virtual rocker, determines the rotation angle of the virtual rocker according to the ending position of the dragging operation in response to the ending of the dragging operation, and generates and sends an angle control instruction carrying the rotation angle, so that the control host controls the rotation of the rotary table according to the rotation angle. For example, assuming that the user drags the virtual stick clockwise to rotate by 30 °, the control host will control the turntable to rotate by 30 ° clockwise.
In practical application, before the data surveying equipment is controlled to collect visual data, the binocular camera and the lidar sensor can be calibrated, so that image data collected by the binocular camera and point cloud data collected by the lidar sensor are in the same target coordinate system. In one embodiment, the calibration control may be provided through a graphical user interface, and the data surveying device may be controlled to calibrate the binocular camera and the lidar sensor based on the position information of the binocular camera and the position information of the lidar sensor in response to a trigger operation for the calibration control. In specific implementation, a camera coordinate system of the binocular camera may be used as a target coordinate system, or a radar coordinate system of the laser radar sensor may be used as the target coordinate system, or a virtual coordinate system is preset as the target coordinate system, for example, a virtual coordinate system is preset as the target coordinate system, an offset between the camera coordinate system and the target coordinate system is determined according to position information of the binocular camera, the camera coordinate system is calibrated according to the offset, so that the camera coordinate system and the target coordinate system coincide, an offset between the radar coordinate system and the target coordinate system is determined according to the position information of the laser radar sensor, the radar coordinate system is calibrated according to the offset, so that the radar coordinate system and the target coordinate system coincide, and calibration of the binocular camera and the laser radar sensor is completed.
Optionally, the calibration process of the binocular camera and the laser radar can be completed before the data surveying equipment leaves a factory, an automatic calibration mode can be adopted for the binocular camera, and a manual calibration mode can be adopted for calibration between the binocular camera and the laser radar and between the laser radar and the rotary table according to the structural position relation of the binocular camera and the laser radar.
On the basis, the embodiment of the invention also provides an implementation mode for controlling the data surveying equipment to acquire the visual data of the scene to be reconstructed, and the surveying instruction can be sent to the control host, so that the control host controls the binocular camera to acquire the image data of the scene to be reconstructed and controls the laser radar sensor to acquire the point cloud data of the scene to be reconstructed. During specific implementation, when the control host receives a surveying instruction, the binocular camera and the laser radar sensor are started, the binocular camera is controlled to collect image data, the laser radar sensor is controlled to collect point cloud data, and the image data and the point cloud data are transmitted to the control host based on a Rostopic communication protocol.
In addition, before the visual data are sent to the server, the user can perform visualization processing on the visual data stored in the control host. Specifically, see the following steps 1 to 3:
And 2, if the file management operation is a viewing operation, providing visual data through a graphical user interface. The graphical user interface comprises an image display area and/or a point cloud display area, wherein the point cloud display area is also the right split screen, the image display area is also the left split screen, namely, the image data is displayed on the left split screen, and the point cloud data is displayed on the right split screen. Preferably, the mobile control terminal is a touch screen.
And 3, responding to the display operation aiming at the visual data to display the visual data with the display effect corresponding to the display operation. The display operation may include a rotation operation, a translation operation, an enlargement operation, and a reduction operation, and the graphical user interface may display controls corresponding to the rotation operation, the translation operation, the enlargement operation, and the reduction operation, for example, when a trigger operation for the control corresponding to the enlargement operation is monitored, the currently displayed point cloud data is enlarged by right split screen to display more details of the point cloud data. During specific implementation, the point cloud data and the image data from the control host can be displayed in real time, and the display visual angle, the expansion and contraction and the like can be adjusted through the touch screen, so that a user can be assisted in judging whether the point cloud data and the image data are suitable or not.
The embodiment of the present invention further provides another three-dimensional scene reconstruction method, which is applied to a server, and referring to a flow diagram of another three-dimensional scene reconstruction method shown in fig. 5, the method mainly includes the following steps S502 to S508:
step S502, acquiring visual data of a scene to be reconstructed. The visual data is obtained by surveying by using the three-dimensional scene reconstruction method provided by the foregoing embodiment, and the visual data includes image data and/or point cloud data.
Step S504, a first target object contained in the scene to be reconstructed is fitted according to the point cloud data. For example, the scene to be reconstructed includes a telegraph pole, and the first target object may include a ground, a pole, a line, and the like. In an embodiment, preprocessing such as splicing operation, filtering processing, down-sampling processing and the like may be performed on the point cloud data to obtain target point cloud data, and then clustering, segmenting and fitting processing are performed on the target point cloud data to obtain a first target object included in a scene to be reconstructed.
Step S506, performing image recognition processing on the image data to obtain a second target object included in the scene to be reconstructed. For example, the scene to be reconstructed includes a telegraph pole, and the second target object may include a special target object such as an insulator or a ground ring. In an embodiment, the image data may be preprocessed by using an image processing function included in the opencv function library, and then the preprocessed image data is subjected to image recognition by using an image recognition function in the object _ detection function library of the tensoflow, so as to obtain the second target object included in the scene to be reconstructed, where the image recognition function may include depth learning and the like.
Step S508, determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object. In one embodiment, since the point cloud data and the image data are located in the same target coordinate system, a matching relationship between the point cloud data and the image data can be determined according to the position information of the first target object and the position information of the second target object, so that a three-dimensional model of a scene to be reconstructed is established on the basis of the matching relationship.
Compared with the prior art in which a scene is reconstructed only according to an image or only according to a point cloud, the three-dimensional scene reconstruction method provided by the embodiment of the invention can significantly improve the three-dimensional scene reconstruction effect.
To facilitate understanding of the foregoing step S504, an embodiment of the present invention further provides an implementation of fitting a first target object included in a scene to be reconstructed according to point cloud data, which is as follows:
And 2, clustering and partitioning the target point cloud data to obtain at least one point cloud cluster. In one embodiment, the target point cloud data may be segmented to separate ground points, line points, pole points, etc. from the target point cloud data, and a plurality of point cloud clusters, such as ground point cloud cluster, line point cloud cluster, and pole point cloud cluster, may be obtained by clustering.
And 3, respectively fitting each point cloud cluster to obtain a first target object contained in the scene to be reconstructed. In one embodiment, for the line point cloud cluster and the rod point cloud cluster, the line and the rod may be obtained by means of cylinder fitting.
To facilitate understanding of the foregoing step S506, an embodiment of the present invention further provides an implementation manner for determining a scene reconstruction result of a scene to be reconstructed according to the first target object and the second target object, where a matching relationship between the first target object and the second target object may be established according to the position information of the first target object and the position information of the second target object, and then the scene reconstruction result of the scene to be reconstructed is determined according to the matching relationship. Illustratively, according to the position information of the insulator, the position information of the ground ring, the position information of the line and the position information of the rod, the insulator and the ground ring are matched to the line and the rod which are closest to the insulator and the ground ring, and finally the ground, the line rod, the insulator, the ground ring, the cross rod and the like are rebuilt.
In order to facilitate understanding of the three-dimensional scene reconstruction method provided by the signing embodiment, an embodiment of the present invention further provides an application example of the three-dimensional scene reconstruction method, referring to a flow diagram of another three-dimensional scene reconstruction method shown in fig. 6, where the method mainly includes the following steps S602 to S622:
step S602, the data surveying equipment calibrates the binocular camera and the laser radar sensor.
In step S604, the data surveying device collects visual data using a binocular camera and a lidar sensor.
Step S606, the server performs filtering processing on the point cloud data.
Step S608, the server performs downsampling processing on the point cloud data.
Step S610, the server performs clustering and segmentation processing on the point cloud data.
Step S612, the server performs registration processing on the point cloud data.
And step S614, the server performs fitting processing on the point cloud data.
In step S616, the server performs preprocessing on the image data.
In step S618, the server performs image recognition processing on the image data.
In step S620, the server performs matching processing on the image recognition result and the point cloud fitting result. The image recognition result comprises the second target object, and the point cloud fitting result comprises the first target object.
And step S622, reconstructing the wire rod model.
In summary, the three-dimensional scene reconstruction method provided by the embodiment of the present invention has at least the following characteristics:
(1) the data acquisition function of the data surveying equipment is realized through an ROS (Robot Operating System) platform, and the data surveying equipment has universality.
(2) The operation is executed through software installed on the mobile control terminal, the mobile control terminal can display visual data, the visual data is managed, and the mobile control terminal is convenient to operate and man-machine friendly.
(3) The processing process of the visual data is based on the open source function library, and the method has universality.
For the three-dimensional scene reconstruction method provided in the foregoing embodiment, an embodiment of the present invention further provides a three-dimensional scene reconstruction device, where the device is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system further includes a data surveying device in communication connection with the mobile control terminal, and the mobile control terminal provides a graphical user interface, and referring to a schematic structural diagram of the three-dimensional scene reconstruction device shown in fig. 7, the device may include the following components:
a control providing module 702, configured to provide an angle adjustment control, a survey control, and a scene reconstruction control through a graphical user interface;
an angle adjustment module 704 for controlling the data survey equipment to adjust the survey angle in response to a trigger operation directed to the angle adjustment control;
a data acquisition module 706, configured to control the data survey equipment to acquire visual data of a scene to be reconstructed in response to a trigger operation for the survey control; wherein the visual data comprises image data and/or point cloud data;
a data sending module 708, configured to, in response to a triggering operation for the scene reconstruction control, control the data surveying device to send the visual data to the server, so as to determine a scene reconstruction result according to the visual data through the server.
The three-dimensional scene reconstruction device provided by the embodiment of the invention establishes the communication connection between the mobile control terminal and the data surveying equipment, so that the mobile control terminal is utilized to perform visual control on the data surveying equipment, the data surveying equipment executes corresponding actions according to the triggering operation acted on each virtual control by a user, and the operation difficulty of three-dimensional scene reconstruction is obviously reduced.
In one embodiment, the data surveying equipment comprises a liftable supporting structure, a control host and a rotary table, wherein the control host is fixedly arranged on the liftable supporting structure, and the rotary table is connected with the control host; the angle adjustment module 704 is further configured to: sending the angle adjusting instruction to a control host machine so as to drive the rotary table to rotate through a surveying angle carried by the control host machine based on the angle adjusting instruction; wherein the survey angle comprises a pitch angle and/or a horizontal angle.
In one embodiment, the data surveying equipment further comprises a holder arranged on the rotary table, a sensor group is arranged in the holder and connected with the control host, and the sensor group comprises a binocular camera and a laser radar sensor; the data acquisition module 706 is further configured to: and sending the surveying instruction to a control host to control the binocular camera to acquire image data of the scene to be reconstructed and control the laser radar sensor to acquire point cloud data of the scene to be reconstructed through the control host.
In one embodiment, the apparatus further includes a calibration module configured to: providing a calibration control through a graphical user interface; and controlling the data surveying equipment to calibrate the binocular camera and the laser radar sensor according to the position information of the binocular camera and the position information of the laser radar sensor in response to the triggering operation aiming at the calibration control.
In one embodiment, the apparatus further includes a management module configured to: controlling the data surveying device to perform a file management operation on the visual data in response to the file management operation on the visual data; the file management operation at least comprises a viewing operation, a copying operation and a deleting operation; if the file management operation is a viewing operation, providing visual data through a graphical user interface; wherein the graphical user interface comprises an image display area and/or a point cloud display area; and responding to the display operation aiming at the visual data to display the visual data with a display effect corresponding to the display operation.
For the three-dimensional scene reconstruction method provided in the foregoing embodiment, an embodiment of the present invention further provides a three-dimensional scene reconstruction device, which is applied to a server, and referring to a schematic structural diagram of another three-dimensional scene reconstruction device shown in fig. 8, the device mainly includes the following components:
a data obtaining module 802, configured to obtain visual data of a scene to be reconstructed; the visual data are obtained by surveying by adopting the three-dimensional scene reconstruction method, and comprise image data and/or point cloud data;
the point cloud fitting module 804 is used for fitting a first target object contained in the scene to be reconstructed according to the point cloud data;
the image identification module 806 is configured to perform image identification processing on the image data to obtain a second target object included in the scene to be reconstructed;
and a scene reconstruction module 808, configured to determine a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object.
Compared with the prior art in which a scene is reconstructed only according to an image or only according to a point cloud, the three-dimensional scene reconstruction device provided by the embodiment of the invention can remarkably improve the three-dimensional scene reconstruction effect.
In one embodiment, the point cloud fitting module 804 is further configured to: preprocessing the point cloud data to obtain target point cloud data; the preprocessing at least comprises splicing operation, filtering processing and down-sampling processing; clustering and dividing the target point cloud data to obtain at least one point cloud cluster; and respectively fitting each point cloud cluster to obtain a first target object contained in the scene to be reconstructed.
In one embodiment, the scene reconstruction module 808 is further configured to: establishing a matching relation between the first target object and the second target object according to the position information of the first target object and the position information of the second target object; and determining a scene reconstruction result of the scene to be reconstructed according to the matching relation.
The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments.
The embodiment of the invention provides a three-dimensional scene reconstruction system, which particularly comprises a processor and a memory; the memory has stored thereon a computer program which, when executed by the processor, performs the method of any of the above embodiments.
Fig. 9 is a schematic structural diagram of a three-dimensional scene reconstruction system according to an embodiment of the present invention, where the three-dimensional scene reconstruction system 100 includes: the system comprises a processor 90, a memory 91, a bus 92 and a communication interface 93, wherein the processor 90, the communication interface 93 and the memory 91 are connected through the bus 92; the processor 90 is arranged to execute executable modules, such as computer programs, stored in the memory 91.
The Memory 91 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 93 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
Bus 92 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 9, but this does not indicate only one bus or one type of bus.
The memory 91 is used for storing a program, the processor 90 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 90, or implemented by the processor 90.
The processor 90 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 90. The Processor 90 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 91, and the processor 90 reads the information in the memory 91 and performs the steps of the above method in combination with the hardware thereof.
The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the foregoing method embodiment, which is not described herein again.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (12)
1. A three-dimensional scene reconstruction method is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system further comprises a data survey device in communication connection with the mobile control terminal, the mobile control terminal provides a graphical user interface, and the method comprises the following steps:
providing an angle adjustment control, a survey control, and a scene reconstruction control through the graphical user interface;
controlling the data survey equipment to adjust a survey angle in response to a triggering operation directed to the angle adjustment control;
controlling the data survey equipment to acquire visual data of a scene to be reconstructed in response to a triggering operation directed to the survey control; wherein the visual data comprises image data and/or point cloud data;
and responding to the triggering operation aiming at the scene reconstruction control, controlling the data surveying equipment to send the visual data to a server so as to determine a scene reconstruction result according to the visual data through the server.
2. The method according to claim 1, wherein the data survey equipment comprises a liftable support structure, a control host and a turntable, the control host is fixedly arranged on the liftable support structure, and the turntable is connected with the control host;
the step of controlling the data survey equipment to adjust a survey angle comprises:
sending an angle adjustment instruction to the control host machine so as to drive the rotary table to rotate through the control host machine based on a survey angle carried by the angle adjustment instruction; wherein the survey angle comprises a pitch angle and/or a horizontal angle.
3. The method according to claim 2, wherein the data surveying equipment further comprises a cradle head disposed on the turntable, a sensor group is disposed in the cradle head, the sensor group is connected with the control host, and the sensor group comprises a binocular camera and a lidar sensor;
the step of controlling the data survey device to collect visual data of a scene to be reconstructed comprises:
sending a surveying instruction to the control host to control the binocular camera to acquire image data of a scene to be reconstructed and control the laser radar sensor to acquire point cloud data of the scene to be reconstructed through the control host.
4. The method of claim 3, wherein prior to the step of responding to the triggering operation for the survey control, the method further comprises:
providing a calibration control through the graphical user interface;
and responding to the trigger operation aiming at the calibration control, and controlling the data surveying equipment to calibrate the binocular camera and the laser radar sensor according to the position information of the binocular camera and the position information of the laser radar sensor.
5. The method of claim 1, wherein prior to the step of responding to the triggering operation for the scene reconstruction control, the method further comprises:
in response to a file management operation directed to the visual data, controlling the data surveying device to perform a file management operation on the visual data; wherein the file management operation at least comprises a viewing operation, a copying operation, a moving operation and a deleting operation;
if the file management operation is the viewing operation, providing the visual data through the graphical user interface; wherein the graphical user interface comprises an image display area and/or a point cloud display area;
and responding to the display operation aiming at the visual data, and displaying the visual data with a display effect corresponding to the display operation.
6. A three-dimensional scene reconstruction method is applied to a server, and comprises the following steps:
acquiring visual data of a scene to be reconstructed; wherein the visual data is obtained by surveying according to the three-dimensional scene reconstruction method of any one of claims 1 to 5, and the visual data comprises image data and/or point cloud data;
fitting a first target object contained in the scene to be reconstructed according to the point cloud data;
carrying out image identification processing on the image data to obtain a second target object contained in the scene to be reconstructed;
and determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object.
7. The method according to claim 6, wherein the step of fitting the point cloud data to a first target object included in the scene to be reconstructed comprises:
preprocessing the point cloud data to obtain target point cloud data; wherein the preprocessing at least comprises splicing operation, filtering processing and down-sampling processing;
clustering and dividing the target point cloud data to obtain at least one point cloud cluster;
and respectively performing fitting processing on each point cloud cluster to obtain a first target object contained in the scene to be reconstructed.
8. The method according to claim 6, wherein the step of determining a scene reconstruction result of the scene to be reconstructed from the first target object and the second target object comprises:
establishing a matching relation between the first target object and the second target object according to the position information of the first target object and the position information of the second target object;
and determining a scene reconstruction result of the scene to be reconstructed according to the matching relation.
9. A three-dimensional scene reconstruction device, which is applied to a mobile control terminal of a three-dimensional scene reconstruction system, the three-dimensional scene reconstruction system further comprises a data survey device in communication connection with the mobile control terminal, the mobile control terminal provides a graphical user interface, and the device comprises:
the control providing module is used for providing an angle adjusting control, a surveying control and a scene rebuilding control through the graphical user interface;
the angle adjusting module is used for responding to the triggering operation aiming at the angle adjusting control and controlling the data surveying equipment to adjust the surveying angle;
the data acquisition module is used for responding to the triggering operation aiming at the surveying control and controlling the data surveying equipment to acquire the visual data of the scene to be reconstructed; wherein the visual data comprises image data and/or point cloud data;
and the data sending module is used for responding to the triggering operation aiming at the scene reconstruction control, controlling the data surveying equipment to send the visual data to a server so as to determine a scene reconstruction result according to the visual data through the server.
10. A three-dimensional scene reconstruction apparatus, wherein the apparatus is applied to a server, and the apparatus comprises:
the data acquisition module is used for acquiring visual data of a scene to be reconstructed; wherein the visual data is obtained by surveying according to the three-dimensional scene reconstruction method of any one of claims 1 to 5, and the visual data comprises image data and/or point cloud data;
the point cloud fitting module is used for fitting a first target object contained in the scene to be reconstructed according to the point cloud data;
the image identification module is used for carrying out image identification processing on the image data to obtain a second target object contained in the scene to be reconstructed;
and the scene reconstruction module is used for determining a scene reconstruction result of the scene to be reconstructed according to the first target object and the second target object.
11. A three-dimensional scene reconstruction system comprising a processor and a memory, the memory storing computer-executable instructions executable by the processor to perform the method of any one of claims 1 to 5 or to perform the method of any one of claims 6 to 8.
12. A computer-readable storage medium having stored thereon computer-executable instructions that, when invoked and executed by a processor, cause the processor to implement the method of any of claims 1 to 5, or cause the processor to implement the method of any of claims 6 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111624638.9A CN113989445B (en) | 2021-12-29 | 2021-12-29 | Three-dimensional scene reconstruction method, device and system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111624638.9A CN113989445B (en) | 2021-12-29 | 2021-12-29 | Three-dimensional scene reconstruction method, device and system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113989445A CN113989445A (en) | 2022-01-28 |
| CN113989445B true CN113989445B (en) | 2022-03-01 |
Family
ID=79734707
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111624638.9A Active CN113989445B (en) | 2021-12-29 | 2021-12-29 | Three-dimensional scene reconstruction method, device and system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113989445B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102692637A (en) * | 2012-05-25 | 2012-09-26 | 西南科技大学 | Teleoperation-device-based virtual reconstruction system and method for nuclear radiation environment |
| CN105184863A (en) * | 2015-07-23 | 2015-12-23 | 同济大学 | Unmanned aerial vehicle aerial photography sequence image-based slope three-dimension reconstruction method |
| CN109461180A (en) * | 2018-09-25 | 2019-03-12 | 北京理工大学 | A kind of method for reconstructing three-dimensional scene based on deep learning |
| CN109801358A (en) * | 2018-12-06 | 2019-05-24 | 宁波市电力设计院有限公司 | A kind of substation's three-dimensional investigation method scanning and put cloud visual fusion based on SLAM |
| CN110009740A (en) * | 2019-04-13 | 2019-07-12 | 中国地质大学（北京） | Geology based on exercise recovery structure is appeared quick three-dimensional reconstructing method |
| CN110132131A (en) * | 2019-05-16 | 2019-08-16 | 西安工业大学 | A kind of measurement method of scene data |
| CN112348952A (en) * | 2020-11-06 | 2021-02-09 | 中铁第一勘察设计院集团有限公司 | Three-dimensional scene construction method for multi-source geographic information data fusion in hard mountainous area |
| CN113192179A (en) * | 2021-04-28 | 2021-07-30 | 沈阳工业大学 | Three-dimensional reconstruction method based on binocular stereo vision |
| CN113362382A (en) * | 2020-03-04 | 2021-09-07 | 华为技术有限公司 | Three-dimensional reconstruction method and three-dimensional reconstruction device |
| CN113806897A (en) * | 2021-08-27 | 2021-12-17 | 江苏齐天电力建设集团有限公司 | Power engineering construction decision optimization method based on three-dimensional GIS technology |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9041771B2 (en) * | 2011-06-08 | 2015-05-26 | City University Of Hong Kong | Automatic switching of a multi-mode display for displaying three-dimensional and two-dimensional images |
| US10970924B2 (en) * | 2018-06-17 | 2021-04-06 | Foresight Ai Inc. | Reconstruction of a scene from a moving camera |
-
2021
- 2021-12-29 CN CN202111624638.9A patent/CN113989445B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102692637A (en) * | 2012-05-25 | 2012-09-26 | 西南科技大学 | Teleoperation-device-based virtual reconstruction system and method for nuclear radiation environment |
| CN105184863A (en) * | 2015-07-23 | 2015-12-23 | 同济大学 | Unmanned aerial vehicle aerial photography sequence image-based slope three-dimension reconstruction method |
| CN109461180A (en) * | 2018-09-25 | 2019-03-12 | 北京理工大学 | A kind of method for reconstructing three-dimensional scene based on deep learning |
| CN109801358A (en) * | 2018-12-06 | 2019-05-24 | 宁波市电力设计院有限公司 | A kind of substation's three-dimensional investigation method scanning and put cloud visual fusion based on SLAM |
| CN110009740A (en) * | 2019-04-13 | 2019-07-12 | 中国地质大学（北京） | Geology based on exercise recovery structure is appeared quick three-dimensional reconstructing method |
| CN110132131A (en) * | 2019-05-16 | 2019-08-16 | 西安工业大学 | A kind of measurement method of scene data |
| CN113362382A (en) * | 2020-03-04 | 2021-09-07 | 华为技术有限公司 | Three-dimensional reconstruction method and three-dimensional reconstruction device |
| CN112348952A (en) * | 2020-11-06 | 2021-02-09 | 中铁第一勘察设计院集团有限公司 | Three-dimensional scene construction method for multi-source geographic information data fusion in hard mountainous area |
| CN113192179A (en) * | 2021-04-28 | 2021-07-30 | 沈阳工业大学 | Three-dimensional reconstruction method based on binocular stereo vision |
| CN113806897A (en) * | 2021-08-27 | 2021-12-17 | 江苏齐天电力建设集团有限公司 | Power engineering construction decision optimization method based on three-dimensional GIS technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113989445A (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11677920B2 (en) | Capturing and aligning panoramic image and depth data | |
| CN108665536B (en) | Three-dimensional and live-action data visualization method and device and computer readable storage medium | |
| US9667862B2 (en) | Method, system, and computer program product for gamifying the process of obtaining panoramic images | |
| CN103841374B (en) | Display method and system for video monitoring image | |
| JP2019532268A (en) | Determination of stereo distance information using an imaging device integrated in the propeller blades | |
| CA2858870C (en) | Multiple-angle imagery of physical objects | |
| US20200242829A1 (en) | Three-dimensional reconstruction method and apparatus for material pile, electronic device, and computer-readable medium | |
| CN110967024A (en) | Method, device, equipment and storage medium for detecting travelable area | |
| CN110072087B (en) | Camera linkage method, device, equipment and storage medium based on 3D map | |
| CN111046762A (en) | Object positioning method, device electronic equipment and storage medium | |
| CN109215111B (en) | Indoor scene three-dimensional modeling method based on laser range finder | |
| DE102018132428A1 (en) | Photomosaic soil mapping | |
| CN104180814A (en) | Navigation method in live-action function on mobile terminal, and electronic map client | |
| CN112284291A (en) | Three-dimensional scanning method and device capable of obtaining physical texture | |
| CN106447728A (en) | Scanning state adjustment method and apparatus | |
| EP2847616A1 (en) | Surveying apparatus having a range camera | |
| CN110895833A (en) | Method and device for three-dimensional modeling of indoor scene | |
| KR20160070874A (en) | Location-based Facility Management System Using Mobile Device | |
| CN113887400A (en) | Obstacle detection method, model training method and device and automatic driving vehicle | |
| JP7351892B2 (en) | Obstacle detection method, electronic equipment, roadside equipment, and cloud control platform | |
| CN113601510B (en) | Robot movement control method, device, system and equipment based on binocular vision | |
| CN110163823B (en) | Multi-view image correction method and system based on capsule robot and oriented to drain pipe | |
| KR102571066B1 (en) | Method of acquiring 3d perceptual information based on external parameters of roadside camera and roadside equipment | |
| US20230249620A1 (en) | Parking process display method and device, and vehicle | |
| CN114549988A (en) | Image processing method and device for intelligent transportation, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |