CN116385621A - Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium - Google Patents
Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium Download PDFInfo
- Publication number
- CN116385621A CN116385621A CN202310361768.0A CN202310361768A CN116385621A CN 116385621 A CN116385621 A CN 116385621A CN 202310361768 A CN202310361768 A CN 202310361768A CN 116385621 A CN116385621 A CN 116385621A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- optimal
- preset
- modified model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 86
- 238000004458 analytical method Methods 0.000 claims abstract description 35
- 238000012545 processing Methods 0.000 claims abstract description 13
- 239000013598 vector Substances 0.000 claims description 41
- 230000000007 visual effect Effects 0.000 claims description 10
- 238000003702 image correction Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 238000012986 modification Methods 0.000 abstract description 9
- 230000004048 modification Effects 0.000 abstract description 9
- 230000008569 process Effects 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002715 modification method Methods 0.000 description 1
- 239000000178 monomer Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Architecture (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Telephone Function (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application provides a mobile phone image mapping method, device, equipment and medium assisted by positioning and attitude determination, and relates to the technical field of data processing. The mobile phone image mapping method assisted by positioning and attitude determination comprises the following steps: and acquiring a plurality of images, selecting at least one candidate image with position information meeting the condition from the plurality of images according to a to-be-modified model area in a preset three-dimensional model, carrying out weighted analysis on the candidate images to obtain an optimal image, and finally carrying out texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimized model. According to the method, the optimal image with the position information meeting the condition is determined from the plurality of images according to the to-be-modified model area in the preset three-dimensional model, so that the workload of manually searching the image corresponding to the to-be-modified model area can be reduced, the image meeting the to-be-modified model area can be determined more accurately, and the modification efficiency of the preset three-dimensional model is improved.
Description
Technical Field
The invention relates to the fields of mapping, three-dimensional modeling and the like, in particular to a mobile phone image mapping method, device, equipment and medium with the assistance of positioning and attitude determination.
Background
With the proposal of concepts such as "smart city", fine three-dimensional models of cities are attracting attention. The ground object image is obtained through the sensor carried by the unmanned aerial vehicle flight platform, and the three-dimensional model of the experimental area is rebuilt by utilizing the three-dimensional modeling algorithm of the multi-view image, so that the method is the current main three-dimensional model production method. However, as the spatial resolution of the unmanned aerial vehicle image in the area near the ground, especially the bottom of a building is low and is often influenced by shielding, the three-dimensional model produced by the method has low texture quality in the area near the ground, the bottom of a building elevation and other areas, has poor model structure, is difficult to meet the texture quality and geometric structure requirements of the model bottom in practical application, and needs to be modified.
The current model modification method acquires a ground visual angle image with high spatial resolution by manually holding a data acquisition device for ground image complement, and then maps the ground visual angle image to a corresponding area of a three-dimensional model. The combined adjustment modeling can be performed on aviation and ground images, but due to the differences of image shooting angles, resolution and overlapping degree, a stable solution is not available for the situation at present; the corresponding relation between the three-dimensional model and the ground image can be found manually, and the texture mapping is performed again on the area with poor texture quality, but a great amount of time is required to be consumed when the corresponding relation between the model and the image is found manually.
Disclosure of Invention
The invention aims to provide a mobile phone image mapping method, device, equipment and medium with the assistance of positioning and gesture determination aiming at the defects in the prior art, so that the workload of manually searching images corresponding to a to-be-modified model area is reduced, and the modification efficiency of a preset three-dimensional model is improved.
In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
in a first aspect, an embodiment of the present application provides a positioning and gesture-determining assisted mobile phone image mapping method, including:
collecting a plurality of images;
selecting at least one candidate image with position information meeting the condition from a plurality of images according to a to-be-modified model area in a preset three-dimensional model;
performing weighted analysis on the candidate images to obtain optimal images;
and performing texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model.
In an alternative embodiment, the image data includes: position data and attitude angle data;
according to the to-be-modified model area in the preset three-dimensional model, selecting at least one candidate image with position information meeting the condition from the plurality of images, wherein the method comprises the following steps:
acquiring the to-be-modified model area in a preset three-dimensional model;
calculating the position information of the to-be-modified model area and determining a coordinate system of the preset three-dimensional model, wherein the position information of the to-be-modified model area comprises: the normal vector and the center point position of the to-be-modified model area;
and selecting at least one candidate image with the position information meeting the condition from the plurality of images according to the position information of the to-be-modified model area and the position data of the plurality of images.
In an optional embodiment, the selecting, according to the position information of the to-be-modified model area and the position data of the plurality of images, at least one candidate image whose position information meets the condition from the plurality of images includes:
according to the attitude angle data of the images, respectively calculating the view angle orientation vector of each image;
calculating an included angle difference value according to the visual angle orientation vector of the image and the normal vector of the to-be-modified model area;
and selecting the images meeting the conditions as the candidate images according to the included angle difference value and a preset threshold, wherein the candidate images comprise the areas corresponding to the to-be-modified model areas.
In an optional embodiment, the performing a weighted analysis on the candidate image to obtain an optimal image includes:
calculating a distance difference value according to the projection center position of the candidate image and the center point position of the to-be-modified model area;
according to the distance difference value and the included angle difference value, carrying out weighted analysis to obtain an analysis result;
and determining an optimal image in the candidate images according to the analysis result.
In an optional embodiment, before performing texture mapping on the to-be-modified model area in the preset three-dimensional model by using the optimal image to obtain an optimized model, the method further includes:
performing image correction on the optimal image according to preset parameters;
obtaining an optimal image to be mapped according to the corrected image data of the optimal image and the normal vector of the plane where the to-be-modified model area is located;
and performing texture mapping on a to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model, wherein the method comprises the following steps:
and performing texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the to-be-mapped optimal image to obtain an optimal model.
In an optional embodiment, after obtaining the optimal image to be mapped according to the corrected pose angle data of the optimal image and the normal vector of the plane where the to-be-modified model area is located, the method further includes:
if the optimal image to be mapped cannot cover the to-be-modified model area, respectively acquiring a plurality of optimal images to be mapped through a plurality of candidate images with overlapping degrees;
and processing and splicing the plurality of optimal images to be mapped to obtain the final optimal image to be mapped.
In an alternative embodiment, the capturing a plurality of images includes:
acquiring a plurality of images through terminal equipment to acquire an image set;
and giving the position data and the attitude angle data of each image in the image set through the position information and the attitude information recorded by the terminal equipment, and generating a pose file corresponding to the image set.
In a second aspect, an embodiment of the present application further provides a positioning and gesture-fixing assisted mobile phone image mapping device, including:
the acquisition module is used for acquiring a plurality of images;
the selection module is used for selecting at least one candidate image with the position information meeting the condition from the plurality of images according to a to-be-modified model area in the preset three-dimensional model;
the acquisition module is used for carrying out weighted analysis on the candidate images to acquire optimal images;
and the mapping module is used for carrying out texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model.
In a third aspect, embodiments of the present application further provide a computer device, including: the mobile phone positioning and attitude determination auxiliary mobile phone image mapping method comprises a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, when the computer equipment runs, the processor and the storage medium are communicated through the bus, and the processor executes the program instructions to execute the steps of the mobile phone positioning and attitude determination auxiliary mobile phone image mapping method according to any one of the first aspect.
In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor performs the steps of the positioning and gesture-determining assisted mobile phone image mapping method according to any one of the first aspect.
The beneficial effects of this application are:
the embodiment of the application provides a mobile phone image mapping method, device, equipment and medium assisted by positioning and attitude determination, which comprises the following steps: and acquiring a plurality of images, selecting at least one candidate image with position information meeting the condition from the plurality of images according to a to-be-modified model area in a preset three-dimensional model, carrying out weighted analysis on the candidate images to obtain an optimal image, and finally carrying out texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimized model. According to the method, the optimal image with the position information meeting the condition is determined from the plurality of images according to the to-be-modified model area in the preset three-dimensional model, so that the workload of manually searching the image corresponding to the to-be-modified model area can be reduced, the image meeting the to-be-modified model area can be determined more accurately, and the modification efficiency of the preset three-dimensional model is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a mobile phone image mapping method with positioning and gesture determination assistance provided in an embodiment of the present application;
fig. 2 is a second flow chart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application;
fig. 3 is a third flow chart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application;
fig. 4 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application;
fig. 5 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application;
FIG. 6 is a flowchart of a mobile phone image mapping method with assistance of positioning and gesture determination according to an embodiment of the present application;
fig. 7 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application;
fig. 8 is a schematic functional block diagram of a mobile phone image mapping device with positioning and gesture determination assistance according to an embodiment of the present application;
fig. 9 is a schematic diagram of a computer device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention.
Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
Furthermore, the terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, without conflict, features in embodiments of the present application may be combined with each other.
In order to more efficiently and conveniently complete image texture mapping, the embodiment of the application provides a mobile phone image mapping method assisted by positioning and gesture determination, position and gesture information of an image can be obtained through the assistance of positioning and gesture determination equipment for the position and gesture of the image acquired by mobile phone equipment, an optimal image in the acquired image is determined according to a to-be-modified model area of a preset three-dimensional model, and finally, the to-be-modified model area is repaired through texture mapping, so that modification efficiency of the preset three-dimensional model is effectively improved.
The mobile phone image mapping method assisted by positioning and attitude determination provided by the embodiment of the application is explained in detail by a specific example with reference to the accompanying drawings. The mobile phone image mapping method assisted by positioning and gesture determination provided by the embodiment of the application can be implemented by pre-installing: the computer equipment for presetting an image mapping algorithm or detecting software is realized by running the algorithm or the software. The computer device may be, for example, a server or a terminal, which may be a user computer. Fig. 1 is a schematic flow chart of a mobile phone image mapping method with positioning and gesture determination assistance according to an embodiment of the present application. As shown in fig. 1, the method includes:
s101, collecting a plurality of images.
In this embodiment, a plurality of images may be acquired through a terminal device, where the terminal device may be a smart phone, a tablet, etc., and if the terminal device is a smart phone, the plurality of images are a plurality of mobile phone images acquired through the smart phone, and specifically, the plurality of mobile phone images are a plurality of Gao Kongjian resolution ground view images.
The execution body of the application may be communicatively connected to the terminal device to obtain the image.
S102, selecting at least one candidate image with the position information meeting the condition from the plurality of images according to a to-be-modified model area in a preset three-dimensional model.
The preset three-dimensional model is a three-dimensional model needing texture modification, the types of the preset three-dimensional model can comprise a three-dimensional model automatically reconstructed by using an unmanned aerial vehicle image, a manually established monomer model and the like, and the preset three-dimensional model can comprise coordinate information to determine corresponding relevant positions and the like. For example, in the process of building the preset three-dimensional model, as the spatial resolution of the unmanned aerial vehicle image in the area near the ground and the bottom of the building is low and is easily shielded, the texture quality of the preset three-dimensional model in the area near the ground and the bottom of the building is poor, and therefore the model area needing to be modified in the preset three-dimensional model, namely the model area to be modified, is marked.
And selecting at least one candidate image with the position information meeting the condition from the acquired multiple images, and modifying the to-be-modified model area in the preset three-dimensional model, wherein the position information meeting the condition indicates that the image data of part of the images in the multiple images are matched with the area data of the to-be-modified model area, and the part of the images are at least one candidate image.
And S103, carrying out weighted analysis on the candidate images to obtain the optimal images.
According to the step S102, at least one candidate image is selected from the plurality of images, and the image data of the candidate image and the area data of the to-be-modified model area are subjected to weighted analysis to obtain the analysis result of each image in the candidate image, so that the candidate image is selected according to the analysis result of each image to obtain an optimal image, wherein the optimal image is indicated as an image with higher matching degree with the to-be-modified model area.
And S104, performing texture mapping on a to-be-modified model area in the preset three-dimensional model by adopting an optimal image to obtain an optimized model.
According to the step S103, an optimal image is obtained, and a preset mapping model is adopted to perform texture mapping on a preset three-dimensional model and the optimal image, wherein different preset three-dimensional models and optimal images can be subjected to texture mapping by adopting different preset mapping models, and if the optimal image is a single image, a rigid model or a non-rigid mapping model can be adopted, and if the optimal image is an optimal image generated by splicing multiple images, the texture deformation can be caused by directly mapping by adopting a rigid transformation model because different candidate images have different deformations in the process of transforming and splicing the multiple images, so that the optimal image generated by splicing the multiple images and the preset three-dimensional model can be subjected to texture mapping by adopting a non-rigid mapping model based on a thin plate spline, and the region and the field of the model to be modified of the preset three-dimensional model are subjected to uniform light and color processing, thereby finally completing the texture mapping, and obtaining the optimal model.
In summary, the embodiment of the present application provides a mobile phone image mapping method with positioning and gesture determination assistance, including: and acquiring a plurality of images, selecting at least one candidate image with position information meeting the condition from the plurality of images according to a to-be-modified model area in a preset three-dimensional model, carrying out weighted analysis on the candidate images to obtain an optimal image, and finally carrying out texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimized model. According to the method, the optimal image with the position information meeting the condition is determined from the plurality of images according to the to-be-modified model area in the preset three-dimensional model, so that the workload of manually searching the image corresponding to the to-be-modified model area can be reduced, the image meeting the to-be-modified model area can be determined more accurately, and the modification efficiency of the preset three-dimensional model is improved.
In the process of collecting the plurality of images, image data of the plurality of images are recorded respectively, wherein the image data comprises: position data and attitude angle data. On the basis of the mobile phone image mapping method assisted by positioning and gesture determination provided by the embodiment, the embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and gesture determination. Fig. 2 is a second flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 2, according to a to-be-modified model area in a preset three-dimensional model, selecting at least one candidate image with position information meeting a condition from a plurality of images, including:
s201, acquiring a to-be-modified model area in a preset three-dimensional model.
S202, calculating position information of a to-be-modified model area and determining a coordinate system of a preset three-dimensional model.
In this embodiment, a preset three-dimensional model to be textured is input first, and a to-be-modified model area is marked in the preset three-dimensional model.
Calculating the position information of the to-be-modified model area, determining and recording a coordinate system of a preset three-dimensional model, wherein the position information of the to-be-modified model area comprises: normal vector and center point position of the model area to be modified.
Meanwhile, the image data of the collected images are converted into a coordinate system of a preset three-dimensional model, a plurality of images which are more similar to the to-be-modified model area are conveniently determined according to the converted image data, a spatial index is built mainly through the image data of the images, and a plurality of images which are more similar to the to-be-modified model area are searched according to the spatial index and an adjacent image set is built.
S203, selecting at least one candidate image with the position information meeting the condition from the plurality of images according to the position information of the to-be-modified model area and the position data of the plurality of images.
Specifically, according to the position information of the to-be-modified model area, namely the normal vector and the center point position of the to-be-modified model area, and the position data of the plurality of images in the adjacent image set, at least one candidate image with the position information meeting the condition is selected from the plurality of images.
According to the method provided by the embodiment of the application, the workload of manually searching the image corresponding to the to-be-modified model area can be reduced, and the image conforming to the to-be-modified model area can be more accurately determined by calculating the position information of the to-be-modified model area and selecting at least one candidate image with the position information conforming to the condition from the images according to the position information of the to-be-modified model area and the position data of the images.
The embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and attitude determination. Fig. 3 is a third flow chart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 3, selecting at least one candidate image whose position information meets a condition from among a plurality of images based on position information of a model area to be modified and position data of the plurality of images, includes:
s301, respectively calculating view angle orientation vectors of the images according to the attitude angle data of the images.
In this embodiment, the attitude angle data of the image includes: pitch angle, yaw angle, roll angle, may be expressed as a= { yaw, pitch, roll }, where yaw is expressed as yaw angle, pitch is expressed as pitch angle, roll is expressed as roll angle. And the visual angle orientation vector to each image can be calculated according to the attitude angle data of the image.
S302, calculating an included angle difference value according to the visual angle orientation vector of the image and the normal vector of the to-be-modified model area.
Wherein the included angle difference value Adif is calculated Rm The method of (2) can be expressed as:
wherein n is m Represented as view angle orientation vectors of an image, N R And the normal vector position data are expressed as normal vector position data of the to-be-modified model area, so that the angle difference value between the angle orientation vector of each image and the normal vector of the to-be-modified model area is calculated.
S303, selecting the images meeting the conditions as candidate images according to the included angle difference value and a preset threshold value.
Setting a preset threshold, discarding the image if the difference value of the included angle between the view angle orientation vector of the image and the normal vector of the to-be-modified model area is larger than the preset threshold, and considering that the angle orientation of the image is close to the normal of the to-be-modified model area if the difference value of the included angle between the view angle orientation vector of the image and the normal vector of the to-be-modified model area is smaller than or equal to the preset threshold.
Optionally, in the process of selecting the image meeting the condition as the candidate image according to the included angle difference value and the preset threshold, after the included angle difference value meets the preset threshold, constructing a view cone according to attitude angle data of the image, judging whether the to-be-modified model area is in a visible area of the view cone, and if the to-be-modified model area is in the visible area of the view cone, determining that the image is the candidate image, wherein the candidate image comprises an area corresponding to the to-be-modified model area. And selecting an image satisfying the above conditions from the plurality of images as a candidate image, wherein the satisfying conditions includes: the difference value of the included angle between the visual angle orientation vector of the image and the normal vector of the model area to be modified is smaller than or equal to a preset threshold value, and the model area to be modified is in the visible area of the view cone constructed by the image through the attitude angle data.
According to the method provided by the embodiment of the application, the visual angle orientation vectors of the images are calculated respectively according to the attitude angle data of the images; calculating an included angle difference value according to the visual angle orientation vector of the image and the normal vector of the to-be-modified model area; according to the included angle difference value and the preset threshold value, selecting the images meeting the conditions as candidate images, so that the workload of manually searching the images corresponding to the to-be-modified model area can be reduced, and the images meeting the to-be-modified model area can be more accurately determined.
The embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and attitude determination. Fig. 4 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 4, performing weighted analysis on the candidate images to obtain an optimal image includes:
s401, calculating a distance difference value according to the projection center position of the candidate image and the center point position of the to-be-modified model area.
In the embodiment of the application, the distance difference value Ddif is calculated Rm The method of (2) can be expressed as:
Ddif Rm ＝|p m -C R |
wherein p is m Represented as the projected center position of the candidate image, C R Represented as the center point position of the model area to be modified, wherein the projection center position of the candidate image is obtained from the position data of the candidate image, thereby calculating the candidate imageAnd a distance difference value between the projection center position of the image and the center point position of the to-be-modified model area.
S402, carrying out weighted analysis according to the distance difference value and the included angle difference value, and obtaining an analysis result.
Specifically, the weighted analysis algorithm W Rm Can be expressed as:
wherein w is A Is the weight corresponding to the difference value of the included angle, w D And carrying out weighted analysis calculation on the distance difference value and the included angle difference value of each candidate image and the to-be-modified model area according to a weighted analysis algorithm for the weight corresponding to the distance difference value, so as to obtain an analysis result of each candidate image.
S403, determining the optimal image in the candidate images according to the analysis result.
And sequencing the candidate images according to the analysis results of the candidate images, so that the candidate image with the highest score in the analysis results is selected as the optimal image.
According to the method provided by the embodiment of the application, the distance difference value is calculated according to the projection center position of the candidate image and the center point position of the to-be-modified model area, weighted analysis is carried out according to the distance difference value and the included angle difference value, an analysis result is obtained, and finally, according to the analysis result, the optimal image is determined in the candidate image, so that the workload of manually searching the image corresponding to the to-be-modified model area can be reduced, and the image conforming to the to-be-modified model area can be determined more accurately.
The embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and attitude determination. Fig. 5 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 5, the texture mapping is performed on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image, and before the optimal model is obtained, the method further includes:
s501, performing image correction on the optimal image according to preset parameters.
In this embodiment, since the influence of different shooting angles exists in the process of acquiring the optimal image through the terminal device, the image is deformed and distorted, so that the optimal image is corrected according to the preset parameters, and the preset parameters may be correction parameters or distortion parameters of the terminal device.
S502, obtaining the optimal image to be mapped according to the corrected image data of the optimal image and the normal vector of the plane where the to-be-modified model area is located.
Specifically, according to the image data of the corrected optimal image and the plane where the to-be-modified model area is located, projecting the corrected optimal image onto the plane, and then generating a parallel projection image of a target area in the to-be-modified model area through rasterization and resampling by using a normal vector of the to-be-modified model area, so as to obtain the to-be-mapped optimal image.
The performing texture mapping on the to-be-modified model region in the preset three-dimensional model by adopting the optimal image to obtain an optimized model comprises the following steps:
s503, performing texture mapping on a to-be-modified model area in a preset three-dimensional model by adopting an optimal image to be mapped to obtain an optimal model.
According to the step S502, an optimal image to be mapped is obtained, and a preset mapping model is adopted to perform texture mapping on a preset three-dimensional model and the optimal image, wherein different preset three-dimensional models and optimal images can be subjected to texture mapping by adopting different preset mapping models, and finally, a region and a neighborhood of the model to be modified of the preset three-dimensional model are subjected to dodging and dodging treatment, and finally, the texture mapping is completed, so that an optimal model is obtained.
In the method provided by the embodiment of the application, the image correction is performed on the optimal image according to the preset parameters, the optimal image to be mapped is obtained according to the corrected posture angle data of the optimal image and the normal vector of the plane where the to-be-modified model area is located, and the texture mapping is performed on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to be mapped, so that the optimal model is obtained. Therefore, the optimal image to be mapped is obtained through correction of the optimal image, so that the optimal image to be mapped is more matched with the area of the model to be modified, and the modification efficiency of the preset three-dimensional model can be improved.
The embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and attitude determination. Fig. 6 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 6, after obtaining the optimal image to be mapped according to the corrected attitude angle data of the optimal image and the normal vector of the plane where the model area to be modified is located, the method further includes:
s601, if the optimal image to be mapped cannot cover the to-be-modified model area, respectively acquiring a plurality of optimal images to be mapped through a plurality of candidate images with overlapping degrees.
In the embodiment of the present application, if the range of the to-be-modified model area is too large, a single image cannot cover the to-be-modified model area completely, then multiple candidate images with overlapping degree may be selected, and multiple optimal images to be mapped with overlapping degree are obtained from the multiple candidate images with overlapping degree by adopting the steps S301-S403.
S602, processing and splicing the plurality of optimal images to be mapped to obtain the final optimal image to be mapped.
And processing and splicing the optimal images to be mapped through the steps of image correction, image matching pair generation, feature extraction matching, image transformation dodging and the like to obtain the final optimal image to be mapped.
In the method provided by the embodiment of the application, if the optimal image to be mapped cannot cover the area of the model to be modified, a plurality of optimal images to be mapped are respectively obtained through a plurality of candidate images with overlapping degree, the optimal images to be mapped are processed and spliced to obtain the final optimal image to be mapped, texture modification of a larger area is completed through single image splicing, multiple texture mapping is not needed, the problem of image distortion of mapping results is avoided, and the automation degree of model modification is improved.
The embodiment of the application also provides another possible implementation mode of the mobile phone image mapping method assisted by positioning and attitude determination. Fig. 7 is a flowchart of a mobile phone image mapping method with positioning and gesture assistance according to an embodiment of the present application. As shown in fig. 7, capturing a plurality of images includes:
s701, acquiring a plurality of images through terminal equipment to acquire an image set.
In this embodiment of the present application, the terminal device may be a smart phone, a tablet, a camera, or the like, and multiple images are collected by the terminal device, and the image set may be represented as i= { I i And recording high-precision position information and attitude angle information of the plurality of images.
S702, position data and attitude angle data of each image in the image set are given through position information and attitude information recorded by the terminal equipment, and a pose file corresponding to the image set is generated.
The terminal equipment has data processing capability and transmission capability, and further comprises positioning and attitude determination equipment, wherein the positioning and attitude determination equipment has the function of integrating real-time differential positioning and inertial measurement units, and is connected with equipment such as a smart phone, a tablet, a camera and the like. Position information and attitude information recorded by the terminal device can be given to position data and attitude angle data of each image in the image set, and the position data can be expressed as p by way of example i ＝{x i ，y i ，z i The attitude angle data may be represented as a i ＝{yaw i ，pitch i ，roll i And generating position data and attitude angle data of each image in the image set into an attitude file corresponding to the image set by using the positioning and attitude determination equipment, so that the position data and the attitude angle data are conveniently derived from the terminal equipment for use.
In the method provided by the embodiment of the application, a plurality of images are acquired by using the terminal equipment, an image set is established, and the position data and the attitude angle data of each image in the image set are generated into the pose file corresponding to the image set according to the positioning and attitude determination equipment in the terminal equipment, so that the corresponding images can be conveniently searched and previewed according to the space position of the to-be-modified model area of the preset three-dimensional model and the pose file corresponding to the image set.
The following further explains the image mapping apparatus, the computer device, and the storage medium provided in any of the embodiments of the present application, and specific implementation processes and technical effects thereof are the same as those of the corresponding method embodiments, and for brevity, reference may be made to corresponding contents in the method embodiments for the parts not mentioned in the present embodiment.
Fig. 8 is a schematic functional block diagram of a mobile phone image mapping device with positioning and gesture determination assistance according to an embodiment of the present application. As shown in fig. 8, the mobile phone image mapping device 100 with positioning and posture assistance includes:
the acquisition module 110 is used for acquiring a plurality of images;
the selecting module 120 is configured to select at least one candidate image with position information meeting the condition from the plurality of images according to a to-be-modified model area in the preset three-dimensional model;
the obtaining module 130 is configured to perform weighted analysis on the candidate images to obtain an optimal image;
the mapping module 140 is configured to perform texture mapping on a to-be-modified model area in the preset three-dimensional model by using the optimal image, so as to obtain an optimized model.
In an alternative embodiment, the selecting module 120 is further configured to obtain a to-be-modified model area in the preset three-dimensional model; calculating the position information of the to-be-modified model area and determining a coordinate system of a preset three-dimensional model, wherein the position information of the to-be-modified model area comprises: normal vector and center point position of the model area to be modified; and selecting at least one candidate image with the position information meeting the condition from the plurality of images according to the position information of the to-be-modified model area and the position data of the images.
In an alternative embodiment, the selection module 120 is further configured to calculate a viewing angle orientation vector of each image according to the pose angle data of the image; calculating an included angle difference value according to the visual angle orientation vector of the image and the normal vector of the to-be-modified model area; and selecting the influence meeting the condition as a candidate image according to the included angle difference value and a preset threshold value, wherein the candidate image comprises a region corresponding to the to-be-modified model region.
In an optional embodiment, the obtaining module 130 is further configured to calculate a distance difference value according to the projection center position of the candidate image and the center point position of the model area to be modified; according to the distance difference value and the included angle difference value, carrying out weighted analysis to obtain an analysis result; and determining the optimal image in the candidate images according to the analysis result.
In an alternative embodiment, the mobile phone image mapping device 100 with positioning and gesture assistance further includes:
the correction module is used for carrying out image correction on the optimal image according to preset parameters; obtaining an optimal image to be mapped according to the corrected image data of the optimal image and the normal vector of the plane where the to-be-modified model area is located; and performing texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the to-be-mapped optimal image to obtain an optimized model.
In an alternative embodiment, the mobile phone image mapping device 100 with positioning and gesture assistance further includes:
the splicing module is used for respectively acquiring a plurality of optimal images to be mapped through a plurality of candidate images with overlapping degree if the optimal images to be mapped cannot cover the area of the model to be modified; and processing and splicing the plurality of optimal images to be mapped to obtain the final optimal image to be mapped.
In an alternative embodiment, the acquisition module 110 is further configured to acquire a plurality of images through the terminal device, and acquire an image set; and giving the position data and the attitude angle data of each image in the image set through the position information and the attitude information recorded by the terminal equipment, and generating a pose file corresponding to the image set.
The foregoing apparatus is used for executing the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, and are not described herein again.
The above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASICs), or one or more microprocessors, or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGAs), etc. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 9 is a schematic diagram of a computer device provided in an embodiment of the present application, where the computer device may be used for positioning and positioning-assisted mobile phone image mapping. As shown in fig. 9, the computer device 200 includes: a processor 210, a storage medium 220, and a bus 230.
The storage medium 220 stores machine-readable instructions executable by the processor 210. When the computer device is running, the processor 210 communicates with the storage medium 220 via the bus 230, and the processor 210 executes the machine-readable instructions to perform the steps of the method embodiments described above. The specific implementation manner and the technical effect are similar, and are not repeated here.
Optionally, the present application further provides a storage medium 220, where the storage medium 220 stores a computer program, which when executed by a processor performs the steps of the above-mentioned method embodiments. The specific implementation manner and the technical effect are similar, and are not repeated here.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (english: processor) to perform some of the steps of the methods according to the embodiments of the invention. And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
The foregoing is merely illustrative of embodiments of the present invention, and the present invention is not limited thereto, and any changes or substitutions can be easily made by those skilled in the art within the technical scope of the present invention, and the present invention is intended to be covered by the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (10)
1. A mobile phone image mapping method assisted by positioning and attitude determination is characterized by comprising the following steps:
collecting a plurality of images;
selecting at least one candidate image with position information meeting the condition from a plurality of images according to a to-be-modified model area in a preset three-dimensional model;
performing weighted analysis on the candidate images to obtain optimal images;
and performing texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model.
2. The method of claim 1, wherein the image data comprises: position data and attitude angle data;
according to the to-be-modified model area in the preset three-dimensional model, selecting at least one candidate image with position information meeting the condition from the plurality of images, wherein the method comprises the following steps:
acquiring the to-be-modified model area in a preset three-dimensional model;
calculating the position information of the to-be-modified model area and determining a coordinate system of the preset three-dimensional model, wherein the position information of the to-be-modified model area comprises: the normal vector and the center point position of the to-be-modified model area;
and selecting at least one candidate image with the position information meeting the condition from the plurality of images according to the position information of the to-be-modified model area and the position data of the plurality of images.
3. The method of claim 2, wherein selecting at least one candidate image from the plurality of images, the position information of which meets the condition, based on the position information of the model area to be modified and the position data of the plurality of images, comprises:
according to the attitude angle data of the images, respectively calculating the view angle orientation vector of each image;
calculating an included angle difference value according to the visual angle orientation vector of the image and the normal vector of the to-be-modified model area;
and selecting the images meeting the conditions as the candidate images according to the included angle difference value and a preset threshold, wherein the candidate images comprise the areas corresponding to the to-be-modified model areas.
4. The method of claim 3, wherein said weighting the candidate images to obtain an optimal image comprises:
calculating a distance difference value according to the projection center position of the candidate image and the center point position of the to-be-modified model area;
according to the distance difference value and the included angle difference value, carrying out weighted analysis to obtain an analysis result;
and determining an optimal image in the candidate images according to the analysis result.
5. The method of claim 1, wherein the performing texture mapping on the to-be-modified model region in the preset three-dimensional model using the optimal image, before obtaining the optimal model, further comprises:
performing image correction on the optimal image according to preset parameters;
obtaining an optimal image to be mapped according to the corrected image data of the optimal image and the normal vector of the plane where the to-be-modified model area is located;
and performing texture mapping on a to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model, wherein the method comprises the following steps:
and performing texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the to-be-mapped optimal image to obtain an optimal model.
6. The method of claim 5, wherein after obtaining the optimal image to be mapped according to the corrected pose angle data of the optimal image and the normal vector of the plane where the to-be-modified model area is located, further comprising:
if the optimal image to be mapped cannot cover the to-be-modified model area, respectively acquiring a plurality of optimal images to be mapped through a plurality of candidate images with overlapping degrees;
and processing and splicing the plurality of optimal images to be mapped to obtain the final optimal image to be mapped.
7. The method of claim 2, wherein the capturing a plurality of images comprises:
acquiring a plurality of images through terminal equipment to acquire an image set;
and giving the position data and the attitude angle data of each image in the image set through the position information and the attitude information recorded by the terminal equipment, and generating a pose file corresponding to the image set.
8. An image mapping apparatus, comprising:
the acquisition module is used for acquiring a plurality of images;
the selection module is used for selecting at least one candidate image with the position information meeting the condition from the plurality of images according to a to-be-modified model area in the preset three-dimensional model;
the acquisition module is used for carrying out weighted analysis on the candidate images to acquire optimal images;
and the mapping module is used for carrying out texture mapping on the to-be-modified model area in the preset three-dimensional model by adopting the optimal image to obtain an optimal model.
9. A computer device, comprising: the mobile phone image mapping method comprises a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, when the computer equipment runs, the processor and the storage medium are communicated through the bus, and the processor executes the program instructions to execute the steps of the mobile phone image mapping method assisted by positioning and gesture determination according to any one of claims 1 to 7.
10. A computer readable storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the positioning and posture-determining auxiliary mobile phone image mapping method according to any one of claims 1 to 7 are executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310361768.0A CN116385621A (en) | 2023-04-06 | 2023-04-06 | Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310361768.0A CN116385621A (en) | 2023-04-06 | 2023-04-06 | Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116385621A true CN116385621A (en) | 2023-07-04 |
Family
ID=86970627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310361768.0A Pending CN116385621A (en) | 2023-04-06 | 2023-04-06 | Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116385621A (en) |
-
2023
- 2023-04-06 CN CN202310361768.0A patent/CN116385621A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106780590B (en) | Method and system for acquiring depth map | |
| CN110427917B (en) | Method and device for detecting key points | |
| JP6057298B2 (en) | Rapid 3D modeling | |
| US20120257016A1 (en) | Three-dimensional modeling apparatus, three-dimensional modeling method and computer-readable recording medium storing three-dimensional modeling program | |
| CN109697728A (en) | Data processing method, device, system and storage medium | |
| US20120069018A1 (en) | Ar process apparatus, ar process method and storage medium | |
| CN113048980B (en) | Pose optimization method and device, electronic equipment and storage medium | |
| CN113361365B (en) | Positioning method, positioning device, positioning equipment and storage medium | |
| CN113256718B (en) | Positioning method and device, equipment and storage medium | |
| CN114494388B (en) | Three-dimensional image reconstruction method, device, equipment and medium in large-view-field environment | |
| CN116433737A (en) | Method and device for registering laser radar point cloud and image and intelligent terminal | |
| CN116129037B (en) | Visual touch sensor, three-dimensional reconstruction method, system, equipment and storage medium thereof | |
| CN114494905A (en) | Building identification and modeling method and device based on satellite remote sensing image | |
| CN113763478A (en) | Unmanned vehicle camera calibration method, device, equipment, storage medium and system | |
| CN115546379A (en) | Data processing method and device and computer equipment | |
| CN115631317A (en) | Tunnel lining ortho-image generation method and device, storage medium and terminal | |
| CN113362445A (en) | Method and device for reconstructing object based on point cloud data | |
| CN109166176B (en) | Three-dimensional face image generation method and device | |
| Deng et al. | Automatic true orthophoto generation based on three-dimensional building model using multiview urban aerial images | |
| CN116912417A (en) | Texture mapping method, device, equipment and storage medium based on three-dimensional reconstruction of human face | |
| CN116385621A (en) | Positioning and attitude-determining auxiliary mobile phone image mapping method, device, equipment and medium | |
| CN114972539A (en) | Machine room camera plane online calibration method, system, computer equipment and medium | |
| CN108921908B (en) | Surface light field acquisition method and device and electronic equipment | |
| CN113421292A (en) | Three-dimensional modeling detail enhancement method and device | |
| CN117671007B (en) | Displacement monitoring method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |