CN114782801A - Machine learning-based automatic control point layout method and device - Google Patents
Machine learning-based automatic control point layout method and device Download PDFInfo
- Publication number
- CN114782801A CN114782801A CN202210214335.8A CN202210214335A CN114782801A CN 114782801 A CN114782801 A CN 114782801A CN 202210214335 A CN202210214335 A CN 202210214335A CN 114782801 A CN114782801 A CN 114782801A
- Authority
- CN
- China
- Prior art keywords
- image
- control point
- image control
- point
- laid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
A control point automatic layout method and device based on machine learning are disclosed, wherein an aerial image to be laid is obtained, and image data associated with a reference image control point is selected from the aerial image to be laid; inputting the image data into a pre-trained point location learning model; utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features; marking the aerial image to be laid by using the image control point points, and generating point location records based on the image control point points; outputting the aerial image to be laid with the image control point points and the point location records, combining machine learning, and utilizing photogrammetry technology to greatly improve the automation degree of links such as control point laying, control point measurement in the three-dimensional reconstruction process and the like.
Description
Technical Field
The invention relates to the field of machine learning, in particular to an automatic aviation image control point layout method and device based on machine learning, a storage medium and computing equipment.
Background
With the continuous development of digital twin city construction and the continuous application of new technologies, the full-automatic and efficient data acquisition becomes a research hotspot, and some scholars in China make some researches and applications of automatic construction by combining with the new technologies.
Jianwangyang et al propose a design and application of a frame-type digital aerial photography instrument image quality inspection system, which realizes inspection of original images, Li Jun et al propose a method for inspecting the aerial photography data quality of unmanned aerial vehicles, find out aiming at automatic inspection of the aerial photography data of unmanned aerial vehicles, keep certain overlapping degree between the photographic images as the basis for forming stereo image pair for mapping, use an image point feature matching method for image matching, accurately identify the image overlapping area through a homography transformation model, effectively solve the difficulty that the overlapping area between the unmanned aerial vehicles is irregular and difficult to identify, verify the practicability of the inspection method based on image matching by using five feature extraction algorithms such as SIFT \ SURF \ ORB \ BRISK \ AKAZE, and the results show that the inspection method based on SIFT can better calculate the overlapping degree between the images in each scale, the speed is slightly slower, while the ORB-based inspection method is the fastest, with the disadvantage of poor reliability in some areas where feature points are not easily identifiable. On the other hand, the unmanned aerial vehicle flight attitude is unstable, and indexes such as a photo inclination angle and a rotation angle of the unmanned aerial vehicle camera shooting are easily caused to be out of limit, the POS data is used for checking the camera station altitude difference, the flight belt curvature, the photo rotation angle, the inclination angle and the like, programming is carried out based on a python programming language to realize calculation, and a set of relatively perfect unmanned aerial vehicle aerial data automatic checking system is completed.
Wu-Gui-Jun, Dong-Ping et al propose a solution for automatic layout of aerial image control points of unmanned aerial vehicles. An orthophoto map (DOM) produced by an unmanned aerial vehicle aerial photography technology by using a low-altitude digital remote sensing image technology can meet the requirement of a certain scale precision, a Position and posture of a Position and Orientation System (also called an IMU/DGPS System) can be measured in real time in the imaging process of a sensor by using a POS System to obtain external Orientation elements of the image, but because the POS data (mainly comprising Position information and posture information) of the unmanned aerial vehicle is not accurate due to the fact that the unmanned aerial vehicle is light and unstable, a large number of image control points (image control points) are needed to improve aerial three-encryption precision when the DOM product is produced by using the unmanned aerial vehicle aerial photography, the image control point arrangement design is used to improve the image control point arrangement efficiency, a network model is automatically designed under the assistance of the POS data by using a program according to the set image control density and the number of image control side-line intervals, an image control distribution collinear map is generated, and the corresponding image point Position of each node on the image control distribution network on the image control distribution is matched based on an equation, and generating a aerial photo image calibrated with the 'image control selectable range'.
However, in the three-dimensional reconstruction automation software such as ContextCapture, Photomesh, smart3d, and the like, in the interior production link, a large amount of interior and exterior staff are still required to manually perform manual control point interpretation and stab selection, and the production automation degree is not high in practice, and needs to be further improved.
In the prior art, there is no technology for improving production automation by using machine learning to output control point positions and related images in an image plane format.
In chinese patent document CN112484704A, a fast mapping method is provided, in order to improve the automation degree of fast mapping, so that mapping is more gradual, and adaptability is improved, a target mapping point is directly calibrated through machine learning, and thus a mapping result is obtained quickly. The target image is identified in a manual judgment mode or a machine learning mode, and then the accurate position information of the target surveying and mapping point is determined in a manual adjustment mode, so that the accuracy and the applicability of real-time calibration and real-time surveying and mapping are improved.
Chinese patent document CN109076173A describes an output image generation method. The ground station converts the position and the posture obtained by the calculation into the position and the posture under the world coordinate system based on the preset GPS information of the image control point, in the implementation mode, the relative position of the image control point in a first image obtained by the shooting of a first shooting device and the relative position of the image control point in a second image obtained by the shooting of a second shooting device are searched in a manual mode, then the relative position and the posture obtained by the calculation are converted into the position and the posture under the world coordinate system based on the relative position of the image control point and the GPS information of the image control point, or the image control point is identified in the area through a preset machine learning model and an optimization algorithm based on the GPS information of the image control point and the difference between the first image obtained by the shooting of the first shooting device and the second image obtained by the shooting of the second shooting device respectively and the area where the image control point possibly exists in the image, the relative positions and postures of the first shooting equipment and the second shooting equipment when the images are re-shot are converted into the positions and postures under the world coordinate system based on the relative positions of the image control points and the GPS information.
The research team of the invention performs enough sample training, and performs image plane formatted output on the control point position output by machine learning and related images by combining the machine learning and photogrammetry technologies, thereby greatly improving the production automation level.
Disclosure of Invention
In view of this, in order to solve the technical problems in the prior art, the present invention aims to provide a control point automatic layout method and apparatus based on machine learning, a storage medium, and a computing device, which can combine machine learning with the traditional photogrammetry technology to process and apply image features, integrate and automatically combine the links of traditional control point layout and control point measurement in the three-dimensional reconstruction process, reduce human intervention, realize software automation processing, perform data control point puncturing on the basis of guaranteeing data accuracy, reduce human intervention, and greatly improve the degree of production automation.
According to one aspect of the invention, a machine learning-based control point automatic layout method is provided, which is characterized by comprising the following steps:
acquiring an aerial image to be laid, and selecting image data associated with a reference image control point from the aerial image to be laid; the reference image control point is determined from the aerial image to be laid in advance;
inputting the image data into a point location learning model trained in advance; utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features;
marking the aerial image to be laid by using the image control point points, and generating point location records based on the image control point points;
and outputting the aerial image to be laid with the image control point points and the point position record.
Preferably, the aerial image to be laid is an aerial image.
Optionally, before the image data is input into a point location learning model trained in advance and the point location learning model is used to learn and obtain the image control point graphic feature corresponding to the image data, the method further includes:
acquiring at least one group of reference images and a plurality of characteristic graphs associated with image control points, manufacturing a mask image corresponding to the reference images by using the characteristic graphs, and determining a pricking point position corresponding to the mask image;
creating a feature data set according to the reference image, the feature graph, the mask image and the puncture point position;
and constructing a machine learning model based on a neural network, and training the machine learning model by using the characteristic data set to obtain a point location learning model which can obtain a corresponding image control point characteristic graph and an image control point location by referring to an image.
Optionally, the obtaining the aerial image to be laid, and the selecting the image data associated with the reference image control point from the aerial image to be laid includes:
acquiring an aerial image to be laid, calculating and reasonably laying reference image control points corresponding to the aerial image to be laid by utilizing the course and the lateral overlapping degree of a photogrammetry principle;
and selecting downward-looking image data associated with the reference image control point from the aerial image to be laid by combining a positioning and attitude determining system.
Optionally, the image data is input into a point location learning model trained in advance; utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features, wherein the method comprises the following steps:
inputting the downward-looking image data into a pre-trained point location learning model, and learning by using the point location learning model to obtain image control point graphic features corresponding to the downward-looking image data;
and automatically determining a first image plane position and a first point number for laying the image control points in the downward-looking image according to the image control point graphic characteristics.
Optionally, after the combined positioning and attitude determination system selects downward-looking image data associated with the reference image control point from the aerial image to be laid, the method further includes:
side-looking image data in the aerial image to be laid are calculated, and side-looking graphic features corresponding to the side-looking image data are obtained;
comparing the image control point graphic features with the side view graphic features, and selecting target image data with the similarity larger than a set threshold value with the feature similarity of the lower view image data from the image side view image data;
and calculating a second image plane position and a second point number for laying image control points in the target image data based on the side view graphic features.
Optionally, the marking the aerial image to be laid by using the image control point location, and generating a point location record based on the image control point location includes:
optimizing image control points under all lenses in the image data by combining the first image plane position, the first point number, the second image plane position and the second point number according to the modes of feature matching and homonymy point matching, and selecting image control point points and corresponding image plane position information which are finally used for marking the image data so as to mark the aerial image to be laid by using the selected image control point points;
and generating point location records according to the image control point locations and the corresponding image plane location information.
Optionally, after the outputting the aerial image to be laid with the image control point location and the point location record, the method further includes:
acquiring a spatial position information table acquired by field operation;
and matching and formatting the spatial position information table and the image plane position information corresponding to the image control point points.
According to a second aspect of the present invention, there is provided an aerial image control point automatic layout device based on machine learning, the device comprising:
the data acquisition module is used for acquiring the aerial image to be laid and selecting image data associated with the reference image control point from the aerial image to be laid; the reference image control point is determined from the aerial image to be laid in advance according to a photogrammetric principle;
the feature learning module is used for inputting the image data into a pre-trained point location learning model and obtaining image control point graphic features corresponding to the image data by utilizing the point location learning model;
the point location determining module is used for determining the image control point location corresponding to the image data according to the image control point graphic feature; marking the aerial image to be laid by using the image control point points, and generating point location records based on the image control point points;
and the result output module is used for outputting the aerial image to be laid with the image control point points and the point position record.
According to a third aspect of the present invention, there is provided a computer readable storage medium for storing program code for performing the method of any one of the first aspects.
According to a fourth aspect of the invention, there is provided a computing device comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the method of any of the first aspects in accordance with instructions in the program code.
The invention provides an automatic aviation image control point layout method and device based on machine learning, a storage medium and computing equipment.
The above description is only an overview of the technical solutions of the present invention, and the present invention can be implemented in accordance with the content of the description so as to make the technical means of the present invention more clearly understood, and the above and other objects, features, and advantages of the present invention will be more clearly understood.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof taken in conjunction with the accompanying drawings.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 is a flow chart of a method for automatically laying control points based on machine learning according to an embodiment of the invention;
FIG. 2 is a schematic flow chart of a point location learning model which is pre-established and trained according to an embodiment of the invention;
FIG. 3 is a diagram illustrating a point location learning model according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating a point location learning model pre-established and trained according to an embodiment of the present invention;
fig. 5 shows a schematic structural diagram of a control point automatic layout device based on machine learning according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
The image control point is the basis for encrypting and mapping aerial photography mapping control. Only if each image control point is arranged according to a certain standard, data can be better processed by the industry, and the three-dimensional model can reach certain precision. The image control point is the basis for encrypting and mapping aerial photography mapping control. Only if each image control point is arranged according to a certain standard, data can be better processed by the interior industry, and the three-dimensional model can reach a certain precision.
The embodiment of the invention provides an automatic layout method of aerial image control points based on machine learning, and referring to fig. 1, the automatic layout method of aerial image control points based on machine learning of the embodiment of the invention can comprise the following steps of S1-S4.
S1, acquiring an aerial image to be laid, and selecting image data associated with a reference image control point from the aerial image to be laid; and the reference image control point is determined from the aerial image to be laid in advance according to the photogrammetry principle.
The aerial image to be laid in the embodiment is based on the aerial image of any area needing to be laid, which is shot by an unmanned aerial vehicle or other equipment. The acquired aerial image to be laid can comprise multiple frames of images with different shooting angles, and after the aerial image to be laid is acquired, image data associated with the reference image control point can be further selected from the aerial image to be laid. The reference control point in this embodiment is determined in advance from the aerial image to be laid according to the principle of photogrammetry. It is understood that the control point to be referred to is a rough position of the image control point determined in advance from the aerial image to be laid according to experience or photogrammetry principles.
S2, inputting the image data into a point location learning model trained in advance; and utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features.
In this embodiment, a machine learning manner is utilized, and a point location learning model is pre-constructed and trained, so that a corresponding image control point graphic feature can be automatically obtained according to image data by utilizing the point location learning model, and an image control point location corresponding to the image data is determined according to the image control point graphic feature. The point location learning model can be obtained by training according to a large number of common feature patterns, so that the image control point feature patterns corresponding to the image data obtained by learning with the point location learning model can be used as reliable feature information to be used as a reference for subsequent information matching.
And S3, marking the aerial image to be laid by using the image control point points, and generating point location records based on the image control point points.
And S4, outputting the aerial image to be laid with the image control point and the point record.
After the image control point graphic features and the corresponding image control point points are obtained by learning through the point location learning model, the aerial image to be laid can be marked through the image control point points, meanwhile, point location records can be generated, and finally the aerial image to be laid with the image control point points and the point location records are output as result data. The method provided by the embodiment combines machine learning and photogrammetry to realize automatic selection of the graphic characteristics and the position of the optimal control point, thereby ensuring the convenience of field measurement control points and the reliability of interior inspection.
In an optional embodiment of the present invention, the image data obtained in the step S1 may be downward-looking image data, that is, the step S1 obtains an aerial image to be laid, and selecting the image data associated with the reference image control point from the aerial image to be laid may include:
and S1-1, acquiring the aerial image to be laid, calculating by using the course and the lateral overlapping degree of the photogrammetry principle, and reasonably laying the reference image control points corresponding to the aerial image to be laid.
Specifically, f, s, w and H respectively represent a focal length, a ground distance, an image plane width and a relative altitude according to a triangle similarity principle, s, w and H can be obtained according to the formula, a three-degree overlapping region and a six-degree overlapping region can be calculated according to different overlapping degrees of the side direction and the heading, and a proper layout point location region is selected as a candidate region of the to-be-punctured point location according to the requirements of different precision grades and across a specified strip base line length and a navigation band.
And S1-2, selecting downward-looking image data associated with the reference image control point from the aerial image to be laid by combining a positioning and attitude determination system.
In the embodiment, the course with the course overlapping degree of 60% -80% and the lane with the side overlapping degree of 15% -60% can be determined according to the course side overlapping degree, the reference control point position is further calculated and reasonably arranged according to the selected lane meeting the requirements, and meanwhile, the image related to the control point position is selected by combining with a Position and Orientation System (POS) to be used as the above mentioned lower-view image data. Generally, the image control points are uniformly distributed according to the whole area of the route, the image control points between the adjacent images and the adjacent routes are shared as much as possible, and the image control point positions are selected as obvious target points on the images.
In the above step S2, the image data is input into a point location learning model trained in advance, optionally, this embodiment may further include S5 that a point location learning model is established and trained in advance, and specifically, as shown in fig. 2, the following steps S5-1 to S5-3 may be included.
S5-1, acquiring at least one group of reference images and a plurality of characteristic graphs related to image control points, making mask images corresponding to the reference images by using the characteristic graphs, and determining the positions of the pricking points corresponding to the mask images. The reference image in this embodiment may be another image in which image control points are already laid, and the feature pattern may be a zebra-crossing corner point, such as a fixed field corner, a field corner, and a grass corner that are approximately rectangular and approximately horizontal, a fixed road intersection that is approximately rectangular and intersects in a horizontal plane, a wall corner of a large building with a flat roof, and the like, and may be used as the feature pattern.
S5-2, creating a feature data set according to the reference image, the feature graph, the mask image and the puncture point position. For the obtained reference image and the feature graph, a mask image corresponding to the reference image can be made, and meanwhile, a feature data set can be created by combining the puncture point position corresponding to the mask image.
S5-3, constructing a machine learning model based on the neural network, and training the machine learning model by using the characteristic data set to obtain a point position learning model capable of obtaining a corresponding image control point characteristic graph and an image control point position by referring to an image.
The model structure mainly comprises four down-sampling blocks and four up-sampling blocks, wherein each down-sampling block is connected through maximum pooling, each up-sampling block is connected through one convolution, the output of each up-sampling block is used as the input of the next layer for correlation, a closed ring is formed from the input to the output of the model, the model is combined by using a basic structure, and the model forms an end-to-end processing flow. The specific structure is shown in fig. 3.
The characteristic data set can be randomly divided into a training set and a verification set, and then the training set and the verification set are used for carrying out multi-round training and verification on the machine learning model, so that a point location learning model meeting requirements is obtained, and then the point location learning model is used for accurately learning characteristic graphs and image control point locations of images.
In the above embodiment, the image data in step S1 may include downward-looking image data, and further, the step S2 may further include inputting the image data into a point location learning model trained in advance, and obtaining the image control point pattern feature corresponding to the image data by using the point location learning model, and further including:
and S2-1, inputting the downward-looking image data into a point location learning model trained in advance, and learning by using the point location learning model to obtain the image control point graphic characteristics corresponding to the downward-looking image data. In practical application, the data characteristics are mainly obtained through priori knowledge obtained by early-stage model learning.
S2-2, automatically determining the first image plane position and the first point number of the image control point laid in the lower-view image according to the image control point graphic feature. In this embodiment, according to the region obtained from the positions of the previously laid image control points, the geometric features and the image plane positions of the control points are obtained by combining the relevant features in the region and the priori knowledge obtained by machine learning, and the positions are sequentially numbered by adopting sequence number self-increment according to a snake-shaped coding mode, so that the point positions are conveniently counted.
In this embodiment, according to the image control point graphic features obtained through the point location learning model, the image plane position and the point number of the arrangement of the downward-looking control points in the downward-looking image can be automatically marked as the first image plane position and the first point number.
In the embodiment of the present invention, in addition to learning the downward-looking image data in the image data, S6 may further learn to obtain corresponding image control points according to the image data of other viewing angles. That is, the present embodiment further includes the steps of:
and S6-1, calculating the side-looking image data in the aerial image to be laid by utilizing a collinear equation of a photogrammetry principle, and acquiring side-looking graphic features corresponding to the side-looking image data. Optionally, when the side view graphic features corresponding to the side view image data are obtained, the side view graphic features can also be obtained by using the point location learning model, or obtained by calculating according to other features.
The collinearity equation is calculated as follows:
wherein R is a rotation matrix, ai、bi、ciAnd (i is 1, 2 and 3) is direction cosine, is a cosine value of an included angle between two coordinate shafting, and consists of a pitch angle psi, a roll angle omega and a sine-cosine of a yaw angle kappa. (X, Y, -f) are the coordinates of the image point in the image space coordinate system, and the coordinates of the image point in the image space auxiliary coordinate system are (X, Y, Z). Wherein:
because the three are collinear, the following three points can be obtained according to the similarity principle of the triangle:
after the above formula is finished, the following formula can be obtained
The image plane coordinate system obtained by arranging all the above formulas is as follows:
XS，YS，ZSthe object space coordinates of the filming points are set;
x, Y and Z are object space coordinates of the object space points.
And converting each point of the image feature into different side-looking images by utilizing the calculation of a collinear equation, thereby obtaining the geometric shapes of the image feature under different lenses.
And S6-2, comparing the image control point graphic features with the side view graphic features, and selecting target image data with the similarity greater than a set threshold with the feature similarity of the lower view image data from the side view image data so as to take the geometric feature points in the target image data as the control points to be stabbed.
And S6-3, calculating a second image plane position and a second point number of the image control points arranged in the target image data based on the side view graphic features.
In this embodiment, the side view graphic feature in S6-1 is compared with the image control point graphic feature obtained in S2-1, and a target image feature with high similarity is selected and a control point to be punctured is determined, so that subsequent point location labeling of the aerial image to be laid is further completed.
Optionally, in the step S3, labeling the aerial image to be laid with the image control point location, and generating a point location record based on the image control point location may include:
and S3-1, optimizing image control points under all lenses in the image data by combining the first image plane position, the first point number, the second image plane position and the second point number according to a characteristic matching and homonymy point matching mode, and selecting the image control point positions finally used for marking the image data and corresponding image plane position information so as to mark the aerial image to be laid by using the selected image control point positions (refer to the step S6-1 for calculating all the point positions).
And S3-2, generating point location records according to the image control point locations and the corresponding image plane location information.
The result data, the image control point distribution sketch map and the format are KMZ, and the distribution situation of the image control points can be known through the image control point distribution sketch map so as to plan the journey. And formatting and outputting the downward-looking images with the labels and the corresponding point location record table, acquiring the geographic coordinate information of the control points for field operation, and performing image plane formatting and outputting the control point locations and the related images which are output by machine learning.
In this embodiment, after the step S4, the method further includes: and S7, performing data organization on the internal operation data and the external operation data.
S7-1, acquiring a spatial position information table acquired by field work;
and S7-2, matching and formatting the spatial position information table and the image plane position information corresponding to the image control point positions for storage. The field GPS measures the point location information according to the point location serial numbers given in the foregoing, stores files corresponding to the coordinate system, matches the field location information to a plurality of corresponding image plane positions by using a serial number matching mode to form one-to-many coordinate serial number matching data, and stores the coordinate serial number matching data according to a format of a space rectangular coordinate system-image plane coordinate system.
And storing the formatted data which is sorted out, customizing and automatically sorting the formatted data according to a format which can be identified by three-dimensional reconstruction software, and outputting information of control points, wherein the information is used for realizing automatic identification of the data which is imported into the software, and the workload of the human common stabbing points is reduced.
The invention provides an automatic layout method of aerial image control points based on machine learning, which realizes automatic selection of graphic features and positions of optimal control points through superposition degree calculation and image feature calculation, thereby ensuring the convenience of field measurement control points and the reliability of interior work inspection. In addition, the method provided by the embodiment can also acquire image features and different types of control points at different visual angles, can output the coordinate positions of the control points in the image planes of different lens images, and can directly utilize the output result for the three-dimensional reconstruction software, thereby reducing human intervention and labor cost.
Based on the same inventive concept, an embodiment of the present invention further provides an automatic layout apparatus for aerial image control points based on machine learning, as shown in fig. 3, the automatic layout apparatus for aerial image control points based on machine learning includes:
the data acquisition module 310 is configured to acquire an aerial image to be laid, and select image data associated with a reference image control point from the aerial image to be laid; the reference image control point is determined from the aerial image to be laid in advance according to a photogrammetric principle;
the feature learning module 320 is configured to input the image data into a point location learning model trained in advance, and learn by using the point location learning model to obtain an image control point graphic feature corresponding to the image data; determining image control point positions corresponding to the image data according to the image control point graphic features;
the marking module 330 is configured to mark the aerial image to be laid with the image control point location, and generate a point location record based on the image control point location;
and the result output module 340 is configured to output the to-be-laid aerial image with the image control point location and the point location record.
In an alternative embodiment of the present invention, as shown in fig. 4, the apparatus for automatically laying out aerial image control points based on machine learning may further include a model training module 350.
A model training module 350, configured to obtain at least one group of reference images and a plurality of feature patterns associated with image control points, make a mask image corresponding to the reference image using the feature patterns, and determine a pricking point position corresponding to the mask image;
creating a feature data set according to the reference image, the feature graph, the mask image and the puncture point position;
and constructing a machine learning model based on a neural network, and training the machine learning model by using the characteristic data set to obtain a point location learning model which can obtain a corresponding image control point characteristic graph and an image control point location by referring to an image. The functions and specific implementation manners of each module in the device for automatically laying out aerial image control points based on machine learning according to this embodiment may refer to the description of the above method for automatically laying out aerial image control points based on machine learning.
An embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium is configured to store a program code, and the program code is configured to execute the method described in the foregoing embodiment.
An embodiment of the present invention further provides a computing device, where the computing device includes a processor and a memory: the memory is used for storing program codes and transmitting the program codes to the processor; the processor is configured to execute the method according to the above embodiment according to the instructions in the program code.
It is clear to those skilled in the art that the specific working processes of the above-described systems, devices, modules and units may refer to the corresponding processes in the foregoing method embodiments, and for the sake of brevity, further description is omitted here.
In addition, the functional units in the embodiments of the present invention may be physically independent of each other, two or more functional units may be integrated together, or all the functional units may be integrated in one processing unit. The integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.
Those of ordinary skill in the art will understand that: the integrated functional units, if implemented in software and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be essentially or wholly or partially embodied in the form of a software product, which is stored in a storage medium and includes several instructions, so that a computing device (for example, a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method according to the embodiments of the present invention when executing the instructions. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
Alternatively, all or part of the steps of the method embodiments may be implemented by hardware (such as a personal computer, a server, or a network device) related to program instructions, which may be stored in a computer-readable storage medium, and when the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method according to the embodiments of the present invention.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments can be modified or some or all of the technical features can be replaced with equivalents within the spirit and principle of the present invention; such modifications or substitutions do not depart from the scope of the present invention.
Claims (10)
1. A control point automatic layout method based on machine learning is characterized by comprising the following steps:
acquiring an aerial image to be laid, and selecting image data associated with a reference image control point from the aerial image to be laid; the reference image control point is determined from the aerial image to be laid in advance;
inputting the image data into a pre-trained point location learning model; utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features;
marking the aerial image to be laid by using the image control point points, and generating point location records based on the image control point points;
and outputting the aerial image to be laid with the image control point points and the point position record.
2. The method according to claim 1, wherein before the image data is input to a point location learning model trained in advance and the point location learning model is used to learn and obtain the image control point graphic feature corresponding to the image data, the method further comprises:
acquiring at least one group of reference images and a plurality of characteristic graphs associated with image control points, manufacturing a mask image corresponding to the reference images by using the characteristic graphs, and determining a pricking point position corresponding to the mask image;
creating a feature data set according to the reference image, the feature graph, the mask image and the puncture point position;
and constructing a machine learning model based on a neural network, and training the machine learning model by using the characteristic data set to obtain a point position learning model capable of obtaining a corresponding image control point characteristic graph and an image control point position by referring to an image.
3. The method as claimed in claim 1, wherein the acquiring the aerial image to be laid, and the selecting the image data associated with the reference image control point from the aerial image to be laid comprises:
acquiring an aerial image to be laid, calculating and reasonably laying reference image control points corresponding to the aerial image to be laid by utilizing the course and the lateral overlapping degree;
and selecting downward-looking image data associated with the reference image control point from the aerial image to be laid by combining a positioning and attitude determination system.
4. The method of claim 3, wherein the image data is input into a pre-trained point location learning model; utilizing the point location learning model to learn to obtain image control point graphic features corresponding to the image data, and determining image control point locations corresponding to the image data according to the image control point graphic features, wherein the method comprises the following steps:
inputting the downward-looking image data into a point location learning model trained in advance, and learning by using the point location learning model to obtain image control point graphic features corresponding to the downward-looking image data;
and automatically determining a first image plane position and a first point number for laying the image control points in the downward-looking image according to the image control point graphic characteristics.
5. The method of claim 4, wherein after the integrated positioning and attitude determination system selects the downward-looking image data associated with the reference image control point from the aerial image to be laid, the method further comprises:
side-looking image data in the aerial image to be laid are calculated, and side-looking graphic features corresponding to the side-looking image data are obtained;
comparing the image control point graph features with the side view graph features, and selecting target image data with the similarity of the features of the side view image data being larger than a set threshold value;
and calculating a second image plane position and a second point number of the image control points distributed in the target image data based on the side view graphic features.
6. The method according to claim 5, wherein the labeling the aerial image to be laid with the image control point locations and generating point location records based on the image control point locations comprises:
optimizing image control points under all lenses in the image data by combining the first image plane position, the first point number, the second image plane position and the second point number according to the modes of feature matching and homonymy point matching, and selecting image control point positions and corresponding image plane position information which are finally used for marking the image data so as to mark the aerial image to be laid by using the selected image control point positions;
and generating a point location record according to the image control point location and the corresponding image plane position information.
7. The method according to any one of claims 1 to 6, wherein after outputting the aerial image to be laid with the image control point location and the point location record, the method further comprises:
acquiring a spatial position information table acquired by field work;
and matching and formatting the spatial position information table and the image plane position information corresponding to the image control point points.
8. An automatic control point layout device based on machine learning, which is characterized by comprising:
the data acquisition module is used for acquiring the aerial image to be laid and selecting image data associated with the reference image control point from the aerial image to be laid; the reference image control point is determined from the aerial image to be laid in advance;
the feature learning module is used for inputting the image data into a pre-trained point location learning model and obtaining image control point graphic features corresponding to the image data by utilizing the point location learning model; determining image control point positions corresponding to the image data according to the image control point graphic features;
the marking module is used for marking the aerial image to be laid by utilizing the image control point points and generating point position records based on the image control point points;
and the result output module is used for outputting the aerial image to be laid with the image control point location and the point location record.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium is configured to store a program code for performing the method of any of claims 1-7.
10. A computing device, comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the method of any one of claims 1-7 according to instructions in the program code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214335.8A CN114782801B (en) | 2022-03-01 | 2022-03-01 | Machine learning-based automatic control point layout method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214335.8A CN114782801B (en) | 2022-03-01 | 2022-03-01 | Machine learning-based automatic control point layout method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114782801A true CN114782801A (en) | 2022-07-22 |
| CN114782801B CN114782801B (en) | 2023-01-10 |
Family
ID=82423292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210214335.8A Active CN114782801B (en) | 2022-03-01 | 2022-03-01 | Machine learning-based automatic control point layout method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114782801B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107270877A (en) * | 2017-06-22 | 2017-10-20 | 中铁大桥勘测设计院集团有限公司 | A kind of banding surveys area's low altitude photogrammetry photo control point method of layout survey |
| CN108763575A (en) * | 2018-06-06 | 2018-11-06 | 湖南省第测绘院 | Photo control point automatically selecting method based on photo control point database |
| CN110321528A (en) * | 2019-07-11 | 2019-10-11 | 生态环境部南京环境科学研究所 | A kind of Hyperspectral imaging heavy metal-polluted soil concentration evaluation method based on semi-supervised geographical space regression analysis |
| CN110335312A (en) * | 2019-06-17 | 2019-10-15 | 武汉大学 | A kind of object space localization method neural network based and device |
| CN110490802A (en) * | 2019-08-06 | 2019-11-22 | 北京观微科技有限公司 | A kind of satellite image Aircraft Targets type identifier method based on super-resolution |
| CN111678503A (en) * | 2020-06-15 | 2020-09-18 | 西安航空职业技术学院 | Unmanned aerial vehicle aerial survey control point arrangement and identification method and system |
| CN111707238A (en) * | 2020-05-29 | 2020-09-25 | 广东省国土资源测绘院 | Method and system for generating aviation digital orthophoto map |
| WO2021229030A1 (en) * | 2020-05-14 | 2021-11-18 | Asml Netherlands B.V. | Method for predicting stochastic contributors |
| CN113920262A (en) * | 2021-10-15 | 2022-01-11 | 中国矿业大学(北京) | Mining area FVC calculation method and system for enhancing edge sampling and improving Unet model |
-
2022
- 2022-03-01 CN CN202210214335.8A patent/CN114782801B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107270877A (en) * | 2017-06-22 | 2017-10-20 | 中铁大桥勘测设计院集团有限公司 | A kind of banding surveys area's low altitude photogrammetry photo control point method of layout survey |
| CN108763575A (en) * | 2018-06-06 | 2018-11-06 | 湖南省第测绘院 | Photo control point automatically selecting method based on photo control point database |
| CN110335312A (en) * | 2019-06-17 | 2019-10-15 | 武汉大学 | A kind of object space localization method neural network based and device |
| CN110321528A (en) * | 2019-07-11 | 2019-10-11 | 生态环境部南京环境科学研究所 | A kind of Hyperspectral imaging heavy metal-polluted soil concentration evaluation method based on semi-supervised geographical space regression analysis |
| CN110490802A (en) * | 2019-08-06 | 2019-11-22 | 北京观微科技有限公司 | A kind of satellite image Aircraft Targets type identifier method based on super-resolution |
| WO2021229030A1 (en) * | 2020-05-14 | 2021-11-18 | Asml Netherlands B.V. | Method for predicting stochastic contributors |
| CN111707238A (en) * | 2020-05-29 | 2020-09-25 | 广东省国土资源测绘院 | Method and system for generating aviation digital orthophoto map |
| CN111678503A (en) * | 2020-06-15 | 2020-09-18 | 西安航空职业技术学院 | Unmanned aerial vehicle aerial survey control point arrangement and identification method and system |
| CN113920262A (en) * | 2021-10-15 | 2022-01-11 | 中国矿业大学(北京) | Mining area FVC calculation method and system for enhancing edge sampling and improving Unet model |
Non-Patent Citations (3)
| Title |
|---|
| SANTIAGO LO´PEZ-TAPIA 等: "Machine learning with high-resolution aerial imagery and data fusion to improve and automate the detection of wetlands", 《INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATIONS AND GEOINFORMATION》 * |
| 宋会莹 等: "航空摄影测量像控布设的新思路与探索", 《江西建材》 * |
| 杨少愚 等: "无人机低空摄影测量在大比例尺测图及精细化建模领域的适用性分析", 《经纬天地》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114782801B (en) | 2023-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109238239B (en) | Digital measurement three-dimensional modeling method based on aerial photography | |
| CN102506824B (en) | Method for generating digital orthophoto map (DOM) by urban low altitude unmanned aerial vehicle | |
| JP5389964B2 (en) | Map information generator | |
| US20090154793A1 (en) | Digital photogrammetric method and apparatus using intergrated modeling of different types of sensors | |
| CN104268935A (en) | Feature-based airborne laser point cloud and image data fusion system and method | |
| CN104457710B (en) | Aviation digital photogrammetry method based on non-metric digital camera | |
| KR100915600B1 (en) | Method for measuring 3-dimensinal coordinates of images using a target for ground control point | |
| KR101258560B1 (en) | Setting method of Ground Control Point by Aerial Triangulation | |
| CN104360362B (en) | Method and system for positioning observed object via aircraft | |
| KR20140061156A (en) | Position detecting method of road traffic sign | |
| CN113920262B (en) | Mining area FVC calculation method and system for enhancing edge sampling and improving Unet model | |
| CN101545776A (en) | Method for obtaining digital photo orientation elements based on digital map | |
| CN107063187A (en) | A kind of height of tree rapid extracting method of total powerstation and unmanned plane image association | |
| CN116758234A (en) | Mountain terrain modeling method based on multipoint cloud data fusion | |
| Su et al. | Developing an unmanned aerial vehicle-based rapid mapping system for traffic accident investigation | |
| CN108253942B (en) | Method for improving oblique photography measurement space-three quality | |
| CN116563377A (en) | Mars rock measurement method based on hemispherical projection model | |
| JP2011112556A (en) | Search target position locating device, method, and computer program | |
| CN110715646B (en) | Map trimming measurement method and device | |
| Maurice et al. | A photogrammetric approach for map updating using UAV in Rwanda | |
| KR101006977B1 (en) | Method for supplementing map data during a productive digital map | |
| CN116594419A (en) | Routing inspection route planning method and device, electronic equipment and storage medium | |
| CN114782801B (en) | Machine learning-based automatic control point layout method and device | |
| KR100871139B1 (en) | Picture log of corrected drawing reference point coodinates input error indication method | |
| Zomrawi et al. | Accuracy evaluation of digital aerial triangulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 22 / F, building 683, zone 2, No. 5, Zhongguancun South Street, Haidian District, Beijing 100086 Applicant after: Terry digital technology (Beijing) Co.,Ltd. Address before: 100089 22 / F, building 683, zone 2, 5 Zhongguancun South Street, Haidian District, Beijing Applicant before: Terra-IT Technology (Beijing) Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |