CN118037863A - Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint - Google Patents
Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint Download PDFInfo
- Publication number
- CN118037863A CN118037863A CN202410430508.9A CN202410430508A CN118037863A CN 118037863 A CN118037863 A CN 118037863A CN 202410430508 A CN202410430508 A CN 202410430508A CN 118037863 A CN118037863 A CN 118037863A
- Authority
- CN
- China
- Prior art keywords
- focal length
- camera
- neural network
- field
- camera internal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 23
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000005457 optimization Methods 0.000 title claims abstract description 14
- 230000000007 visual effect Effects 0.000 title claims abstract description 13
- 230000006870 function Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 230000002349 favourable effect Effects 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 20
- 238000004422 calculation algorithm Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 5
- 238000002474 experimental method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Landscapes
- Studio Devices (AREA)
Abstract
The invention belongs to the technical field of zoom camera calibration, and discloses a neural network optimization automatic zoom camera internal parameter calibration method based on visual field constraint, which comprises the following steps: constraining the relation between the field of view and the focal length; step 2: optimizing the relation between the focal length and the camera internal parameters based on a neural network; step 3: automatic focusing; the invention firstly fixes the relation between the view field and the focal length, is favorable for acquiring the target pixel coordinates more accurately, and provides guarantee for final high-precision gesture solving; then, the corresponding relation between the focal length and the internal reference is optimized by utilizing the neural network, so that the accuracy is high and the efficiency is high; finally, automatic focusing is carried out, so that the complexity of repeated manual focusing is avoided.
Description
Technical Field
The invention belongs to the technical field of zoom camera calibration, and particularly relates to a neural network optimization automatic zoom camera internal parameter calibration method based on visual field constraint.
Background
The accuracy of automatic hole making of the aerial assembly robot is affected by the attitude information of the aerial assembly robot, so that the attitude measurement of the assembly robot is particularly important. At present, visual measurement has the advantages of non-contact, low cost, rich information and the like. Camera calibration plays a very important role in vision measurement. Camera calibration can be classified into fixed focus and zoom calibration. The fixed-focus lens is widely applied to the field of short-distance measurement and has higher calibration precision. There are several algorithms for calibrating the internal and external parameters of fixed-focus cameras: calibration algorithms based on direct linear Transformation (DIRECT LINEAR Transformation, DLT), two-step calibration algorithms based on radial constraints, zhang Zhengyou calibration algorithms, and calibration algorithms independent of camera distortion. Abdel-Aziz, y.i., karara, h.m., et al, in 2015, issued a direct linear transformation method from comparator coordinates to object space coordinates on photogrammetry engineering and remote sensing as the earliest camera calibration method for close-range photogrammetry. Zhang Zhengyou proposes a widely used calibration method that uses a checkerboard or circular pattern as a template to capture multiple images from different perspectives to determine the camera's internal and external parameters. Zoom lenses have inherent advantages in terms of flexibility and controllability as compared to fixed focus lenses. However, in a wide range of measurement fields, calibration challenges of zoom lens cameras limit their wide application. The simplest method for calibrating the zoom lens is to perform single focal length calibration under each zoom and focus setting, and then store the setting value and the calibration result in a lookup table. In order to reduce the calibration workload, y. -s. Chen et al published a simple and efficient method of calibrating an electric zoom lens on image and vision calculations in 2001, which selects a set of samples of lens settings for calibration, and then further constructs a sparse table to interpolate the required internal parameters. M. Sarkis et al published in 2009, automated science and engineering theory of the institute of electrical and electronics engineers, a method for calibrating an automatic zoom camera with a moving least squares method, by fitting a continuous local function of internal parameters in zoom and focus settings, calibrating a zoom lens based on a moving least squares scheme. T, xian et al issue a novel dynamic zoom calibration technology of a multi-view 3D modeling system based on stereoscopic vision on International optical engineering society in 2004, and propose a calibration method based on perspective projection, which uses polynomial fitting to simulate all parameters of a camera under zooming.
However, in many cases, the polynomial fails to find the optimal solution. It would be further advantageous to use neural network optimization methods. Focal length variations can lead to field of view variations and picture blurring.
Disclosure of Invention
The invention aims to provide a visual field constraint-based neural network optimization automatic zoom camera internal parameter calibration method.
In order to achieve the above purpose, the invention adopts the following technical scheme:
a neural network optimization automatic zoom camera internal parameter calibration method based on visual field constraint comprises the following steps:
Step 1: constraining the relationship of field of view to focal length, camera field of view to focal length: wherein S is the pixel size, x res is the resolution, f is the focal length, D is the distance between the camera and the target, and H x is the camera field of view;
step 2: optimizing the relation between the focal length and the camera internal parameters based on a neural network;
Firstly, acquiring a focal length range of a zoom camera, fixing a plurality of focal lengths at equal intervals according to the focal length range, acquiring 15-30 pictures of a shooting target under each focal length, wherein the target occupies 20% of a field of view at the center of each picture, and acquiring a plurality of groups of data sets of focal lengths and camera internal parameter relations by adopting a Zhang Zhengyou camera calibration method; wherein the camera internal parameters comprise distortion coefficients and principal point abscissas and ordinates;
Secondly, dividing a plurality of groups of data sets of focal length and camera internal reference relations into a training set and a testing set according to the proportion of 2:1, and then carrying out normalization processing on the data sets;
Then, fitting the data set by using polynomial, gaussian and trigonometric functions respectively, and performing curve optimization by using a multi-layer perceptron so as to realize calibration of internal parameters of the camera;
Step 3: automatic focusing; setting a threshold value of image definition, fixing a current focal length according to the relation between a visual field and the focal length, changing image distances at equal intervals, shooting a picture under each image distance, acquiring the definition of the picture by utilizing an image definition function, comparing the image definition with the threshold value, and changing the image distance to acquire the picture and the image definition when the image definition is smaller than the threshold value until the image definition is larger than the threshold value, wherein automatic focusing is completed.
Further, in step 2, the parameters of the multi-layer perceptron are set: the training round number is set to 10000, the characteristic number range is 1-14, the learning rate is set to 0.01, and the batch size is set to 32.
Further, the threshold value in step 3 is set to 80.
The invention firstly fixes the relation between the view field and the focal length, is favorable for acquiring the target pixel coordinates more accurately, and provides guarantee for final high-precision gesture solving; then, the corresponding relation between the focal length and the internal reference is optimized by utilizing the neural network, so that the accuracy is high and the efficiency is high; finally, automatic focusing is carried out, so that the complexity of repeated manual focusing is avoided.
The invention combines the constrained fixed view field of the view field and the focal length and the automatic focusing algorithm to make the picture clear, wherein the automatic focusing algorithm can be integrated into camera software to realize more efficient and accurate automatic camera calibration.
Drawings
FIG. 1 is a schematic flow chart of the present invention.
Fig. 2 is a graph showing the effect of fitting distortion coefficients according to an embodiment of the present invention.
Fig. 3 is a graph of a fitting effect of principal point abscissa in an embodiment of the present invention, where (a) is a polynomial and gaussian fitting effect, and (b) is a trigonometric fitting effect.
Fig. 4 is a graph of a fitting effect of principal points on the ordinate, wherein (a) is a polynomial and gaussian fitting effect, and (b) is a trigonometric fitting effect.
Fig. 5 is a graph of a fitting effect of a focal length along an X direction according to an embodiment of the present invention, where (a) is a polynomial and gaussian fitting effect, and (b) is a trigonometric fitting effect.
Fig. 6 is a graph of a fitting effect of a focal length along a Y direction according to an embodiment of the present invention, where (a) is a polynomial and gaussian fitting effect, and (b) is a trigonometric fitting effect.
Fig. 7 is a graph of a fit of distortion coefficients for an embodiment of the present invention.
Fig. 8 is a graph of a fit of the principal point abscissa of an embodiment of the present invention.
Fig. 9 is a graph of a fit of the ordinate of the principal points of an embodiment of the present invention.
Fig. 10 is a graph of a fit of focal length along the X-direction in an embodiment of the present invention.
FIG. 11 is a graph of a fit of focal length along the Y direction for an embodiment of the present invention.
Fig. 12 is an autofocus experiment of fig. 1 according to an embodiment of the present invention.
Fig. 13 is an autofocus experiment of fig. 2 according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the neural network optimization automatic zoom camera internal parameter calibration method based on field constraint provided in this embodiment includes the following steps:
Step 1: constraining the relationship of field of view to focal length
Relationship between camera field of view and focal length: Wherein S is the pixel size, x res is the resolution, and S and x res are both the camera attributes; f is a focal length, D is a distance between the camera and the target, and the distance is obtained through a laser tracker; h x is the camera field of view.
The target size of this embodiment is 800mm×800mm, the ratio of the field of view is about 20%, and the focal length is solved according to the relationship between the camera field of view and the focal length.
Step 2: optimizing focal length and camera internal reference relation based on neural network
The embodiment is based on Zhang Zhengyou camera calibration method, and utilizes a neural network to optimize a fitting curve, thereby improving the internal parameter calibration precision of the zoom camera, and the specific process is as follows:
firstly, acquiring a focal length range of a zoom camera, fixing a plurality of focal lengths at equal intervals according to the focal length range, acquiring 15-30 pictures of a shooting target under each focal length, wherein the target occupies 20% of a field of view in the center of the camera in each picture, and acquiring a plurality of groups of data sets of focal lengths and camera internal reference relations by adopting a Zhang Zhengyou camera calibration method; wherein the camera internal parameters comprise distortion coefficients and principal point abscissas and ordinates;
in this embodiment, taking a certain camera as an example, the focal length range of the camera is 16 mm-96 mm, and 30 focal lengths are equally spaced apart, so as to obtain the internal reference relationship between 30 groups of focal lengths and the camera.
Secondly, dividing a plurality of groups of data sets of focal lengths and camera internal reference relations into a training set and a testing set according to the proportion of 2:1, and carrying out normalization processing on the data sets to enable the data sets to be more concentrated so as to improve convergence speed and prediction accuracy;
Then, fitting the data set by using polynomial, gaussian and trigonometric functions respectively, and performing curve optimization by using a multi-layer perceptron so as to realize calibration of internal parameters of the camera;
Parameter setting of the multi-layer perceptron: the training round number is set to 10000, the characteristic number range is 1-14, the learning rate is set to 0.01, and the batch size is set to 32; and updating and solving the optimal weight parameter W and the bias parameter b by using the minimized root mean square error loss function, evaluating the fitting effect, visualizing the result, and selecting the best fitting curve with the smallest loss function.
Fig. 2 shows the effect of fitting the distortion coefficients of the present embodiment, fig. 3 (a) and (b) show the effect of fitting the principal point on the abscissa, fig. 4 (a) and (b) show the effect of fitting the focal length in the X direction, and fig. 5 (a) and (b) show the effect of fitting the focal length in the Y direction. As can be seen from fig. 2 to 6, the polynomial fitting effect is the best except that the ordinate of the principal point is the gaussian fitting effect.
FIG. 7 is a graph showing the relationship between distortion coefficients and focal length of the neural network after optimization y=-0.014x+0.568x2-0.525x3+0.362x4-0.556x5+0.639x6+0.581x7-0.025x8.
FIG. 8 is a graph of a neural network optimized fit with principal point abscissa, the relationship between principal point abscissa and focal length being y=1.563x+1.024x2+0.257x3-1.875x4-0.892x5-0.044x6-1.7x7+1.581x8+0.647x9.
FIG. 9 is a graph of a neural network optimized fit with a principal point ordinate, the relationship between principal point ordinate and focal length being y=-0.432exp(-(x-0.0)2/(2×12.52))+0.775exp(-(x-12.5)2/(2×12.52))-1.285exp(-(x-25.0)2/(2×12.52)).
Fig. 10 is a graph of a neural network optimized fit of a focal length along the X direction, where the relationship between the focal length X direction and the focal length is y=0.333x+0.277X 2-0.911x3+1.0x4+1.1x5-0.78x6.
FIG. 11 is a graph of a neural network optimized fit with focal length along the Y-direction, the relationship between the focal length Y-direction and the focal length being y=0.788x-0.808x2+0.353x3+0.744x4-0.363x5-0.025x6+0.764x7-0.411x8.
Step 3: automatic focusing
Setting a threshold value of image definition, fixing a current focal length according to the relation between a visual field and the focal length, changing image distances at equal intervals, shooting a picture under each image distance, acquiring the definition of the picture by utilizing an image definition function, comparing the image definition with the threshold value, and changing the image distance to acquire the picture and the image definition when the image definition is smaller than the threshold value until the image definition is larger than the threshold value, wherein automatic focusing is completed. The threshold value of this embodiment is set to 80, and the experimental chart of auto-focusing is shown in fig. 12 and 13, fig. 12 is a experimental chart of auto-focusing with steps of 6 and fig. 13 is a experimental chart of auto-focusing with steps of 10.
The foregoing is merely a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any modification and substitution based on the technical scheme and the inventive concept provided by the present invention should be covered in the scope of the present invention.
Claims (3)
1. The neural network optimization automatic zoom camera internal parameter calibration method based on visual field constraint is characterized by comprising the following steps of:
Step 1: constraining the relationship of field of view to focal length, camera field of view to focal length: wherein S is the pixel size, x res is the resolution, f is the focal length, D is the distance between the camera and the target, and H x is the camera field of view;
step 2: optimizing the relation between the focal length and the camera internal parameters based on a neural network;
Firstly, acquiring a focal length range of a zoom camera, fixing a plurality of focal lengths at equal intervals according to the focal length range, acquiring 15-30 pictures of a shooting target under each focal length, wherein the target occupies 20% of a field of view at the center of each picture, and acquiring a plurality of groups of data sets of focal lengths and camera internal parameter relations by adopting a Zhang Zhengyou camera calibration method; wherein the camera internal parameters comprise distortion coefficients and principal point abscissas and ordinates;
Secondly, dividing a plurality of groups of data sets of focal length and camera internal reference relations into a training set and a testing set according to the proportion of 2:1, and then carrying out normalization processing on the data sets;
Then, fitting the data set by using polynomial, gaussian and trigonometric functions respectively, and performing curve optimization by using a multi-layer perceptron so as to realize calibration of internal parameters of the camera;
Step 3: automatic focusing; setting a threshold value of image definition, fixing a current focal length according to the relation between a visual field and the focal length, changing image distances at equal intervals, shooting a picture under each image distance, acquiring the definition of the picture by utilizing an image definition function, comparing the image definition with the threshold value, and changing the image distance to acquire the picture and the image definition when the image definition is smaller than the threshold value until the image definition is larger than the threshold value, wherein automatic focusing is completed.
2. The visual field constraint-based neural network optimization automatic zoom camera internal parameter calibration method according to claim 1, wherein in step 2, the parameters of the multi-layer perceptron are set: the training round number is set to 10000, the characteristic number range is 1-14, the learning rate is set to 0.01, and the batch size is set to 32.
3. The visual field constraint-based neural network optimized auto-zoom camera internal parameter calibration method according to claim 1, wherein the threshold in step 3 is set to 80.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410430508.9A CN118037863B (en) | 2024-04-11 | 2024-04-11 | Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410430508.9A CN118037863B (en) | 2024-04-11 | 2024-04-11 | Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118037863A true CN118037863A (en) | 2024-05-14 |
| CN118037863B CN118037863B (en) | 2024-06-18 |
Family
ID=90987771
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410430508.9A Active CN118037863B (en) | 2024-04-11 | 2024-04-11 | Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118037863B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104851104A (en) * | 2015-05-29 | 2015-08-19 | 大连理工大学 | Flexible-target-based close-range large-field-of-view calibrate method of high-speed camera |
| CN105389808A (en) * | 2015-10-27 | 2016-03-09 | 郑州大学 | Camera self-calibration method based on two vanishing points |
| CN106960456A (en) * | 2017-03-28 | 2017-07-18 | 长沙全度影像科技有限公司 | A kind of method that fisheye camera calibration algorithm is evaluated |
| CN112950671A (en) * | 2020-08-06 | 2021-06-11 | 郑锴 | Real-time high-precision parameter measurement method for moving target by unmanned aerial vehicle |
| US20220368883A1 (en) * | 2020-04-09 | 2022-11-17 | Looking Glass Factory, Inc. | System and method for generating light field images |
| CN117495975A (en) * | 2023-10-23 | 2024-02-02 | 神力视界(深圳)文化科技有限公司 | Zoom lens calibration method and device and electronic equipment |
| CN117671020A (en) * | 2022-09-07 | 2024-03-08 | 比亚迪股份有限公司 | Calibration method of fish-eye camera, panoramic image acquisition method and vehicle |
-
2024
- 2024-04-11 CN CN202410430508.9A patent/CN118037863B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104851104A (en) * | 2015-05-29 | 2015-08-19 | 大连理工大学 | Flexible-target-based close-range large-field-of-view calibrate method of high-speed camera |
| CN105389808A (en) * | 2015-10-27 | 2016-03-09 | 郑州大学 | Camera self-calibration method based on two vanishing points |
| CN106960456A (en) * | 2017-03-28 | 2017-07-18 | 长沙全度影像科技有限公司 | A kind of method that fisheye camera calibration algorithm is evaluated |
| US20220368883A1 (en) * | 2020-04-09 | 2022-11-17 | Looking Glass Factory, Inc. | System and method for generating light field images |
| CN112950671A (en) * | 2020-08-06 | 2021-06-11 | 郑锴 | Real-time high-precision parameter measurement method for moving target by unmanned aerial vehicle |
| CN117671020A (en) * | 2022-09-07 | 2024-03-08 | 比亚迪股份有限公司 | Calibration method of fish-eye camera, panoramic image acquisition method and vehicle |
| CN117495975A (en) * | 2023-10-23 | 2024-02-02 | 神力视界(深圳)文化科技有限公司 | Zoom lens calibration method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 殷晨晖;褚鑫磊;杨珊;李丽莹;隋国荣;: "单摄像机立体视觉测量***的高精度变焦标定技术", 光学技术, no. 06, 15 November 2019 (2019-11-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118037863B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109767476B (en) | Automatic focusing binocular camera calibration and depth calculation method | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN109873948B (en) | Intelligent automatic focusing method and device for optical microscope and storage device | |
| CN111932636B (en) | Calibration and image correction method and device for binocular camera, storage medium, terminal and intelligent equipment | |
| CN111080705B (en) | Calibration method and device for automatic focusing binocular camera | |
| CN111899164B (en) | Image splicing method for multi-focal-segment scene | |
| CN103475820B (en) | PI method for correcting position and system in a kind of video camera | |
| CN111998797B (en) | Scanning track planning method of photographing type three-dimensional scanner | |
| CN115086516A (en) | Camera system automatic focusing method based on deep learning model | |
| CN112465918B (en) | Microscopic vision calibration method based on Tsai calibration | |
| CN118037863B (en) | Neural network optimization automatic zooming camera internal parameter calibration method based on visual field constraint | |
| CN117495975A (en) | Zoom lens calibration method and device and electronic equipment | |
| CN111754587B (en) | Zoom lens rapid calibration method based on single-focus focusing shooting image | |
| CN115810055A (en) | Annular structure light calibration method based on planar checkerboard | |
| CN116051651A (en) | Equivalent multi-vision model of variable-vision imaging system and calibration method thereof | |
| JP5925109B2 (en) | Image processing apparatus, control method thereof, and control program | |
| CN105651699A (en) | Dynamic focus following method based on area-array camera | |
| CN112749464B (en) | Reference optical fiber distribution design method of optical fiber spectrum telescope | |
| CN112819900B (en) | Method for calibrating internal azimuth, relative orientation and distortion coefficient of intelligent stereography | |
| CN115239816A (en) | Camera calibration method, system, electronic device and storage medium | |
| CN110689582B (en) | Total station camera calibration method | |
| CN109859263B (en) | Wide-view angle positioning method based on fisheye lens | |
| CN104811604B (en) | Image acquiring device and its image deformation bearing calibration | |
| JPH0252204A (en) | Measuring instrument for three-dimensional coordinate | |
| CN111462240B (en) | Target positioning method based on multi-monocular vision fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |