Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a terminal to which a distortion parameter optimizing device in a hardware operating environment according to an embodiment of the present invention belongs.
The terminal of the embodiment of the invention can be a PC, and can also be a mobile terminal device with a display function, such as a smart phone, a tablet computer, an electronic book reader, an MP3(Moving Picture Experts Group Audio Layer III, dynamic video Experts compress standard Audio Layer 3) player, an MP4(Moving Picture Experts Group Audio Layer IV, dynamic video Experts compress standard Audio Layer 4) player, a portable computer, and the like.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Optionally, the terminal may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display screen according to the brightness of ambient light, and a proximity sensor that may turn off the display screen and/or the backlight when the mobile terminal is moved to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), detect the magnitude and direction of gravity when the mobile terminal is stationary, and can be used for applications (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer and tapping) and the like for recognizing the attitude of the mobile terminal; of course, the mobile terminal may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and an optimization program of distortion parameters.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be used to invoke an optimization routine for the distortion parameters stored in the memory 1005.
In this embodiment, the apparatus for optimizing distortion parameters includes: a memory 1005, a processor 1001 and an optimization program for a distortion parameter stored on the memory and executable on the processor, wherein:
when the processor 1001 calls the optimization program of the distortion parameter stored in the memory 1005, the following operations are performed:
acquiring a first parameter set group corresponding to a fisheye image, wherein the first parameter set group comprises a plurality of sets of parameter sets;
calculating first linear degree errors corresponding to the parameter sets in the first parameter set group according to a first preset sequence on the basis of the parameter sets in the first parameter set group and the first characteristic point set of the fisheye image;
generating a second parameter set group corresponding to the fisheye image based on parameter sets corresponding to first linear degree errors meeting preset conditions in the first parameter set group, wherein the second parameter set group comprises multiple groups of parameter sets;
calculating second straightness errors corresponding to the parameter sets in the second parameter set group according to a second preset sequence and based on the parameter sets in the second parameter set group and the first characteristic point set of the fisheye image;
and acquiring a parameter set corresponding to the minimum straightness error in the second straightness errors, and taking the distortion parameters in the acquired parameter set as target distortion parameters of the fisheye image.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
acquiring a first characteristic point set of the fisheye image according to a first preset rule;
calculating a second feature point set of the target image according to a first preset sequence based on the parameter sets in the first parameter set group and the acquired first feature point set;
and calculating the first linear degree error based on the second characteristic point set obtained by calculation.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
acquiring a preset central point coordinate of the target image;
and calculating the second feature point set according to a first preset sequence based on the parameter sets in the first parameter set group, the first feature point set and the preset central point coordinate.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
calculating a first linear equation of each row and a second linear equation of each column corresponding to the second feature point set according to a first preset rule;
calculating a row error according to the first linear equation and the coordinates of each characteristic point in the second characteristic point set, and calculating a column error according to the second linear equation and the coordinates of each characteristic point in the second characteristic point set;
calculating the first linearity error based on each of the row errors and each of the column errors.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
calculating a third feature point set of the target image based on the parameter sets in the second parameter set group and the first feature point set in sequence according to a second preset sequence;
and calculating the second straightness error based on the third feature point set obtained by calculation.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
acquiring a preset central point coordinate of the target image;
and calculating the third feature point set according to a second preset sequence based on the parameter sets in the second parameter set group, the first feature point set and the preset central point coordinate.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
calculating the value range of the distortion center coordinate based on the center coordinate of the fisheye image and the adjustment range of the distortion center;
calculating the value range of the distortion radius based on the reference radius of the fisheye image and the scaling coefficient of the distortion radius;
and acquiring the first parameter set based on the value range of the distortion center coordinate, the value range of the distortion radius and the first adjustment step length.
Further, the processor 1001 may call an optimization program of the distortion parameter stored in the memory 1005, and further perform the following operations:
acquiring a parameter set corresponding to the smallest preset number of straightness errors in the first straightness errors;
and generating the second parameter set group according to the acquired parameter sets and the second adjustment step length in sequence.
A first embodiment of the present invention provides a method for optimizing a distortion parameter, and referring to fig. 2, fig. 2 is a schematic flow chart of the first embodiment of the method for optimizing a distortion parameter of the present invention, where the method for optimizing a distortion parameter includes:
step S100, acquiring a first parameter set group corresponding to a fisheye image, wherein the first parameter set group comprises a plurality of sets of parameter sets;
distortion correction of fisheye images firstly requires building a geometric model of a fisheye lens, and then plane perspective projection correction is carried out according to the geometric relationship. The equidistant projection model is adopted in the embodiment, and the geometric model is widely used in lens manufacturing and is a geometric model with a good effect when fisheye image correction is carried out. Because the equidistant projection model is adopted in the fisheye image correction method, the distortion parameters of the fisheye image correction method mainly comprise the distortion center coordinate and the distortion radius of the fisheye image, and the distortion center coordinate can be refined into the X coordinate in the horizontal direction and the Y coordinate in the vertical direction. The first parameter set group corresponding to the fisheye image comprises a plurality of sets of parameters, wherein each set of parameters comprises 3 distortion parameters, and the first distortion parameter refers to a distortion parameter used when a distortion parameter range is determined for the first time.
In this embodiment, a distortion center coordinate value range and a distortion radius value range may be set according to a preset rule, and a first adjustment step length is set according to parameters such as resolution of a fisheye image, and preferably, resolution of the fisheye image in this embodiment is 1280 × 720, so that the first adjustment step length is set to 10. Wherein, the calculation formulas of the distortion center point coordinate value range and the distortion radius value range are respectively as follows:
CenterX∈[ImgCenterX-Range,ImgCenterX+Range]
CenterY∈[ImgCenterY-Range,ImgCenterY+Range]
R∈[REFER_R*MinCoe,REFER_R*MaxCoe]
where Center X and Center Y are the calculated distortion Center X coordinate and Y coordinate, R is the calculated distortion radius, imgcenter X and imgcenter Y are the Center X coordinate and Y coordinate of the fisheye image to be corrected, and REFER _ F is the reference radius set by the user according to the size of the fisheye image to be corrected, and preferably, the resolution of the fisheye image to be corrected in this embodiment is 1280 × 720, so the reference radius is set to 600. Range is an adjustment Range of the distortion center, and preferably, Range in the present embodiment is 100. MaxCoe and MinCoe are respectively scaling coefficients of distortion radius, and preferably, MaxCoe and MinCoe in the present embodiment are respectively 1.2 and 0.8. For example, the center coordinates of a fisheye image to be corrected with a resolution of 1280 × 720 are (640, 360), the adjustment range of the distortion center is 100, the value range of the X coordinate of the distortion center point can be (540, 740), and the value range of the Y coordinate of the distortion center point is (260, 460); the reference radius is 600, and then multiplied by the scaling factor to obtain the distortion radius (480, 720). In this embodiment, the adjustment step is 10, and the value sets of the three first distortion parameters can be obtained by traversing the value ranges of the three first distortion parameters according to the adjustment step, where the value sets are the value set of the X coordinate of the distortion center point, that is, the center X belongs to {540, 550, 560.. 730, 740}, the value set of the Y coordinate of the distortion center point, that is, the center Y belongs to {260, 270, 280.. 450, 460}, and the value set of the distortion radius R, that is, the R belongs to {480, 490, 500.. 700, 710, 720 }.
And then, combining elements in the value sets of the three first distortion parameters to enable each element to be combined with elements in the other two sets possibly, wherein the total combined number is equal to the number of the elements in the value set of the X coordinate of the distortion central point multiplied by the number of the elements in the value set of the Y coordinate of the distortion central point multiplied by the number of the elements in the value set of the distortion radius R. For example, the value set of the X coordinate of the distortion center point includes 2 elements, which are [1, 2] respectively, the value set of the Y coordinate of the distortion center point includes 2 elements, which are [3, 4] respectively, and the value set of the distortion radius includes 2 elements [5, 6], and then after the value set of the first distortion parameter is arranged and combined, the combination of all possible distortion parameters that can be obtained includes: (1,3,5),(2,3,5),(1,4,5),(2,4,5),(1,3,6),(2,3,6),(1,4,6),(2,4,6). All possible combinations obtained after arranging and combining the value sets of all the first distortion parameters are the first parameter set group, wherein the first parameter set group comprises a plurality of groups of parameter sets, and each parameter set comprises three parameters which are a distortion central point X coordinate, a distortion central point Y coordinate and a distortion radius R.
Step S200, calculating first linear degree errors corresponding to the parameter sets in the first parameter set group according to a first preset sequence on the basis of the parameter sets in the first parameter set group and the first characteristic point set of the fisheye image;
in this embodiment, the first feature point set can be obtained by calibrating the checkerboard corner points on the fisheye image to be corrected. Preferably, in this embodiment, a black-and-white chessboard calibration manner is adopted to calibrate the feature point set on the fisheye image to be corrected and determine the feature point set as the first feature point set. M rows and N columns of feature point sets { (u) can be acquired on the fisheye image to be correctedij,vij)},0≤i<M,0≤j<N, wherein uijIs the abscissa, v, of the point in the fish-eye imageijThe ordinate of the point on the fisheye image.
The processor selects all parameter sets from the first parameter set group in sequence according to a first preset sequence, and calculates a second feature point set corresponding to the first feature point set based on the parameter sets. The first preset sequence is that after a value is taken out from a value set of a distortion central point X coordinate from a small order to a large order, a value is taken out from a value set of a distortion central point Y coordinate and a set of a distortion radius R from a small order to a large order in sequence to combine to generate a parameter set in a first parameter set group until all possible conditions are taken out, and the first preset sequence is easy to understand, wherein three parameters in the first distortion parameter can be taken out smoothly from a large order to a small order or smoothly from a small order to a large order, so that the first preset sequence can be any combination of the three parameter value sequences, the process is similar to the process, and the description is omitted. For a particular set of parameters (CenterX, CenterY, R), the corresponding points in the second set of feature points for each point in the first set of feature points can be calculated according to the following formula:
wherein, (u, v) is a point on the fisheye image to be corrected, (x, y) is a point on the corrected image, (amdlmgcenterx, amdlmgcentery) is a preset center coordinate of the corrected image, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, any one fisheye image feature point { (u) extracted from the first feature point set can be calculatedij,vij) The points in the corrected image second feature point set (x)ij,yij)},0≤i<M,0≤j<N。
And calculating the straightness error based on the calculated second characteristic point set. Because the distortion of the center of the fisheye image is relatively minimum, M straight-line equations f can be determined by selecting two points at the middle of each linem(x),0≤m<And M, based on the M linear equations and all the element points of the second characteristic point set, calculating horizontal direction linearity errors of all the points according to the following formula:
similarly, selecting the two points at the middle of each column can determine N linear equations fn(x),0≤n<Based on the N linear equations and all the element points of the second feature point set, the vertical straightness errors of all the points can be calculated according to the following formula:
based on the above-mentioned straightness error in the horizontal direction and the straightness error in the vertical direction, the total straightness error can be obtained according to the following formula:
Error=Errorhor+Errorver
and the total linearity error obtained by calculation is the linearity error of the specific parameter set. For any parameter set in the first parameter set group, the corresponding first linearity error can be calculated according to the above steps and formulas.
Step 300, generating a second parameter set group corresponding to the fisheye image based on a parameter set corresponding to a first linear degree error meeting a preset condition in the first parameter set group, wherein the second parameter set group comprises a plurality of groups of parameter sets;
and after the first linearity error groups corresponding to all the parameter sets in the first parameter set group are obtained, sorting the first linearity error groups, and outputting the parameter sets corresponding to the first Z least linearity errors. Preferably, in this embodiment, three sets of parameter sets with the smallest straightness error are output after the sorting is finished, and then the parameter sets are readjusted according to a preset rule, so as to obtain a value range of a new parameter set. In this embodiment, a user may set the distortion center coordinates and the adjustment range of the distortion radius in the parameter set according to a preset formula, and set a second adjustment step, preferably, the second adjustment step in this embodiment is set to 1. The preset rules mainly include 3 formulas:
CenterX∈[CenterXk-NEW_RANGE,CenterXk+NEW_RANGE]
CenterY∈[CenterYk-NEW_RANGE,CenterYk+NEW_RANGE]
R∈[R-NEW_RANGE,R+NEW_RANGE]
preferably, NEW _ RANGE is set to 10 in this embodiment, that is, the NEW value RANGE is adjusted to 10, and the parameter adjustment STEP length is 1 each time, that is, PARAMS _ STEP is 1. The second distortion parameter refers to a distortion parameter obtained based on the first distortion parameter, and includes 3 parameters of a distortion center X coordinate, a distortion center Y coordinate, and a distortion radius. For the distortion parameter in each parameter set, the processor may sequentially traverse the entire value range from small to large according to the second adjustment step length within the adjustment range of the distortion parameter. And for each distortion parameter, after all possible values are taken, putting all the taken values into a created set, and generating a second parameter set corresponding to a second distortion parameter of the fisheye image, wherein the second parameter set comprises a value set of an X coordinate of a distortion center, a value set of a Y coordinate of the distortion center and a value set of a distortion radius. For example, the first 3 parameter sets with the smallest straightness error are (1, 4, 7), (2, 5, 8), (3, 6, 9), which means that the distortion center X coordinate corresponding to the first three smallest straightness errors includes [1, 2, 3], the distortion center Y coordinate includes [4, 5, 6], the distortion radius R includes [7, 8, 9], a distortion parameter value range is generated according to a preset rule for each value, a value range of { -9, 11] can be generated when the distortion center X coordinate takes 1, and simultaneously, since the second adjustment step size is 1, a value range of { -9, -8, }, 0, 1, 2,..,. 10, 11} of the distortion center X coordinate corresponding to the distortion center X coordinate taking 2 can be obtained, correspondingly, a value range of { -8 of the distortion center X coordinate corresponding to the distortion center X coordinate taking 2 can be obtained, 7, 1, 2, 3, 11, 12}, where the distortion center X coordinate is 3, the range of values of the corresponding distortion center X coordinate is { -7, -6, · 2, 3, 4, · 12, 13], and after the summation, the set of values of the distortion center X coordinate in the second distortion parameter is [ -9, -8,. once, 0, 1, 2, 3, 4,. once, 12, 13 ]. Similarly, the set of values of the Y coordinate of the distortion center in the second distortion parameter is [ -6, -5,. multidot.3, 4, 5, 6, 7,. multidot.15, 16], and the set of values of the distortion radius R in the second distortion parameter is [ -3, -2,. multidot.6, 7, 8, 9, 10,. multidot.18, 19 ]. After the value sets of the first distortion parameter are arranged and combined, all possible combinations of the distortion parameters that can be obtained are a first parameter set group, in this embodiment, a total of 23 × 23 × 23 parameter sets are provided, and each parameter set includes three parameters, namely, a distortion center point X coordinate, a distortion center point Y coordinate, and a distortion radius R.
Step S400, calculating second straightness errors corresponding to the parameter sets in the second parameter set group according to a second preset sequence on the basis of the parameter sets in the second parameter set group and the first characteristic point set of the fisheye image;
the second preset sequence is that values are sequentially selected from parameter sets contained in the second distortion parameter from small to large to form a set, specifically, after the values are taken from a value set of the X coordinate of the distortion central point from small to large, then sequentially taking out a value from the value set of the Y coordinate of the distortion central point and the set of the distortion radius R according to the sequence from small to large to combine to generate a parameter set in a first parameter set group until all possible conditions are taken out, it is easy to understand that all three parameters in the first distortion parameter can be successfully evaluated from large to small or from small to large, therefore, the first preset sequence can be any combination of the values of the three parameters, the process is similar to the above process, and it is not repeated here to obtain the first feature point set { (u) of M rows and N columns on the fisheye image to be corrected according to the preset method.ij,vij)},0≤i<M,0≤j<N, wherein uijIs the abscissa, v, of the point in the fish-eye imageijThe ordinate of the point on the fisheye image.
The processor may sequentially select all parameter sets from the second parameter set according to a second preset order, and for a specific parameter set, may calculate a point in the third feature point set corresponding to each point in the first feature point set according to the following formula:
wherein (u, v) is a point on the fisheye image to be corrected(x, y) is a point on the corrected image, (amdlmgcenterx, amdlmgcentery) is a preset corrected image center point coordinate, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, any one fisheye image feature point { (u) extracted from the first feature point set can be calculatedij,vij) The points in the corrected image third feature point set (x)ij,yij)},0≤i<M,0≤j<N。
And calculating the straightness error based on the third feature point set obtained by calculation. Because the distortion of the center of the fisheye image is relatively minimum, the two points at the middle of each line are selected to determine M linear equations fm (x), M is more than or equal to 0 and less than M, and based on the M linear equations and all element points of the third feature point set, the horizontal direction linearity errors of all the points can be calculated according to the following formula:
similarly, selecting the two points at the middle of each column can determine N linear equations fn (x), N is greater than or equal to 0 and is less than N, and based on the N linear equations and all the element points of the third feature point set, the vertical direction straightness errors of all the points can be calculated according to the following formula:
based on the above-mentioned straightness error in the horizontal direction and the straightness error in the vertical direction, the total straightness error can be obtained according to the following formula:
Error=Errorhor+Errorver
and the total linearity error obtained by calculation is the linearity error of the specific parameter set. For any parameter set in the first parameter set, the corresponding second straightness error can be calculated according to the steps and the formula.
And step S500, acquiring a parameter set corresponding to the smallest straightness error in the second straightness errors, and taking the distortion parameters in the acquired parameter set as target distortion parameters of the fisheye image.
And after second straightness error groups corresponding to all the parameter sets in the second parameter set group are obtained, sorting the second straightness error groups, and outputting the parameter set corresponding to the minimum straightness error. The distortion parameters contained in the parameter set are determined as target distortion parameters of the fisheye image.
In the optimization method of distortion parameters in this embodiment, a first parameter set group corresponding to a fisheye image is obtained, where the first parameter set group includes multiple sets of parameter sets; then, calculating first linear degree errors corresponding to the parameter sets in the first parameter set group according to a first preset sequence on the basis of the parameter sets in the first parameter set group and the first characteristic point set of the fisheye image; then, generating a second parameter set group corresponding to the fisheye image based on the parameter set corresponding to the first linear degree error meeting preset conditions in the first parameter set group, wherein the second parameter set group comprises multiple groups of parameter sets; then, according to a second preset sequence, calculating second straightness errors corresponding to the parameter sets in the second parameter set group on the basis of the parameter sets in the second parameter set group and the first characteristic point set of the fisheye image; and then acquiring a parameter set corresponding to the smallest straightness error in the second straightness errors, and taking the distortion parameters in the acquired parameter set as target distortion parameters of the fisheye image, so that the optimization precision and accuracy of the distortion parameters are improved, the defect that the conventional method depends on the circular contour of the fisheye image is avoided, the straight line constraint can be effectively utilized to generate an image with a correction effect which is more consistent with the expectation, and a solid foundation is provided for subsequent image registration.
Based on the first embodiment, a second embodiment of the method for optimizing distortion parameters of the present invention is proposed, and referring to fig. 3, in this embodiment, step S200 includes:
step S210, acquiring a first characteristic point set of the fisheye image according to a first preset rule;
the method for obtaining the first characteristic point set is to calibrate the chessboard distribution corner points on the fisheye image to be corrected according to a preset method, so as to obtain the characteristic point set of the fisheye image to be corrected. Preferably, in this embodiment, a black-and-white chessboard calibration method is adopted to calibrate the feature point set on the fisheye image to be corrected and determine the feature point set as the first feature point set. M rows and N columns of feature point sets { (uij, vij) } can be obtained on the fisheye image to be corrected, i is more than or equal to 0 and less than M, j is more than or equal to 0 and less than N, wherein uij is the abscissa of the point on the fisheye image, and vij is the ordinate of the point on the fisheye image.
Step S220, calculating a second feature point set of the target image according to a first preset sequence based on the parameter sets in the first parameter set group and the acquired first feature point set;
the processor selects all parameter sets from the first parameter set group in sequence according to a first preset sequence, and calculates a second feature point set corresponding to the first feature point set based on the parameter sets. The first preset sequence is that after a value is taken out from a value set of a distortion central point X coordinate from a small order to a large order, a value is taken out from a value set of a distortion central point Y coordinate and a set of a distortion radius R from a small order to a large order in sequence to combine to generate a parameter set in a first parameter set group until all possible conditions are taken out, and the first preset sequence is easy to understand, wherein three parameters in the first distortion parameter can be taken out smoothly from a large order to a small order or smoothly from a small order to a large order, so that the first preset sequence can be any combination of the three parameter value sequences, the process is similar to the process, and the description is omitted. For a particular set of parameters (CenterX, CenterY, R), the corresponding points in the second set of feature points for each point in the first set of feature points can be calculated according to the following formula:
wherein, (u, v) is a point on the fisheye image to be corrected, (x, y) is a point (AmdImgCenterX, AmdImgCenterY) on the corrected image is a preset center coordinate of the corrected image, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, the corrected image second feature point set point { (xij, yij) } corresponding to any one fisheye image feature point { (uij, vij) } extracted from the first feature point set can be calculated, i is greater than or equal to 0 and less than M, and j is greater than or equal to 0 and less than N.
Step S230, calculating the first linear degree error based on the calculated second feature point set.
And calculating the straightness error based on the calculated second characteristic point set. Because the distortion of the center of the fisheye image is relatively minimum, the two points at the middle of each line are selected to determine M linear equations fm (x), M is more than or equal to 0 and less than M, and based on the M linear equations and all element points of the second characteristic point set, the horizontal direction linearity errors of all the points can be calculated according to the following formula:
similarly, selecting the two points at the middle of each column can determine N linear equations fn (x), N is greater than or equal to 0 and is less than N, and based on the N linear equations and all the element points of the second feature point set, the vertical direction straightness errors of all the points can be calculated according to the following formula:
based on the above-mentioned straightness error in the horizontal direction and the straightness error in the vertical direction, the total straightness error can be obtained according to the following formula:
Error=Errorhor+Errorver
and the total linearity error obtained by calculation is the linearity error of the specific parameter set. For any parameter set in the first parameter set group, the corresponding first linearity error can be calculated according to the above steps and formulas.
In the optimization method of distortion parameters in this embodiment, a first feature point set of the fisheye image is obtained according to a first preset rule; then, calculating a second feature point set of the target image according to a first preset sequence based on the parameter sets in the first parameter set group and the acquired first feature point set in sequence; and then, based on the second characteristic point set obtained by calculation, the first linearity error is calculated, so that the accuracy of the second parameter set group is improved, the accuracy of the first linearity error can be improved, and the optimization precision and accuracy of the distortion parameter are further improved.
Based on the second embodiment, a third embodiment of the method for optimizing distortion parameters of the present invention is proposed, and referring to fig. 4, in this embodiment, step S220 includes:
step S221, acquiring a preset central point coordinate of the target image;
because the corrected target image corresponding to the fisheye image needs to meet the specific size requirement, the size of the corrected target image can be judged according to specific conditions before the optimization process of the distortion parameter, and then the preset central point coordinate of the target image is determined.
Step S222, sequentially calculating the second feature point set based on the parameter sets in the first parameter set group, the first feature point set, and the preset central point coordinate according to a first preset order.
The processor selects all parameter sets from the first parameter set group in sequence according to a first preset sequence, and calculates a second feature point set corresponding to the first feature point set based on the parameter sets. The first preset sequence is that after a value is taken out from a value set of a distortion central point X coordinate from a small order to a large order, a value is taken out from a value set of a distortion central point Y coordinate and a set of a distortion radius R from a small order to a large order in sequence to combine to generate a parameter set in a first parameter set group until all possible conditions are taken out, and the first preset sequence is easy to understand, wherein three parameters in the first distortion parameter can be taken out smoothly from a large order to a small order or smoothly from a small order to a large order, so that the first preset sequence can be any combination of the three parameter value sequences, the process is similar to the process, and the description is omitted. For a particular set of parameters (CenterX, CenterY, R), the corresponding points in the second set of feature points for each point in the first set of feature points can be calculated according to the following formula:
wherein, (u, v) is a point on the fisheye image to be corrected, (x, y) is a point on the corrected image, (amdlmgcenterx, amdlmgcentery) is a preset center coordinate of the corrected image, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, the corrected image second feature point set point { (xij, yij) } corresponding to any one fisheye image feature point { (uij, vij) } extracted from the first feature point set can be calculated, i is greater than or equal to 0 and less than M, and j is greater than or equal to 0 and less than N.
In the method for optimizing the distortion parameter in the embodiment, the coordinates of the preset central point of the target image are obtained; then, calculating a second feature point set according to a first preset sequence based on the parameter sets in the first parameter set group, the first feature point set and the preset central point coordinate in sequence; the second characteristic point set corresponding to the first characteristic point set is calculated based on each parameter set in the second parameter set group, the second characteristic point set can be accurately obtained, the accuracy of the second parameter set group is improved, the accuracy of the first linear error can be improved, and the optimization precision and accuracy of the distortion parameters are further improved.
Based on the second embodiment, a fourth embodiment of the method for optimizing distortion parameters of the present invention is proposed, and referring to fig. 5, in this embodiment, step S230 includes:
step S231, calculating a first linear equation of each row and a second linear equation of each column corresponding to the second feature point set according to a second preset rule;
because the distortion of the center of the fisheye image is relatively minimum, M straight-line equations fm (x) can be determined by selecting two points at the middle of each line, wherein M is more than or equal to 0 and less than M; similarly, selecting the two points at the middle of each column can determine N linear equations fn (x), where N is greater than or equal to 0 and less than N. The second preset rule is an equation that two points in a plane can determine a straight line, and the two-point equation is the prior art and is not described herein again.
If the number of the characteristic points of the row or the column is even, selecting two points in the middle to calculate a linear equation; if the number of the feature points of the row or the column is odd, the feature points on two sides of the most middle point are selected to calculate a linear equation.
Step S232, calculating a row error according to the first linear equation and the coordinates of each feature point in the second feature point set, and calculating a column error according to the second linear equation and the coordinates of each feature point in the second feature point set;
based on the M linear equations and all the element points of the second feature point set, the horizontal straightness errors of all the points can be calculated according to the following formula:
similarly, based on the N linear equations and all the element points of the second feature point set, the vertical straightness errors of all the points can be calculated according to the following formula:
step S233, calculating the first linear error based on each of the row errors and each of the column errors.
Based on the above-mentioned straightness error in the horizontal direction and the straightness error in the vertical direction, the total straightness error can be obtained according to the following formula:
Error=Errorhor+Errorver
and the total linearity error obtained by calculation is the linearity error of the specific parameter set. For any parameter set in the first parameter set group, the corresponding first linearity error can be calculated according to the above steps and formulas.
In the method for optimizing distortion parameters in this embodiment, first a first linear equation of each row and a second linear equation of each column corresponding to the second feature point set are calculated according to a second preset rule; then calculating a row error according to the first linear equation and the coordinates of each characteristic point in the second characteristic point set, and calculating a column error according to the second linear equation and the coordinates of each characteristic point in the second characteristic point set; and then calculating the first linear error based on each row error and each column error. The method for comparing the first linearity error is adopted to carry out the first screening on all the parameter sets, the parameter sets meeting the conditions are reserved for carrying out the next screening, the optimization efficiency of the distortion parameters is improved, the accuracy of the first linearity error is improved, the optimization precision and accuracy of the distortion parameters are further improved, and the time of technicians and the limited resources of a processor are saved.
A fifth embodiment of the method for optimizing a distortion parameter according to the present invention is proposed based on the first embodiment, and referring to fig. 6, in this embodiment, step S400 includes:
step S410, calculating a third feature point set of the target image according to a second preset sequence and based on the parameter sets in the second parameter set group and the first feature point set in sequence;
the second preset sequence refers to that values are sequentially selected from parameter sets contained in the second distortion parameter from small to large to form a set, specifically, after the values are sequentially taken out from the value set of the X coordinate of the distortion center point from small to large, the values are sequentially taken out from the value set of the Y coordinate of the distortion center point and the set of the distortion radius R from small to large to be combined to generate a parameter set in the first parameter set group until all possible conditions are taken out.
After the first feature point set is obtained, the processor sequentially selects all parameter sets from the second parameter set group according to a second preset sequence, and for a specific parameter set, the corresponding point in the third feature point set corresponding to each point in the first feature point set can be calculated according to the following formula:
wherein, (u, v) is a point on the fisheye image to be corrected, (x, y) is a point on the corrected image, (amdlmgcenterx, amdlmgcentery) is a preset center coordinate of the corrected image, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, the corrected image third feature point set point { (xij, yij) } corresponding to any one fisheye image feature point { (uij, vij) } extracted from the first feature point set can be calculated, i is greater than or equal to 0 and less than M, and j is greater than or equal to 0 and less than N.
Step S420, calculating the second straightness error based on the calculated third feature point set.
And calculating the straightness error based on the third feature point set obtained by calculation. Because the distortion of the center of the fisheye image is relatively minimum, the two points at the middle of each line are selected to determine M linear equations fm (x), M is more than or equal to 0 and less than M, and based on the M linear equations and all element points of the third feature point set, the horizontal direction linearity errors of all the points can be calculated according to the following formula:
similarly, selecting the two points at the middle of each column can determine N linear equations fn (x), N is greater than or equal to 0 and is less than N, and based on the N linear equations and all the element points of the third feature point set, the vertical direction straightness errors of all the points can be calculated according to the following formula:
based on the above-mentioned straightness error in the horizontal direction and the straightness error in the vertical direction, the total straightness error can be obtained according to the following formula:
Error=Errorhor+Errorver
and the total linearity error obtained by calculation is the linearity error of the specific parameter set. For any parameter set in the first parameter set, the corresponding second straightness error can be calculated according to the steps and the formula.
In the method for optimizing distortion parameters in this embodiment, first, a third feature point set of a target image is calculated based on parameter sets in a second parameter set and the first feature point set in sequence according to a second preset order; and then calculating the second straightness error based on the third feature point set obtained by calculation. The second straightness error is calculated based on the second parameter set, the obtained parameter set is small in range and high in precision, parameter set data which are not contained in the first parameter set can be obtained, optimizing precision and accuracy of distortion parameters are improved, and accuracy of correction of the fisheye image is further improved.
Based on the fifth embodiment, a sixth embodiment of the method for optimizing distortion parameters of the present invention is proposed, and referring to fig. 7, in this embodiment, step S410 includes:
step S411, acquiring a preset central point coordinate of the target image;
because the corrected target image corresponding to the fisheye image needs to meet the specific size requirement, the size of the corrected target image can be judged according to specific conditions before the optimization process of the distortion parameter, and then the preset central point coordinate of the target image is determined.
Step S412, sequentially calculating the third feature point set based on the parameter sets in the second parameter set group, the first feature point set, and the preset central point coordinate according to a second preset order.
The processor may sequentially select all parameter sets from the second parameter set according to a second preset order, and for a specific parameter set, may calculate a point in the third feature point set corresponding to each point in the first feature point set according to the following formula:
wherein, (u, v) is a point on the fisheye image to be corrected, (x, y) is a point on the corrected image, (amdlmgcenterx, amdlmgcentery) is a preset center coordinate of the corrected image, (CenterX, CenterY) is a distortion center coordinate, and R is a distortion radius. According to the formula, the corrected image third feature point set point { (xij, yij) } corresponding to any one fisheye image feature point { (uij, vij) } extracted from the first feature point set can be calculated, i is greater than or equal to 0 and less than M, and j is greater than or equal to 0 and less than N.
In the method for optimizing the distortion parameter in the embodiment, the coordinates of the preset central point of the target image are obtained; and then, calculating the third feature point set according to a first preset sequence based on the parameter sets in the second parameter set group, the first feature point set and the preset central point coordinate. The second parameter set is obtained based on the first parameter set optimization, so that the third feature point set can be optimized on the basis of the second feature point set, the precision of the third feature point set is improved, and the optimization quality of the distortion parameter is improved.
Based on the first embodiment, a seventh embodiment of the method for optimizing distortion parameters of the present invention is proposed, and referring to fig. 8, in this embodiment, step S100 includes:
step S110, calculating a value range of the distortion center coordinate based on the center coordinate of the fisheye image and the adjustment range of the distortion center;
the user can set the value range of the distortion center coordinate based on the center point coordinate of the fisheye image to be corrected and the adjustment range of the distortion center. The adjustment range of the distortion center can be set by the user, and preferably, the adjustment range of the distortion center in this embodiment is 100. When the X coordinate and the Y coordinate of the central point of the fisheye image to be corrected are obtained, the processor may calculate the value ranges of the X coordinate and the Y coordinate of the distortion center according to the following formula:
CenterX∈[ImgCenterX-Range,ImgCenterX+Range]
CenterY∈[ImgCenterY-Range,ImgCenterY+Range]
R∈[REFER_R*MinCoe,REFER_R*MaxCoe]
wherein, Center X and Center Y are the X coordinate and Y coordinate of the distortion Center, R is the distortion radius, ImgCenter X and ImgCenter Y are the X coordinate and Y coordinate of the Center point of the fisheye image to be corrected, and Range is the adjusting Range of the distortion Center. For example, the coordinates of the center of a fisheye image to be corrected with a resolution of 1280 × 720 are (640, 360), the adjustment range of the distortion center is 100, so that the value range of the X coordinate of the distortion center point is (540, 740), and the value range of the Y coordinate of the distortion center point is (260, 460).
Step S120, calculating a value range of the distortion radius based on the reference radius of the fisheye image and the scaling coefficient of the distortion radius;
the user can set the range of values of the distortion radius based on the reference radius and the scaling factor of the distortion radius. The adjustment range of the reference radius may be set by the user himself/herself, and preferably, since the resolution of the fisheye image to be corrected used in the present embodiment is 1280 × 720, the reference radius is set to 600, and at the same time, MaxCoe and MinCoe are set to 1.2 and 0.8, respectively. When the reference radius and the distortion parameter of the fisheye image to be corrected are obtained, the processor can calculate the value range of the distortion radius R according to the following formula:
R∈[REFER_R*MinCoe,REFER_R*MaxCoe]
wherein, REFER _ F is a reference radius set by the user according to the size of the fisheye image to be corrected. For example, a fisheye image to be corrected with a resolution of 1280 × 720 has center coordinates of (640, 360), and MaxCoe and MinCoe are scaling coefficients of distortion radius, respectively. For example, the reference radius is 600, and then multiplied by the scaling factor to obtain the distortion radius, which has a value range of (480, 720).
And step S130, acquiring the first parameter set group based on the value range of the distortion center coordinate, the value range of the distortion radius and the first adjustment step length.
In this embodiment, a user may set a distortion center coordinate value range and a distortion radius value range according to a preset rule, and set a first adjustment step length at the same time, preferably, the first adjustment step length in this embodiment is set to 10. For each distortion parameter, the processor may traverse the entire value range from small to large in sequence according to the first adjustment step length within the value range of the distortion parameter. And then, carrying out permutation and combination operation on the elements in the three first distortion parameter sets, so that each element can carry out all possible combinations with the elements in the other two sets, wherein the total combination number is equal to the number of the elements in the distortion central point X coordinate value set multiplied by the number of the elements in the distortion central point Y coordinate value set multiplied by the number of the elements in the distortion radius R value set. For example, the value set of the X coordinate of the distortion center point includes 2 elements, which are [1, 2] respectively, the value set of the Y coordinate of the distortion center point includes 2 elements, which are [3, 4] respectively, and the value set of the distortion radius includes 2 elements [5, 6], and then after the value set of the first distortion parameter is arranged and combined, the combination of all possible distortion parameters that can be obtained includes: (1,3,5),(2,3,5),(1,4,5),(2,4,5),(1,3,6),(2,3,6),(1,4,6),(2,4,6). All possible combinations obtained after arranging and combining the value sets of all the first distortion parameters are the first parameter set group, wherein the first parameter set group comprises a plurality of groups of parameter sets, and each parameter set comprises three parameters which are a distortion central point X coordinate, a distortion central point Y coordinate and a distortion radius R.
In the optimization method of the distortion parameter in this embodiment, first, a value range of a distortion center coordinate is calculated based on a center coordinate of the fisheye image and an adjustment range of a distortion center; then, calculating a value range of the distortion radius based on the reference radius of the fisheye image and the scaling coefficient of the distortion radius; and then acquiring the first parameter set based on the value range of the distortion center coordinate, the value range of the distortion radius and the first adjustment step length. The first parameter set is estimated through an empirical method, so that the basic requirements of image correction are met, the process of obtaining distortion parameters is simplified through a scientific means, and the optimization precision of the distortion parameters is improved.
Based on the above embodiments, an eighth embodiment of the method for optimizing distortion parameters of the present invention is provided, and referring to fig. 9, in this embodiment, step S300 includes:
step S310, acquiring parameter sets corresponding to the smallest preset number of straightness errors in the first straightness errors;
and after the first linearity error groups corresponding to all the parameter sets in the first parameter set group are obtained, sorting the first linearity error groups, and outputting the parameter sets corresponding to the first Z least linearity errors. Preferably, in this embodiment, three sets of parameter sets with the smallest straightness error are output after the sorting is finished, and then the parameter sets are readjusted according to a preset rule, so as to obtain a value range of a new parameter set.
And step S320, sequentially generating the second parameter set group according to the acquired parameter sets and a second adjustment step size.
In this embodiment, a user may set the distortion center coordinates and the adjustment range of the distortion radius in the parameter set according to a preset formula, and set a second adjustment step, preferably, the second adjustment step in this embodiment is set to 1. The preset rules mainly include 3 formulas:
CenterX∈[CenterXk-NEW_RANGE,CenterXk+NEW_RANGE]
CenterY∈[CenterYk-NEW_RANGE,CenterYk+NEW_RANGE]
R∈[R-NEW_RANGE,R+NEW_RANGE]
preferably, NEW _ RANGE is set to 10 in this embodiment, that is, the NEW value RANGE is adjusted to 10, and the parameter adjustment STEP length is 1 each time, that is, PARAMS _ STEP is 1. The second distortion parameter refers to a distortion parameter obtained based on the first distortion parameter, and includes 3 parameters of a distortion center X coordinate, a distortion center Y coordinate, and a distortion radius. For the distortion parameter in each parameter set, the processor may sequentially traverse the entire value range from small to large according to the second adjustment step length within the adjustment range of the distortion parameter. And for each distortion parameter, after all possible values are taken, putting all the taken values into a created set, and generating a second parameter set corresponding to a second distortion parameter of the fisheye image, wherein the second parameter set comprises a value set of an X coordinate of a distortion center, a value set of a Y coordinate of the distortion center and a value set of a distortion radius. For example, the first 3 parameter sets with the smallest straightness error are (1, 4, 7), (2, 5, 8), (3, 6, 9), which means that the distortion center X coordinate corresponding to the first three smallest straightness errors includes [1, 2, 3], the distortion center Y coordinate includes [4, 5, 6], the distortion radius R includes [7, 8, 9], a distortion parameter value range is generated according to a preset rule for each value, a value range of { -9, 11] can be generated when the distortion center X coordinate takes 1, and simultaneously, since the second adjustment step size is 1, a value range of { -9, -8, }, 0, 1, 2,..,. 10, 11} of the distortion center X coordinate corresponding to the distortion center X coordinate taking 2 can be obtained, correspondingly, a value range of { -8 of the distortion center X coordinate corresponding to the distortion center X coordinate taking 2 can be obtained, 7, 1, 2, 3, 11, 2, and the range of the distortion center X coordinate corresponding to the distortion center X coordinate when the distortion center X coordinate is 3 is { -7, -6, · 2, 3, 4, · 12, 13], after the summation, the set of the distortion center X coordinates in the second distortion parameter is [ -9, -8,. 0, 1, 2, 3, 4,. page.,. 12, 13 ]. Similarly, the set of values of the Y coordinate of the distortion center in the second distortion parameter is [ -6, -5,. multidot.3, 4, 5, 6, 7,. multidot.15, 16], and the set of values of the distortion radius R in the second distortion parameter is [ -3, -2,. multidot.6, 7, 8, 9, 10,. multidot.18, 19 ]. After the value sets of the first distortion parameter are arranged and combined, all possible combinations of the distortion parameters that can be obtained are a first parameter set group, in this embodiment, a total of 23 × 23 × 23 parameter sets are provided, and each parameter set includes three parameters, namely, a distortion center point X coordinate, a distortion center point Y coordinate, and a distortion radius R.
In the method for optimizing distortion parameters in this embodiment, a parameter set corresponding to a minimum preset number of straightness errors in the first straightness errors is obtained; and then sequentially generating the second parameter set group according to the acquired parameter sets and the second adjustment step length. After a certain number of optimal solutions are obtained, the optimal solutions are refined, and a second parameter set is recalculated by using a smaller adjustment step length, so that the correction method is more refined, the precision of the second parameter set is improved, and the optimization precision of the distortion parameters is further improved.
The present invention also provides a computer-readable storage medium, in this embodiment, a program for optimizing a distortion parameter is stored on the computer-readable storage medium, wherein:
when the optimizing program of the distortion parameter is executed by the processor, the following steps are realized:
acquiring a first parameter set group corresponding to a fisheye image, wherein the first parameter set group comprises a plurality of sets of parameter sets;
calculating first linear degree errors corresponding to the parameter sets in the first parameter set group according to a first preset sequence on the basis of the parameter sets in the first parameter set group and the first characteristic point set of the fisheye image;
generating a second parameter set group corresponding to the fisheye image based on parameter sets corresponding to first linear degree errors meeting preset conditions in the first parameter set group, wherein the second parameter set group comprises multiple groups of parameter sets;
calculating second straightness errors corresponding to the parameter sets in the second parameter set group according to a second preset sequence and based on the parameter sets in the second parameter set group and the first characteristic point set of the fisheye image;
and acquiring a parameter set corresponding to the minimum straightness error in the second straightness errors, and taking the distortion parameters in the acquired parameter set as target distortion parameters of the fisheye image.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
acquiring a first characteristic point set of the fisheye image according to a first preset rule;
calculating a second feature point set of the target image according to a first preset sequence based on the parameter sets in the first parameter set group and the acquired first feature point set;
and calculating the first linear degree error based on the second characteristic point set obtained by calculation.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
acquiring a preset central point coordinate of the target image;
and calculating the second feature point set according to a first preset sequence based on the parameter sets in the first parameter set group, the first feature point set and the preset central point coordinate.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
calculating a first linear equation of each row and a second linear equation of each column corresponding to the second feature point set according to a second preset rule;
calculating a row error according to the first linear equation and the coordinates of each characteristic point in the second characteristic point set, and calculating a column error according to the second linear equation and the coordinates of each characteristic point in the second characteristic point set;
calculating the first linearity error based on each of the row errors and each of the column errors.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
calculating a third feature point set of the target image based on the parameter sets in the second parameter set group and the first feature point set in sequence according to a second preset sequence;
and calculating the second straightness error based on the third feature point set obtained by calculation.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
acquiring a preset central point coordinate of the target image;
and calculating the third feature point set according to a second preset sequence based on the parameter sets in the second parameter set group, the first feature point set and the preset central point coordinate.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
acquiring a parameter set corresponding to the smallest preset number of straightness errors in the first straightness errors;
and generating the second parameter set group according to the acquired parameter sets and the first adjustment step length in sequence.
Further, when the optimization program of the distortion parameter is executed by the processor, the following steps are also realized:
calculating the value range of the distortion center coordinate based on the center coordinate of the fisheye image and the adjustment range of the distortion center;
calculating the value range of the distortion radius based on the reference radius of the fisheye image and the scaling coefficient of the distortion radius;
and acquiring the first parameter set based on the value range of the distortion center coordinate, the value range of the distortion radius and the second adjustment step length.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.