CN114842071B - Automatic conversion method for image viewing angle - Google Patents
Automatic conversion method for image viewing angle Download PDFInfo
- Publication number
- CN114842071B CN114842071B CN202210439518.XA CN202210439518A CN114842071B CN 114842071 B CN114842071 B CN 114842071B CN 202210439518 A CN202210439518 A CN 202210439518A CN 114842071 B CN114842071 B CN 114842071B
- Authority
- CN
- China
- Prior art keywords
- image
- coordinate system
- individual
- individuals
- conversion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 60
- 238000000034 method Methods 0.000 title claims abstract description 28
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 238000005457 optimization Methods 0.000 claims abstract description 9
- 230000000007 visual effect Effects 0.000 claims abstract description 9
- 239000013598 vector Substances 0.000 claims description 48
- 230000035772 mutation Effects 0.000 claims description 19
- 230000009466 transformation Effects 0.000 claims description 15
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 238000002474 experimental method Methods 0.000 abstract description 3
- 238000012545 processing Methods 0.000 description 16
- 230000002068 genetic effect Effects 0.000 description 6
- 238000013519 translation Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 241000283070 Equus zebra Species 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009396 hybridization Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000013522 software testing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Mathematical Optimization (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Algebra (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an automatic conversion method of image visual angles, which comprises the following steps: loading the camera shooting image and the plane image, clicking the camera shooting image and the plane image, and selecting a target camera shooting image and a target plane image in a path; and connecting corresponding points of the image pickup image and the plane image, optimizing the optimal parameters of a conversion matrix by applying a differential evolution algorithm, and converting the image pickup image into the plane image through the conversion matrix. In modeling, it is particularly important to estimate the true position of an object, and errors generated by directly extracting the position from an image to estimate the true position of the object are relatively large. According to the invention, a differential evolution algorithm is adopted to convert the real position estimation problem of the object into the parameter optimization problem, and the differential evolution algorithm is utilized to search the optimal solution of the parameters of the visual angle conversion matrix, so that the conversion from the pixel coordinate system to the plane 2D coordinate system is realized, and the optimal coordinate system conversion effect is obtained. Experiments show that the method can effectively estimate the real position of the target in the image.
Description
Technical Field
The invention belongs to the field of software testing and intelligent computing, and particularly relates to an automatic conversion method for an image viewing angle.
Background
The differential evolution algorithm (DIFFERENTIAL EVOLUTION, DE) was proposed in 1997 by Rainer Storn and KENNETH PRICE on the basis of evolutionary ideas such as genetic algorithm, essentially a multi-objective (continuous variable) optimization algorithm (MOEAs) for solving the overall optimal solution in multidimensional space. The source of the differential evolution concept is an early proposed genetic algorithm (GeneticAlgorithm, GA), and genetic operators are designed by simulating hybridization (cross server), mutation (mutation) and replication (reproduction) in genetics.
The differential evolution algorithm is a self-organizing minimization method that utilizes two randomly selected different vectors in a population to interfere with existing vectors, each vector in the population being interfered. It generates a new parameter vector by adding the weighted difference vector between two members of the population to the third member, which becomes a variant. Mixing the parameters of the variation vector with other predetermined target vector parameters according to a certain rule to generate an experimental vector, wherein the operation is crossed; if the cost function of the experimental vector is lower than the cost function of the target, the experimental vector replaces the target vector in the next generation, and the operation becomes selection;
Compared with a genetic algorithm, the differential evolution algorithm has the advantages that an initial population is randomly generated at the same point, the fitness value of each individual in the population is used as a selection standard, and the main process also comprises three steps of mutation, crossover and selection. The difference is that the genetic algorithm controls the crossover of the father and the probability value of the filial generation selected after mutation according to the fitness value, and the probability of the individual with large fitness value being selected in the maximization problem is correspondingly larger. The variation vector of the differential evolution algorithm is generated by a parent differential vector, and is intersected with a parent individual vector to generate a new individual vector, and the new individual vector is directly selected with the parent individual. It is apparent that the differential evolution algorithm has a more remarkable approximation effect than the genetic algorithm.
The differential evolution algorithm is mainly used for solving the real number optimization problem. The algorithm is a group-based self-adaptive global optimization algorithm, belongs to one of evolution algorithms, and is widely applied to various fields such as data mining, pattern recognition, digital filter design, artificial neural networks, electromagnetics and the like due to the characteristics of simple structure, easiness in implementation, rapid convergence, strong robustness and the like.
Data-driven modeling methods have now become a popular and powerful method to simulate complex systems, such as crowd system (A.Lerner,C.Yiorgos,and L.Dani."Crowds by example,"Computer Graphics Forum.Vol.26.No.3.Blackwell Publishing Ltd,2007.)、 traffic system (D.Wilkie,S.Jason,L.Ming,"Flow reconstruction for datadriven traffic animation,"ACM Transactions on Graphics(TOG).32.4(2013):89.), for example. The key idea of the data-driven approach is to train components of the model with data so that the trained model can fit the real world data. One of the most commonly used data formats is an image. With the rapid development of pattern recognition technology image processing, it becomes feasible to extract the positional image of the object (D.Ryan,et al."Crowd counting using multiple local features,"In Digital Image Computing:Techniques and Applications,2009.DICTA'09.,pp.81-88.IEEE,2009.). however, the obtained image is often distorted, as the image is often not recorded in a perfect top-down view. The position of the object in the distorted image is inaccurate. Training models based on inaccurate data will reduce the accuracy of the training model.
Disclosure of Invention
According to the invention, the Chart evolutionary algorithm is used for searching the optimal solution of the parameters of the visual angle conversion matrix, the problem of estimating the real position of the object is converted into the problem of optimizing the parameters, and the differential evolutionary algorithm is used for searching the optimal solution of the parameters of the visual angle conversion matrix, so that the real position of the object can be accurately estimated, and the problem of inaccurate position of the acquired object caused by image distortion in the image processing problem is solved.
The invention is realized at least by one of the following technical schemes.
An automatic conversion method of image visual angles comprises the following steps:
Loading the camera shooting image and the plane image, clicking the camera shooting image and the plane image, and selecting a target camera shooting image and a target plane image in a path;
And connecting corresponding points of the image pickup image and the plane image, optimizing the optimal parameters of a conversion matrix by applying a differential evolution algorithm, and converting the image pickup image into the plane image through the conversion matrix.
Further, a point in the image is a point in three-dimensional space that is regarded as a point in world coordinates, a point in the plan view is regarded as a point in a pixel coordinate system, and the conversion of the corresponding points of the two coordinate systems is determined by a conversion matrix.
Further, the coordinates of the world coordinate system obtained by dotting and the coordinates in the corresponding pixel coordinate system are utilized to obtain accurate transformation matrix parameters, so that the transformation from the coordinates of the world coordinate system to the coordinates of the pixel coordinate system is realized, and the real position of the object is accurately estimated.
Further, the differential evolution algorithm comprises the following steps:
1) Initialization of
M individuals are randomly and uniformly generated in the solution space, each individual is a set of parameters of a transformation matrix, and each individual consists of n-dimensional vectors:
Xi(0)＝{xi,1(0),xi,2(0),xi,3(0),…,xi,n(0)}
i＝1,2,3,…,M
Wherein M is the number of randomly quasi-swapped individuals, namely the number of randomly generated conversion matrixes; x i (0) is the ith individual of the 0 th generation group in the solution space, namely the ith conversion matrix in the 0 th generation group; x i,n (0) is the nth dimension vector in X i (0), namely the nth parameter in the ith conversion matrix in the 0 th generation population;
Performing mutation operation;
The individual variation is realized through a differential strategy, wherein the differential strategy is to randomly select two different individuals in the population, scale the vector difference of the individuals and then perform vector synthesis with the individuals to be mutated; in the g-th iteration, 3 individuals X p1(g)、Xp2(g)、Xp3 (g) are randomly selected from the population, and the selected individuals are different, and the variation vectors generated by the three individuals are:
ΔP2,P3(g)＝Xp2(g)-Xp3(g)
Hi(g)＝Xp1(g)+F·ΔP2,P3(g)
Wherein Δp 2,P3(g)＝Xp2(g)-Xp3 (g) is a differential vector and F is a scaling factor; crossover operation
Where cr is the probability of crossing between [0,1], V i,j is the value of the j-th dimension of V i of crossing individuals; h i,j is the variant; x i,j (g) is the value of the j-th vector of the i-th individual in the current g-th generation population;
2) Selection operation
For each individual, the resulting solution is better or equal to the solution obtained for the individual through mutation, crossover, and selection to achieve full optimization, X i (g+1) is the ith individual in the (g+1) th generation population; v i (g) is the crossing individual of the ith individual of the g generation population obtained by the crossing operation; x i (g) is the ith individual of the g generation population.
Further, the j-th dimension value of the i-th individual is taken as follows:
Xi,j(0)＝Lj_min+rand(0,1)(Lj_max-Lj_min)
i＝1,2,3,…,M
j＝1,2,3,…,n
wherein L j_min is the lower bound of the j-th vector value and L j_max is the upper bound of the j-th vector value.
Further, the scaling factor F is a random number between [0,1 ].
Further, the adaptive adjustment of the scaling factor F is as follows:
Sequencing three randomly selected individuals in the mutation operator from good to bad to obtain X b、Xm,、Xw, wherein the corresponding fitness is f b、fm、fw, and the mutation operator is as follows:
Vi＝Xb+Fi·(Xm-Xw)
Wherein, F i is the scaling factor of the ith individual in the population, and meanwhile, the value of F is adaptively changed according to two individuals generating a differential vector:
Fl＝0.1,Fu＝0.9
Where F l is the lower bound of the scaling factor and F u is the upper bound of the scaling factor.
Further, the mutation operator is a fixed constant or has the capability of self-adapting to mutation step length and search direction, and corresponding automatic adjustment is performed according to different objective functions.
Further, the objective function is defined as the sum of Euclidean distances of all points on both sides in the image coordinate system and their corresponding coordinates in the real world coordinate system:
φ(S)＝∑i|Xi-X′i|
Where X i is a point on both sides of the image and X' i is the corresponding point in the real world coordinate system.
Further, the adaptive adjustment of the crossover probability cr is as follows:
Where f i is the fitness value of individual x i, f min and f max are the fitness values of the worst and best individuals, respectively, in the current population, Is the average of the fitness of the current population, cr l and cr u are the lower and upper limits of cr, respectively.
The differential evolution algorithm is a heuristic algorithm. In the searching process, the differential evolution algorithm can finish parameter optimization of a search matrix by using the differential evolution algorithm based on the coordinate positions according to the corresponding positions between images provided by a user, so that the corresponding positions of the target positions in the images are extracted. In order to further improve the global searching capability and efficiency of the algorithm, the searching accuracy of the algorithm can be improved by extracting the corresponding positions of the target positions of as many groups as possible.
Compared with the prior art, the invention has the beneficial effects that:
It is proposed to estimate the true position of an object based on an evolutionary algorithm. Some reference information of the image is used to estimate a transformation matrix that converts the image position to a real world position. The problem of inaccurate estimated object coordinates caused by image distortion is effectively solved.
Drawings
FIG. 1 is a flowchart illustrating an exemplary embodiment of an image conversion system;
FIG. 2 is an interface diagram of an image conversion system;
FIG. 3 is an original diagram of example 2;
FIG. 4 is a graph of the results after conversion in example 2.
Detailed Description
The method of the present invention is further described below with reference to the accompanying drawings.
The invention is applicable to processing an original image, describing and interpreting specific information in the image to apply to a certain transformation. For example: the noise reduction, contrast, rotation operations, computation density operations may be performed at the pixel level. The conversion of the image viewing angle allows the user to complete the operation without having to have a complicated knowledge of the image, as long as the user has knowledge of the characteristics of the object.
In modeling, it is particularly important to estimate the true position of an object, and errors generated by directly extracting the position from an image to estimate the true position of the object are relatively large. In the automatic conversion method of the image view angle, an evolutionary algorithm is adopted to convert the real position estimation problem of the object into a parameter optimization problem, and a differential evolutionary algorithm is utilized to search the optimal solution of the parameters of the view angle conversion matrix, so that the conversion from a pixel coordinate system to a plane 2D coordinate system is realized, and the optimal coordinate system conversion effect is obtained. Experiments show that the method can effectively estimate the real position of the target in the image.
Example 1
(1) Loading the image pickup image and the plane image, clicking the image pickup image and the plane image, and selecting a target image pickup image and a target plane image from a local path of a computer where the image is located;
(2) And drawing points according to the corresponding points in the image pickup image and the plane image, wherein the points in the image pickup image are points in a three-dimensional space (points in world coordinates), the points in the plane image are points in a pixel coordinate system, and the conversion of the corresponding points of the two coordinate systems is determined by a conversion matrix. The more the corresponding point groups of the connected image and plane diagram are, the more the number of checking groups provided for the differential evolution algorithm is, so that the more accurate the conversion matrix parameters converted by operation are.
The preferred content of this embodiment is the 18 parameters of the quasi-transform matrix. Therefore, the more points are drawn, the more corresponding points of the world coordinate system on pixel coordinates are obtained, the larger the search space is, the more accurate the corresponding points are drawn, and the more accurate the calculated conversion matrix parameters are;
(3) After the dot drawing is finished, clicking a run DE button (the DE button shown in figure 2), starting operation of the conversion matrix, and starting display operation progress by a progress bar;
(4) After the operation is finished, a dialog box is popped up to transform the matrix search completion-! "
(5) Clicking a confirmation button in the dialog box, and at the moment, clicking any point in the image, displaying a point on the plane corresponding to the point on the plane;
(6) While the transformation matrix is being calculated, 18 parameters of the transformation matrix are output in the disk.
Image processing refers to the technique of analyzing, processing, and handling an image to meet visual, psychological, or other requirements. Image processing is one application of signal processing in the field of images. Most of the images are currently stored in digital form, and thus image processing is often referred to as digital image processing. In addition, the treatment method based on the optical theory still plays an important role. Image processing is a subclass of signal processing and is also closely related to the fields of computer science, artificial intelligence, and the like.
Many of the conventional methods and concepts of one-dimensional signal processing can still be directly applied to image processing, such as noise reduction, quantization, and the like. However, the image belongs to a two-dimensional signal, and has a special side, and the processing mode and angle are different from those of a one-dimensional signal.
The image processing according to the present invention mainly processes coordinate information of a three-dimensional stereoscopic image into coordinates in a two-dimensional plan view. Four coordinate systems are involved in this image processing: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system. The image viewing angle conversion is the conversion between the pixel coordinate system and the world coordinate system.
A conversion relation exists between the coordinates of the world coordinate system (three-dimensional stereogram) and the coordinates of the corresponding points in the plane pixel coordinate system (two-dimensional plane diagram), and the conversion relation is completed by a conversion matrix containing 18 parameters. The specific conversion is described in detail in the following conversion of the world coordinate system to the pixel coordinate system. The optimized content in the invention is that the numerical value of 18 parameters of the conversion matrix containing 18 parameters is optimized by using a differential evolution algorithm, so that the coordinate of a world coordinate system obtained by dotting and the coordinate in a pixel coordinate system corresponding to the coordinate are utilized to obtain the accurate conversion matrix parameter, thereby accurately converting the coordinate of the world coordinate system into the coordinate of the pixel coordinate system and further accurately estimating the real position of an object.
(1) World coordinate system and camera coordinate system
From the world coordinate system to the camera coordinate system, rotation and translation are involved (in fact all movements can also be described by rotation matrices and translation vectors). And rotating around different coordinate axes by different angles to obtain corresponding rotation matrixes. The conversion relationship between the two coordinate systems is as follows:
where (X C,YC,ZC) is the coordinates of the object in the camera coordinate system and (X W,YW,ZW) is the coordinate representation of the object in the world coordinate system.
From the world coordinate system to the camera coordinate system, rotation and translation are involved (in fact all movements can also be described by rotation matrices and translation vectors). And rotating around different coordinate axes by different angles to obtain corresponding rotation matrixes.
Assuming that the angles phi, omega and theta from the world coordinate system to the camera coordinate system are required to be rotated around X, Y, Z axes respectively, then
T is a translation coordinate matrix:
(2) Camera coordinate system and image coordinate system
The method comprises the steps of converting from 3D to 2D from a camera coordinate system to an image coordinate system, belonging to perspective projection relation. The conversion relation between the two coordinate systems is that
Wherein (X C,YC,ZC) is the coordinates of the object in the camera coordinate system, and f is the camera focal length.
(3) Image coordinate system and pixel coordinate system
The pixel coordinate system and the image coordinate system are both on the imaging plane, except that the respective origin and measurement units are different. The origin of the image coordinate system is the intersection of the camera optical axis and the imaging plane, typically the midpoint of the imaging plane or the point PRINCIPAL POINT. The unit of the image coordinate system is mm, which belongs to the physical unit, and the unit of the pixel coordinate system is pixel, which usually describes that a pixel point is a few rows and a few columns. The transition between the two is as follows: where dx and dy represent how many mm each column and each row represent, respectively, i.e. 1pixel = dx mm, s' represents a non-orthogonal induced skew factor.
Wherein the method comprises the steps ofS=s' f, (u, v) is the point coordinate on the pixel coordinate system and (u 0,v0) is the point coordinate on the plane coordinate system.
By the conversion of the four coordinate systems, it is possible to obtain how a point is converted from the world coordinate system to the pixel coordinate system.
Wherein the method comprises the steps ofIs a matrix of 3 x 4.
The invention optimizes 18 parameters in the M 1,M2,M3 conversion matrix through a differential evolution algorithm.
FIG. 1 is a flow chart illustrating the problem of optimizing test case prioritization by the algorithm of the present invention. The following describes the specific implementation of the whole algorithm in terms of the content steps of the flow chart:
1) Initialization of
M individuals (each individual being a parameter of a set of transformation matrices) are randomly generated uniformly in the solution space, each individual consisting of n-dimensional vectors (in the present invention, the value of n is 18, i.e. 18 parameters of the transformation matrices):
Xi(0)＝{xi,1(0),xi,2(0),xi,3(0),…,xi,n(0)}
i＝1,2,3,…,M
Wherein M is the number of randomly quasi-swapped individuals, namely the number of randomly generated conversion matrixes; x i (0) is the ith individual of the 0 th generation group in the solution space, namely the ith conversion matrix in the 0 th generation group; x i,n (0) is the nth dimension vector in X i (0), namely the nth parameter in the ith conversion matrix in the 0 th generation population;
the j-th dimension value of the i-th individual is taken as follows:
Xi,j(0)＝Lj_min+rand(0,1)(Lj_max-Lj_min)
i＝1,2,3,…,M
j＝1,2,3,…,n
Wherein L j_min is the lower bound of the j-th vector value, and L j_max is the upper bound of the j-th vector value; (in the present invention, n has a value of 18, i.e., 18 parameters of the transformation matrix)
Performing mutation operation;
The individual variation is realized through a differential strategy, wherein the differential strategy is to randomly select two different individuals in the population, scale the vector difference of the two different individuals and then perform vector synthesis with the individual to be mutated. In the g-th iteration, 3 individuals X p1(g)、Xp2(g)、Xp3 (g) are randomly selected from the population, and the selected individuals are different, and the variation vectors generated by the three individuals are:
ΔP2,P3(g)＝Xp2(g)-Xp3(g)
Hi(g)＝Xp1(g)+F·ΔP2,P3(g)
Wherein Δp 2,P3(g)＝Xp2(g)-Xp3 (g) is a differential vector and F is a scaling factor;
the scaling factor F is set to a random number between [0,1];
Adaptive adjustment of F:
Sequencing three randomly selected individuals in the mutation operator from good to bad to obtain X b、Xm,、Xw, wherein the corresponding fitness is f b、fm、fw, and the mutation operator is as follows:
Vi＝Xb+Fi·(Xm-Xw)
f i is the scale factor of the ith individual in the population;
Meanwhile, the value of F is changed in a self-adaptive manner according to two individuals generating differential vectors:
Fl＝0.1,Fu＝0.9
f l is the lower bound of the scaling factor, F u is the upper bound of the scaling factor;
2) Crossover operation
Where cr is the probability of crossing between [0,1], V i,j is the value of the j-th dimension of V i of crossing individuals; h i,j is the variant; x i,j (g) is the value of the j-th vector of the i-th individual in the current g-th generation population;
wherein the adaptive adjustment of the parameter cr is as follows:
Where f i is the fitness value of individual x i, f min and f max are the fitness values of the worst and best individuals, respectively, in the current population, Is the average value of the fitness of the current population, cr l and cr u are the lower and upper limits of cr, respectively;
3) Selection operation
For each individual, the resulting solution is better or equal to the individual's crossover by variation, chosen to be fully optimal. (X i (g+1) is the ith individual in the (g+1) th generation population, V i (g) is the crossed individual of the ith individual in the g generation population obtained by the cross operation, and X i (g) is the ith individual in the g generation population.
The parameters of the invention are set as follows: population number n=100, termination condition is 25000 iterations. The final results show that the algorithm of the present invention did not fall into local optimization over multiple experiments. In the initial stage, under the condition that the corresponding points of the given image pickup image and the plane image are more, the image visual angle conversion effect is very good, and the accuracy of the image visual angle conversion is higher as the corresponding points of the given image pickup image and the plane image are increased. This illustrates the use of differential evolution algorithms for images.
Example 2
(1) The method is applied to the data set of the central station scene in New York. This dataset was obtained from 33 minutes of video (original video and extracted track from download http:// www.ee.cuhk.edu.hk/xgwang/grandcentral. Html).
(2) The dataset includes 40000 person movement tracking record tuples (over 6000000 tuples). Each tuple consists of an ID, an x value, a y value, and a timestamp. The reference information in the present embodiment is coordinates of four corners of the station square. First, coordinates of four corners are estimated from the trajectory approximation:
C1＝Cright,top＝(544,431)
C2＝Cright,bottom＝(695,0)
C3＝Cleft,top＝(170,431)
C4＝Cleft,bottom＝(3,0)
the true positions of the four corners are
R1＝Rright,top＝(34，48)
R2＝Rright,bottom＝(34,0)
R3＝Rleft,top＝(3,48)
R4＝Rleft,bottom＝(3,0)
(3) According to the above-described conversion of the world coordinate system into the pixel coordinate system, there are 18 parameters to be determined.
The 18 parameters in the transformation matrix are described as:
Defining an objective function as TM 1、TM2、TM3 corresponds to the three conversion matrices described above for M 1、M2、M3, respectively. C 1、C2,、C3、C4 corresponds to the estimated coordinates of the points corresponding to the upper right corner, lower right corner, upper left corner and lower left corner, respectively. R 1、R2、R3、R4 corresponds to the true coordinates of the points corresponding to the upper right corner, the lower right corner, the upper left corner and the lower left corner respectively. Phi (S) is the objective function.
(4) The DE is used to search for the optimal values of the 18 parameters to minimize the objective function. In order to improve the robustness of the differential evolution algorithm, the scaling factor F and the crossover rate CR randomly select each generation and individual from [0,1], and the finally calculated quasi-transform matrix parameters are as follows:
(5) Fig. 3 shows a rough simulation of the original data and the transformed trajectory, respectively. It can be observed that the present invention converts the original planar pattern of the pixel coordinate system into a rectangle like a trapezoid, and can reflect two-dimensional information more directly when seen from the right. The above results show that the method can effectively estimate the true position of the object in the image.
Example 3
(1) The invention is applied to the scenario where the data set crosses the road. The video duration is 60 seconds, involving about 150 pedestrians. The pavement area of the crosswalk is a rectangle of 11m×31m, and the position of the pedestrian is recorded every 0.5 s. Comparison results in that the two sides of the zebra stripe rectangle are not parallel in the result of the linear fitting method, but nearly parallel with our method. The method used in the present invention can produce more realistic results. .
(2) In the study of this example, the reference information was two sides of the zebra stripe region. The goal is to find the correct parameters in the transformation matrix to transform the warped trapezoids into standard rectangles. Thus, the objective function is defined as the sum of the Euclidean distances of all points on both sides in the image coordinate system and their corresponding coordinates in the real world coordinate system:
φ(S)＝∑i|Xi-X′i|
Where X i is a point on both sides of the image and X' i is the corresponding point in the real world coordinate system.
(3) The DE is used to search for the optimal values of the 18 parameters to minimize the objective function.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (1)
1. An automatic conversion method for image visual angles is characterized in that: the method comprises the following steps:
Loading the camera shooting image and the plane image, clicking the camera shooting image and the plane image, and selecting a target camera shooting image and a target plane image in a path; the points in the image are points in the three-dimensional space and are regarded as points in world coordinates, the points in the plane view are regarded as points in a pixel coordinate system, and the conversion of the corresponding points of the two coordinate systems is determined by a conversion matrix; obtaining accurate transformation matrix parameters by utilizing the coordinates of the world coordinate system obtained by dotting and the coordinates in the corresponding pixel coordinate system, thereby realizing the transformation from the coordinates of the world coordinate system to the coordinates of the pixel coordinate system and further accurately estimating the real position of the object;
Connecting corresponding points of the image pickup image and the plane image, optimizing optimal parameters of a conversion matrix by applying a differential evolution algorithm, and converting the image pickup image into the plane image through the conversion matrix;
the differential evolution algorithm comprises the following steps:
1) Initialization of
M individuals are randomly and uniformly generated in the solution space, each individual is a set of parameters of a transformation matrix, and each individual consists of n-dimensional vectors:
Xi(0)＝{xi,1(0),xi,2(0),xi,3(0),...,xi,n(0)}
i＝1，2，3，...，M
Wherein M is the number of randomly quasi-swapped individuals, namely the number of randomly generated conversion matrixes; x i (0) is the ith individual of the 0 th generation group in the solution space, namely the ith conversion matrix in the 0 th generation group; x i,n (0) is the nth dimension vector in X i (0), namely the nth parameter in the ith conversion matrix in the 0 th generation population;
2) Mutation operation
The individual variation is realized through a differential strategy, wherein the differential strategy is to randomly select two different individuals in the population, scale the vector difference of the individuals and then perform vector synthesis with the individuals to be mutated; in the g-th iteration, 3 individuals X p1(g)、Xp2(g)、Xp3 (g) are randomly selected from the population, and the selected individuals are different, and the variation vectors generated by the three individuals are:
ΔP2,P3(g)＝Xp2(g)-Xp3(g)
Hi(g)＝Xp1(g)+F·ΔP2,P3(g)
Wherein Δp 2,P3(g)＝Xp2(g)-Xp3 (g) is a differential vector and F is a scaling factor; crossover operation
Where cr is the probability of crossing between [0,1], V i,j is the value of the j-th dimension of V i of crossing individuals; h i,j is the variant; x i,j (g) is the value of the j-th vector of the i-th individual in the current g-th generation population;
the scaling factor F is a random number between [0,1], and the adaptive adjustment of the scaling factor F is as follows:
Sequencing three randomly selected individuals in the mutation operator from good to bad to obtain X b、Xm、Xw, wherein the corresponding fitness is f b、fm、fw, and the mutation operator is as follows:
Vi＝Xb+Fi·(Xm-Xw)
Wherein, F i is the scaling factor of the ith individual in the population, and meanwhile, the value of F is adaptively changed according to two individuals generating a differential vector:
＝0.1，Fu＝0.9
wherein, As a lower bound of the scaling factor, F u is an upper bound of the scaling factor;
The adaptive adjustment of the crossover probability cr is as follows:
Where f i is the fitness value of individual x i, f min and f max are the fitness values of the worst and best individuals, respectively, in the current population, Is the average value of the fitness of the current population,/>And/>The lower and upper limits of cr, respectively;
3) Selection operation
For each individual, the resulting solution is better or equal to the solution obtained for the individual through mutation, crossover, and selection to achieve full optimization, X i (g+1) is the ith individual in the (g+1) th generation population; v i (g) is the crossing individual of the ith individual of the g generation population obtained by the crossing operation; x i (g) is the ith individual of the g generation population;
the j-th dimension value of the i-th individual is taken as follows:
Xi,j(0)＝Lj_min+rand(0,1)(Lj_max-Lj_min)
i＝1，2，3，...，M
j＝1，2，3，...，n
Wherein L j_min is the lower bound of the j-th vector value, and L j_max is the upper bound of the j-th vector value;
The mutation operator is a fixed constant or has the capability of self-adapting to mutation step length and search direction, and corresponding automatic adjustment is carried out according to different objective functions;
the objective function is defined as the sum of the Euclidean distances of all points on both sides in the image coordinate system and their corresponding coordinates in the real world coordinate system:
φ(S)＝∑i|Xi-X′i|
Where X i is a point on both sides of the image and X' i is the corresponding point in the real world coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210439518.XA CN114842071B (en) | 2022-04-25 | 2022-04-25 | Automatic conversion method for image viewing angle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210439518.XA CN114842071B (en) | 2022-04-25 | 2022-04-25 | Automatic conversion method for image viewing angle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114842071A CN114842071A (en) | 2022-08-02 |
| CN114842071B true CN114842071B (en) | 2024-05-24 |
Family
ID=82565826
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210439518.XA Active CN114842071B (en) | 2022-04-25 | 2022-04-25 | Automatic conversion method for image viewing angle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114842071B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102779207A (en) * | 2012-06-19 | 2012-11-14 | 北京航空航天大学 | Wing profile optimal design method of parallel difference evolutionary algorithm based on open computing language (Open CL) |
| CN105631844A (en) * | 2014-10-28 | 2016-06-01 | 上海大学 | Image camera calibration method |
| CN111416972A (en) * | 2020-01-21 | 2020-07-14 | 同济大学 | Three-dimensional imaging system and method based on axially adjustable cascade rotating mirror |
| WO2021218157A1 (en) * | 2020-04-28 | 2021-11-04 | 福州大学 | Discrete differential evolution-based multi-strategy optimization x-architecture minimum tree construction method |
-
2022
- 2022-04-25 CN CN202210439518.XA patent/CN114842071B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102779207A (en) * | 2012-06-19 | 2012-11-14 | 北京航空航天大学 | Wing profile optimal design method of parallel difference evolutionary algorithm based on open computing language (Open CL) |
| CN105631844A (en) * | 2014-10-28 | 2016-06-01 | 上海大学 | Image camera calibration method |
| CN111416972A (en) * | 2020-01-21 | 2020-07-14 | 同济大学 | Three-dimensional imaging system and method based on axially adjustable cascade rotating mirror |
| WO2021218157A1 (en) * | 2020-04-28 | 2021-11-04 | 福州大学 | Discrete differential evolution-based multi-strategy optimization x-architecture minimum tree construction method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114842071A (en) | 2022-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111709409B (en) | Face living body detection method, device, equipment and medium | |
| CN109558862A (en) | The people counting method and system of attention refinement frame based on spatial perception | |
| JP2023545199A (en) | Model training method, human body posture detection method, apparatus, device and storage medium | |
| CN112085835B (en) | Three-dimensional cartoon face generation method and device, electronic equipment and storage medium | |
| CN110910391A (en) | Video object segmentation method with dual-module neural network structure | |
| CN111414953A (en) | Point cloud classification method and device | |
| CN112989085A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN115311502A (en) | Remote sensing image small sample scene classification method based on multi-scale double-flow architecture | |
| CN114913552A (en) | Three-dimensional human body density corresponding estimation method based on single-view-point cloud sequence | |
| CN114494395A (en) | Depth map generation method, device and equipment based on plane prior and storage medium | |
| CN114882493A (en) | Three-dimensional hand posture estimation and recognition method based on image sequence | |
| CN114842071B (en) | Automatic conversion method for image viewing angle | |
| KR20210058638A (en) | Apparatus and method for image processing | |
| CN115908697A (en) | Generation model based on point cloud probability distribution learning and method thereof | |
| CN115688234A (en) | Building layout generation method, device and medium based on conditional convolution | |
| CN115359508A (en) | Performing complex optimization tasks with increased efficiency by expert neuron optimization | |
| CN114863132A (en) | Method, system, equipment and storage medium for modeling and capturing image spatial domain information | |
| CN116433852B (en) | Data processing method, device, equipment and storage medium | |
| CN113850900B (en) | Method and system for recovering depth map based on image and geometric clues in three-dimensional reconstruction | |
| CN118096836B (en) | Target tracking method and system based on rotary constant-change network and triple-concentration mechanism | |
| Lin et al. | Bio-inspired multi-level interactive contour detection network | |
| Yuan | A 3D photo-realistic environment simulator for mobile robots | |
| Huang et al. | Estimating the real positions of objects in images by using evolutionary algorithm | |
| Sun et al. | Multi-view 3D Reconstruction Based on View Selection Network | |
| Lin et al. | Enhance Local Feature Consistency with Structure Similarity Loss for 3D Semantic Segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |