CN103516960A - Shipborne video image stabilization method based on ship motion posture prediction - Google Patents

Abstract

The invention relates to a shipborne video image stabilization method based on ship motion posture prediction. The method is characterized in that a ship motion posture prediction method based on a least squares support vector machine (Least squares support vector machine, LSSVM) of particle swarm optimization (Particle swarm optimization, PSO) is provided to predict the ship motion posture, so as to acquire ship motion vector prediction data; according to the ship motion vector prediction data, the motion vector data of images are calculated; the image compensation vector is calculated in advance; and motion compensation is carried out on the images frame by frame. According to the method, a particle swarm optimization algorithm is introduced to carry out parameter selection on the least squares support vector machine; the ship motion posture prediction accuracy is improved; the image motion vector estimation accuracy is improved; the video image stabilization quality is improved; the image compensation vector is calculated in advance; the real-time of an image stabilization algorithm is enhanced; and the problem of lag, which is caused due to the fact that the traditional method estimates the image motion vector according to a fluctuated video, is solved.

Description

A kind of boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll

Technical field

The present invention relates to a kind of boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll.

Background technology

Boat-carrying photo electric imaging system, because operating distance is far away, the impact that is subject to its carrier (boats and ships) attitude to change and vibrate, image sequence interframe changes greatly, causes image blurring, unstable., there is many negative consequences in these unsettled sequence of video images: weapon hydraulic performance decline; Manual observation difficulty, and very easily cause visual fatigue, cause failing to judge and judging by accident; Image detail can not clearly be expressed, and causes image processing algorithm difficulty to strengthen; Be unfavorable for the storage of the digital video frequency recording equipment of rear end, compression ratio reduces; Vision application to obtaining information from image has a negative impact, and as the identification of moving target becomes difficult, the tracking of target also can be affected, and even causes lose objects.Therefore, for this boat-carrying photo electric imaging system, there is the video sequence of irregular movement, be necessary to carry out steady picture and process, remove unnecessary randomized jitter and disturb, be beneficial to subsequent video such as supervision, target detection, identification and tracking and process.While utilizing boats and ships imaging system to carry out real-time scouting observation to distant object on sea, need to adopt long focal length lens, but focal length is longer, video camera is small rock obtaining the stability influence of image sequence larger, boats and ships as camera system carrier are affected by wave simultaneously, attitude alters a great deal, and image sequence is rocked more outstanding.Therefore, the sequence of video images of boat-carrying imaging system is carried out to steady picture and become particularly important.

If camera system platform is unstable, same object will be imaged on the different region of image, and on monitor, picture with the aid of pictures will thicken.At this time need to utilize certain matching criterior, find match block corresponding in reference frame and present frame, determine image motion vector.Then image motion vector is analyzed and processed, computed image compensation vector, mobile current frame pixel in the opposite direction, makes itself and former frame picture registration, reaches the object of image compensation, the image sequence of stable output.As can be seen here, motion estimation algorithm is the core of image stabilization system fast and accurately, and it has occupied the main operation time of image stabilization system, has determined speed and the precision of image stabilization system.Existing is surely to calculate according to the motion vector of the video estimated image having fluctuateed as algorithm, and algorithm is complicated, lacks real-time, and the compensation motion vector of image exists the problem lagging behind.

Summary of the invention

The object of the invention is to provide a kind of boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll, and the compensation motion vector that effectively solves image exists the problem lagging behind.

Realize the object of the invention technical scheme:

A boat-carrying Video Stabilization method for Ship Movement Prediction and Anti-roll, is characterized in that:

Step 1: Ship Motion Attitude is forecast, obtain ship motion vector prediction data;

Step 2: according to ship motion vector prediction data, calculate the motion vector data of image;

Step 3: the image motion vector data according to obtaining, calculate in advance image compensation vector, image is carried out to motion compensation frame by frame.

In step 1, adopt the Forecasting Methodology based on population-least square method supporting vector machine to forecast Ship Motion Attitude.

In step 2, establish (x ₀, y ₀) be the rectangular coordinate system coordinate of target in reference frame image, its polar coordinates are (r ₀, θ ₀); (x _k, _yk) be the rectangular coordinate system coordinate of target in k two field picture, its polar coordinates are (r _k, θ _k), the motion vector of k two field picture (Δ x, Δ y, Δ θ) is obtained by yaw angle, roll angle and the pitch angle of boats and ships, wherein, and Δ x=r _k-r ₀the yaw angle of corresponding boats and ships, Δ _y=(r _k-r ₀) θ ₀the roll angle of corresponding boats and ships, Δ θ=θ _k-θ ₀the pitch angle of corresponding boats and ships.

In step 3, the start frame of selecting video, as benchmark image, is usingd benchmark image as initial time, and the image compensation vector according to obtaining, carries out motion compensation frame by frame, and every 10 frames adopt 1 start frame.

The beneficial effect that the present invention has:

The present invention, according to ship motion vector prediction data, calculates the vector motion forecast data of image; According to the image vector exercise data obtaining, can calculate in advance image compensation vector, image is carried out to motion compensation frame by frame, strengthen the real-time of steady picture algorithm, effectively solve conventional method and according to the motion vector of the video estimated image having fluctuateed, have the problem lagging behind.

The Forecasting Methodology that the present invention is based on population-least square method supporting vector machine is forecast Ship Motion Attitude, can improve Ship Movement Prediction and Anti-roll precision, improves boat-carrying electronic steady image image motion vector and estimates accuracy, improves Video Stabilization quality.

The start frame of selecting video of the present invention is as benchmark image, using benchmark image as initial time, according to the image vector Motion prediction data that obtain, carry out frame by frame motion compensation, and every 10 frames adopt 1 start frame, avoid adopting stabilizer frame to compensate always, the image information loss after compensation is increased gradually, the problem that produces error accumulation, improves steady image quality.

Accompanying drawing explanation

Fig. 1 is Video Stabilization method flow diagram of the present invention;

Fig. 2 is graph of a relation between target component variation and ship motion;

Fig. 3 is the Y-PSNR comparison diagram of stable front and back image sequence.

Embodiment

Step 1: the Forecasting Methodology based on population-least square method supporting vector machine is forecast Ship Motion Attitude, obtains ship motion vector motion forecast data.

For Ship Motion Attitude data { x ₁, x ₂, x ₃..., x _n, i=1,2 ..., n, { x _nforecast desired value, set up input x={x _n-1, x _n-2, x _n-3..., x _n-mand output y={x _nbetween mapping relations.

In forecasting model, the sample of PSO-LSSVM study is:

$X=\left[\begin{array}{cccc}{x}_1& x_2& ...& x_m\\ x_1& x_3& ...& x_{m+1}\\ ...& ...& ...& ...\\ x_{n-m}& x_{n-m+1}& ...& x_{n-1}\end{array}\right]Y=\left[\begin{array}{c}{x}_{m+1}\\ x_{m+2}\\ ...\\ x_n\end{array}\right]---\left(1\right)$

Wherein, m is input dimension, and n is training sample number.

After having trained, to following a step of forecasting, be:

${\hat{x}}_{n+1}=f(x_n,x_{n-1},...x_{n-m+1})---\left(2\right)$

In formula, x _nthe actual value that represents n data,

the predicted value that represents n data.

Step 2: according to ship motion vector motion forecast data, obtain the vector motion forecast data of image.

Obtaining on the basis of Ship Motion vector motion forecast data, the Ship Motion vector forecasting, after coordinate transform, is transformed to the motion vector of image, can calculate in advance image compensation vector.

Owing to inevitably wanting the depth information of lose objects in target projection process, for further simplification, its motion can be described with a translation vector and a spin matrix around optical axis (z axle), motion by video camera or carrier (naval vessel) causes the conversion such as image generation translation being shot, rotation to be called as " affine transformation ", and in homogeneous coordinate system, the reduced form of model is as follows:

$\left[\begin{array}{c}{x}^{\′}\\ y^{\′}\end{array}\right]=\left[\begin{array}{cc}1& -\mathrm{\Δ\θ}\\ \mathrm{\Δ\θ}& 1\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]+\left[\begin{array}{c}\mathrm{\Δx}\\ \mathrm{\Δy}\end{array}\right]---\left(3\right)$

In formula, θ is the angle that image rotates around optical axis, and Δ x, Δ y are image displacement in the horizontal and vertical directions.

On the basis of above-mentioned theory, the parameter of setting up target changes and ship motion relational model, and it is example that the imaging system of take is loaded in side in the starboard of host's ship, as shown in Figure 2.

If (x ₀, y ₀) be the rectangular coordinate system coordinate of target in reference frame (the 0th frame) image, its polar coordinates are (r ₀, θ ₀); (x _k, y _k) be the rectangular coordinate system coordinate of target in k two field picture, its polar coordinates are (r _k, θ _k).After motion model simplification due to image, only to translational motion with around rotatablely moving of optical axis, compensate, so here using the center of reference picture as initial point, the current frame image of deriving is to the transformation relation of reference frame image.

First, by straight line L _kwith respect to initial point rotation Δ θ=θ _k-θ ₀after, use straight line

represent, according to rotation of coordinate relation, conversion is expressed as follows:

$\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]=\left[\begin{array}{cc}\cos ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)& -\sin ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)\\ \sin ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)& \cos ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)\end{array}\right]\left[\begin{array}{c}{x}_k\\ y_k\end{array}\right]---\left(4\right)$

In formula,

(x _k, y _k) be respectively straight line

and L _kon a certain point coordinates.Then, then by straight line

move to straight line L ₀position, convert as follows:

$\left[\begin{array}{c}{x}_0\\ y_0\end{array}\right]=\left[\begin{array}{c}(r_0-r_k)\cos {\mathrm{\θ}}_0\\ (r_0-r_k)\sin {\mathrm{\θ}}_0\end{array}\right]+\left[\begin{array}{c}{x}_k^{\′}\\ y_k^{\′}\end{array}\right]---\left(5\right)$

Wherein, (x ₀, y ₀) be straight line L ₀on point, formula (4) substitution (5) can be obtained by straight line L _kto L ₀transformation model:

$\left[\begin{array}{c}{x}_0\\ y_0\end{array}\right]=\left[\begin{array}{cc}\cos ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)& -\sin ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)\\ \sin ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)& \cos ({\mathrm{\θ}}_0-{\mathrm{\θ}}_k)\end{array}\right]\left[\begin{array}{c}{x}_k\\ y_k\end{array}\right]+\left[\begin{array}{c}(r_0-r_k)\cos {\mathrm{\θ}}_0\\ (r_0-r_k)\sin {\mathrm{\θ}}_0\end{array}\right]$

(6)

After simplification, be:

$\left[\begin{array}{c}{x}_0\\ y_0\end{array}\right]=\left[\begin{array}{cc}1& {\mathrm{\θ}}_0-{\mathrm{\θ}}_k\\ {\mathrm{\θ}}_0-{\mathrm{\θ}}_k& 1\end{array}\right]\left[\begin{array}{c}{x}_k\\ y_k\end{array}\right]+\left[\begin{array}{c}{r}_0-r_k\\ (r_0-r_k){\mathrm{\θ}}_0\end{array}\right]---\left(7\right)$

By above formula, can be obtained the motion vector (Δ x, Δ y, Δ θ) of k two field picture, wherein Δ x=r _k-r ₀for image displacement in the horizontal direction, be that the yawing by boats and ships causes, i.e. Δ x=r _k-r ₀the yaw angle of corresponding boats and ships; Δ y=(r _k-r ₀) θ ₀for image displacement in vertical direction, be that the rolling by boats and ships causes, i.e. Δ y=(r _k-r ₀) θ ₀the roll angle of corresponding boats and ships, Δ θ=θ _k-θ ₀for the angle that image rotates around the origin of coordinates, reflected the pitching angle of boats and ships in interval during this period of time.Therefore, obtain image motion vector need forecast the yawing of Ship Motion, roll and pitch.By forecasting Ship Motion Attitude, be converted to after the motion vector (Δ x, Δ y, Δ θ) of image, can carry out boat-carrying sequence of video images inter motion compensation.

Step 3: according to the image vector Motion prediction data that obtain, image is carried out to motion compensation frame by frame.

Obtain after image motion vector, computed image compensation vector, the start frame of selecting video is as benchmark image, using benchmark image as initial time, carry out frame by frame motion compensation, the 1st frame of compensation original series obtains the 2nd stable frame, then compensates the 2nd stable frame and obtain the 3rd stable frame, thereby can obtain whole stable image sequence.Because each stabilizer frame is all to be compensated by last stabilizer frame, in critical sequences, there will not be unexpected motion excursion phenomenon, if but adopt stabilizer frame to compensate, the image information loss after compensation will increase gradually; always can produce error accumulation, therefore stipulate that every 10 frames adopt 1 start frame.

Below in conjunction with specific embodiment, the present invention is further described.

The ship motion yaw angle of take forecast is example, utilizes PSO-LSSVM to carry out the forecast of ship bow cradle angle, and using yaw angle ψ (k) as input, input vector can be expressed as:

X＝[ψ(k),ψ(k-1),...,ψ(k-n+1)] ^T (8)

Wherein, the order that n is yaw angle.

PSO-LSSVM is output as ψ (k+1).Given training sample is:

$D=\{({\mathrm{<figure-callout\; id="1"\; label="X"\; filenames="HDA0000396930590000021.png"\; state="\{\{state\}\}">X</figure-callout>}}_1,{\mathrm{\&psi;}}_1),(X_2,{\mathrm{\&psi;}}_2),...,(X_l,{\mathrm{\&psi;}}_l)\}\&Subset;R^N\&times;R---\left(9\right)$

Wherein, X ₁, ..., X _lfor k arrives k+1-l input vector constantly:

X ₁＝[ψ(k),ψ(k-1),...,ψ(k-n+1)] ^T

X ₂＝[ψ(k+1),ψ(k),...,ψ(k-n+2)] ^T

.

.

.

X _l＝[ψ(k+l-1),ψ(k+l-2),...,ψ(k-n+l)] ^T

L=1000 wherein, n=3.PSO-LSSVM is 3 input 1 outputs, utilizes front 3 moment data to forecast next yaw angle data constantly.Other parameter of population arranges as follows: particle number 30, study factor c ₁=c ₂=2, inertia weight w is reduced to 0.2 with iterations from 0.9 linearity, maximum iteration time 100 times.Least square method supporting vector machine kernel functional parameter σ ²with penalty factor γ respectively in [0,20] and [1,2000] interior initialization.The course of required ship motion, roll and pitch angular data obtain according to emulation, and because video was 25 frame/seconds, so the ship motion sampling time is made as 0.04s, simulation time is 100s, totally 2500 groups of data, wherein, front 1000 groups of data are training data, and remaining data are check data.

In like manner, forecast roll angle and the pitch angle of ship motion.

According to ship motion vector and image motion vector transformational relation, the yawing of the ship motion forecasting, roll and pitch angle are converted to image motion vector, can carry out the motion compensation of video sequence interframe according to the boat-carrying video sequence interframe motion compensation process in embodiment, obtain stable video sequence.

The video sequence of experiment for image stabilization is the gray level image video of resolution 720x480.Employing Y-PSNR PSNR is to one section of video analysis of stablizing before and after processing, steady as the PSNR value between the frame of video of front and back by calculating, and can find out the stablizing effect of the rear image sequence of steady picture processing.Fig. 3 is the PSNR value before and after Video Stabilization is processed, in Fig. 3, abscissa is the frame number of video sequence, ordinate is the PSNR value of video sequence, and the solid line of image below is the PSNR value curve of original video sequence interframe, and the dotted line of top is surely as the PSNR value curve of rear video sequence.As can be seen from Figure, the PSNR value of the video sequence after steady picture is obviously greater than the front video sequence of steady picture.PSNR value is larger, illustrates that between stable rear picture frame, gray-scale deviation amount is less, and image stabilization effect is better.

Claims (4)

1. the boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll, is characterized in that:

Step 1: Ship Motion Attitude is forecast, obtain ship motion vector prediction data;

Step 2: according to ship motion vector prediction data, calculate the motion vector data of image;

Step 3: the image motion vector data according to obtaining, calculate in advance image compensation vector, image is carried out to motion compensation frame by frame.

2. the boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll according to claim 1, is characterized in that: in step 1, the Forecasting Methodology based on population-least square method supporting vector machine is forecast Ship Motion Attitude.

3. the boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll according to claim 2, is characterized in that: in step 2, establish (x ₀, y ₀) be the rectangular coordinate system coordinate of target in reference frame image, its polar coordinates are (r ₀, θ ₀); (x _k, _yk) be the rectangular coordinate system coordinate of target in k two field picture, its polar coordinates are (r _k, θ _k), the motion vector of k two field picture (Δ x, Δ y, Δ θ) is obtained by yaw angle, roll angle and the pitch angle of boats and ships, wherein, and Δ x=r _k-r ₀the yaw angle of corresponding boats and ships, Δ _y=(r _k-r ₀) θ ₀the roll angle of corresponding boats and ships, Δ θ=θ _k-θ ₀the pitch angle of corresponding boats and ships.

4. the boat-carrying Video Stabilization method based on Ship Movement Prediction and Anti-roll according to claim 3, it is characterized in that: in step 3, the start frame of selecting video is as benchmark image, using benchmark image as initial time, according to the image motion vector forecast data obtaining, carry out frame by frame motion compensation, and every 10 frames adopt 1 start frame.

Images