CN112164092B - Generalized Markov dense optical flow determination method and system - Google Patents

Generalized Markov dense optical flow determination method and system Download PDF

Info

Publication number
CN112164092B
CN112164092B CN202011087976.9A CN202011087976A CN112164092B CN 112164092 B CN112164092 B CN 112164092B CN 202011087976 A CN202011087976 A CN 202011087976A CN 112164092 B CN112164092 B CN 112164092B
Authority
CN
China
Prior art keywords
optical flow
markov
value
generalized
variational
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011087976.9A
Other languages
Chinese (zh)
Other versions
CN112164092A (en
Inventor
江少锋
杨素华
张聪炫
陈震
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanchang Hangkong University
Original Assignee
Nanchang Hangkong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanchang Hangkong University filed Critical Nanchang Hangkong University
Priority to CN202011087976.9A priority Critical patent/CN112164092B/en
Publication of CN112164092A publication Critical patent/CN112164092A/en
Application granted granted Critical
Publication of CN112164092B publication Critical patent/CN112164092B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/29—Graphical models, e.g. Bayesian networks
    • G06F18/295—Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a generalized Markov dense optical flow determination method and a generalized Markov dense optical flow determination system. The method comprises the steps of obtaining any two continuous frames of images; calculating a data item of a Markov variational optical flow model from the two frames of images; calculating an initial optical flow value between the two frames of images according to the data items of the Markov variable optical flow model; calculating a value of an optical flow delta field from the initial optical flow value; reconstructing a Markov variable optical flow model according to the value of the optical flow incremental field, and determining a generalized Markov variable optical flow model; calculating data items of a generalized Markov variational optical flow model according to the two frames of images; dense optical flow values are calculated from data items of a generalized Markov variational optical flow model. The invention realizes the calculation of the dense optical flow field and improves the calculation precision of the variational optical flow.

Description

Generalized Markov dense optical flow determination method and system
Technical Field
The invention relates to the field of optical flow calculation, in particular to a generalized Markov dense optical flow determining method and system.
Background
The image sequence variable optical flow computing technology has made some progress in moving object detection, and is also applied to the fields of social production, life and the like and plays an important role. The markov optical flow model is an important method for calculating the optical flow of the image, but the method needs to discretize the size of the optical flow, so that only a sparse optical flow field can be obtained, and the sampling width during discretization limits the calculation accuracy. Therefore, it is necessary to invent a markov variational optical flow determination method or system capable of realizing dense optical flow field calculation, which can not only obtain the dense optical flow field, but also improve the calculation accuracy.
Disclosure of Invention
The invention aims to provide a method and a system for determining generalized Markov dense optical flow, which are used for realizing the calculation of a dense optical flow field and improving the calculation precision of the variational optical flow.
In order to achieve the purpose, the invention provides the following scheme:
a generalized markov dense optical flow determination method, comprising:
acquiring any two continuous frames of images;
calculating a data item of a Markov variational optical flow model from the two frames of images;
calculating an initial optical flow value between the two frames of images according to the data items of the Markov variable optical flow model;
calculating a value of an optical flow delta field from the initial optical flow value;
reconstructing a Markov variable optical flow model according to the value of the optical flow incremental field, and determining a generalized Markov variable optical flow model;
calculating data items of a generalized Markov variational optical flow model according to the two frames of images;
dense optical flow values are calculated from data items of a generalized Markov variant optical flow model.
Optionally, the calculating a value of an optical flow increment field according to the initial optical flow value specifically includes:
using the formula i p ＝k 2 (f p L/2) determining the value of the optical flow delta field for a pixel p of an image; wherein, the i p Value of the optical flow delta field for pixel p, f p Is the initial optical flow value, k, of pixel point p 2 The increment step length is L, and the number of sampling points is L.
Optionally, the calculating a data item of the generalized markov variational optical flow model according to the two frames of images specifically includes:
using a formula
Figure BDA0002720934160000021
Calculating data items of the generalized Markov variational optical flow model; wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002720934160000022
for data items of a generalized Markov variational optical flow model, tau being a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of vertical direction of dense light flow, I 1 And I 2 The two images are respectively front and back frames, u is the component of the optical flow in the horizontal direction, v is the component of the optical flow in the vertical direction,
Figure BDA0002720934160000023
is an optical flow vector.
Optionally, the calculating a dense optical flow value according to the data item of the generalized markov variational optical flow model specifically includes:
calculating information to be transmitted by each image pixel by adopting a BP algorithm according to the data items of the generalized Markov variable optical flow model;
iteratively updating the message of each pixel point in parallel according to the information to obtain an updated message;
simultaneously transmitting the updated information to the upper direction, the lower direction, the left direction and the right direction of each image pixel point in parallel, returning the step of iteratively updating the information of each pixel point in parallel according to the information to obtain the updated information, and determining the confidence coefficient vector of the pixel point until N times of updating;
and determining the optical flow value of the pixel point according to the minimum confidence coefficient in the confidence coefficient vector of the pixel point.
A generalized markov dense optical flow determination system, comprising:
the image acquisition module is used for acquiring any two continuous frames of images;
the data item calculation module of the Markov variable optical flow model is used for calculating the data item of the Markov variable optical flow model according to the two frames of images;
an initial optical flow value calculating module, configured to calculate an initial optical flow value between the two frames of images according to the data item of the markov variable fractional optical flow model;
a value calculation module of the optical flow increment field, which is used for calculating the value of the optical flow increment field according to the initial optical flow value;
the generalized Markov variational optical flow model determining module is used for reconstructing a Markov variational optical flow model according to the value of the optical flow incremental field and determining the generalized Markov variational optical flow model;
the data item calculation module of the generalized Markov variational optical flow model is used for calculating the data item of the generalized Markov variational optical flow model according to the two frames of images;
and the dense optical flow value calculation module is used for calculating the dense optical flow values according to the data items of the generalized Markov variable optical flow model.
Optionally, the value calculating module of the optical flow incremental field specifically includes:
a value calculation unit of the optical flow increment field for using the formula i p ＝k 2 (f p L/2) determining the value of the optical flow increment field for a pixel point p of an image; wherein, the i p Value of the optical flow delta field for pixel p, f p Is the initial optical flow value, k, of pixel point p 2 For incremental step length, L is the number of sampling points.
Optionally, the data item calculation module of the generalized markov variational optical flow model specifically includes:
data item calculation unit of generalized Markov variational optical flow model for utilizing formula
Figure BDA0002720934160000031
Calculating data items of the generalized Markov variational optical flow model; wherein the content of the first and second substances,
Figure BDA0002720934160000032
for data items of a generalized Markov variational optical flow model, tau being a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of vertical direction of dense light flow, I 1 And I 2 The two images are respectively front and back frames, u is the component of the optical flow in the horizontal direction, v is the component of the optical flow in the vertical direction,
Figure BDA0002720934160000033
is an optical flow vector.
Optionally, the dense optical flow value module specifically includes:
the transmitted information determining unit is used for calculating the information to be transmitted of each image pixel by adopting a BP algorithm according to the data item of the generalized Markov variable optical flow model;
the information updating unit is used for iteratively updating the information of each pixel point in parallel according to the information to obtain updated information;
the confidence coefficient vector determining unit is used for simultaneously transmitting the updated information to the upper direction, the lower direction, the left direction and the right direction of each image pixel point in parallel, returning the step of iteratively updating the information of each pixel point in parallel according to the information to obtain the updated information, and determining the confidence coefficient vector of the pixel point until the updating is carried out for N times;
and the optical flow value determining unit is used for determining the optical flow value of the pixel point according to the minimum confidence coefficient in the confidence coefficient vector of the pixel point.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a generalized Markov dense optical flow determining method and system, wherein the first step is that a Markov variable optical flow model calculating method acquires optical flow between two continuous frame images, but the optical flow is discrete and sparse; and a second step of establishing an optical flow increment field by adopting a small step length by taking the optical flow calculation result of the first step as a center, redefining a data item and a smooth item in the Markov variable-split optical flow model by using the increment field, establishing a generalized Markov variable-split optical flow model, and optimizing the generalized Markov optical flow model by adopting a BP (back propagation) model to obtain a dense optical flow field. The method can obtain a dense optical flow field and improve the calculation accuracy.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of a generalized Markov dense optical flow determination method provided by the present invention;
FIG. 2 is a diagram illustrating a conventional parabolic lower bound method;
FIG. 3 is a schematic diagram of a lower bound parabola method provided by the present invention;
FIG. 4(a) is a schematic diagram of a 10 th frame of a Yosmenite valley image, FIG. 4((b) is a color diagram of a motion optical flow vector of the image, FIG. 4(c) is a schematic diagram of a calculation result of a conventional Markov optical flow model, and FIG. 4 (d) is a schematic diagram of a calculation result of a generalized Markov optical flow model according to the present invention;
FIG. 5 is a block diagram of a generalized Markov dense optical flow determination system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a method and a system for determining generalized Markov dense optical flow, which are used for realizing the calculation of a dense optical flow field and improving the calculation precision of the variational optical flow.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Fig. 1 is a schematic flow chart of a generalized markov dense optical flow determination method provided by the present invention, and as shown in fig. 1, the generalized markov dense optical flow determination method provided by the present invention includes:
s101, acquiring any two continuous frames of images.
S102, calculating data items of the Markov variational optical flow model according to the two frames of images.
S103, calculating an initial optical flow value between the two frames of images according to the data items of the Markov variable optical flow model. The Markov variable optical flow model inputs two continuous frame images in a motion video and outputs motion vectors and motion optical flows (u, v) of a moving object in the x direction and the y direction in the two frame images.
The Markov variational optical flow model is as follows:
Figure BDA0002720934160000051
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002720934160000052
is an optical flow vector, u is a component of the optical flow in the horizontal direction, v is a component of the optical flow in the vertical direction,
Figure BDA0002720934160000053
l is the number of sample points in the x and y directions, k 1 Is the sampling width.
First item
Figure BDA0002720934160000054
For smoothing terms, indicating the marking of the optical flow o p And o q Simultaneously giving the cost of two adjacent image pixel points p and q,
Figure BDA0002720934160000055
and (4) representing the neighborhood of the image pixel point, and taking four neighborhoods, namely an upper neighborhood, a lower neighborhood, a left neighborhood and a right neighborhood in two dimensions. The second term is a data item representing the optical flow o p A cost is assigned to the p-point.
The data items of the Markov variational optical flow model are calculated by adopting the following formula:
Figure BDA0002720934160000056
where τ is the cutoff value, x p ,y p Is the coordinate of pixel point p, I 1 And I 2 The two images are respectively the front and the back frames, and are both known quantities. The lowest energy of each pixel point is output.
Calculating the information of image pixel points, adopting a Max-Product method in a BP algorithm, and defining the information transmitted among the image pixel points as follows:
Figure BDA0002720934160000061
wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0002720934160000062
optical flow o representing image pixel point p and its direction neighborhood point q q The information that is to be propagated is,
Figure BDA0002720934160000063
representing the neighborhood of p points excluding q points. To calculate the formula, a dimensionality reduction method is adopted. The dimension reduction method converts the two-dimensional model of the formula into a two-dimensional modelThe dimension model is used for reducing the dimension and calculating, and a parabola lower bound method is adopted during the one-dimensional calculation. Two groups of parabolas obtained by a dimensionality reduction method according to the formula are output as the minimum value of an envelope formed by the parabolas.
As a specific embodiment, the information of the image pixels is calculated, a Max-Product method in a BP algorithm is adopted, and the information transmitted between the image pixels is defined as follows:
Figure BDA0002720934160000064
wherein the content of the first and second substances,
Figure BDA0002720934160000065
mark o for expressing image pixel point p and its adjacent area point q q The information that is to be propagated is,
Figure BDA0002720934160000066
indicating that p points exclude the neighborhood of q points. To calculate the formula, a dimensionality reduction method is adopted. The dimension reduction method converts the two-dimensional model of the formula into a one-dimensional model represented by the following formula twice to reduce the dimension:
Figure BDA0002720934160000067
to calculate
Figure BDA0002720934160000068
And
Figure BDA0002720934160000069
the parabolic lower bound method (see fig. 2) is used. From fig. 2 it can be seen that parabolas 0, 2, 4 form the lower boundary of the set of parabolas, with parabolas 1 and 3 being deleted. The lower bound of the parabola can be found by intersecting the parabolas.
To calculate
Figure BDA00027209341600000610
For example, the detailed calculation steps are as follows:
(1) two arrays Z and V are established, array Z being used to place the intersection of the two parabolas and V being used to place the center of the parabola that forms the lower bound of the parabola.
(2) Let Z [0 ]]＝-∞，Z[1]＝+∞，V[0]Setting the vernier k to 0; p is a radical of n ＝1。
(3) From V [ k ]]In the first parabola
Figure BDA0002720934160000071
Read in the next parabola
Figure BDA0002720934160000072
Calculating the intersection point S of the two parabolas, if S is more than or equal to Z [ k ]]Entering the next step; if S is<Z[k]And if k is equal to k-1, returning to the step (3) for calculation again.
(4) k is k +1 and p is n Store in V [ k ]]Store S in Z [ k ]]Is prepared from Z [ k +1 ]]＝+∞；p n ＝p n +1, return to step (3) until p n ＝L。
(5) Let the cursor k equal to 0;
Figure BDA0002720934160000073
(6) when in use
Figure BDA0002720934160000074
Computing information
Figure BDA0002720934160000075
If it is not
Figure BDA0002720934160000076
Finishing the calculation, otherwise, returning to the step (6) again to calculate the next value; when in use
Figure BDA0002720934160000077
If so, k is k +1, and the process returns to step (6).
After the information is calculated according to the formula, the information of each point is transmitted to four directions, namely, up, down, left and right, respectively, the updated information is returned after the information is transmitted, and then the updated information is transmitted, so that the information is not updated after the information is transmitted R times, and the information vector (confidence) of the obtained pixel point is as follows:
Figure BDA0002720934160000078
vector b q (o q ) O corresponding to the minimum confidence in q The optical flow values of the image pixels.
And S104, calculating the value of the optical flow increment field according to the initial optical flow value.
Using the formula i p ＝k 2 (f p L/2) determining the value of the optical flow delta field for a pixel p of an image; wherein, the i p Value of the optical flow increment field, f, for pixel point p p Is the initial optical flow value, k, of pixel point p 2 For incremental step length, L is the number of sampling points.
And S105, reconstructing the Markov variable optical flow model according to the value of the optical flow incremental field, and determining the generalized Markov variable optical flow model.
The generalized Markov variational optical flow model is:
Figure BDA0002720934160000079
wherein o is p And o q As initial optical flow values, i, of pixels p and q p And i q Respectively correspond to o p And o q Increment of (i) p ＝k 2 (f p -L/2)，fp＝1，2，…，L，f p To obtain a dense flow of light, let k be 2 ＜＜k 1 . It can be seen that if i p And i q Are all equal to 0, the markov variational optical flow model.
And S106, calculating a data item of the generalized Markov variational optical flow model according to the two frames of images.
Using formulas
Figure BDA0002720934160000081
Calculating the generalized MalkeData items of a Fround-variant optical flow model; wherein the content of the first and second substances,
Figure BDA0002720934160000082
is a data item of a generalized Markov variant optical flow model, tau is a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of dense optical flow vertical direction, I 1 And I 2 The two images are respectively a front frame image and a back frame image, u is a component of an optical flow in the horizontal direction, v is a component of the optical flow in the vertical direction,
Figure BDA0002720934160000083
is an optical flow vector.
As a specific embodiment, information of image pixels is calculated, and in the BP algorithm, the information transmitted between new image pixels is calculated as follows:
Figure BDA0002720934160000084
in order to calculate the information, the invention adopts a dimension reduction method to convert the two-dimensional model of the formula into a two-time one-dimensional model shown as the following formula to reduce the dimension:
Figure BDA0002720934160000085
to calculate
Figure BDA0002720934160000086
And
Figure BDA0002720934160000087
the determination was performed using a modified parabolic lower bound method (see fig. 3). By comparing fig. 2 and fig. 3, it is apparent that the shape, position and intersection of the parabola are changed due to the introduction of the incremental field in the generalized markov variational optical flow model.
To calculate
Figure BDA0002720934160000091
For example, the detailed calculation steps are as follows:
(1) two arrays Z and V are established, array Z being used to place the intersection of the two parabolas and V being used to place the center of the parabola that forms the lower bound of the parabola.
(2) Let Z [0 ]]＝-∞，Z[1]＝+∞，V[0]Setting the vernier k to 0; p is a radical of n ＝1；
(3) From V [ k ]]In the first parabola
Figure BDA0002720934160000092
Read in the next parabola
Figure BDA0002720934160000093
Calculating the intersection point S of the two parabolas, if S is more than or equal to Z [ k ]]Entering the next step; if S is<Z[k]If k is k-1, returning to the step (3) for calculation again;
(4) k is k +1 and p is n Store in V [ k ]]Store S in Z [ k ]]Let Z [ k +1 ]]＝+∞；p n ＝p n +1, return to step (3) until p n ＝L。
(5) Let the cursor k equal to 0;
Figure BDA0002720934160000094
(6) when in use
Figure BDA0002720934160000095
Time, calculate
Figure BDA0002720934160000096
Figure BDA0002720934160000097
If it is not
Figure BDA0002720934160000098
Finishing the calculation, otherwise, returning to the step (6) to calculate the next value again; when Z [ k ]]<
Figure BDA0002720934160000099
Then, k is k +1, and go back to (6)
S107, dense optical flow values are calculated according to data items of the generalized Markov variable optical flow model.
S107 specifically comprises the following steps:
and calculating the information to be transmitted by each image pixel by adopting a BP algorithm according to the data items of the generalized Markov variable optical flow model.
Using formulas
Figure BDA00027209341600000910
The information to be conveyed for each image pixel is calculated.
And iteratively updating the message of each pixel point in parallel according to the information to obtain the updated message.
And simultaneously transmitting the updated information to the upper direction, the lower direction, the left direction and the right direction of each image pixel point in parallel, returning the step of updating the information of each pixel point according to the information in parallel and in an iterative manner to obtain the updated information, and determining the confidence coefficient vector of the pixel point until N times of updating. The confidence vectors are:
Figure BDA00027209341600000911
a specific example is provided to further support the present invention. FIG. 4 is a schematic diagram illustrating an optical flow calculation evaluation comparison of a Yosmenite valley image, where, from the calculation result, an optical flow field obtained by a conventional Markov variational optical flow model is sparse, and an obvious unsmooth phenomenon exists in the optical flow calculation result; the optical flow field obtained by the generalized Markov variable optical flow model calculation is dense, the optical flow calculation result is smooth and is close to the real optical flow result, and the error is small.
Fig. 5 is a schematic structural diagram of a generalized markov dense optical flow determination system provided by the present invention, and as shown in fig. 5, the generalized markov dense optical flow determination system provided by the present invention includes: an image acquisition module 501, a data item computation module for a markov variant optical flow model 502, an initial optical flow value computation module 503, a value computation module for an optical flow incremental field 504, a generalized markov variant optical flow model determination module 505, a data item computation module for a generalized markov variant optical flow model 506, and a dense optical flow value computation module 507.
The image obtaining module 501 is configured to obtain any two consecutive frames of images.
The data item computation module 502 of the markov variational optical flow model is used for computing the data items of the markov variational optical flow model from the two frames of the image.
The initial optical flow value calculating module 503 is used to calculate the initial optical flow value between the two frames of images according to the data items of the markov variable fractional optical flow model.
The value calculation module 504 of the incremental optical flow field is configured to calculate the value of the incremental optical flow field from the initial optical flow values.
The generalized Markov variational optical flow model determining module 505 is configured to reconstruct a Markov variational optical flow model from the values of the optical flow incremental field and determine the generalized Markov variational optical flow model.
The data item computation module 506 of the generalized Markov variational optical flow model is used for computing the data item of the generalized Markov variational optical flow model according to the two frames of the images.
The dense optical flow value calculation module 507 is configured to calculate dense optical flow values according to data items of the generalized Markov variational optical flow model.
The value calculation module 504 of the optical flow incremental field specifically includes: a value calculation unit of the optical flow increment field.
The value calculation unit of the optical flow increment field is used for utilizing the formula i p ＝k 2 (f p L/2) determining the value of the optical flow delta field for a pixel p of an image; wherein, the i p Value of the optical flow delta field for pixel p, f p Is the initial optical flow value, k, of pixel point p 2 The increment step length is L, and the number of sampling points is L.
The data item calculating module 506 of the generalized markov variational optical flow model specifically includes: and a data item calculation unit of the generalized Markov variational optical flow model.
Broad sense of Ma Er KeThe data item calculation unit of the Fround variation optical flow model is used for utilizing a formula
Figure BDA0002720934160000111
Calculating data items of the generalized Markov variational optical flow model; wherein the content of the first and second substances,
Figure BDA0002720934160000112
for data items of a generalized Markov variational optical flow model, tau being a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of vertical direction of dense light flow, I 1 And I 2 The two images are respectively front and back frames, u is the component of the optical flow in the horizontal direction, v is the component of the optical flow in the vertical direction,
Figure BDA0002720934160000113
is an optical flow vector.
The dense optical flow value module 507 specifically includes: a passed information determining unit, an information updating unit, a confidence vector determining unit, and an optical flow value determining unit.
And the transmitted information determining unit is used for calculating the information to be transmitted of each image pixel by adopting a BP algorithm according to the data item of the generalized Markov variable spectral optical flow model.
And the information updating unit is used for iteratively updating the information of each pixel point in parallel according to the information to obtain the updated information.
And the confidence coefficient vector determining unit is used for simultaneously transmitting the updated information to the upper direction, the lower direction, the left direction and the right direction of each image pixel point in parallel, returning the step of iteratively updating the information of each pixel point in parallel according to the information to obtain the updated information, and determining the confidence coefficient vector of the pixel point until N times of updating.
And the optical flow value determining unit is used for determining the optical flow value of the pixel point according to the minimum confidence coefficient in the confidence coefficient vectors of the pixel point.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the description of the method part.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to help understand the method and the core concept of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the above, the present disclosure should not be construed as limiting the invention.

Claims (6)

1. A generalized markov dense optical flow determination method, comprising:
acquiring any two continuous frames of images;
calculating a data item of a Markov variational optical flow model from the two frames of images;
calculating an initial optical flow value between the two frames of images according to the data items of the Markov variable optical flow model;
calculating a value of an optical flow delta field from the initial optical flow value;
reconstructing a Markov variable optical flow model according to the value of the optical flow increment field, and determining a generalized Markov variable optical flow model;
calculating data items of a generalized Markov variational optical flow model according to the two frames of images;
calculating dense optical flow values according to data items of the generalized Markov variable optical flow model;
the calculating a value of an optical flow increment field according to the initial optical flow value specifically includes:
using the formula i p ＝k 2 (f p L/2) determining the value of the optical flow delta field for a pixel p of an image; wherein, the i p Value of the optical flow delta field for pixel p, f p Is the initial optical flow value, k, of pixel point p 2 For incremental step length, L is the number of sampling points.
2. The method of claim 1, wherein the computing the data items of the generalized markov variant optical flow model from the two frames of images comprises:
using formulas
Figure FDA0003680588800000011
Calculating data items of the generalized Markov variational optical flow model; wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003680588800000012
for data items of a generalized Markov variational optical flow model, tau being a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of dense optical flow vertical direction, I 1 And I 2 The two images are respectively front and back frames, u is the component of the optical flow in the horizontal direction, v is the component of the optical flow in the vertical direction,
Figure FDA0003680588800000013
is an optical flow vector.
3. The generalized markov dense optical flow determination method according to claim 2, wherein the calculating of the dense optical flow values from the data items of the generalized markov variational optical flow model specifically comprises:
calculating information to be transmitted by each image pixel by adopting a BP algorithm according to the data items of the generalized Markov variable optical flow model;
iteratively updating the message of each pixel point in parallel according to the information to obtain an updated message;
simultaneously transmitting the updated information to the upper direction, the lower direction, the left direction and the right direction of each image pixel point in parallel, returning the step of iteratively updating the information of each pixel point in parallel according to the information to obtain the updated information, and determining the confidence coefficient vector of the pixel point until N times of updating;
and determining the optical flow value of the pixel point according to the minimum confidence coefficient in the confidence coefficient vector of the pixel point.
4. A generalized markov dense optical flow determination system, comprising:
the image acquisition module is used for acquiring any two continuous frames of images;
the data item calculation module of the Markov variational optical flow model is used for calculating the data item of the Markov variational optical flow model according to the two frames of images;
an initial optical flow value calculation module, configured to calculate an initial optical flow value between the two frames of images according to the data items of the markov variational optical flow model;
a value calculation module of the optical flow increment field, which is used for calculating the value of the optical flow increment field according to the initial optical flow value;
the generalized Markov variational optical flow model determining module is used for reconstructing a Markov variational optical flow model according to the value of the optical flow incremental field and determining the generalized Markov variational optical flow model;
the data item calculation module of the generalized Markov variational optical flow model is used for calculating the data item of the generalized Markov variational optical flow model according to the two frames of images;
a dense optical flow value calculation module for calculating dense optical flow values from data items of the generalized Markov variational optical flow model;
the value calculation module of the optical flow incremental field specifically includes:
a value calculation unit of the optical flow increment field for utilizing the formula i p ＝k 2 (f p L/2) determining the value of the optical flow delta field for a pixel p of an image; wherein, the i p Value of the optical flow delta field for pixel p, f p Is the initial optical flow value, k, of pixel point p 2 For incremental step length, L is the number of sampling points.
5. The generalized Markov dense optical flow determination system of claim 4 wherein the data item computation module of the generalized Markov variational optical flow model specifically comprises:
data item calculation unit of generalized Markov variational optical flow model for utilizing formula
Figure FDA0003680588800000021
Calculating data items of the generalized Markov variational optical flow model; wherein, the first and the second end of the pipe are connected with each other,
Figure FDA0003680588800000031
is a data item of a generalized Markov variant optical flow model, tau is a truncation value, x p ,y p Is the coordinate of pixel point p, u d For the horizontal component of the dense light flow, v d Component of vertical direction of dense light flow, I 1 And I 2 The two images are respectively front and back frames, u is the component of the optical flow in the horizontal direction, v is the component of the optical flow in the vertical direction,
Figure FDA0003680588800000032
is an optical flow vector.
6. The generalized markov dense optical flow determination system of claim 5, wherein the dense optical flow values module comprises in particular:
the transmitted information determining unit is used for calculating the information to be transmitted of each image pixel by adopting a BP algorithm according to the data item of the generalized Markov variable optical flow model;
the information updating unit is used for iteratively updating the information of each pixel point in parallel according to the information to obtain updated information;
a confidence vector determining unit, configured to transmit the updated message in parallel to the upper, lower, left, and right directions of each image pixel point at the same time, and return to the step of iteratively updating the message of each pixel point in parallel according to the information to obtain an updated message, and determine a confidence vector of a pixel point until N times of updating;
and the optical flow value determining unit is used for determining the optical flow value of the pixel point according to the minimum confidence coefficient in the confidence coefficient vector of the pixel point.
CN202011087976.9A 2020-10-13 2020-10-13 Generalized Markov dense optical flow determination method and system Active CN112164092B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011087976.9A CN112164092B (en) 2020-10-13 2020-10-13 Generalized Markov dense optical flow determination method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011087976.9A CN112164092B (en) 2020-10-13 2020-10-13 Generalized Markov dense optical flow determination method and system

Publications (2)

Publication Number Publication Date
CN112164092A CN112164092A (en) 2021-01-01
CN112164092B true CN112164092B (en) 2022-09-27

Family

ID=73866586

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011087976.9A Active CN112164092B (en) 2020-10-13 2020-10-13 Generalized Markov dense optical flow determination method and system

Country Status (1)

Country Link
CN (1) CN112164092B (en)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663445A (en) * 2012-03-29 2012-09-12 中国科学院上海光学精密机械研究所 Image understanding system based on layered temporal memory algorithm and image understanding method thereof
CN102938153A (en) * 2012-11-21 2013-02-20 清华大学 Video image splitting method based on restrain spectral clustering and markov random field
CN104463191A (en) * 2014-10-30 2015-03-25 华南理工大学 Robot visual processing method based on attention mechanism
CN104835181A (en) * 2015-05-23 2015-08-12 温州大学 Object tracking method based on ordering fusion learning
CN105426820A (en) * 2015-11-03 2016-03-23 中原智慧城市设计研究院有限公司 Multi-person abnormal behavior detection method based on security monitoring video data
CN105635808A (en) * 2015-12-31 2016-06-01 电子科技大学 Video splicing method based on Bayesian theory
CN105869178A (en) * 2016-04-26 2016-08-17 昆明理工大学 Method for unsupervised segmentation of complex targets from dynamic scene based on multi-scale combination feature convex optimization
CN106611157A (en) * 2016-11-17 2017-05-03 中国石油大学(华东) Multi-people posture recognition method based on optical flow positioning and sliding window detection
CN107253192A (en) * 2017-05-24 2017-10-17 湖北众与和智能装备科技有限公司 It is a kind of based on Kinect without demarcation human-computer interactive control system and method
CN107464228A (en) * 2017-08-24 2017-12-12 深圳市唯特视科技有限公司 A kind of image analysis method based on adaptive regularization
CN108596965A (en) * 2018-03-16 2018-09-28 天津大学 A kind of light field image depth estimation method
CN108776971A (en) * 2018-06-04 2018-11-09 南昌航空大学 A kind of variation light stream based on layering nearest-neighbor determines method and system
CN108876808A (en) * 2018-06-06 2018-11-23 南昌航空大学 A kind of parallel markov variation light stream determines method and system

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8432434B2 (en) * 2011-07-08 2013-04-30 Mitsubishi Electric Research Laboratories, Inc. Camera and method for focus based depth reconstruction of dynamic scenes
US10514770B2 (en) * 2016-06-17 2019-12-24 Texas Instruments Incorporated Hidden Markov model-based gesture recognition with FMCW radar

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102663445A (en) * 2012-03-29 2012-09-12 中国科学院上海光学精密机械研究所 Image understanding system based on layered temporal memory algorithm and image understanding method thereof
CN102938153A (en) * 2012-11-21 2013-02-20 清华大学 Video image splitting method based on restrain spectral clustering and markov random field
CN104463191A (en) * 2014-10-30 2015-03-25 华南理工大学 Robot visual processing method based on attention mechanism
CN104835181A (en) * 2015-05-23 2015-08-12 温州大学 Object tracking method based on ordering fusion learning
CN105426820A (en) * 2015-11-03 2016-03-23 中原智慧城市设计研究院有限公司 Multi-person abnormal behavior detection method based on security monitoring video data
CN105635808A (en) * 2015-12-31 2016-06-01 电子科技大学 Video splicing method based on Bayesian theory
CN105869178A (en) * 2016-04-26 2016-08-17 昆明理工大学 Method for unsupervised segmentation of complex targets from dynamic scene based on multi-scale combination feature convex optimization
CN106611157A (en) * 2016-11-17 2017-05-03 中国石油大学(华东) Multi-people posture recognition method based on optical flow positioning and sliding window detection
CN107253192A (en) * 2017-05-24 2017-10-17 湖北众与和智能装备科技有限公司 It is a kind of based on Kinect without demarcation human-computer interactive control system and method
CN107464228A (en) * 2017-08-24 2017-12-12 深圳市唯特视科技有限公司 A kind of image analysis method based on adaptive regularization
CN108596965A (en) * 2018-03-16 2018-09-28 天津大学 A kind of light field image depth estimation method
CN108776971A (en) * 2018-06-04 2018-11-09 南昌航空大学 A kind of variation light stream based on layering nearest-neighbor determines method and system
CN108876808A (en) * 2018-06-06 2018-11-23 南昌航空大学 A kind of parallel markov variation light stream determines method and system

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
Jiang S F.Parallel computing of signed distance function in level set based on dimension reduction.《Journal of Image and Graphics》.2018,第174-181页. *
刘润邦等.基于视觉对比机制的红外弱小目标检测方法.《激光与红外》.2017,(第09期), *
张聪炫等.单目图像序列光流三维重建技术研究综述.《电子学报》.2016,(第12期), *
曾庆山等.基于光流共生矩阵的人群行为异常检测.《郑州大学学报(工学版)》.2017,(第03期), *
江少锋 等.并行马尔可夫随机场计算变分光流方法.《中国图象图形学报》.2018,第1852-1863页. *
郑思凡等.双粒度光流流形学习的刮刷总成摆杆摆幅检测.《华南理工大学学报(自然科学版)》.2020,(第01期), *
黎海雪等.基于马尔科夫随机场的图像超分辨技术研究综述.《通信技术》.2018,(第10期), *

Also Published As

Publication number Publication date
CN112164092A (en) 2021-01-01

Similar Documents

Publication Publication Date Title
CN110782490B (en) Video depth map estimation method and device with space-time consistency
WO2022033048A1 (en) Video frame interpolation method, model training method, and corresponding device
CN109800692B (en) Visual SLAM loop detection method based on pre-training convolutional neural network
CN111144463B (en) Hyperspectral image clustering method based on residual subspace clustering network
CN111145255B (en) Pose calculation method and system combining deep learning and geometric optimization
CN111640187B (en) Video stitching method and system based on interpolation transition
CN105427308A (en) Sparse and dense characteristic matching combined image registration method
CN112233249A (en) B spline surface fitting method and device based on dense point cloud
CN112270692B (en) Monocular video structure and motion prediction self-supervision method based on super-resolution
CN110473151B (en) Partition convolution and correlation loss based dual-stage image completion method and system
CN111879258A (en) Dynamic high-precision three-dimensional measurement method based on fringe image conversion network FPTNet
CN109299669B (en) Video face key point detection method and device based on double intelligent agents
CN112233179A (en) Visual odometer measuring method
Lewis et al. Scattered data interpolation and approximation for computer graphics
CN113838102B (en) Optical flow determining method and system based on anisotropic dense convolution
CN115690908A (en) Three-dimensional gesture attitude estimation method based on topology perception
CN114140601A (en) Three-dimensional grid reconstruction method and system based on single image under deep learning framework
CN112164092B (en) Generalized Markov dense optical flow determination method and system
CN110390336B (en) Method for improving feature point matching precision
CN116310349B (en) Large-scale point cloud segmentation method, device, equipment and medium based on deep learning
CN112396167A (en) Loop detection method for fusing appearance similarity and spatial position information
CN115222947B (en) Rock joint segmentation method and device based on global self-attention transformation network
CN115966004A (en) Human face key point detection method based on attention-directed lightweight network
CN114419347A (en) Spatial curve shape matching method based on deep learning
CN113593007A (en) Single-view three-dimensional point cloud reconstruction method and system based on variational self-coding

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant