CN110378936A - Optical flow computation method, apparatus and electronic equipment - Google Patents
Optical flow computation method, apparatus and electronic equipment Download PDFInfo
- Publication number
- CN110378936A CN110378936A CN201910692802.6A CN201910692802A CN110378936A CN 110378936 A CN110378936 A CN 110378936A CN 201910692802 A CN201910692802 A CN 201910692802A CN 110378936 A CN110378936 A CN 110378936A
- Authority
- CN
- China
- Prior art keywords
- prediction network
- image
- flow information
- target video
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
A kind of optical flow computation method, apparatus and electronic equipment are provided in the embodiment of the present disclosure, belong to data processing field, this method comprises: the first image collection that time interval in target video is [t-N, t+N] is input in the first prediction network, the first Optic flow information is obtained；Optical flow computation is carried out to the second image collection that time interval in target video is [t-M, t+M], obtains the second Optic flow information using the second prediction network serial with the first prediction network based on first Optic flow information；By adjusting the value of M, N and t, all video frames in the target video is made to carry out optical flow computation by the first prediction network and the second prediction network；Based on the second Optic flow information that the second prediction network obtains, the light stream value of the target video is determined.By the processing scheme of the disclosure, the Optic flow information of image can be accurately calculated.
Description
Technical field
This disclosure relates to technical field of data processing more particularly to a kind of optical flow computation method, apparatus and electronic equipment.
Background technique
Image procossing, which refers to, to be analyzed image, processed and is handled, it is made to meet vision, psychology or other requirements
Technology.Image procossing is an application of the signal processing on image domains.Most of image is in digital form at present
Storage, thus image procossing refers to Digital Image Processing in many cases.In addition, the processing method based on optical theory is still occupied
Consequence.
Image procossing is the subclass of signal processing, in addition also has close pass with fields such as computer science, artificial intelligence
System.The method and concept of traditional one-dimensional signal processing still can be much directly applied on image procossing, such as noise reduction, amount
Change etc..It is compared however, image belongs to 2D signal with one-dimensional signal, the one side that it has oneself special, the mode and angle of processing
Also different.
Light stream (Optical flow) is about the concept in the object of which movement detection in the ken.For describing relative to sight
The movement of observed object caused by the movement for the person of examining, surface or edge.Optical flow method template identify, computer vision and its
It is highly useful in his image processing field, it can be used for calculating, the fortune of motion detection, object cutting, collision time and object expansion
Dynamic compensation coding, or three-dimensional measurement etc. is carried out by body surface and edge.In actual optical flow computation, how
The accuracy for improving optical flow computation, becomes the technical issues that need to address.
Summary of the invention
In view of this, the embodiment of the present disclosure provides a kind of optical flow computation method, apparatus and electronic equipment, at least partly solve
Problems of the prior art.
In a first aspect, the embodiment of the present disclosure provides a kind of optical flow computation method, comprising:
The first image collection that time interval in target video is [t-N, t+N] is input in the first prediction network, is obtained
To the first Optic flow information, N is the numerical value less than t；
Based on first Optic flow information, using the second prediction network serial with the first prediction network, to target
Time interval is that second image collection of [t-M, t+M] carries out optical flow computation in video, obtains the second Optic flow information, and M is less than N
Numerical value；
By adjusting the value of M, N and t, make all video frames in the target video by the first prediction network
Optical flow computation is carried out with the second prediction network；
After the video frame carries out optical flow computation by the first prediction network and the second prediction network,
Based on the second Optic flow information that the second prediction network obtains, the light stream value of the target video is determined.
According to a kind of specific implementation of the embodiment of the present disclosure, second obtained based on the second prediction network
Optic flow information determines the light stream value of the target video, comprising:
Based on second Optic flow information, network is predicted using the third serial with the second prediction network, to target
Time interval is that the third image collection of [t-L, t+L] carries out optical flow computation in video, obtains third Optic flow information, and L is less than M
Value；
Based on the third Optic flow information, the light stream value of the target video is determined.
It is described to be predicted by described first in the video frame according to a kind of specific implementation of the embodiment of the present disclosure
After network and the second prediction network carry out optical flow computation, the second light stream obtained based on the second prediction network is believed
It ceases, after the light stream value for determining the target video, which comprises
The loss function different with the second prediction network settings to the first prediction network；
Based on the loss function, the first prediction network and the second prediction network are trained；
Using the first prediction network and the second prediction network after training, the Optic flow information of video to be predicted is counted
It calculates.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, obtain the first Optic flow information, comprising:
Image associated layers are set in the first prediction network；
Based on described image associated layers, the characteristics of image of the first image set is extracted；
By way of spatial convolution operation, the correlation of the characteristics of image for the first image set extracted is determined
Property；
The correlation of characteristics of image based on the first image set, it is determined whether calculate first Optic flow information.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, obtain the first Optic flow information, comprising:
It is arranged multiple deconvolution ReLU layers in the first prediction network；
For each deconvolution ReLU layers, while inputting the output of deconvolution ReLU layers of preceding layer, also input deconvolution
Characteristic layer in the light stream and respective modules of the low scale of ReLU layers of preceding layer prediction.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described be based on first Optic flow information, using with institute
The second serial prediction network of the first prediction network is stated, is second image set of [t-M, t+M] to time interval in target video
It closes and carries out optical flow computation, comprising:
Multiple convolutional layers are set in the second prediction network；
Based on the multiple convolutional layer, image characteristics extraction is carried out to second image collection；
Based on the feature of the second image collection extracted, the Optic flow information of second image collection is determined.
It is described to be based on the multiple convolutional layer according to a kind of specific implementation of the embodiment of the present disclosure, to described second
Image carries out image characteristics extraction, comprising:
Multiple convolutional layers are configured by concatenated mode；
Sample level is set among concatenated the multiple convolutional layer, the number of the number of the sample level than the convolutional layer
Mesh is one few；
The final result that the convolutional layer being successively serially arranged and sample level are calculated, as second image collection
Characteristics of image.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described be based on first Optic flow information, using with institute
The second serial prediction network of the first prediction network is stated, is second image set of [t-M, t+M] to time interval in target video
It closes and carries out optical flow computation, comprising:
The fisrt feature matrix for indicating first Optic flow information and second image collection and the second spy are obtained respectively
Levy matrix；
The fisrt feature matrix and the second characteristic matrix are normalized, third feature matrix is obtained；
Light stream using the third feature matrix as the input of the second prediction network, to second image collection
Information is predicted.
It is described using the third feature matrix as described second according to a kind of specific implementation of the embodiment of the present disclosure
The input for predicting network, predicts the Optic flow information of second image collection, comprising:
Using convolutional layer, batch normalization layer and ReLu layers being serially arranged in the second prediction network, to described the
Three eigenmatrixes are calculated, and optical flow computation result is obtained；
The second Optic flow information that the optical flow computation result is obtained as the second prediction neural network forecast.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, before obtaining the first Optic flow information, the method also includes:
Utilize formula v (out)=v (in)γImage rectification is executed to the image in the first image collection, wherein v (in) is
Image before correction, v (out) are the image after correction, correction coefficient of the γ between 0 and 1.
Second aspect, the embodiment of the present disclosure provide a kind of optical flow computation device, comprising:
First input module, for the first image collection that time interval in target video is [t-N, t+N] to be input to
In first prediction network, the first Optic flow information is obtained, N is the numerical value less than t；
Second input module utilizes serial with the first prediction network for being based on first Optic flow information
Two prediction networks carry out optical flow computation to the second image collection that time interval in target video is [t-M, t+M], obtain second
Optic flow information, M are the numerical value less than N；
Adjust module makes all video frames in the target video pass through institute for the value by adjusting M, N and t
It states the first prediction network and the second prediction network carries out optical flow computation；
Execution module, for being carried out by the first prediction network and the second prediction network in the video frame
After optical flow computation, based on the second Optic flow information that the second prediction network obtains, the light stream value of the target video is determined.
The third aspect, the embodiment of the present disclosure additionally provide a kind of electronic equipment, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out the light in any implementation of aforementioned first aspect or first aspect
Flow calculation methodologies.
Fourth aspect, the embodiment of the present disclosure additionally provide a kind of non-transient computer readable storage medium, the non-transient meter
Calculation machine readable storage medium storing program for executing stores computer instruction, and the computer instruction is for making the computer execute aforementioned first aspect or the
Optical flow computation method in any implementation of one side.
5th aspect, the embodiment of the present disclosure additionally provide a kind of computer program product, which includes
The calculation procedure being stored in non-transient computer readable storage medium, the computer program include program instruction, when the program
When instruction is computer-executed, the computer is made to execute the light stream in aforementioned first aspect or any implementation of first aspect
Calculation method.
Optical flow computation scheme in the embodiment of the present disclosure, including by time interval in target video is [t-N, t+N] the
One image collection is input in the first prediction network, obtains the first Optic flow information, and N is the numerical value less than t；
Based on first Optic flow information, using the second prediction network serial with the first prediction network, to target
Time interval is that second image collection of [t-M, t+M] carries out optical flow computation in video, obtains the second Optic flow information, and M is less than N
Numerical value；By adjusting the value of M, N and t, make all video frames in the target video by the first prediction network
Optical flow computation is carried out with the second prediction network；Network and described second is predicted in advance by described first in the video frame
After survey grid network carries out optical flow computation, based on the second Optic flow information that the second prediction network obtains, the target view is determined
The light stream value of frequency.By the scheme of the disclosure, the Optic flow information of image can be accurately calculated.
Detailed description of the invention
It, below will be to needed in the embodiment attached in order to illustrate more clearly of the technical solution of the embodiment of the present disclosure
Figure is briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present disclosure, for this field
For those of ordinary skill, without creative efforts, it can also be obtained according to these attached drawings other attached drawings.
Fig. 1 is a kind of optical flow computation flow diagram that the embodiment of the present disclosure provides；
Fig. 2 is a kind of optical flow computation configuration diagram that the embodiment of the present disclosure provides；
Fig. 3 is another optical flow computation flow diagram that the embodiment of the present disclosure provides；
Fig. 4 is another optical flow computation flow diagram that the embodiment of the present disclosure provides；
Fig. 5 is a kind of optical flow computation apparatus structure schematic diagram that the embodiment of the present disclosure provides；
Fig. 6 is the electronic equipment schematic diagram that the embodiment of the present disclosure provides.
Specific embodiment
The embodiment of the present disclosure is described in detail with reference to the accompanying drawing.
Illustrate embodiment of the present disclosure below by way of specific specific example, those skilled in the art can be by this specification
Disclosed content understands other advantages and effect of the disclosure easily.Obviously, described embodiment is only the disclosure
A part of the embodiment, instead of all the embodiments.The disclosure can also be subject to reality by way of a different and different embodiment
It applies or applies, the various details in this specification can also be based on different viewpoints and application, in the spirit without departing from the disclosure
Lower carry out various modifications or alterations.It should be noted that in the absence of conflict, the feature in following embodiment and embodiment can
To be combined with each other.Based on the embodiment in the disclosure, those of ordinary skill in the art are without creative efforts
Every other embodiment obtained belongs to the range of disclosure protection.
It should be noted that the various aspects of embodiment within the scope of the appended claims are described below.Ying Xian
And be clear to, aspect described herein can be embodied in extensive diversified forms, and any specific structure described herein
And/or function is only illustrative.Based on the disclosure, it will be understood by one of ordinary skill in the art that one described herein
Aspect can be independently implemented with any other aspect, and can combine the two or both in these aspects or more in various ways.
For example, carry out facilities and equipments in terms of any number set forth herein can be used and/or practice method.In addition, can make
With other than one or more of aspect set forth herein other structures and/or it is functional implement this equipment and/or
Practice the method.
It should also be noted that, diagram provided in following embodiment only illustrates the basic structure of the disclosure in a schematic way
Think, component count, shape and the size when only display is with component related in the disclosure rather than according to actual implementation in schema are drawn
System, when actual implementation kenel, quantity and the ratio of each component can arbitrarily change for one kind, and its assembly layout kenel can also
It can be increasingly complex.
In addition, in the following description, specific details are provided for a thorough understanding of the examples.However, fields
The skilled person will understand that the aspect can be practiced without these specific details.
The embodiment of the present disclosure provides a kind of optical flow computation method.Optical flow computation method provided in this embodiment can be by a meter
Device is calculated to execute, which can be implemented as software, or be embodied as the combination of software and hardware, which can
To be integrally disposed in server, terminal device etc..
Referring to Fig. 1, a kind of optical flow computation method that the embodiment of the present disclosure provides includes the following steps:
The first image collection that time interval in target video is [t-N, t+N] is input to the first prediction network by S101
In, the first Optic flow information is obtained, N is the numerical value less than t.
The information of target video record different objects in for some time, passes through object present in analysis target video
Mobile message can determine the Optic flow information for the object for including in target video.
First image collection is in the case that time interval is shorter, for the image of same or similar scene capture.Make
For a kind of situation, the first image collection can be multiple video frames adjacent in one section of video.Due in the first image collection
Image is in adjacent states on time dimension, it is possible thereby to calculate the light stream letter of the adjacent image in adjacent time interval
Breath.
The first prediction network can be set referring to fig. 2 in order to facilitate the Optic flow information calculated between the first image collection,
First prediction network can be a kind of neural network framework being arranged based on convolutional neural networks.For example, the first prediction network
It may include convolutional layer, pond layer, sample level.
Convolutional layer major parameter includes the size of convolution kernel and the quantity of input feature vector figure, if each convolutional layer may include
The characteristic pattern of dry same size, for same layer characteristic value by the way of shared weight, the convolution kernel in every layer is in the same size.Volume
Lamination carries out convolutional calculation to input picture, and extracts the spatial layout feature of input picture.
It can be connect with sample level behind the feature extraction layer of convolutional layer, sample level is used to ask the part of input picture flat
Mean value simultaneously carries out Further Feature Extraction, by the way that sample level to be connect with convolutional layer, neural network model can be guaranteed for input
Image has preferable robustness.
In order to accelerate the training speed of the first prediction network, pond layer is additionally provided with behind convolutional layer, pond layer uses
The mode in maximum pond handles the output result of convolutional layer, can preferably extract the Invariance feature of input picture.
It in addition to this, can also be in the described first pre- survey grid in order to be associated calculating for the first image collection
Image associated layers are arranged in network can extract the characteristics of image of the first image set, pass through space by image associated layers
The mode of convolution algorithm determines the correlation of the characteristics of image of the first image and second image extracted, so as to
The correlation of characteristics of image in based on the first image collection, it is determined whether calculate first Optic flow information.
Alternatively situation can also be arranged multiple deconvolution ReLU layers in the first prediction network, for
Each deconvolution ReLU layers, while inputting the output of deconvolution ReLU layers of preceding layer, also input deconvolution ReLU layers of preceding layer
Characteristic layer in the light stream and respective modules of the low scale of prediction, to ensure that each layer of warp lamination in refinement, can obtain
The abstracted information of deep layer and the tool image information of shallow-layer, to make up the information lost by the diminution of feature space scale.
It can be first image set of [t-N, t+N] by time interval in target video in the video frame of selection input
Conjunction is input in the first prediction network, obtains the first Optic flow information, wherein N is the numerical value less than t.
S102 is based on first Optic flow information, right using the second prediction network serial with the first prediction network
Time interval is that second image collection of [t-M, t+M] carries out optical flow computation in target video, obtains the second Optic flow information, M is
Numerical value less than N.
In order to further improve the accuracy of optical flow computation, referring to fig. 2, can also be arranged serial with the first prediction network
Second prediction network.Second prediction network can be a kind of neural network framework being arranged based on convolutional neural networks.Example
Such as, the second prediction network may include convolutional layer, pond layer, sample level.
Convolutional layer major parameter includes the size of convolution kernel and the quantity of input feature vector figure, if each convolutional layer may include
The characteristic pattern of dry same size, for same layer characteristic value by the way of shared weight, the convolution kernel in every layer is in the same size.Volume
Lamination carries out convolutional calculation to input picture, and extracts the spatial layout feature of input picture.
It can be connect with sample level behind the feature extraction layer of convolutional layer, sample level is used to ask the part of input picture flat
Mean value simultaneously carries out Further Feature Extraction, by the way that sample level to be connect with convolutional layer, neural network model can be guaranteed for input
Image has preferable robustness.
In order to accelerate the training speed of the second prediction network, pond layer is additionally provided with behind convolutional layer, pond layer uses
The mode in maximum pond handles the output result of convolutional layer, can preferably extract the Invariance feature of input picture.
Using the second prediction network serial with the first prediction network, characteristics of image is carried out to second image and is mentioned
During taking, multiple convolutional layers can be set in the second prediction network, by multiple convolutional layers, to second figure
Image in image set conjunction carries out image characteristics extraction.
Specifically, being based on multiple convolutional layers, image characteristics extraction is carried out to the image in second image collection, it can be with
It is configured using by multiple convolutional layers by concatenated mode, while sampling is set among concatenated the multiple convolutional layer
Layer, the number of the sample level one fewer than the number of the convolutional layer.Finally by the convolutional layer being successively serially arranged and sampling
The final result that layer is calculated, the characteristics of image as second image collection.
S103 makes all video frames in the target video pass through described first pre- by adjusting the value of M, N and t
Survey grid network and the second prediction network carry out optical flow computation.
By reading the total duration of target video, the value of M, N and t can be determined based on total duration, by adjusting M, N and
The value of t can make video frame all in target video all pass through the first prediction network and the second prediction network progress light stream meter
It calculates, so that the optical flow computation value of video frame is more accurate in target video.
When being directed to the value of the first prediction network and second prediction network selection M, N and t, the first prediction network and second is in advance
The value of t may be the same or different in survey grid network.
S104 carries out optical flow computation by the first prediction network and the second prediction network in the video frame
Later, the second Optic flow information obtained based on the second prediction network, determines the light stream value of the target video.
After obtaining the second Optic flow information, the second Optic flow information can be directly based upon to determine the light stream value of target video,
Second Optic flow information can also be processed again, by the second Optic flow information after processing, to determine the light of target video
Flow valuve.
Alternatively situation, can also be based on the second Optic flow information come to the first prediction network and the second prediction network
It is trained, by the way that loss function can be set during training, by loss function to the first prediction network and second
The accuracy for the second Optic flow information that prediction network query function obtains is judged.So, it is calculated by successive ignition training,
When the accuracy of second Optic flow information is met the requirements, the training to the first prediction network and the second prediction network is completed.
After the first prediction network and the second prediction network training are completed, the first prediction trained and completed can be utilized
Network and the second prediction network, carry out Optic flow information prediction to the video frame in target video frame.
By the scheme of the disclosure, multiple prediction networks can be utilized, different numbers is set in different prediction networks
The accuracy of light stream prediction is improved by being equipped with for multiple prediction networks according to processing task.
According to a kind of specific implementation of the embodiment of the present disclosure, referring to figs. 2 and 3, based on the second prediction network
The second obtained Optic flow information during the light stream value for determining the target video, can also include the following steps:
S301 is based on second Optic flow information, predicts network using the third serial with the second prediction network, right
Time interval is that the third image collection of [t-L, t+L] carries out optical flow computation in target video, obtains third Optic flow information,
In, L is the value less than M.
Third prediction network can be a kind of neural network framework being arranged based on convolutional neural networks.For example, second
Predict that network may include convolutional layer, pond layer, sample level.
Convolutional layer major parameter includes the size of convolution kernel and the quantity of input feature vector figure, if each convolutional layer may include
The characteristic pattern of dry same size, for same layer characteristic value by the way of shared weight, the convolution kernel in every layer is in the same size.Volume
Lamination carries out convolutional calculation to input picture, and extracts the spatial layout feature of input picture.
It can be connect with sample level behind the feature extraction layer of convolutional layer, sample level is used to ask the part of input picture flat
Mean value simultaneously carries out Further Feature Extraction, by the way that sample level to be connect with convolutional layer, neural network model can be guaranteed for input
Image has preferable robustness.
In order to accelerate the training speed of the second prediction network, pond layer is additionally provided with behind convolutional layer, pond layer uses
The mode in maximum pond handles the output result of convolutional layer, can preferably extract the Invariance feature of input picture.
After third predicts that network settings are completed, it can be based on second Optic flow information, predicted using with described second
The serial third of network predicts network, carries out light stream to the third image collection that time interval in target video is [t-L, t+L]
It calculates, obtains third Optic flow information, wherein L is the value less than M.
S302 is based on the third Optic flow information, determines the light stream value of the target video.
After obtaining third Optic flow information, third Optic flow information can be directly based upon to determine the light stream value of target video,
Third Optic flow information can also be processed again, by the third Optic flow information after processing, to determine the light of target video
Flow valuve.
Alternatively situation, can also be based on third Optic flow information come to the first prediction network, the second prediction network
It is trained with third prediction network, by the way that loss function can be set during training, by loss function to first
The accuracy for the third Optic flow information that prediction network, the second prediction network and third prediction network query function obtain is judged.This
Sample one is calculated by successive ignition training, when the accuracy of the third Optic flow information is met the requirements, is completed pre- to first
The training of survey grid network, the second prediction network and third prediction network.
After first predicts that network, the second prediction network and third prediction network training are completed, training can be utilized
The first prediction network, the second prediction network and the third completed predict network, carry out light stream to the video frame in target video frame
Information prediction.
Referring to fig. 4, according to a kind of specific implementation of the embodiment of the present disclosure, it is described pass through in the video frame it is described
First prediction network and it is described second prediction network carry out optical flow computation after, based on it is described second prediction network obtain second
Optic flow information, after the light stream value for determining the target video, which comprises
S401, the loss function different with the second prediction network settings to the first prediction network；
S402 is based on the loss function, is trained to the first prediction network and the second prediction network；
S403, using the first prediction network and the second prediction network after training, to the Optic flow information of video to be predicted into
Row calculates.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, obtain the first Optic flow information, may include steps of:
Firstly, image associated layers are arranged in the first prediction network.
By the way that image associated layers are arranged, the incidence relation between image in the first image collection can be calculated.Make
For an example, image associated layers can be completed by way of similarity calculation function is arranged.
Next, being based on described image associated layers, the characteristics of image of the first image set is extracted.
It can be first to the image between the first image collection before the calculating of memory relevance by image associated layers
Feature extracts, for example, the mode of convolutional layer can be set in image associated layers, it is specific by being arranged in convolutional layer
Convolution kernel, to extract the characteristics of image of the first image collection.
Next, determining the characteristics of image for the first image set extracted by way of spatial convolution operation
Correlation.
The characteristics of image of first image collection can be described by way of eigenmatrix, at this time, it is only necessary to be calculated
Relevance between the corresponding eigenmatrix of first image collection, can obtain the correlation in the first image collection between image
Property.
Finally, the correlation of the characteristics of image based on the first image set, it is determined whether calculate first light stream
Information.
After getting the correlation between the first image collection, correlation can be normalized, by sentencing
Whether the correlation after disconnected normalization is greater than preset value, further to calculate the first Optic flow information.For example, when normalization
When correlation afterwards is greater than preset value, the calculating of the first Optic flow information is carried out, correlation after normalization is no more than default
When value, then the calculating without the first Optic flow information.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, obtain the first Optic flow information, comprising: it is described first prediction network
Multiple deconvolution ReLU layers of middle setting；For each deconvolution ReLU layers, the same of the output of deconvolution ReLU layers of preceding layer is inputted
When, the characteristic layer in the light stream and respective modules of the low scale of deconvolution ReLU layers of preceding layer prediction is also inputted, it is each to ensure
Layer warp lamination can obtain the abstracted information of deep layer and the tool image information of shallow-layer, in refinement to make up because of feature space ruler
The diminution of degree and the information lost.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described be based on first Optic flow information, using with institute
The second serial prediction network of the first prediction network is stated, is second image set of [t-M, t+M] to time interval in target video
It closes and carries out optical flow computation, comprising: multiple convolutional layers are set in the second prediction network；It is right based on the multiple convolutional layer
Second image collection carries out image characteristics extraction；Based on the feature of the second image collection extracted, described second is determined
The Optic flow information of image collection.
It is described to be based on the multiple convolutional layer according to a kind of specific implementation of the embodiment of the present disclosure, to described second
Image carries out image characteristics extraction, comprising: is configured multiple convolutional layers by concatenated mode；Concatenated the multiple
Setting sample level among convolutional layer, the number of the sample level one fewer than the number of the convolutional layer；It will successively serial setting
Convolutional layer and the final result that is calculated of sample level, the characteristics of image as second image collection.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described be based on first Optic flow information, using with institute
The second serial prediction network of the first prediction network is stated, is second image set of [t-M, t+M] to time interval in target video
It closes and carries out optical flow computation, comprising: obtain the fisrt feature for indicating first Optic flow information and second image collection respectively
Matrix and second characteristic matrix；The fisrt feature matrix and the second characteristic matrix are normalized, obtain
Three eigenmatrixes；Using the third feature matrix as the input of the second prediction network, to second image collection
Optic flow information is predicted.
It is described using the third feature matrix as described second according to a kind of specific implementation of the embodiment of the present disclosure
The input for predicting network, predicts the Optic flow information of second image collection, comprising: utilizes the second prediction network
In the convolutional layer, batch normalization layer and ReLu layers that are serially arranged, the third feature matrix is calculated, light stream meter is obtained
Calculate result；The second Optic flow information that the optical flow computation result is obtained as the second prediction neural network forecast.
According to a kind of specific implementation of the embodiment of the present disclosure, it is described by time interval in target video be [t-N, t+
N] the first image collection be input in the first prediction network, before obtaining the first Optic flow information, the method also includes: utilize
Formula v (out)=v (in)γImage rectification is executed to the image in the first image collection, before wherein v (in) is correction
Image, v (out) are the image after correction, correction coefficient of the γ between 0 and 1.
Corresponding with above method embodiment, referring to Fig. 5, the embodiment of the present disclosure additionally provides a kind of optical flow computation device
50, comprising:
First input module 501, for inputting the first image collection that time interval in target video is [t-N, t+N]
Into the first prediction network, the first Optic flow information is obtained, N is the numerical value less than t；
Second input module 502, for being based on first Optic flow information, using serial with the first prediction network
Second prediction network carries out optical flow computation to the second image collection that time interval in target video is [t-M, t+M], obtains the
Two Optic flow informations, M are the numerical value less than N；
Adjusting module 503 passes through all video frames in the target video for the value by adjusting M, N and t
The first prediction network and the second prediction network carry out optical flow computation；
Execution module 504, for predicting network by the first prediction network and described second in the video frame
After carrying out optical flow computation, based on the second Optic flow information that the second prediction network obtains, the light of the target video is determined
Flow valuve.
Fig. 5 shown device can it is corresponding execute above method embodiment in content, what the present embodiment was not described in detail
Part, referring to the content recorded in above method embodiment, details are not described herein.
Referring to Fig. 6, the embodiment of the present disclosure additionally provides a kind of electronic equipment 60, which includes:
At least one processor；And
The memory being connect at least one processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one processor, and the instruction is by least one processor
It executes, so that at least one processor is able to carry out optical flow computation method in preceding method embodiment.
The embodiment of the present disclosure additionally provides a kind of non-transient computer readable storage medium, and the non-transient computer is readable to deposit
Storage media stores computer instruction, and the computer instruction is for executing the computer in preceding method embodiment.
The embodiment of the present disclosure additionally provides a kind of computer program product, and the computer program product is non-temporary including being stored in
Calculation procedure on state computer readable storage medium, the computer program include program instruction, when the program instruction is calculated
When machine executes, the computer is made to execute the optical flow computation method in preceding method embodiment.
Below with reference to Fig. 6, it illustrates the structural schematic diagrams for the electronic equipment 60 for being suitable for being used to realize the embodiment of the present disclosure.
Electronic equipment in the embodiment of the present disclosure can include but is not limited to such as mobile phone, laptop, Digital Broadcasting Receiver
Device, PDA (personal digital assistant), PAD (tablet computer), PMP (portable media player), car-mounted terminal are (such as vehicle-mounted
Navigation terminal) etc. mobile terminal and such as number TV, desktop computer etc. fixed terminal.Electronics shown in Fig. 6
Equipment is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in fig. 6, electronic equipment 60 may include processing unit (such as central processing unit, graphics processor etc.) 601,
It can be loaded into random access storage according to the program being stored in read-only memory (ROM) 602 or from storage device 608
Program in device (RAM) 603 and execute various movements appropriate and processing.In RAM 603, it is also stored with the behaviour of electronic equipment 60
Various programs and data needed for making.Processing unit 601, ROM 602 and RAM 603 are connected with each other by bus 604.It is defeated
Enter/export (I/O) interface 605 and is also connected to bus 604.
In general, following device can connect to I/O interface 605: including such as touch screen, touch tablet, keyboard, mouse, figure
As the input unit 606 of sensor, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaking
The output device 607 of device, vibrator etc.；Storage device 608 including such as tape, hard disk etc.；And communication device 609.It is logical
T unit 609 can permit electronic equipment 60 and wirelessly or non-wirelessly be communicated with other equipment to exchange data.Although showing in figure
The electronic equipment 60 with various devices is gone out, it should be understood that being not required for implementing or having all devices shown.
It can alternatively implement or have more or fewer devices.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communication device 609, or from storage device 608
It is mounted, or is mounted from ROM 602.When the computer program is executed by processing unit 601, the embodiment of the present disclosure is executed
Method in the above-mentioned function that limits.
It should be noted that the above-mentioned computer-readable medium of the disclosure can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In open, computer-readable signal media may include in a base band or as the data-signal that carrier wave a part is propagated,
In carry computer-readable program code.The data-signal of this propagation can take various forms, including but not limited to
Electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable and deposit
Any computer-readable medium other than storage media, the computer-readable signal media can send, propagate or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF (radio frequency) etc. are above-mentioned
Any appropriate combination.
Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not
It is fitted into the electronic equipment.
Above-mentioned computer-readable medium carries one or more program, when said one or multiple programs are by the electricity
When sub- equipment executes, so that the electronic equipment: obtaining at least two internet protocol addresses；Send to Node evaluation equipment includes institute
State the Node evaluation request of at least two internet protocol addresses, wherein the Node evaluation equipment is internet from described at least two
In protocol address, chooses internet protocol address and return；Receive the internet protocol address that the Node evaluation equipment returns；Its
In, the fringe node in acquired internet protocol address instruction content distributing network.
Alternatively, above-mentioned computer-readable medium carries one or more program, when said one or multiple programs
When being executed by the electronic equipment, so that the electronic equipment: receiving the Node evaluation including at least two internet protocol addresses and request；
From at least two internet protocol address, internet protocol address is chosen；Return to the internet protocol address selected；Wherein,
The fringe node in internet protocol address instruction content distributing network received.
The calculating of the operation for executing the disclosure can be write with one or more programming languages or combinations thereof
Machine program code, above procedure design language include object oriented program language-such as Java, Smalltalk, C+
+, it further include conventional procedural programming language-such as " C " language or similar programming language.Program code can
Fully to execute, partly execute on the user computer on the user computer, be executed as an independent software package,
Part executes on the remote computer or executes on a remote computer or server completely on the user computer for part.
In situations involving remote computers, remote computer can pass through the network of any kind --- including local area network (LAN)
Or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
Being described in unit involved in the embodiment of the present disclosure can be realized by way of software, can also be by hard
The mode of part is realized.Wherein, the title of unit does not constitute the restriction to the unit itself under certain conditions, for example, the
One acquiring unit is also described as " obtaining the unit of at least two internet protocol addresses ".
It should be appreciated that each section of the disclosure can be realized with hardware, software, firmware or their combination.
The above, the only specific embodiment of the disclosure, but the protection scope of the disclosure is not limited thereto, it is any
Those familiar with the art is in the technical scope that the disclosure discloses, and any changes or substitutions that can be easily thought of, all answers
Cover within the protection scope of the disclosure.Therefore, the protection scope of the disclosure should be subject to the protection scope in claims.
Claims (13)
1. a kind of optical flow computation method characterized by comprising
The first image collection that time interval in target video is [t-N, t+N] is input in the first prediction network, obtains the
One Optic flow information, N are the numerical value less than t；
Based on first Optic flow information, using the second prediction network serial with the first prediction network, to target video
Middle time interval is that second image collection of [t-M, t+M] carries out optical flow computation, obtains the second Optic flow information, and M is the number less than N
Value；
By adjusting the value of M, N and t, make all video frames in the target video by the first prediction network and institute
It states the second prediction network and carries out optical flow computation；
After the video frame carries out optical flow computation by the first prediction network and the second prediction network, it is based on
The second Optic flow information that the second prediction network obtains, determines the light stream value of the target video.
2. the method according to claim 1, wherein second light obtained based on the second prediction network
Stream information determines the light stream value of the target video, comprising:
Based on second Optic flow information, network is predicted using the third serial with the second prediction network, to target video
Middle time interval is that the third image collection of [t-L, t+L] carries out optical flow computation, obtains third Optic flow information, and L is less than M's
Value；
Based on the third Optic flow information, the light stream value of the target video is determined.
3. the method according to claim 1, wherein described pass through the described first pre- survey grid in the video frame
After network and the second prediction network carry out optical flow computation, based on obtained second Optic flow information of the second prediction network,
After the light stream value for determining the target video, which comprises
The loss function different with the second prediction network settings to the first prediction network；
Based on the loss function, the first prediction network and the second prediction network are trained；
Using the first prediction network and the second prediction network after training, the Optic flow information of video to be predicted is calculated.
4. the method according to claim 1, wherein it is described by time interval in target video be [t-N, t+N]
The first image collection be input in the first prediction network, obtain the first Optic flow information, comprising:
Image associated layers are set in the first prediction network；
Based on described image associated layers, the characteristics of image of the first image set is extracted；
By way of spatial convolution operation, the correlation of the characteristics of image for the first image set extracted is determined；
The correlation of characteristics of image based on the first image set, it is determined whether calculate first Optic flow information.
5. the method according to claim 1, wherein it is described by time interval in target video be [t-N, t+N]
The first image collection be input in the first prediction network, obtain the first Optic flow information, comprising:
It is arranged multiple deconvolution ReLU layers in the first prediction network；
For each deconvolution ReLU layers, while inputting the output of deconvolution ReLU layers of preceding layer, also input deconvolution ReLU
Characteristic layer in the light stream and respective modules of the low scale of layer preceding layer prediction.
6. the method according to claim 1, wherein it is described be based on first Optic flow information, using with it is described
The second serial prediction network of first prediction network, is second image collection of [t-M, t+M] to time interval in target video
Carry out optical flow computation, comprising:
Multiple convolutional layers are set in the second prediction network；
Based on the multiple convolutional layer, image characteristics extraction is carried out to second image collection；
Based on the feature of the second image collection extracted, the Optic flow information of second image collection is determined.
7. according to the method described in claim 6, it is characterized in that, described be based on the multiple convolutional layer, to second figure
As carrying out image characteristics extraction, comprising:
Multiple convolutional layers are configured by concatenated mode；
Sample level is set among concatenated the multiple convolutional layer, and the number of the sample level is fewer than the number of the convolutional layer
One；
The final result that the convolutional layer being successively serially arranged and sample level are calculated, the figure as second image collection
As feature.
8. the method according to claim 1, wherein it is described be based on first Optic flow information, using with it is described
The second serial prediction network of first prediction network, is second image collection of [t-M, t+M] to time interval in target video
Carry out optical flow computation, comprising:
The fisrt feature matrix and second feature square for indicating first Optic flow information and second image collection are obtained respectively
Battle array；
The fisrt feature matrix and the second characteristic matrix are normalized, third feature matrix is obtained；
Using the third feature matrix as the input of the second prediction network, to the Optic flow information of second image collection
It is predicted.
9. according to the method described in claim 8, it is characterized in that, described pre- using the third feature matrix as described second
The Optic flow information of second image collection is predicted in the input of survey grid network, comprising:
Using convolutional layer, batch normalization layer and ReLu layer being the serially arranged in the second prediction network, to the third spy
Sign matrix is calculated, and optical flow computation result is obtained；
The second Optic flow information that the optical flow computation result is obtained as the second prediction neural network forecast.
10. the method according to claim 1, wherein it is described by time interval in target video be [t-N, t+N]
The first image collection be input in the first prediction network, before obtaining the first Optic flow information, the method also includes:
Utilize formula v (out)=v (in)γImage rectification is executed to the image in the first image collection, wherein v (in) is correction
Image before, v (out) are the image after correction, correction coefficient of the γ between 0 and 1.
11. a kind of optical flow computation device characterized by comprising
First input module, for the first image collection that time interval in target video is [t-N, t+N] to be input to first
It predicts in network, obtain the first Optic flow information, N is the numerical value less than t；
Second input module, for being based on first Optic flow information, in advance using serial with the first prediction network second
Survey grid network carries out optical flow computation to the second image collection that time interval in target video is [t-M, t+M], obtains the second light stream
Information, M are the numerical value less than N；
Module is adjusted, for the value by adjusting M, N and t, all video frames in the target video is made to pass through described the
One prediction network and the second prediction network carry out optical flow computation；
Execution module, for carrying out light stream by the first prediction network and the second prediction network in the video frame
After calculating, based on the second Optic flow information that the second prediction network obtains, the light stream value of the target video is determined.
12. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
At least one processor；And
The memory being connect at least one described processor communication；Wherein,
The memory is stored with the instruction that can be executed by least one described processor, and described instruction is by described at least one
It manages device to execute, so that at least one described processor is able to carry out optical flow computation side described in aforementioned any claim 1-10
Method.
13. a kind of non-transient computer readable storage medium, which stores computer instruction,
The computer instruction is for making the computer execute optical flow computation method described in aforementioned any claim 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910692802.6A CN110378936B (en) | 2019-07-30 | 2019-07-30 | Optical flow calculation method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910692802.6A CN110378936B (en) | 2019-07-30 | 2019-07-30 | Optical flow calculation method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378936A true CN110378936A (en) | 2019-10-25 |
| CN110378936B CN110378936B (en) | 2021-11-05 |
Family
ID=68256937
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910692802.6A Active CN110378936B (en) | 2019-07-30 | 2019-07-30 | Optical flow calculation method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378936B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178277A (en) * | 2019-12-31 | 2020-05-19 | 支付宝实验室(新加坡)有限公司 | Video stream identification method and device |
| CN112232283A (en) * | 2020-11-05 | 2021-01-15 | 深兰科技（上海）有限公司 | Bubble detection method and system based on optical flow and C3D network |
Citations (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101770568A (en) * | 2008-12-31 | 2010-07-07 | 南京理工大学 | Target automatically recognizing and tracking method based on affine invariant point and optical flow calculation |
| CN102609688A (en) * | 2012-02-02 | 2012-07-25 | 杭州电子科技大学 | Multidirectional moving population flow estimation method on basis of generalized regression neural network |
| CN104021525A (en) * | 2014-05-30 | 2014-09-03 | 西安交通大学 | Background repairing method of road scene video image sequence |
| CN104636751A (en) * | 2014-12-11 | 2015-05-20 | 广东工业大学 | Crowd abnormity detection and positioning system and method based on time recurrent neural network |
| CN104807611A (en) * | 2015-05-04 | 2015-07-29 | 中国科学技术大学 | Flue gas velocity field and turbulence field experimental measurement device and method based on video |
| CN105025271A (en) * | 2015-07-28 | 2015-11-04 | 深圳英飞拓科技股份有限公司 | Behavior monitoring method and device |
| CN105426883A (en) * | 2015-12-25 | 2016-03-23 | 中国科学院深圳先进技术研究院 | Video classified rapid identification method and device |
| CN105447886A (en) * | 2014-08-26 | 2016-03-30 | 上海云罡同汇视觉艺术设计有限公司 | Dynamic cinema playback control method |
| CN105956618A (en) * | 2016-04-27 | 2016-09-21 | 云南昆钢集团电子信息工程有限公司 | Converter steelmaking blowing state recognition system and method based on image dynamic and static characteristics |
| CN106548153A (en) * | 2016-10-27 | 2017-03-29 | 杭州电子科技大学 | Video abnormality detection method based on graph structure under multi-scale transform |
| CN106658023A (en) * | 2016-12-21 | 2017-05-10 | 山东大学 | End-to-end visual odometer and method based on deep learning |
| US20170193310A1 (en) * | 2015-12-31 | 2017-07-06 | Pinhole (Beijing) Technology Co., Ltd. | Method and apparatus for detecting a speed of an object |
| US20170249590A1 (en) * | 2015-11-03 | 2017-08-31 | Bullet Scanning, Llc | System and method for inventory identification and quantification |
| CN107292912A (en) * | 2017-05-26 | 2017-10-24 | 浙江大学 | A kind of light stream method of estimation practised based on multiple dimensioned counter structure chemistry |
| CN108197566A (en) * | 2017-12-29 | 2018-06-22 | 成都三零凯天通信实业有限公司 | Monitoring video behavior detection method based on multi-path neural network |
| CN108229338A (en) * | 2017-12-14 | 2018-06-29 | 华南理工大学 | A kind of video behavior recognition methods based on depth convolution feature |
| US20180205922A1 (en) * | 2012-11-29 | 2018-07-19 | University Of Maryland, College Park | Techniques to extract enf signals from video image sequences exploiting the rolling shutter mechanism; and a new video synchronization approach by matching the enf signals extracted from soundtracks and image sequences |
| CN108519631A (en) * | 2018-02-22 | 2018-09-11 | 青岛心中有数科技有限公司 | Precipitation intensity prediction technique |
| CN108596157A (en) * | 2018-05-14 | 2018-09-28 | 三峡大学 | A kind of crowd's agitation scene detection method and system based on motion detection |
| CN108681695A (en) * | 2018-04-26 | 2018-10-19 | 北京市商汤科技开发有限公司 | Video actions recognition methods and device, electronic equipment and storage medium |
| CN109389072A (en) * | 2018-09-29 | 2019-02-26 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN109414145A (en) * | 2018-06-28 | 2019-03-01 | 广州艾若博机器人科技有限公司 | Speed-measuring method, skidding detection method based on light stream sensor, mobile electronic equipment, path correction method and device |
| US20190079536A1 (en) * | 2017-09-13 | 2019-03-14 | TuSimple | Training and testing of a neural network system for deep odometry assisted by static scene optical flow |
| CN109711316A (en) * | 2018-12-21 | 2019-05-03 | 广东工业大学 | A kind of pedestrian recognition methods, device, equipment and storage medium again |
| CN109740499A (en) * | 2018-12-28 | 2019-05-10 | 北京旷视科技有限公司 | Methods of video segmentation, video actions recognition methods, device, equipment and medium |
| CN109819321A (en) * | 2019-03-13 | 2019-05-28 | 中国科学技术大学 | A kind of video super-resolution Enhancement Method |
| CN110046568A (en) * | 2019-04-11 | 2019-07-23 | 中山大学 | A kind of video actions recognition methods based on Time Perception structure |
| US20190228264A1 (en) * | 2017-03-08 | 2019-07-25 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for training neural network model used for image processing, and storage medium |
-
2019
- 2019-07-30 CN CN201910692802.6A patent/CN110378936B/en active Active
Patent Citations (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101770568A (en) * | 2008-12-31 | 2010-07-07 | 南京理工大学 | Target automatically recognizing and tracking method based on affine invariant point and optical flow calculation |
| CN102609688A (en) * | 2012-02-02 | 2012-07-25 | 杭州电子科技大学 | Multidirectional moving population flow estimation method on basis of generalized regression neural network |
| US20180205922A1 (en) * | 2012-11-29 | 2018-07-19 | University Of Maryland, College Park | Techniques to extract enf signals from video image sequences exploiting the rolling shutter mechanism; and a new video synchronization approach by matching the enf signals extracted from soundtracks and image sequences |
| CN104021525A (en) * | 2014-05-30 | 2014-09-03 | 西安交通大学 | Background repairing method of road scene video image sequence |
| CN105447886A (en) * | 2014-08-26 | 2016-03-30 | 上海云罡同汇视觉艺术设计有限公司 | Dynamic cinema playback control method |
| CN104636751A (en) * | 2014-12-11 | 2015-05-20 | 广东工业大学 | Crowd abnormity detection and positioning system and method based on time recurrent neural network |
| CN104807611A (en) * | 2015-05-04 | 2015-07-29 | 中国科学技术大学 | Flue gas velocity field and turbulence field experimental measurement device and method based on video |
| CN105025271A (en) * | 2015-07-28 | 2015-11-04 | 深圳英飞拓科技股份有限公司 | Behavior monitoring method and device |
| US20170249590A1 (en) * | 2015-11-03 | 2017-08-31 | Bullet Scanning, Llc | System and method for inventory identification and quantification |
| US20180357600A1 (en) * | 2015-11-03 | 2018-12-13 | Bullet Scanning, Llc | System and method for inventory identification and quantification |
| CN105426883A (en) * | 2015-12-25 | 2016-03-23 | 中国科学院深圳先进技术研究院 | Video classified rapid identification method and device |
| US20170193310A1 (en) * | 2015-12-31 | 2017-07-06 | Pinhole (Beijing) Technology Co., Ltd. | Method and apparatus for detecting a speed of an object |
| CN105956618A (en) * | 2016-04-27 | 2016-09-21 | 云南昆钢集团电子信息工程有限公司 | Converter steelmaking blowing state recognition system and method based on image dynamic and static characteristics |
| CN106548153A (en) * | 2016-10-27 | 2017-03-29 | 杭州电子科技大学 | Video abnormality detection method based on graph structure under multi-scale transform |
| CN106658023A (en) * | 2016-12-21 | 2017-05-10 | 山东大学 | End-to-end visual odometer and method based on deep learning |
| US20190228264A1 (en) * | 2017-03-08 | 2019-07-25 | Tencent Technology (Shenzhen) Company Limited | Method and apparatus for training neural network model used for image processing, and storage medium |
| CN107292912A (en) * | 2017-05-26 | 2017-10-24 | 浙江大学 | A kind of light stream method of estimation practised based on multiple dimensioned counter structure chemistry |
| US20190079536A1 (en) * | 2017-09-13 | 2019-03-14 | TuSimple | Training and testing of a neural network system for deep odometry assisted by static scene optical flow |
| CN108229338A (en) * | 2017-12-14 | 2018-06-29 | 华南理工大学 | A kind of video behavior recognition methods based on depth convolution feature |
| CN108197566A (en) * | 2017-12-29 | 2018-06-22 | 成都三零凯天通信实业有限公司 | Monitoring video behavior detection method based on multi-path neural network |
| CN108519631A (en) * | 2018-02-22 | 2018-09-11 | 青岛心中有数科技有限公司 | Precipitation intensity prediction technique |
| CN108681695A (en) * | 2018-04-26 | 2018-10-19 | 北京市商汤科技开发有限公司 | Video actions recognition methods and device, electronic equipment and storage medium |
| CN108596157A (en) * | 2018-05-14 | 2018-09-28 | 三峡大学 | A kind of crowd's agitation scene detection method and system based on motion detection |
| CN109414145A (en) * | 2018-06-28 | 2019-03-01 | 广州艾若博机器人科技有限公司 | Speed-measuring method, skidding detection method based on light stream sensor, mobile electronic equipment, path correction method and device |
| CN109389072A (en) * | 2018-09-29 | 2019-02-26 | 北京字节跳动网络技术有限公司 | Data processing method and device |
| CN109711316A (en) * | 2018-12-21 | 2019-05-03 | 广东工业大学 | A kind of pedestrian recognition methods, device, equipment and storage medium again |
| CN109740499A (en) * | 2018-12-28 | 2019-05-10 | 北京旷视科技有限公司 | Methods of video segmentation, video actions recognition methods, device, equipment and medium |
| CN109819321A (en) * | 2019-03-13 | 2019-05-28 | 中国科学技术大学 | A kind of video super-resolution Enhancement Method |
| CN110046568A (en) * | 2019-04-11 | 2019-07-23 | 中山大学 | A kind of video actions recognition methods based on Time Perception structure |
Non-Patent Citations (2)
| Title |
|---|
| RANJAN A等: "Optical flow estimation using a spatial pyramid network", 《PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| 刘双叶: "基于视频的行为识别", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178277A (en) * | 2019-12-31 | 2020-05-19 | 支付宝实验室(新加坡)有限公司 | Video stream identification method and device |
| CN111178277B (en) * | 2019-12-31 | 2023-07-14 | 支付宝实验室(新加坡)有限公司 | Video stream identification method and device |
| CN112232283A (en) * | 2020-11-05 | 2021-01-15 | 深兰科技（上海）有限公司 | Bubble detection method and system based on optical flow and C3D network |
| CN112232283B (en) * | 2020-11-05 | 2023-09-01 | 深兰科技（上海）有限公司 | Bubble detection method and system based on optical flow and C3D network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378936B (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108491816A (en) | The method and apparatus for carrying out target following in video | |
| CN110399848A (en) | Video cover generation method, device and electronic equipment | |
| CN112258512B (en) | Point cloud segmentation method, device, equipment and storage medium | |
| CN110381368A (en) | Video cover generation method, device and electronic equipment | |
| CN112101305B (en) | Multi-path image processing method and device and electronic equipment | |
| CN110287891A (en) | Gestural control method, device and electronic equipment based on human body key point | |
| CN110619314A (en) | Safety helmet detection method and device and electronic equipment | |
| CN110288037A (en) | Image processing method, device and electronic equipment | |
| CN110222726A (en) | Image processing method, device and electronic equipment | |
| CN110378947A (en) | 3D model reconstruction method, device and electronic equipment | |
| CN110189394A (en) | Shape of the mouth as one speaks generation method, device and electronic equipment | |
| CN112115900B (en) | Image processing method, device, equipment and storage medium | |
| CN110555861B (en) | Optical flow calculation method and device and electronic equipment | |
| CN110278447A (en) | Video pushing method, device and electronic equipment based on continuous feature | |
| CN108171167B (en) | Method and apparatus for exporting image | |
| CN110378936A (en) | Optical flow computation method, apparatus and electronic equipment | |
| CN110209658A (en) | Data cleaning method and device | |
| CN112232311A (en) | Face tracking method and device and electronic equipment | |
| CN110069997A (en) | Scene classification method, device and electronic equipment | |
| CN110287350A (en) | Image search method, device and electronic equipment | |
| CN109977925A (en) | Expression determines method, apparatus and electronic equipment | |
| CN110337027A (en) | Video generation method, device and electronic equipment | |
| CN110197459A (en) | Image stylization generation method, device and electronic equipment | |
| CN110300329A (en) | Video pushing method, device and electronic equipment based on discrete features | |
| CN109635821A (en) | Feature extracting method, device, equipment and the readable medium of image-region |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Patentee before: Tiktok vision (Beijing) Co.,Ltd. |