CN113630601B - Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium - Google Patents
Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium Download PDFInfo
- Publication number
- CN113630601B CN113630601B CN202110732236.4A CN202110732236A CN113630601B CN 113630601 B CN113630601 B CN 113630601B CN 202110732236 A CN202110732236 A CN 202110732236A CN 113630601 B CN113630601 B CN 113630601B
- Authority
- CN
- China
- Prior art keywords
- current block
- motion vector
- affine motion
- distortion cost
- affine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000033001 locomotion Effects 0.000 title claims abstract description 215
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 title claims abstract description 126
- 238000000034 method Methods 0.000 title claims abstract description 56
- 239000013598 vector Substances 0.000 claims abstract description 95
- 230000003247 decreasing effect Effects 0.000 claims abstract description 5
- 238000004590 computer program Methods 0.000 claims description 9
- 230000007423 decrease Effects 0.000 claims description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000005457 optimization Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/567—Motion estimation based on rate distortion criteria
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application provides an affine motion estimation method, an affine motion estimation device, affine motion estimation equipment and a storage medium. Wherein the method comprises the following steps: initializing a current block to obtain attribute information of the current block; determining an initial predicted motion vector of the current block according to the attribute information; carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector; based on a linear equation set for calculating the affine motion vector, the affine motion vector is iteratively updated along the direction of decreasing the mean square error gradient, and the optimal affine motion vector of the current block is obtained after iteration is completed.
Description
Technical Field
The present disclosure relates to the field of video encoding and decoding technologies, and in particular, to an affine motion estimation method, apparatus, device, and storage medium.
Background
The main role of video codec technology is to pursue as high a video reconstruction quality and as high a compression ratio as possible within the available computing resources. Such as advanced video coding (Advanced Video Coding, AVS for short).
The motion compensation technology based on Affine (Affine) is a displacement transformation model for irregular motions such as fade-in/fade-out, rotation, scaling and the like, and solves the problem of inaccurate motion compensation of the displacement transformation model. The Affine-based motion compensation technique includes an AffineMerge mode (Affine merge mode) and an Affine motion estimation mode (Affine Motion Estimation), both of which are included in the process of inter-mode selection. Affine motion estimation works together with normal motion estimation to calculate the rate-distortion cost RDCost. Affine motion estimation is a new tool for inter prediction, increasing the temporal complexity of the encoder and requiring more hardware resources.
Disclosure of Invention
An object of the present application is to provide an affine motion estimation method, apparatus, device and storage medium to reduce complexity of affine motion estimation and to facilitate hardware implementation.
A first aspect of the present application provides an affine motion estimation method, including:
initializing a current block to obtain attribute information of the current block;
determining an initial predicted motion vector of the current block according to the attribute information;
carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector;
and iteratively updating the affine motion vector along the direction of decreasing the mean square error gradient based on the linear equation set for calculating the affine motion vector, and obtaining the optimal affine motion vector of the current block after the iteration is completed.
A second aspect of the present application provides an affine motion estimation device comprising:
the initialization module is used for initializing the current block to obtain attribute information of the current block;
a determining module, configured to determine an initial predicted motion vector of the current block according to the attribute information;
the affine motion estimation module is used for carrying out affine motion estimation on the current block based on the initial prediction motion vector to obtain an affine motion vector;
and the iteration updating module is used for iteratively updating the affine motion vector along the direction of the decline of the mean square error gradient based on the linear equation set for calculating the affine motion vector, and obtaining the optimal affine motion vector of the current block after the iteration is completed.
A third aspect of the present application provides an affine motion estimation device comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to perform the method of the first aspect of the present application.
A fourth aspect of the present application provides a computer readable medium having stored thereon computer readable instructions executable by a processor to implement the method of the first aspect of the present application.
Compared with the prior art, the affine motion estimation method provided by the application initializes the current block to obtain the attribute information of the current block; determining an initial predicted motion vector of the current block according to the attribute information; carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector; based on a linear equation set for calculating the affine motion vector, the affine motion vector is iteratively updated along the direction of gradient descent of the mean square error, and the optimal affine motion vector of the current block is obtained after iteration is completed.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 shows a flow chart of an affine motion estimation method provided by the present application;
FIG. 2 illustrates a flow chart of a specific affine motion estimation method provided herein;
FIG. 3 shows a flow chart for calculating rate distortion costs provided herein;
FIG. 4 is a schematic diagram of an affine motion compensated pipeline architecture provided herein;
FIG. 5 shows a flow chart for iteratively updating affine motion vectors provided herein;
FIG. 6 shows a schematic diagram of a division operation of three stages prior to a system of equations solving step;
fig. 7 shows a schematic diagram of an affine motion estimation device provided by the present application;
fig. 8 shows a schematic diagram of an affine motion estimation device provided by the present application;
fig. 9 shows a schematic diagram of a computer readable storage medium provided herein.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
It is noted that unless otherwise indicated, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs.
In addition, the terms "first" and "second" etc. are used to distinguish different objects and are not used to describe a particular order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
To further illustrate aspects of embodiments of the present application, the following description is provided with reference to the accompanying drawings. It will be appreciated that in the following embodiments, the same or corresponding contents may be referred to each other, and are not described in detail for simplicity of description.
Some technical terms in the present application are first described as follows:
affine motion estimation, affine motion estimation.
affine motion compensation, affine motion compensation.
CU coding unit, i.e. coding unit.
ME motion estimation, motion estimation.
mv motion vector, motion vector.
mvd motion vector, motion vector difference.
RDO rate distortion optimization, rate distortion optimization.
satd: sum of absolute transformed difference, absolute transformed difference sum.
The embodiment of the application provides an affine motion estimation method and device, affine motion estimation equipment and a computer readable storage medium, and the affine motion estimation method and device are described below with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of an affine motion estimation method according to some embodiments of the present application is shown, and the method can be applied to an AVS3 hardware encoder.
As shown in fig. 1, the affine motion estimation method described above may include the following steps S101 to S104:
step S101: initializing a current block to obtain attribute information of the current block;
specifically, the attribute information of the current block includes information such as the size of the current block and the position of the current block in the current frame. This step mainly involves data splitting (partition) after which the data blocks can be processed in parallel.
Step S102: determining an initial predicted motion vector of the current block according to the attribute information;
referring to fig. 2, a flowchart of a specific affine motion estimation method according to some embodiments of the present application is shown. Specifically, step S102 may be implemented as:
according to the attribute information, obtaining a translational motion vector of the current block and an affine motion vector of an adjacent block;
performing motion compensation on the current block according to affine motion vectors of adjacent blocks, and calculating a first rate distortion cost after motion compensation;
performing motion compensation on the current block according to the translational motion vector of the current block, and calculating a second rate distortion cost after the motion compensation;
performing motion compensation on the current block according to the zero vector, and calculating a third rate distortion cost after motion compensation;
and selecting the minimum rate-distortion cost from the first rate-distortion cost, the second rate-distortion cost and the third rate-distortion cost, and taking the motion vector corresponding to the minimum rate-distortion cost as the initial prediction motion vector of the current block.
In some embodiments of the present application, the above manner of motion compensation for the current block may be as follows:
dividing the current block into a plurality of sub-blocks;
and performing motion compensation on the plurality of sub-blocks by adopting a pipeline structure.
Referring to fig. 3, a flow chart of calculating rate distortion costs is shown as provided by some embodiments of the present application.
Specifically, in fig. 3, the motion compensation is used to interpolate the predicted pixel after motion according to the current coordinate, the motion vector and the reference pixel. Currently, motion compensation is an optimization bottleneck of the number of clock cycles (cycles) and hardware resources in affine motion estimation, and the application designs a flow structure of affine motion compensation for this, and its microstructure is shown in fig. 4.
In fig. 4, taking the 32x32 CU size as an example, the motion compensation is performed by the 64 4x4 sub-block pipeline structure within the CU. Wherein the motion compensation of each sub-block is divided into four steps. First, calculating coordinates (shown as a coord calc in fig. 4) of a reference block corresponding to the sub-block; a second step of acquiring pixel values (denoted as split ref in fig. 4) of the reference block based on the coordinates of the reference block; third, calculating a predicted pixel (denoted mc in fig. 4) based on the pixel value interpolation of the reference block; fourth, the predicted pixel (denoted as write pred in fig. 4) is stored. The above four steps of each 4x4 sub-block are performed in a serial manner; and performing respective motion compensation between the sub-blocks according to a running water structure.
If motion compensation is performed block by block in units of 4x4 sub-blocks, the hardware speed is slower; if the motion compensation processes of all the 4x4 sub-blocks are parallel, the problem of overlarge hardware area is caused, so that the motion compensation pipeline structure is adopted, and the balance of the hardware area and the hardware speed is realized.
Step S103: carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector;
step S104: and iteratively updating the affine motion vector along the direction of decreasing the mean square error gradient based on the linear equation set for calculating the affine motion vector, and obtaining the optimal affine motion vector of the current block after the iteration is completed.
Specifically, the iterative update of the affine motion vector is a key step of affine motion estimation, and the affine motion vector is iteratively updated along the direction in which the Mean Square Error (MSE) gradient decreases, and the optimal affine motion vector finally converged is the motion vector output by affine motion estimation.
Fig. 5 is a flowchart of iteratively updating affine motion vectors in the present application. The method optimizes the equation set solving step, wherein the equation set solving step comprises 7-stage 64-bit division operation before optimization, and each stage of division operation needs to consume more cycles. Fig. 6 is a schematic diagram of a division operation of three stages before the solving step of the equation set.
The method is used for optimizing the cycle of the solving step of the equation set. First, the division operation of the previous stage is combined to the final stage, that is, the number of division operation stages is reduced, but since the previous stage does not perform division operation, intermediate variable data overflows, so that the data precision of the participation operation needs to be reduced first. The divisor and the dividend of the division operation are replaced by the numbers corresponding to the power of 2 nearest to the divisor (for example, 9 is replaced by the power of 3 times of 2, namely 8), the division operation can be replaced by the shift operation, and the optimization method can cause the loss of part of algorithm precision, but greatly reduces the cycle number and saves hardware resources.
In the affine motion vector prediction process of gradient-based affine motion estimation, more division operations are involved, hardware speed is affected, and as divisors and dividends are variables, the operation method of the method is to find 2 power numbers adjacent to the divisors and dividends respectively, shift operations are used for replacing division operations, hardware speed is improved, and hardware resources are reduced.
The affine motion estimation method provided by the application initializes the current block to obtain attribute information of the current block; determining an initial predicted motion vector of the current block according to the attribute information; carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector; based on a linear equation set for calculating the affine motion vector, the affine motion vector is iteratively updated along the direction of decreasing the mean square error gradient, and the optimal affine motion vector of the current block is obtained after iteration is completed.
In the above-described embodiments, an affine motion estimation method is provided, and correspondingly, the present application also provides an affine motion estimation device. Referring to fig. 7, a schematic diagram of an affine motion estimation device according to some embodiments of the present application is shown. Since the apparatus embodiments are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points. The device embodiments described below are merely illustrative.
As shown in fig. 7, the affine motion estimating device 10 includes:
an initialization module 101, configured to initialize a current block to obtain attribute information of the current block;
a determining module 102, configured to determine an initial predicted motion vector of the current block according to the attribute information;
an affine motion estimation module 103, configured to perform affine motion estimation on the current block based on the initial predicted motion vector, to obtain an affine motion vector;
and the iteration updating module 104 is used for iteratively updating the affine motion vector along the direction of the gradient decline of the mean square error based on the linear equation set for calculating the affine motion vector, and obtaining the optimal affine motion vector of the current block after the iteration is completed.
According to some embodiments of the present application, the determining module 102 is specifically configured to:
according to the attribute information, obtaining a translational motion vector of the current block and an affine motion vector of an adjacent block;
performing motion compensation on the current block according to affine motion vectors of adjacent blocks, and calculating a first rate distortion cost after motion compensation;
performing motion compensation on the current block according to the translational motion vector of the current block, and calculating a second rate distortion cost after the motion compensation;
performing motion compensation on the current block according to the zero vector, and calculating a third rate distortion cost after motion compensation;
and selecting the minimum rate-distortion cost from the first rate-distortion cost, the second rate-distortion cost and the third rate-distortion cost, and taking the motion vector corresponding to the minimum rate-distortion cost as the initial prediction motion vector of the current block.
According to some embodiments of the present application, the manner in which the determining module 102 performs motion compensation on the current block is as follows:
dividing the current block into a plurality of sub-blocks;
and performing motion compensation on the plurality of sub-blocks by adopting a pipeline structure.
According to some embodiments of the present application, the attribute information includes a size of the current block and a position of the current block in the current frame.
The affine motion estimation device provided by the embodiment of the present application has the same beneficial effects as the affine motion estimation method provided by the foregoing embodiment of the present application because of the same inventive concept.
The present embodiment also provides an affine motion estimation device, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, etc., corresponding to the affine motion estimation method provided by the foregoing embodiment, to perform the affine motion estimation method described above.
Referring to fig. 8, a schematic diagram of an affine motion estimation device according to some embodiments of the present application is shown. As shown in fig. 8, the affine motion estimating device 20 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 being connected by the bus 202; the memory 201 stores a computer program executable on the processor 200, and the processor 200 executes the affine motion estimation method according to any one of the foregoing embodiments of the present application when the computer program is executed.
The memory 201 may include a high-speed random access memory (RAM: random Access Memory), and may further include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the system network element and at least one other network element is implemented via at least one communication interface 203 (which may be wired or wireless), the internet, a wide area network, a local network, a metropolitan area network, etc. may be used.
Bus 202 may be an ISA bus, a PCI bus, an EISA bus, or the like. The buses may be classified as address buses, data buses, control buses, etc. The memory 201 is configured to store a program, and the processor 200 executes the program after receiving an execution instruction, and the motion estimation method disclosed in any of the foregoing embodiments of the present application may be applied to the processor 200 or implemented by the processor 200.
The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 200 or by instructions in the form of software. The processor 200 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but may also be a Digital Signal Processor (DSP), application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be embodied directly in hardware, in a decoded processor, or in a combination of hardware and software modules in a decoded processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and in combination with its hardware, performs the steps of the above method.
The affine motion estimation device provided by the embodiment of the present application and the affine motion estimation method provided by the embodiment of the present application are the same inventive concept, and have the same advantageous effects as the method adopted, operated or implemented by the same.
The present embodiment also provides a computer readable storage medium corresponding to the motion estimation method provided in the foregoing embodiment, referring to fig. 9, the computer readable storage medium is shown as an optical disc 30, on which a computer program (i.e. a program product) is stored, where the computer program, when executed by a processor, performs the affine motion estimation method provided in any of the foregoing embodiments.
It should be noted that examples of the computer readable storage medium may also include, but are not limited to, a phase change memory (PRAM), a Static Random Access Memory (SRAM), a Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a flash memory, or other optical or magnetic storage medium, which will not be described in detail herein.
The computer-readable storage medium provided by the above-described embodiments of the present application and the affine motion estimation method provided by the embodiments of the present application have the same advantageous effects as the method adopted, operated or implemented by the application program stored therein, for the same inventive concept.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the embodiments, and are intended to be included within the scope of the claims and description.
Claims (5)
1. An affine motion estimation method comprising:
initializing a current block to obtain attribute information of the current block; the attribute information includes the size of the current block and the position of the current block in the current frame;
determining an initial predicted motion vector of the current block according to the attribute information;
carrying out affine motion estimation on the current block based on the initial predicted motion vector to obtain an affine motion vector;
based on a linear equation set for calculating the affine motion vector, iteratively updating the affine motion vector along the direction of decreasing the mean square error gradient, and obtaining an optimal affine motion vector of the current block after iteration is completed;
the determining the initial prediction motion vector of the current block according to the attribute information comprises the following steps:
according to the attribute information, obtaining a translational motion vector of the current block and an affine motion vector of an adjacent block;
performing motion compensation on the current block according to affine motion vectors of adjacent blocks, and calculating a first rate distortion cost after motion compensation;
performing motion compensation on the current block according to the translational motion vector of the current block, and calculating a second rate distortion cost after the motion compensation;
performing motion compensation on the current block according to the zero vector, and calculating a third rate distortion cost after motion compensation;
selecting a minimum rate distortion cost from the first rate distortion cost, the second rate distortion cost and the third rate distortion cost, and taking a motion vector corresponding to the minimum rate distortion cost as an initial prediction motion vector of the current block;
the manner of motion compensation for the current block is as follows:
dividing the current block into a plurality of sub-blocks;
the method for performing motion compensation on the plurality of sub-blocks by adopting a pipeline structure specifically comprises the following steps:
the first step, calculating coordinates of a reference block corresponding to the sub-block;
step two, obtaining pixel values of the reference block according to the coordinates of the reference block;
thirdly, calculating a predicted pixel based on pixel value interpolation of the reference block;
fourth, storing the predicted pixel;
the four steps of each sub-block are performed in a serial manner; and performing respective motion compensation between the sub-blocks according to a running water structure.
2. The method of claim 1, wherein in the solving of the system of linear equations, both the divisor and the dividend of the division operation are replaced with numbers corresponding to powers of 2 nearest thereto to replace the division operation with a shift operation.
3. An affine motion estimation device, characterized by comprising:
the initialization module is used for initializing the current block to obtain attribute information of the current block; the attribute information includes the size of the current block and the position of the current block in the current frame;
a determining module, configured to determine an initial predicted motion vector of the current block according to the attribute information;
the affine motion estimation module is used for carrying out affine motion estimation on the current block based on the initial prediction motion vector to obtain an affine motion vector;
the iteration updating module is used for iteratively updating the affine motion vector along the direction of the gradient decline of the mean square error based on the linear equation set for calculating the affine motion vector, and obtaining the optimal affine motion vector of the current block after the iteration is completed;
the determining module is specifically configured to:
according to the attribute information, obtaining a translational motion vector of the current block and an affine motion vector of an adjacent block;
performing motion compensation on the current block according to affine motion vectors of adjacent blocks, and calculating a first rate distortion cost after motion compensation;
performing motion compensation on the current block according to the translational motion vector of the current block, and calculating a second rate distortion cost after the motion compensation;
performing motion compensation on the current block according to the zero vector, and calculating a third rate distortion cost after motion compensation;
selecting a minimum rate distortion cost from the first rate distortion cost, the second rate distortion cost and the third rate distortion cost, and taking a motion vector corresponding to the minimum rate distortion cost as an initial prediction motion vector of the current block;
the mode of the determining module for performing motion compensation on the current block is as follows:
dividing the current block into a plurality of sub-blocks;
the method for performing motion compensation on the plurality of sub-blocks by adopting a pipeline structure specifically comprises the following steps:
the first step, calculating coordinates of a reference block corresponding to the sub-block;
step two, obtaining pixel values of the reference block according to the coordinates of the reference block;
thirdly, calculating a predicted pixel based on pixel value interpolation of the reference block;
fourth, storing the predicted pixel;
the four steps of each sub-block are performed in a serial manner; and performing respective motion compensation between the sub-blocks according to a running water structure.
4. An affine motion estimation device comprising: memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor executes to implement the method according to any of claims 1 to 2 when the computer program is run.
5. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, is to implement the method of any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732236.4A CN113630601B (en) | 2021-06-29 | 2021-06-29 | Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732236.4A CN113630601B (en) | 2021-06-29 | 2021-06-29 | Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113630601A CN113630601A (en) | 2021-11-09 |
| CN113630601B true CN113630601B (en) | 2024-04-02 |
Family
ID=78378647
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110732236.4A Active CN113630601B (en) | 2021-06-29 | 2021-06-29 | Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113630601B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103458245A (en) * | 2013-08-30 | 2013-12-18 | 上海高清数字科技产业有限公司 | Water-flowing design method and system of decoder motion compensation module |
| CN106658012A (en) * | 2017-01-06 | 2017-05-10 | 华南理工大学 | Parallel pipeline task division method for VP9 decoder |
| CN109640097A (en) * | 2018-12-07 | 2019-04-16 | 辽宁师范大学 | The video Affine motion estimation method of adaptive factor |
| CN109660800A (en) * | 2017-10-12 | 2019-04-19 | 北京金山云网络技术有限公司 | Method for estimating, device, electronic equipment and computer readable storage medium |
| WO2019160860A1 (en) * | 2018-02-14 | 2019-08-22 | Futurewei Technologies, Inc. | Adaptive interpolation filter |
| WO2019199152A1 (en) * | 2018-04-14 | 2019-10-17 | 엘지전자 주식회사 | Method and device for processing video signal by using affine prediction |
| CN110383839A (en) * | 2017-03-14 | 2019-10-25 | 高通股份有限公司 | Affine motion information export |
| WO2019231256A1 (en) * | 2018-05-30 | 2019-12-05 | 엘지전자 주식회사 | Method and device for processing video signal using affine motion prediction |
| CN110891176A (en) * | 2018-09-10 | 2020-03-17 | 华为技术有限公司 | Motion vector prediction method and device based on affine motion model |
| CN111050168A (en) * | 2019-12-27 | 2020-04-21 | 浙江大华技术股份有限公司 | Affine prediction method and related device thereof |
| US10671076B1 (en) * | 2017-03-01 | 2020-06-02 | Zoox, Inc. | Trajectory prediction of third-party objects using temporal logic and tree search |
| CN111479110A (en) * | 2020-04-15 | 2020-07-31 | 郑州轻工业大学 | Fast affine motion estimation method for H.266/VVC |
| WO2020242350A2 (en) * | 2019-09-30 | 2020-12-03 | Huawei Technologies Co., Ltd. | Usage of dct based interpolation filter and enhanced bilinear interpolation filter in affine motion compensation |
| CN112118451A (en) * | 2019-06-20 | 2020-12-22 | 杭州海康威视数字技术股份有限公司 | Encoding and decoding method, device and equipment |
| CN112703733A (en) * | 2018-09-21 | 2021-04-23 | 交互数字Vc控股公司 | Translation and affine candidates in a unified list |
| CN112868234A (en) * | 2019-09-24 | 2021-05-28 | 深圳市大疆创新科技有限公司 | Motion estimation method, system and storage medium |
| CN112956190A (en) * | 2018-09-17 | 2021-06-11 | 高通股份有限公司 | Affine motion prediction |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8665958B2 (en) * | 2008-01-29 | 2014-03-04 | Electronics And Telecommunications Research Institute | Method and apparatus for encoding and decoding video signal using motion compensation based on affine transformation |
-
2021
- 2021-06-29 CN CN202110732236.4A patent/CN113630601B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103458245A (en) * | 2013-08-30 | 2013-12-18 | 上海高清数字科技产业有限公司 | Water-flowing design method and system of decoder motion compensation module |
| CN106658012A (en) * | 2017-01-06 | 2017-05-10 | 华南理工大学 | Parallel pipeline task division method for VP9 decoder |
| US10671076B1 (en) * | 2017-03-01 | 2020-06-02 | Zoox, Inc. | Trajectory prediction of third-party objects using temporal logic and tree search |
| CN110383839A (en) * | 2017-03-14 | 2019-10-25 | 高通股份有限公司 | Affine motion information export |
| CN109660800A (en) * | 2017-10-12 | 2019-04-19 | 北京金山云网络技术有限公司 | Method for estimating, device, electronic equipment and computer readable storage medium |
| WO2019160860A1 (en) * | 2018-02-14 | 2019-08-22 | Futurewei Technologies, Inc. | Adaptive interpolation filter |
| WO2019199152A1 (en) * | 2018-04-14 | 2019-10-17 | 엘지전자 주식회사 | Method and device for processing video signal by using affine prediction |
| WO2019231256A1 (en) * | 2018-05-30 | 2019-12-05 | 엘지전자 주식회사 | Method and device for processing video signal using affine motion prediction |
| CN110891176A (en) * | 2018-09-10 | 2020-03-17 | 华为技术有限公司 | Motion vector prediction method and device based on affine motion model |
| CN112956190A (en) * | 2018-09-17 | 2021-06-11 | 高通股份有限公司 | Affine motion prediction |
| CN112703733A (en) * | 2018-09-21 | 2021-04-23 | 交互数字Vc控股公司 | Translation and affine candidates in a unified list |
| CN109640097A (en) * | 2018-12-07 | 2019-04-16 | 辽宁师范大学 | The video Affine motion estimation method of adaptive factor |
| CN112118451A (en) * | 2019-06-20 | 2020-12-22 | 杭州海康威视数字技术股份有限公司 | Encoding and decoding method, device and equipment |
| CN112868234A (en) * | 2019-09-24 | 2021-05-28 | 深圳市大疆创新科技有限公司 | Motion estimation method, system and storage medium |
| WO2020242350A2 (en) * | 2019-09-30 | 2020-12-03 | Huawei Technologies Co., Ltd. | Usage of dct based interpolation filter and enhanced bilinear interpolation filter in affine motion compensation |
| CN111050168A (en) * | 2019-12-27 | 2020-04-21 | 浙江大华技术股份有限公司 | Affine prediction method and related device thereof |
| CN111479110A (en) * | 2020-04-15 | 2020-07-31 | 郑州轻工业大学 | Fast affine motion estimation method for H.266/VVC |
Non-Patent Citations (4)
| Title |
|---|
| An improved frame-level pipelined architecture for high resolution video motion estimation;Wei-feng He等;2007 IEEE International Symposium on Circuits and Systems;20070530;全文 * |
| Efficient Macroblock Pipeline Structure in High Definition AVS Video Encoder VLSI Architecture;贾惠柱等;2010 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS;20200101;全文 * |
| 基于FPGA的HEVC后处理CNN硬件加速器研究;严伟等;计算机研究与发;20181215;全文 * |
| 端云融合***中运动补偿研究与实现;徐一凡;中国优秀硕士学位论文全文数据库 信息科技辑;20210601;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113630601A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3611507B2 (en) | Motion vector detection device | |
| US8213515B2 (en) | Interpolated skip mode decision in video compression | |
| US10412409B2 (en) | Encoding system using motion estimation and encoding method using motion estimation | |
| US7778494B2 (en) | FIR-based interpolation in advanced video codecs on VLIW processor | |
| CN109348234B (en) | Efficient sub-pixel motion estimation method and system | |
| CN111798545A (en) | Method and device for playing skeleton animation, electronic equipment and readable storage medium | |
| US20240037700A1 (en) | Apparatus and method for efficient motion estimation | |
| US10924753B2 (en) | Modular motion estimation and mode decision engine | |
| JP6781823B2 (en) | Interframe predictive coding method and equipment | |
| JP4547503B2 (en) | Arithmetic coding apparatus, arithmetic coding method, arithmetic coding program, and computer-readable recording medium storing the program | |
| CN113630601B (en) | Affine motion estimation method, affine motion estimation device, affine motion estimation equipment and storage medium | |
| CN112203091B (en) | Motion vector prediction method, system and computer medium based on quadratic polynomial | |
| Babionitakis et al. | A real-time motion estimation FPGA architecture | |
| Kim et al. | MESIP: A configurable and data reusable motion estimation specific instruction-set processor | |
| Ismail et al. | High performance architecture for real-time HDTV broadcasting | |
| CN110691253A (en) | Encoding and decoding method and device based on inter-frame prediction | |
| CN111343465A (en) | Electronic circuit and electronic device | |
| JP2003530780A (en) | Method of determining similarity between a data sample and interpolation of another data sample | |
| CN116389743A (en) | Image encoding method, image encoding device, chip, terminal, storage medium, and program product | |
| CN110038301B (en) | Data processing method and device, electronic equipment and storage medium | |
| JP2007124605A (en) | Method and apparatus for calculating cost functions and interpolation method thereof | |
| CN113489986A (en) | Integer pixel motion estimation method and device, electronic equipment and medium | |
| CN108322759B (en) | Pixel value acquisition method and device and electronic equipment | |
| CN102377995A (en) | Fractional pixel motion estimation method and system thereof | |
| CN101882112A (en) | Reading method of four-byte character, device thereof and decoder thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |