WO2007057986A1 - 動きベクトル算出装置および動きベクトル算出方法 - Google Patents
動きベクトル算出装置および動きベクトル算出方法 Download PDFInfo
- Publication number
- WO2007057986A1 WO2007057986A1 PCT/JP2006/300533 JP2006300533W WO2007057986A1 WO 2007057986 A1 WO2007057986 A1 WO 2007057986A1 JP 2006300533 W JP2006300533 W JP 2006300533W WO 2007057986 A1 WO2007057986 A1 WO 2007057986A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- motion vector
- global
- macroblock
- vector calculation
- calculating
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/43—Hardware specially adapted for motion estimation or compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/527—Global motion vector estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/533—Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/56—Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
Definitions
- the present invention relates to a global motion vector calculation technique for calculating a motion vector used for inter-frame prediction when coding an input image.
- encoding is performed by compressing the amount of moving image information mainly using the following three image conversions. That is, the first image conversion using intra-image (spatial) correlation, the second image conversion using inter-image (temporal) correlation, and the third using variable-length encoding according to the appearance probability of the code.
- Image conversion in the second image conversion using temporal correlation, the moving direction and moving amount of the image between frames (input images) that differ in time are calculated as “motion vectors” in units of macroblocks.
- the amount of moving picture information can be reduced by representing a frame with pixel information and motion vector information of the macroblock.
- FIG. 1 is a diagram for explaining another example of motion vector calculation for each macroblock using the global motion vector.
- the global motion vector (0101) that contains the macroblock ⁇ that contains the macroblock ⁇ whose motion vector is to be detected is The calculation is the same as the motion vector calculation. Then, it is considered that the movement of the macroblock ⁇ is within a predetermined range including the movement destination of the global macroblock ⁇ ′ indicated by this global motion beta.
- the ability to get S Therefore, by calculating the range shown in (2) of Fig. 1 as the search range for the motion vector of macroblock ⁇ , the motion vector (0102) for macroblock a can be calculated without unnecessarily widening the search range. It can be done.
- Patent Document 1 WO00Z05899
- the above conventional techniques have the following problems. That is, as shown by the solid line in FIG. 2, if the calculated global motion beta is incorrect (correct, global motion vector is a dashed arrow), it is determined based on the incorrect global motion vector.
- the search range is naturally different from the search range that should be determined. Therefore, there is a problem that the motion vector of the macroblock searched and calculated in the wrong search range is likely to be erroneously calculated.
- the motion vector calculation error may cause a reduction in the efficiency of moving picture coding. This is because the motion vector calculated by such an error usually takes an extreme value. This is because the motion vector is calculated by referring to a completely different block due to an error. If the absolute value of the vector is extremely large, the Golomb code that achieves high-efficiency coding as the absolute value of the vector is close to 0 is a factor that reduces the coding efficiency. It can be both.
- the present invention uses a global motion vector knowledge calculated in units of global macroblocks to determine a motion vector search range of a macroblock, and calculates a motion vector.
- a vector calculation device and further includes a motion vector calculation device having the following functions for calculating the global motion vector.
- a global macroblock is divided into, for example, four blocks (midnor blocks) a, b, c, d, and the middle motion that is the motion vector in that middle block.
- Detect vectors From the detected middle motion vector, the middle motion that seems to have taken an abnormal value by, for example, “extreme vector exclusion”, “median processing”, or “abnormal value exclusion using standard deviation” processing, etc. Exclude vectors The Then, based on the remaining middle motion vector, this function calculates a more accurate global motion vector 0301 as shown in (2) of FIG.
- the present invention provides a dividing unit that divides a global macroblock into a plurality of middle blocks, a middle motion vector detecting unit that detects a middle motion vector for each divided middle block, and a global macroblock.
- a global motion vector calculation unit that calculates a global motion vector based on a middle motion vector of a middle block included in the global macroblock.
- Embodiment 1 Outline of Embodiment 1 An example of a motion vector calculation apparatus according to the present embodiment is to determine a global motion vector calculated in units of global macroblocks and determine a motion block search range for macroblocks.
- This is a motion vector calculation device that calculates a motion vector by using the motion vector detection device, and further includes the following functions. That is, when the global motion vector is calculated, the global macro block is changed to the middle block.
- This is a motion vector calculation device having a function of calculating a more accurate global motion vector using a middle motion vector that is a motion vector in the middle block.
- global macroblock refers to a block that is larger than a macroblock that is a unit of motion vector detection used in interframe prediction in moving picture coding, and the size of the block depends on the design as appropriate. I don't mind being able to set it.
- the search range for motion vector detection is determined using the global motion vector detected in units of global macroblocks. In this way, in the calculation of motion vectors in units of macroblocks, the search range of the macroblock movement destination can be narrowed down by the global motion vector, so that the processing load on the entire apparatus can be suppressed.
- the global macro block is divided into middle blocks, and a more accurate global motion vector is calculated using the middle motion vector.
- FIG. 4 is a diagram illustrating an example of functional blocks in the motion vector calculation apparatus according to the present embodiment.
- the “motion vector calculation device” (0400) of the present embodiment includes a “splitting unit” (0401), a “middle motion vector detection unit” (0402), and a “global motion vector calculation unit”. (0403).
- a motion vector calculation device using a global motion vector a search range based on the global motion vector is determined, and a motion vector using the search range is determined. Needless to say, processing such as calculation is performed.
- an arithmetic unit such as a CPU, a main memory, a bus, a storage device such as a hard disk or a nonvolatile memory, a storage medium such as a CD-ROM or a DVD-ROM, etc.
- Hardware components such as media reading drives, transmission / reception ports for various communications and printing devices, other peripheral devices, driver programs for controlling these hardware, other application programs, and interfaces used for information input Hue Source.
- the present invention can be realized not only as an apparatus but also as a method.
- a part of such an invention can be configured as software.
- a software product used for causing a computer to execute such software and a recording medium in which the product is fixed to a recording medium are naturally included in the technical scope of the present invention (the same applies throughout this specification). is there).
- the "divider” (0401) has a function of dividing a global macroblock into a plurality of middle blocks. It should be noted that the number of divisions and the division shape can be arbitrarily set.
- the “division” is not only a dynamic division in which the coordinates of the divided middle block are calculated based on the coordinate information of the global macroblock, for example, but also the following static division. It is okay. That is, a “global macro block middle block table” in which the coordinate information of the global macro block and the coordinate information of the divided middle block are tabulated in advance is stored in a storage device or the like.
- the table is referenced using the global macro block indicated by the coordinate information as a key, and the coordinate information of the corresponding middle block that has already been divided is acquired.
- the global macroblock is formed into a rectangular shape having a predetermined long side. And how to divide it into middle blocks. Note that these global macroblocks and a plurality of midole blocks generated by dividing the global macroblocks are retained by associating their respective identification information.
- the “middle motion vector detection unit” (0402) has a function of detecting a middle motion vector for each divided middle block. This middle motion vector detection process is similar to the normal motion beta detection process. For example, the absolute difference between each middle pixel and the block in the reference frame of the middle block is calculated for each corresponding pixel. Take the value and calculate the sum as a correlation coefficient.
- the entire reference block is searched while searching in units of one pixel or half pixel, and the correlation coefficient is calculated.
- the block having the smallest correlation coefficient is specified as the movement destination of the middle block. .
- the middle motion vector is detected from the direction and amount of movement of the middle block to the destination, it will be good.
- the correlation coefficients of any three blocks are obtained.
- the middle motion vector may be detected by processing called so-called log search, etc., by narrowing down the destination block based on the magnitude relationship.
- the search amount when the search amount is set to “Z”, the amount of calculation for the motion destination search of the global macroblock whose horizontal size is “2a” X vertical size is “2b” "4ab XZ”.
- the global macroblock is divided into four middle blocks with vertical size “a” X horizontal size “b”, the computational amount of the motion destination search is “Z” because the search amount is the same. , “Ab XZX 4”.
- the method of calculating the global motion vector using the middle motion vector according to the present embodiment can be implemented almost the same as the calculation processing amount of the conventional global motion vector calculation.
- the "global motion vector calculation unit” (0403) has a function of calculating a global motion vector of one global macroblock based on a middle motion vector of a middle block included in the global macroblock. .
- This embodiment is characterized in that the global motion vector is calculated by the global motion vector calculation unit using a calculation process different from that of the conventional motion vector detection. In other words, it is detected by taking an extreme value by referring to a completely different block by “exclusion of extreme value vector”, “median processing”, or “exclusion of abnormal value using standard deviation”.
- the global motion vector is calculated after excluding middle motion vectors that have abnormal values. The As a result, the global motion vector can be calculated more accurately than in the past.
- FIGS an example of global motion vector calculation processing in the global motion vector calculation unit excluding the middle motion vector nore taking these extreme values
- FIG. 7 is a diagram for explaining an example of a global motion vector calculation process in the global motion vector calculation unit.
- the motion vector calculation apparatus of the present embodiment is characterized by having its “global motion vector calculation unit” (0703) force and “extreme value exclusion averaging means” (0704).
- "Extreme Exclusion Averaging Means” (0704) calculates the global motion vector by averaging the sum of vector values excluding the middle motion vector that takes the maximum value and the middle motion vector that takes the minimum value from the middle motion vectors. It has a function to issue. For example, as shown in FIG. 7, the global macroblock is divided into nine middle blocks, and the middle motion vectors a, b, and i are detected.
- each vector quantity is calculated and compared, and it is determined that among the nine middle motion vectors, the betalet having the largest beta quantity is “i” and the vector having the smallest vector quantity is “a”. It was done. Then, this extreme value exclusion averaging means averages the sum of the remaining middle motion vectors 3, c, ..., h, excluding the maximum vector "i” and the minimum vector "a”, and global motion It asks for a vector.
- FIG. 8 is a diagram for explaining another example of the global motion vector calculation processing in the global motion vector calculation unit.
- the motion vector calculation apparatus of the present embodiment is characterized by having its “global motion vector nore calculation unit” (0803) force and “median averaging means” (0804).
- the “median averaging means” (0804) has a function of calculating a global motion vector by performing median processing on the middle motion vector.
- “median processing” is a process in which, for example, vector components are arranged in order of magnitude, and the value located at the center (the average of the two central values if the number is an even number) is calculated as the average value. . For example, as shown in FIG.
- middle motion vectors a, b, and i of nine middle blocks obtained by dividing a global macro block are detected. Then, the vector components are arranged in ascending order of their values. Then, since the median can be specified as “4, 3”, the global motion vector component is set to “4, 3”. "And so on.” In this median processing, it may be possible to perform processing such as obtaining the median value of the remaining vector components by excluding the vector having the extreme value before obtaining the median value of the components.
- FIG. 9 is a diagram for explaining another example of the global motion vector calculation process in the global motion vector calculation unit.
- the motion vector calculation apparatus of the present embodiment is characterized by having its “global motion vector calculation unit” (0903) force and “standard deviation utilization averaging means” (0904).
- the “standard deviation utilization averaging means” (0904) has a function of calculating a global motion vector by averaging the sum of the middle motion vector values having a value within ⁇ 3 ⁇ of the middle block motion vectors.
- “ ⁇ ” is the standard deviation, that is, the square root of the variance calculated by the average value of “the square of (the difference between each vector value and its average value)”.
- the middle motion vector not included in ⁇ 3 is an abnormal value that appears with a probability of 0 ⁇ 3%. Therefore, middle motion vectors that take these abnormal values, such as a and i, are excluded, and the sum of the remaining middle motion vectors b, c, ..., h is averaged to obtain a global motion vector. .
- the middle motion vector having an extreme vector value can be excluded, and therefore, a more accurate global motion vector can be calculated. Then, the search range for calculating the motion vector can be determined more accurately based on the accurate and accurate global motion vector, and the motion vector can be calculated.
- FIG. 10 is a schematic diagram showing an example of the hardware configuration of the motion vector calculation device of the present embodiment.
- the functions of the dividing unit, the middle motion vector detecting unit, and the global motion vector calculating unit in the motion vector calculating apparatus of the present embodiment are “calculators” ( 1001), “main memory” (1002) for storing data for that purpose, “image input means” (1003) for inputting an image subject to high-efficiency encoding, and “Video memory "(1004) etc. It is realized by. They are connected to each other via a data communication path such as a system bus (1005) to transmit and receive information.
- a data communication path such as a system bus (1005) to transmit and receive information.
- the "main memory” provides a work area that is a work area of a program executed by the computing unit, and a data area that stores data to be processed by the program.
- a plurality of memory addresses are assigned to the “main memory” and “video memory”, and programs executed by the “calculator” specify the memory addresses and access them to each other. It is possible to exchange and process.
- an image to be subjected to high-efficiency encoding is input from the “image input unit”.
- the input images here are various, such as moving image data that is encoded with analog broadcast waves received by a TV tuner without compression, or uncompressed encoded moving image data that is shot with a digital video camera. good.
- an encoded moving image obtained by developing moving image data encoded with high efficiency may be used.
- the images that make up the moving image input here are input to a predetermined memory address of the video memory via the system bus, for example, input image 1 (first frame of moving image data) to memory address A, and input to memory address B.
- Image 2 second frame of moving image data
- ... and so on are stored.
- the input image data is sequentially stored at a predetermined memory address of the "main memory”, and DCT conversion processing, quantization processing, and the like are performed by the arithmetic processing of the "calculator”.
- First image conversion using correlation is performed.
- image inverse transform inverse DCT transform, inverse quantization
- image motion detection is performed to perform image compression by inter-frame prediction, and the motion detection processing is performed using the inversely converted image and the input image.
- global motion vector calculation processing is further performed to reduce the amount of calculation processing and improve detection accuracy in this motion detection processing.
- the global motion vector for the input image 1 of the global macroblock of the input image 2 is calculated so that the configuration matches the macroblock used for the motion prediction in the forward prediction process of inter-frame prediction. .
- the middle motion vector detection process and the global motion vector based on the middle motion vector are performed as follows. of Perform the calculation process.
- the global motion vector calculation process and the middle motion vector detection process include a method using the inversely transformed image and the input image, and a method using only the input image. The detection method using only images will be described as an example.
- the middle block dividing process of the global macroblock is performed. Therefore, the global motion block coordinate information for calculating the global motion vector in input image 2 is first acquired by the instructions of the global motion vector calculation program developed in the main memory peak area, and the main memory is obtained. Stored at memory address 1. Then, for example, according to the instruction indicating that “obtain the coordinate information of the middle block obtained by dividing the global macroblock using the coordinate information of the global macroblock stored in memory address 1 as a key” The coordinate information of the corresponding middle block 1, 2, 3, 4 is acquired by referring to “Global Macro Block-Middle Block Tape Nore”. The coordinate information of the middle block is stored in memory addresses 2, 3, 4, and 5 of the main memory, respectively.
- a middle motion vector detection process in the middle block stored in each memory address of the main memory is performed.
- the detection of the middle movement beta is preferably performed by the same process as that of the conventional movement beta. Therefore, in order to calculate the above-described correlation coefficient, the pixel information of the middle block 1 is acquired based on the coordinate information. Specifically, based on the coordinate information stored in the memory address 2, the pixel information constituting the middle block 1 of the input image 2 stored in the video memory is stored in the predetermined memory address of the main memory. A command such as “Se” is sent and executed.
- identification information of the input image before or after the reference destination of the middle block 1, for example, the input image 1, is specified and specified memory in the main memory via the system bus or the like. Stored in the address. Then, in the same way as the Midoret block, in order to calculate the correlation number, the pixel information constituting the block is acquired and stored in the main memory based on the coordinate information of the block 1 of the input image 1. . Also, the force that does not move (search) the block in half-pixel units (or 1 pixel or 1Z4 pixel), and so on. The pixel information constituting the memory is acquired and stored in the main memory.
- the pixel information of the middle block and the pixel information of those blocks are specified by the memory address.
- the difference value between the corresponding pixels of the middle block 1 and the block 1 of the input image 1 (reference destination) Calculate the sum of absolute values and store the calculation result (correlation coefficient) at a predetermined memory address in the main memory. Store to address.
- the correlation coefficients are calculated for the other blocks 2, 3,... Of the other input image 1, and stored in another predetermined memory address of the main memory.
- each correlation coefficient is specified by the memory address. For example, “Compare the size of the operation results (number of correlations) stored in the main memory and specify the minimum correlation coefficient”.
- the correlation coefficient magnitude comparison is executed by the arithmetic processing in the arithmetic unit. Then, the correlation coefficient that takes the minimum is specified, and the block force associated with the correlation coefficient is specified as the movement destination of the middle block 1 of the input image 2 in the input image 1. Then, the middle motion vector 1 is calculated from the coordinate information of the middle block 1 and the block specified as the movement destination by the operation of the calculator, and stored in the memory address 11 of the main memory. The same middle motion vector calculation processing is performed for middle blocks 2, 3, and 4. Similarly, other global macroblocks ⁇ , ⁇ ,... Of input image 2 are subjected to middle block division processing and middle motion solid tone detection processing.
- a global motion vector is calculated using the detected middle motion vector.
- “extreme vector exclusion” extreme exclusion averaging means
- “median processing” median processing means
- “abnormal value exclusion using standard deviation” Standard deviation use average method. So, for example, for extreme value vector exclusion processing, “Middle motion vector vector size comparison is performed and the middle motion vector excluding the middle motion vector that takes the maximum value and the middle motion vector that takes the minimum value.
- the magnitude comparison results of each vector component are stored in the memory address in order, and the middle number of vectors (if the number is even, the middle two numbers) is stored in order. It is recommended that an instruction that uses a global motion vector component as the component (or the average of the components when the number of vectors is an even number) and an arithmetic unit for this are performed.
- the standard deviation ⁇ of the vector value is calculated, and the average of middle motion vectors having vector values within ⁇ 3 ⁇ is used as the global motion vector. It is preferable that the instruction and the arithmetic unit for the instruction are performed.
- the coordinates of the search range are calculated by the calculation in the arithmetic unit, similarly to the conventional motion vector detection using the global motion vector.
- Information, etc. are calculated, and search based on the information and calculation of the motion vector are performed.
- FIG. 11 is a flowchart showing an example of a process flow in the motion vector calculation apparatus of the present embodiment.
- the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.
- the global macro block is divided into a plurality of middle blocks (step S1101), and a middle motion vector is detected for each divided middle block (step S1102).
- a global motion vector of one global macroblock is calculated based on the middle motion vector of the middle block included in the detected global macroblock (step S1103).
- the motion vector search range of the macroblock is determined based on the calculated global motion vector (step S 1104), and the motion vector is calculated using the search range (step S 1105).
- FIG. 12 is a diagram for explaining an example of an outline of a motion vector calculation apparatus according to the present embodiment.
- the global motion vector of the block including the human moving in the direction of arrow P, the moving object, and the background is, for example, vector G
- the vector and the movement of the person and the background are combined. Therefore, there is a possibility that false detection may occur in motion vector calculation in the macro block of the human part included in this global macro block. Therefore, in this embodiment, as shown in FIG.
- the global macroblock is composed of blocks that contain a large number of moving objects (which seems to be), blocks that contain too much, and blocks that contain too much. Divide the global macroblocks in a non-uniform manner.
- the global motion vector in the block including the moving object can be calculated as a vector that matches the movement of the moving object while suppressing the influence of the motion vector such as the background that does not move, such as the vector F. .
- the influence of the motion vector of the moving object can be suppressed, and it can be calculated as a vector that matches the background that does not move (here, the pan movement of the camera).
- FIG. 13 is a diagram illustrating an example of functional blocks in the motion vector calculation apparatus according to the present embodiment.
- the “motion vector calculation device” (1300) of this embodiment uses the global motion vector calculated in units of global macroblocks to determine the motion vector search range of macroblocks.
- This is a motion vector calculation device for calculating a motion vector, and further includes a “frame dividing unit” (1301) and a “predetermined divided global motion vector calculating unit” (1302).
- the "frame dividing unit” (1301) has a function of dividing a frame area into a plurality of unequal global macroblocks according to a predetermined rule. This division is performed in advance, for example. There is a method of dividing according to a predetermined mode. For example, assuming that an important subject is located in the central part of a normal moving image, the central part of the frame is a global macroblock that is subdivided, and the peripheral part of the frame is a global macroblock that has a larger area. Even if it is a division of,.
- the division processing in this frame division unit follows the unequal division pattern as described above, for example, a table indicating the coordinates of each global macroblock is prepared and used for the processing in this frame division unit. Method. In other words, as an example of the “predetermined rule”, there is a coordinate table that holds coordinate information indicating the division pattern.
- the frame may be divided into global macroblocks that are broadly divided in the horizontal direction.
- a global macro block that mainly includes catchers, batters, and pitchers according to the typical scene composition.
- information specifying the type of moving image to be encoded may be acquired by selecting a mode by the user, and the division pattern as described above may be selected.
- a certain record holds a motion vector history for each block of a predetermined frame (the past 10 frames, etc.), and based on the history, a region with a large amount of motion is subdivided and the amount of motion is small. Even if you select a rule that divides the area into large areas, it's okay. Similarly, a rule such as using a history and combining similar motion vectors into a new global motion vector may be used.
- the “predetermined global motion vector calculation unit” (1302) has a function of calculating the global motion vector of the global macroblock obtained by the division.
- This global motion betaton calculation process can be calculated by the same process as the conventional global motion vector calculation process using the correlation coefficient, except that the shape of the global motion betatone is not uniform. Therefore, the explanation is omitted.
- FIG. 15 shows the motion vector calculation of this embodiment. It is the schematic showing an example of the hardware constitutions in an output device. An example of the operation of each hardware component in the global motion vector calculation process of this embodiment will be described using this figure.
- the functions of the frame dividing unit and the pre-divided global motion vector detecting unit in the motion vector calculating apparatus of the present embodiment are the “calculator” (1501) for calculating data, and therefore “Main memory” (1502) for storing the data of the image, “image input means” (1503) for inputting the image to be subjected to high-efficiency encoding, and “video memory” (1504) for storing the input image ), And a storage device (1506) that stores and stores a coordinate table indicating an uneven division pattern of the global macroblock. These are connected to each other through a data communication path such as a system bus (1505) to transmit and receive information.
- a data communication path such as a system bus (1505) to transmit and receive information.
- an image to be subjected to high-efficiency encoding is input from the “image input unit”.
- the input image here may be various as in the case of the first embodiment, such as analog data encoded video data.
- the images constituting the moving image input are transferred to a predetermined memory address of the video memory via the system bus, for example, input image 1 (first frame of moving image data) at memory address A, and memory address B.
- Input image 2 (second frame of moving image data), etc. are stored.
- the input image data is sequentially stored at a predetermined memory address of the "main memory”, and DCT conversion processing, quantization processing, and the like are performed by the arithmetic processing of the "calculator", so that the image (space) ) First image conversion using correlation is performed.
- an image inverse transform (inverse DCT transform, inverse quantization) process is performed to generate an inversely transformed image.
- global motion vector calculation processing is performed using the inversely transformed image and the input image or a plurality of input images.
- the global motion vector calculation processing is performed based on the coordinate information of the global macroblock 1 (after the predetermined division) stored in the memory address of the main memory. Done.
- the calculation process is similar to the conventional global motion vector calculation process, for example, using a correlation coefficient that is the sum of absolute values of the difference values of the corresponding pixels.
- the calculated global motion vector is stored in the memory address 11 of the main memory. For other global macroblocks 2, 3,..., The global motion vector is calculated in the same way and stored in a predetermined memory address.
- FIG. 16 is a flowchart showing an example of a process flow in the motion vector calculation apparatus of the present embodiment.
- the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.
- the frame region is divided into a plurality of unequal global macroblocks according to a predetermined rule (step S1601), and the global motion vector of the global macroblock obtained by the division is calculated (step S1601). S1602).
- a motion vector search range of the macroblock is determined based on the calculated global motion vector (step S1603), and a motion vector is calculated using the search range (step S1604).
- Embodiment 3 Overview of Embodiment 3 [0049] This embodiment is based on Embodiment 2, and holds a plurality of rules, and selectively applies the rules to make the frames uneven. It is further characterized by the ability to divide into global macroblocks.
- FIG. 17 is a diagram illustrating an example of functional blocks in the motion vector calculation apparatus according to the present embodiment.
- the “motion vector calculation device” (1700) of the present embodiment is based on the second embodiment, and the “frame dividing unit” (1701) and the “predetermined divided global motion vector calculation”. Part "(1702). Since these “frame dividing unit” and “predetermined divided global motion vector calculating unit” have already been described in the second embodiment, description thereof will be omitted.
- the feature point of the motion vector calculation apparatus of this embodiment is that it further includes a “rule holding unit” (1703) and a “selection unit” (1704).
- the “rule holding unit” (1703) has a function of holding a plurality of the predetermined rules.
- This rule holding unit can be realized by a magnetic recording device such as a hard disk drive, a non-volatile memory, or an optical medium such as a DVD disk and its reading drive.
- FIG. 18 is a conceptual diagram for explaining an example of the predetermined rule.
- the predetermined rule is, for example, a rule that specifies a division pattern of a global macroblock.
- a division pattern is set in association with the mode specified by the user, and based on information indicating the mode input from the interface, for example, a normally shot video
- the frame is divided based on the coordinate information indicated by pattern 1 or the like.
- pattern matching processing on the input image in FIG. 18, for example, a position where an important subject is present is detected, and the position indicated by the division pattern set according to the detection result is detected.
- the frame is divided based on the mark information.
- FIG. 19 is a diagram for explaining an example of another predetermined rule.
- the middle block vector motion described in the first embodiment is used. That is, as shown in (1) of FIG. 19, first, each middle motion vector is calculated for each middle block. Then, the vector values are compared between adjacent or discrete blocks, and the middle block having approximate vectors within a predetermined range, for example, 1, 2, 9, 10 is grouped as a global macroblock A. That ’s right.
- the predetermined rule in this example defines a range for the approximation of the vector, it is acceptable.
- the predetermined rule force as described above is selected in the next selection unit and applied as a frame division rule.
- the “selection unit” (1704) has a function of selecting one rule from a plurality of held rules and using it as a rule in the frame division unit.
- the selection criterion may be based on, for example, mode information selected and input by the user via the interface, or position information within the frame of an important subject detected by pattern matching.
- the selection by the selection unit is a configuration in which a division method is selected, such as whether or not to use global macroblock division using middle blocks. Also good.
- the above-mentioned information as a selection criterion may be acquired as an “attribute value” by the motion vector calculation apparatus of the present embodiment, and the selection may be performed based on the attribute value.
- FIG. 20 shows an example of functional blocks in a motion vector calculation apparatus having an “attribute value acquisition unit” that acquires this attribute value.
- “dynamic vector calculation device” (2000) includes “frame division unit” (2001), “global motion vector calculation unit after predetermined division” (2002), It has a “rule holding unit” (2003) and a “selecting unit” (2004). Furthermore, it has an “attribute value acquisition unit” (2005).
- the “attribute value acquisition unit” (2005) has a function of acquiring an attribute value of a moving image for which a motion vector is to be calculated.
- the “attribute value” is a value indicating the mode information selected and input by the user via the interface, a value indicating the position of the important subject in the frame detected by pattern matching, or the like. Glo using blocks For example, a value indicating whether or not to use division of one macroblock is used. Then, by selecting a rule according to the acquired attribute value, it becomes possible to divide the global macroblock according to the input image.
- FIG. 21 is a flowchart showing an example of the process flow in the motion vector calculation apparatus of the present embodiment.
- the steps shown below may be processing steps that constitute a program for controlling a computer recorded on a medium.
- an attribute value of a moving image for which a motion vector is to be calculated is acquired (step S2101), and one rule is selected from a plurality of previously held rules based on the attribute value. (Step S2102).
- the frame region is divided into a plurality of unequal global macroblocks according to the selected normal (step S2103), and the global motion vector of the global macroblock obtained by the division is calculated (step S2104). ).
- a motion vector search range of the macroblock is determined (step S2105), and a motion vector is calculated using the search range (step S2106).
- the motion vector calculation device selects a rule to be applied to the division of the frame region according to the attribute value of the image. I can do it. Therefore, it is possible to perform unequal division of global macroblocks that match the input image, and to determine a search range that more accurately corresponds to the movement of each macroblock included in each global macroblock.
- the power S can be calculated.
- FIG. 1 is a diagram for explaining another example of motion vector calculation in units of macroblocks using a global motion vector, which is a conventional technique.
- FIG. 2 A diagram for explaining the problems in the calculation of motion vectors in macroblock units using global motion vectors in the prior art
- FIG. 3 is a conceptual diagram for explaining an example of a motion vector calculation apparatus according to the first embodiment.
- FIG. 4 is a diagram illustrating an example of functional blocks in the motion vector calculation apparatus according to the first embodiment.
- FIG. 5 is a diagram for explaining an example of division in the dividing unit of the motion vector calculating apparatus according to the first embodiment.
- FIG. 7 is a diagram for explaining an example of a global motion vector calculation process in the global motion vector calculation unit of the motion vector calculation device according to the first embodiment.
- FIG. 8 is a diagram for explaining another example of the global motion betaton calculation process in the global motion vector calculation unit of the motion vector calculation apparatus according to the first embodiment.
- FIG. 9 is a diagram for explaining another example of the global motion vector calculation process in the global motion vector calculation unit of the motion vector calculation apparatus of the first embodiment.
- 10 Motion vector calculation of the first embodiment. Schematic diagram showing an example of the hardware configuration of the device
- FIG. 11 A flow chart showing an example of the processing flow in the motion vector calculation apparatus of the first embodiment.
- FIG. 17 A diagram showing an example of a functional block in the motion vector calculation device of the third embodiment. [18] For explaining an example of the predetermined rule held in the rule holding unit of the motion vector calculation device of the third embodiment.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Image Analysis (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/064,445 US20090268819A1 (en) | 2005-11-15 | 2006-01-17 | Motion vector calculation device and motion vector calculation method |
EP06711813A EP1950972A1 (en) | 2005-11-15 | 2006-01-17 | Motion vector calculation device and motion vector calculation method |
CN2006800003643A CN101080930B (zh) | 2005-11-15 | 2006-01-17 | 运动矢量计算装置及运动矢量计算方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005329770A JP2007142521A (ja) | 2005-11-15 | 2005-11-15 | 動きベクトル算出装置および動きベクトル算出方法 |
JP2005-329770 | 2005-11-15 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2007057986A1 true WO2007057986A1 (ja) | 2007-05-24 |
Family
ID=38048378
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2006/300533 WO2007057986A1 (ja) | 2005-11-15 | 2006-01-17 | 動きベクトル算出装置および動きベクトル算出方法 |
Country Status (7)
Country | Link |
---|---|
US (1) | US20090268819A1 (ja) |
EP (1) | EP1950972A1 (ja) |
JP (1) | JP2007142521A (ja) |
KR (1) | KR20070088322A (ja) |
CN (1) | CN101080930B (ja) |
TW (1) | TWI348870B (ja) |
WO (1) | WO2007057986A1 (ja) |
Families Citing this family (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8085849B1 (en) * | 2006-11-03 | 2011-12-27 | Keystream Corporation | Automated method and apparatus for estimating motion of an image segment using motion vectors from overlapping macroblocks |
CN101816180B (zh) * | 2007-08-06 | 2013-01-16 | 汤姆森特许公司 | 利用多个视图间参考画面的运动跳跃模式的方法和装置 |
JP5286805B2 (ja) * | 2008-01-31 | 2013-09-11 | 沖電気工業株式会社 | 動きベクトル検出装置及び方法、動画像符号化装置及び方法、並びに、動画像復号化装置及び方法 |
JP2011135230A (ja) * | 2009-12-22 | 2011-07-07 | Panasonic Corp | 画像符号化方法、画像符号化装置及び撮像システム |
US8837592B2 (en) | 2010-04-14 | 2014-09-16 | Mediatek Inc. | Method for performing local motion vector derivation during video coding of a coding unit, and associated apparatus |
US8681866B1 (en) | 2011-04-28 | 2014-03-25 | Google Inc. | Method and apparatus for encoding video by downsampling frame resolution |
US9106787B1 (en) | 2011-05-09 | 2015-08-11 | Google Inc. | Apparatus and method for media transmission bandwidth control using bandwidth estimation |
US8856624B1 (en) | 2011-10-27 | 2014-10-07 | Google Inc. | Method and apparatus for dynamically generating error correction |
US9185429B1 (en) | 2012-04-30 | 2015-11-10 | Google Inc. | Video encoding and decoding using un-equal error protection |
KR101939628B1 (ko) | 2012-05-30 | 2019-01-17 | 삼성전자주식회사 | 모션 검출 방법 및 모션 검출기 |
US9172740B1 (en) | 2013-01-15 | 2015-10-27 | Google Inc. | Adjustable buffer remote access |
US9311692B1 (en) | 2013-01-25 | 2016-04-12 | Google Inc. | Scalable buffer remote access |
US9225979B1 (en) | 2013-01-30 | 2015-12-29 | Google Inc. | Remote access encoding |
JP6918661B2 (ja) * | 2017-09-22 | 2021-08-11 | キヤノン株式会社 | 符号化装置、符号化方法及びプログラム |
CN108646931B (zh) * | 2018-03-21 | 2022-10-14 | 深圳市创梦天地科技有限公司 | 一种终端控制方法及终端 |
CN110995849B (zh) * | 2019-12-11 | 2022-04-19 | 鲁东大学 | 一种矢量数据位置信息的传输与存储方法 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH043595A (ja) * | 1990-04-19 | 1992-01-08 | Nec Corp | 動画像の動き情報検出装置および動画像の動き補償フレーム間予測符号化装置 |
JPH08149481A (ja) * | 1994-11-25 | 1996-06-07 | Mitsubishi Electric Corp | 映像信号符号化・復号化装置及び符号化・復号化方法 |
JPH10191347A (ja) * | 1996-12-27 | 1998-07-21 | Casio Comput Co Ltd | 動き検出装置、動き検出方法、及び、記憶媒体 |
WO2000005899A1 (en) * | 1998-07-22 | 2000-02-03 | Mitsubishi Denki Kabushiki Kaisha | Image encoding system |
JP2000278694A (ja) * | 1999-03-19 | 2000-10-06 | Canon Inc | 符号化装置、画像処理装置、画像処理システム、符号化方法、及び記憶媒体 |
JP2001169288A (ja) * | 1999-12-07 | 2001-06-22 | Nec Corp | 動きベクトル探索装置および方法 |
JP2004229150A (ja) * | 2003-01-27 | 2004-08-12 | Nec Corp | 動きベクトル探索方法および装置 |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR0154921B1 (ko) * | 1994-12-30 | 1998-11-16 | 배순훈 | 동영상 부호화 장치에 이용하기 위한 움직임 추정기 |
JPH09168152A (ja) * | 1995-10-26 | 1997-06-24 | Hyundai Electron Ind Co Ltd | 動画像処理における動きの推定装置及びその方法 |
EP1422928A3 (en) * | 2002-11-22 | 2009-03-11 | Panasonic Corporation | Motion compensated interpolation of digital video signals |
US8107535B2 (en) * | 2003-06-10 | 2012-01-31 | Rensselaer Polytechnic Institute (Rpi) | Method and apparatus for scalable motion vector coding |
-
2005
- 2005-11-15 JP JP2005329770A patent/JP2007142521A/ja active Pending
-
2006
- 2006-01-17 US US12/064,445 patent/US20090268819A1/en not_active Abandoned
- 2006-01-17 WO PCT/JP2006/300533 patent/WO2007057986A1/ja active Application Filing
- 2006-01-17 EP EP06711813A patent/EP1950972A1/en not_active Withdrawn
- 2006-01-17 CN CN2006800003643A patent/CN101080930B/zh not_active Expired - Fee Related
- 2006-01-17 KR KR20067026952A patent/KR20070088322A/ko not_active IP Right Cessation
- 2006-04-28 TW TW095115350A patent/TWI348870B/zh not_active IP Right Cessation
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH043595A (ja) * | 1990-04-19 | 1992-01-08 | Nec Corp | 動画像の動き情報検出装置および動画像の動き補償フレーム間予測符号化装置 |
JPH08149481A (ja) * | 1994-11-25 | 1996-06-07 | Mitsubishi Electric Corp | 映像信号符号化・復号化装置及び符号化・復号化方法 |
JPH10191347A (ja) * | 1996-12-27 | 1998-07-21 | Casio Comput Co Ltd | 動き検出装置、動き検出方法、及び、記憶媒体 |
WO2000005899A1 (en) * | 1998-07-22 | 2000-02-03 | Mitsubishi Denki Kabushiki Kaisha | Image encoding system |
JP2000278694A (ja) * | 1999-03-19 | 2000-10-06 | Canon Inc | 符号化装置、画像処理装置、画像処理システム、符号化方法、及び記憶媒体 |
JP2001169288A (ja) * | 1999-12-07 | 2001-06-22 | Nec Corp | 動きベクトル探索装置および方法 |
JP2004229150A (ja) * | 2003-01-27 | 2004-08-12 | Nec Corp | 動きベクトル探索方法および装置 |
Also Published As
Publication number | Publication date |
---|---|
TWI348870B (en) | 2011-09-11 |
TW200719727A (en) | 2007-05-16 |
CN101080930A (zh) | 2007-11-28 |
JP2007142521A (ja) | 2007-06-07 |
US20090268819A1 (en) | 2009-10-29 |
CN101080930B (zh) | 2010-06-16 |
EP1950972A1 (en) | 2008-07-30 |
KR20070088322A (ko) | 2007-08-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2007057986A1 (ja) | 動きベクトル算出装置および動きベクトル算出方法 | |
US9800889B2 (en) | Video decoding device, video decoding method, video decoding program, and video decoding integrated circuit | |
JP4528441B2 (ja) | ブロック整合法及び統合投射法を用いた階層的動き評価処理及び装置 | |
US10440383B2 (en) | Image predictive encoding and decoding system | |
US8300695B2 (en) | Image encoding apparatus and image encoding method | |
CN102036067B (zh) | 运动图像编码设备及其控制方法 | |
US20060083310A1 (en) | Adaptive overlapped block matching for accurate motion compensation | |
JP2015536092A (ja) | 標準に準拠した、モデルベースの映像符号化及び映像復号化 | |
US20070217515A1 (en) | Method for determining a search pattern for motion estimation | |
US11310524B2 (en) | Method and apparatus for determining motion vector of affine code block | |
CN110996099B (zh) | 一种视频编码方法、***及设备 | |
US20040218675A1 (en) | Method and apparatus for determining reference picture and block mode for fast motion estimation | |
US20160261869A1 (en) | Content-adaptive b-picture pattern video encoding | |
JP3979768B2 (ja) | デジタル映像分割方法及びその装置 | |
JP2008042424A (ja) | 画像マッチング装置及び画像マッチング方法 | |
CN110876082B (zh) | 一种视频帧的复杂度的确定方法及装置 | |
Jubran et al. | Sequence-level reference frames in video coding | |
US20090268822A1 (en) | Motion vector detection by stepwise search | |
US10075691B2 (en) | Multiview video coding method using non-referenced view video group | |
KR102245740B1 (ko) | 깊이 영상에서 신축을 이용한 움직임 추정 방법 및 장치 | |
US11956441B2 (en) | Identifying long term reference frame using scene detection and perceptual hashing | |
JP2008072608A (ja) | 画像符号化装置及び画像符号化方法 | |
CN107431821A (zh) | 高效低复杂度视频压缩 | |
JP2006108932A (ja) | 複数参照画像を用いた動画像符号化装置、動画像復号装置、動画像符号化方法、動画像復号方法、動画像符号化プログラム及び動画像復号プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200680000364.3 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020067026952 Country of ref document: KR |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2006711813 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWP | Wipo information: published in national office |
Ref document number: 2006711813 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 12064445 Country of ref document: US |