US20190273944A1 - Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device - Google Patents

Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device Download PDF

Info

Publication number
US20190273944A1
US20190273944A1 US16/418,738 US201916418738A US2019273944A1 US 20190273944 A1 US20190273944 A1 US 20190273944A1 US 201916418738 A US201916418738 A US 201916418738A US 2019273944 A1 US2019273944 A1 US 2019273944A1
Authority
US
United States
Prior art keywords
motion vector
image
information
virtual
present
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/418,738
Other languages
English (en)
Inventor
Donggyu SIM
Seanae Park
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Golden Wave Partners Co Ltd
Original Assignee
Golden Wave Partners Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Golden Wave Partners Co Ltd filed Critical Golden Wave Partners Co Ltd
Assigned to KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION reassignment KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PARK, SEANAE, SIM, DONGGYU
Publication of US20190273944A1 publication Critical patent/US20190273944A1/en
Assigned to GOLDENWAVEPARTNERS CO., LTD. reassignment GOLDENWAVEPARTNERS CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION
Priority to US17/100,117 priority Critical patent/US11218725B2/en
Priority to US17/552,678 priority patent/US11711540B2/en
Priority to US18/332,136 priority patent/US20230328273A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/521—Processing of motion vectors for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/55—Motion estimation with spatial constraints, e.g. at image or region borders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding

Definitions

  • the present invention relates to an image encoding and decoding technique in a high-quality video compression method and apparatus for an omnidirectional security camera. And more particularly, the present invention relates to a method and apparatus for efficiently transmitting a differential motion vector, calculating an actual motion vector through a transmitted differential motion vector, and thereby performing motion compensation.
  • An effective motion vector transmission technique is an important technique for improving inter prediction performance.
  • An object of Some embodiments of the present invention is to effectively compress image data acquired via an omnidirectional security camera.
  • an apparatus and method for decoding an image adaptively sets a prediction candidate of a motion vector to an image using a virtual motion vector, and performs motion compensation after calculating an actual motion vector using the prediction candidate and a transmitted differential motion vector.
  • an embodiment of the present invention includes a parsing unit for parsing image information and camera information, an information acquisition unit for calculating and predicting image information using parsed information, a virtual coordinate determination unit for determining a virtual image coordinate system using image information, a motion vector prediction candidate setting unit for setting a motion vector prediction candidate in a virtual coordinate, a virtual motion vector calculation unit for calculating a virtual motion vector by using a predictive motion vector and a transmitted differential motion vector, a motion vector conversion unit for converting the virtual motion vector into an actual motion vector in an image, and a motion compensation performing unit for performing motion compensation using an actual motion vector.
  • the present invention determines a virtual coordinate by reflecting characteristics of an image, calculates a virtual motion vector using a predictive motion vector and a differential motion vector in a virtual coordinate, and then performs motion compensation.
  • FIG. 1 is a block diagram showing a configuration of a video decoding apparatus according to an embodiment of the present invention.
  • FIG. 2 illustrates a position of a neighboring block to be used as a candidate of a predictive motion vector in a motion vector prediction according to an embodiment of the present invention.
  • FIG. 3 is an embodiment in which there is no candidate of a predictive motion vector according to an embodiment of the present invention.
  • FIG. 4 illustrates a relationship between a virtual coordinate of a predictive motion vector and an actual image coordinate according to an embodiment of the present invention.
  • FIG. 5 illustrates a method of performing inter prediction in an embodiment of the present invention.
  • FIG. 6 illustrates a method of performing inter prediction in an embodiment of the present invention.
  • FIG. 7 illustrates a process of calculating a motion vector to perform motion compensation in an embodiment of the present invention.
  • FIG. 8 is a diagram for explaining the concept of virtual coordinates in an embodiment of the present invention.
  • FIG. 9 illustrates various types of omnidirectional projection in an embodiment of the present invention.
  • FIG. 10 illustrates a method of constructing a frame using a projected image in an embodiment of the present invention.
  • FIG. 11 illustrates a method of constructing a frame using a projected image in an embodiment of the present invention.
  • step or “step of ⁇ ” used in the present specification does not imply a step for ⁇ .
  • first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.
  • each component shown in the embodiments of the present invention is shown independently to represent different characteristic functions, and it does not mean that each component is composed of separate hardware or one software constituent unit. That is, each constituent unit is described separately for convenience of explanation, and at least two constituent units of constituent units may be combined to form one constituent unit or one constituent unit may be divided into a plurality of constituent units to perform a function.
  • the integrated embodiments and the separate embodiments of each of these components are also included in the scope of the present invention without departing from the essence of the present invention.
  • the video decoding apparatus may be a device included in the server terminal such as a personal security camera, a private security system, a military security camera, a military security system, a personal computer (PC), a notebook computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, and a service server.
  • the video decoding apparatus may be various devices including a user terminal such as various devices, a communication device such as a wired/wireless communication network, Communication modem to perform communication etc., various programs for inter-prediction or intra-prediction or for decoding an image, a memory for storing data, and a microprocessor for calculating and controlling by executing a program.
  • an image encoded into a bitstream by an encoder may be transmitted in real time or in non-real time via a wired or wireless communication network such as the internet, a local area wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, or via a cable, Universal Serial Bus (USB), and the like to an image decoding apparatus.
  • the encoded image may be decoded and restored into an image, and then reproduced.
  • a moving picture may be composed of a series of pictures, and each picture may be divided into a coding unit such as a block.
  • a coding unit such as a block.
  • FIG. 7 illustrates a process for performing motion compensation according to an embodiment of the present invention.
  • the decoder parses information of an image acquisition camera and image information from the bitstream transmitted from the encoder ( 701 ).
  • the information may be transmitted in a sequence unit, in an SEI message unit, or in an image group or a single image unit.
  • the information of the camera included in the bitstream may include the number of cameras that acquire an image at the same time, a position of the camera, an angle of the camera, a type of the camera, and a resolution of the camera.
  • the image information may include a resolution, a size, bit-depth, a projection shape, a preprocessing type, related coefficient information, and virtual coordinate-related information for the image acquired through the camera.
  • all of the information may be transmitted. Only a part of the information may be transmitted and the other part of the information may be calculated or derived by the decoder. In addition to the above-mentioned information, information required by the decoder may be transmitted together.
  • the decoder obtains information for decoding from the transmitted and parsed information ( 702 ).
  • the transmitted information may be directly used as information for decoding, or the information for decoding may be derived or calculated using the transmitted information.
  • information which is related to whether a motion vector of a block decoded at the boundary of the image opposite to the boundary block of the image described in FIGS. 3 and 4 is to be included in the candidate group when the predictive motion vector group is determined and whether the embodiments in which a reference block illustrated in FIG. 6 is divided by a picture boundary are applied, may be information transmitted or acquired through the corresponding image information. The divided blocks may exist at a boundary different from each other.
  • the decoder determines a virtual coordinate based on the acquired information ( 703 ).
  • step or “step of ⁇ ” used in the present specification does not imply a step for ⁇ .
  • first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.
  • each component shown in the embodiments of the present invention is shown independently to represent different characteristic functions, and it does not mean that each component is composed of separate hardware or one software constituent unit. That is, each constituent unit is described separately for convenience of explanation, and at least two constituent units of constituent units may be combined to form one constituent unit or one constituent unit may be divided into a plurality of constituent units to perform a function.
  • the integrated embodiments and the separate embodiments of each of these components are also included in the scope of the present invention without departing from the essence of the present invention.
  • the video decoding apparatus may be a device included in the server terminal such as a personal security camera, a private security system, a military security camera, a military security system, a personal computer (PC), a notebook computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, and a service server.
  • the video decoding apparatus may be various devices including a user terminal such as various devices, a communication device such as a wired/wireless communication network, Communication modem to perform communication etc., various programs for inter-prediction or intra-prediction or for decoding an image, a memory for storing data, and a microprocessor for calculating and controlling by executing a program.
  • an image encoded into a bitstream by an encoder may be transmitted in real time or in non-real time via a wired or wireless communication network such as the internet, a local area wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, or via a cable, Universal Serial Bus (USB), and the like to an image decoding apparatus.
  • the encoded image may be decoded and restored into an image, and then reproduced.
  • a moving picture may be composed of a series of pictures, and each picture may be divided into a coding unit such as a block.
  • a coding unit such as a block.
  • FIG. 1 illustrates a decoding apparatus for performing image decoding on a block-by-block basis using division information of a block according to an embodiment of the present invention.
  • the decoding apparatus may include at least one of an entropy decoding unit 110 , an inverse quantization unit 120 , an inverse transform unit 130 , an inter prediction unit 140 , an intra prediction unit 150 , an in-loop filter unit 160 , or a reconstructed image storage unit 170 .
  • the entropy decoding unit 110 decodes the input bitstream 100 and outputs decoded information such as syntax elements and quantized coefficients.
  • the output information includes various information for performing decoding and may include information on the image and image acquisition cameras.
  • the image information and image acquisition information may be transmitted in various forms and units and may be extracted from a bitstream or may be calculated or predicted using information extracted from a bitstream.
  • the inverse quantization unit 120 and the inverse transformation unit 130 receive the quantized coefficient, perform inverse-quantization and inverse-transform, and output a residual signal.
  • the inter prediction unit 140 calculates a motion vector using a differential motion vector extracted from the bitstream and a predictive motion vector, and generates a prediction signal by performing motion compensation using the reconstructed image stored in the reconstructed image storage unit 170 .
  • accurate prediction of the predictive motion vector may be a very important factor in efficient motion vector transmission because it can reduce the amount of differential motion vector.
  • the motion vector of the neighboring block of the current block to be decoded are used as the candidate of the predictive motion vector as shown in FIG. 2 .
  • FIG. 2 is an embodiment of the present invention.
  • the shape of a decoding block and the position relationship between a motion vector candidate and a current decoding block may vary according to an embodiment of the present invention. In FIG.
  • the shape of the decoding block may be a square, a non-square having an arbitrary size or a block having an arbitrary shape according to an embodiment.
  • the motion vector candidate may be determined in various forms according to the shape of the decoding block and the coordinate within the image.
  • the motion vector candidate may be representative of a motion vector of a neighboring block of a current block to be decoded, a motion vector of a co-located block of a reference image, a motion vector of a chrominance component corresponding to the decoding block, a motion vector of a neighboring block of a chrominance component corresponding to the decoding block, a motion vector resulting from scaling, based on a temporal position relation between a reference image and a decoding image, a motion vector of a neighboring block of the decoding block.
  • FIG. 3 illustrates a case where there is no motion vector of a spatial neighboring block according to the position relationship of the current decoding block in the image or the characteristics of the image when constructing the predictive motion vector candidate group.
  • a gray hatched block represents a block that does not exist or does not have a motion vector.
  • the presence or absence of a candidate for a spatial predictive motion vector of a neighboring block may vary.
  • FIG. 3 illustrates four embodiments. For example, if the current block to be decoded is positioned at the right edge of the image, the block with the hatched position cannot exist in the image and the motion vector cannot exist, as illustrated in FIG. 3A . In this case, a motion vector of a block in a different position may be used as illustrated in FIG. 4 . As illustrated in FIG.
  • the decoding block located at the right edge of the image does not have the decoding block at the hatched position, but the decoding block at the R position exists. Therefore, the motion vector of the decoding block at the R position may also be used as the predictive motion vector candidate.
  • the embodiment of FIG. 4 may be applied to the case of FIG. 3 (B), (C) and (D). This is a predictable embodiment by a person having ordinary knowledge, and a detailed description thereof will be omitted.
  • the motion vector may be calculated by obtaining the predictive motion vector through this process and adding the differential motion vector, which is transmitted through the bitstream, to the predictive motion vector.
  • the motion compensation of the inter prediction unit 140 is performed based on the obtained motion vector and the reference image.
  • the encoder may transmit the syntax including the related information to the decoder in order to use the motion vector of the block located away from the current decoding block rather than the motion vector of the neighboring block as the predictive motion vector.
  • This transmission my be available at various levels, such as a sequence unit, a frame unit, a slice unit, a tile unit.
  • sequence, frame, slice, and tile may be replaced with other term that denote a group of coding units.
  • Information whether to use the embodiment of the present invention and the related information may be directly transmitted according to the embodiment, or the decoder may calculate and estimate using other information transmitted from the encoder.
  • the embodiment of the present invention may be equally applied not only to the determination of the predictive motion vector candidate group but also to the motion vector merging (MV merge).
  • An merging candidate motion vector is required for motion vector merging in the encoder, and a predictive motion vector candidate group in the embodiment of the present invention may be used as a candidate group for motion vector merging. That is, in the decoder according to the embodiment of the present invention, when the current decoding block corresponds to the motion vector merging block using the same motion vector as the neighboring block, the current decoding block may be merged with one of the motion vector candidate blocks described with reference to FIG. 3 and FIG. 4 . The corresponding information may be obtained from the decoder through parsing and decoding of the bitstream.
  • the intra prediction unit 150 generates a prediction signal of a current block by performing spatial prediction using pixel values of a decoded neighboring block adjacent to the current block to be decoded.
  • the prediction signals output from the inter prediction unit 140 and the intra prediction unit 150 are summed with the residual signal, and the reconstructed image generated through the summing is transmitted to the in-loop filter unit 160 .
  • the reconstructed picture to which the filtering is applied in the in-loop filter unit 160 is stored in the reconstructed image storage unit 170 and may be used as a reference picture in the inter prediction unit 140 .
  • FIG. 5 illustrates an embodiment of motion compensation for a block applied in inter prediction.
  • FIG. 5A shows motion compensation for a P slice when only one reference image is used
  • FIG. 5B shows motion compensation for a B slice when two reference images are used.
  • the reference image may be one of frames which are decoded previously regardless of POC and stored in the reference image frame buffer.
  • the related information is transmitted from the encoder to the decoder together with index information and motion information (differential motion vector, merge index, scale information, etc.) and the block may be decoded using the same.
  • the reference block as shown in FIG. 5 is generally located inside the reference image.
  • the motion vector calculated for the reference between images is shown in the same form as FIG. 6 .
  • the reference block indicated by the motion vector may be referred. If the correlation between the left edge and the right edge of the image is high depending on the characteristics of the image, the encoding efficiency may be improved through the embodiment of the present invention.
  • the regions B and C of FIG. 6 are located on both edges at different positions in the image plane, but they are blocks located at the same position in the x-coordinate. When the both edges are connected to each other, the shape becomes as shown in FIG. 8A . That is, according to the embodiment of the present invention, it is possible to perform motion compensation in a form in which both edges having high correlation are connected.
  • the motion compensation may be performed in the form shown in FIG. 8B .
  • the embodiment of the present invention may be performed regardless of the number of reference images.
  • the embodiment of the present invention may be applied to a first case where one of the two reference blocks is referred to within the image and the other one is referred to at the image edge as shown in FIG. 6B or a second case where both reference blocks are referred to at the image edge.
  • the virtual coordinate is set by connecting the boundaries of the image each other.
  • the boundaries of the image are connected each other to form an annular shape.
  • the motion vector may appears beyond the boundary or across the boundary.
  • only one boundary may be connected to each other to have a virtual coordinate.
  • the virtual coordinate setting may adaptively appear according to the image. If the virtual coordinate are obtained, the PMV candidate setting 704 is possible according to the virtual coordinate.
  • the motion vector in the virtual coordinate 705 is calculated through the predictive motion vector and the transmitted differential motion vector. Then, a virtual motion vector is calculated as a motion vector in a plane image ( 706 ), and then a reference region determination and compensation is performed using the corresponding motion vector ( 707 ). If the virtual coordinate and the actual coordinate are the same, it may be performed without the virtual coordinate setting step. For the convenience of the embodiment, the motion compensation is performed by calculating MV through the virtual coordinate. However, It is possible to perform the motion vector calculation without the virtual coordinate according to the embodiment. That is, according to the embodiment, it is also possible to calculate, based on a method of converting coordinates using table mapping, the motion vector without the virtual coordinate and perform motion compensation in a reference image.
  • the table that maps the coordinates may include the virtual coordinate design.
  • the encoder may transmit the image information including the coordinate setting or coordinate mapping table for the virtual coordinate design.
  • the decoder may perform conversion between the actual coordinate and the virtual coordinate in the image using the coordinate mapping table transmitted from the encoder.
  • virtual coordinates or coordinates may be fixed by appointments between the encoder and the decoder and the fixed virtual coordinate value may be used.
  • the decoder performs motion vector calculation and motion compensation using only predetermined virtual coordinate.
  • the encoder may transmit information indicating the corresponding virtual coordinate to the decoder, or the decoder may obtain information relating to the virtual coordinate by predicting based on the decoded image.
  • FIG. 9 is various embodiments in which an image of an omnidirectional camera is projected.
  • FIG. 9A illustrates a projection onto a cube.
  • the number of sensors may be six so as to match the number of the respective projected planes, but fewer or more cases are possible.
  • images of six planes are generated.
  • one face may be composed of one frame as shown in FIG. 10A , or one frame may be constructed and transmitted using six images. At this time, the position of the six faces in FIG. 10B may vary depending on the embodiment.
  • the encoder since the corresponding information may be used when the virtual coordinate is set, the encoder must transmit the corresponding information to the decoder through the bitstream.
  • the decoder may obtain the information at ( 701 ) and ( 702 ) and use it at the time of virtual coordinate design. Of course, this information may be omitted if the information is predetermined by a promise of the encoder and the decoder.
  • the decoder may obtain the information through the promised matter even if it is not received from the encoder.
  • FIG. 10C corresponds to an embodiment constructing the projected image into one frame in case that the image is projected onto a figure having 12 faces.
  • the embodiment relate to a method of projecting am image or images obtained by a camera having a plurality of sensors at the same time and constructing one frame for convenience of compression and transmission.
  • the method has various forms according to the number of camera sensors and the projection type, and may vary depending on the embodiment.
  • FIG. 11 shows another embodiment relating to a projection type and a method of constructing a frame.
  • the black shaded portion is the portion where the acquired image is projected and the actual image data exists
  • the white portion is the portion where the image data does not exist.
  • the data may not exist in a form filled with a general rectangular frame.
  • the encoder must transmit the corresponding information to the decoder.
  • a white portion may be padded to form a rectangular frame, and then the frame may be encoded/decoded.
  • only image data may be encoded/decoded without padding.
  • the encoder/decoder needs to know and use the related information.
  • the related information may be transmitted from the encoder to the decoder, or the related information may be determined by the promise of the encoder and the decoder.
  • the present invention may be used in manufacturers such as broadcasting equipment manufacturing, terminal manufacturing, and industries related to original technology in video encoding/decoding related industries.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
US16/418,738 2016-11-22 2019-05-21 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device Abandoned US20190273944A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/100,117 US11218725B2 (en) 2016-11-22 2020-11-20 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US17/552,678 US11711540B2 (en) 2016-11-22 2021-12-16 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US18/332,136 US20230328273A1 (en) 2016-11-22 2023-06-09 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
KR1020160155541A KR20180057789A (ko) 2016-11-22 2016-11-22 전방향 카메라에서 효과적인 차분 움직임 백터 전송방법을 이용한 비디오 코딩 방법 및 방법 및 장치
KR10-2016-0155541 2016-11-22
PCT/KR2016/013592 WO2018097351A1 (ko) 2016-11-22 2016-11-24 전방향 카메라에서 효과적인 차분 움직임 백터 전송방법을 이용한 비디오 코딩 방법 및 방법 및 장치

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2016/013592 Continuation WO2018097351A1 (ko) 2016-11-22 2016-11-24 전방향 카메라에서 효과적인 차분 움직임 백터 전송방법을 이용한 비디오 코딩 방법 및 방법 및 장치

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/100,117 Continuation US11218725B2 (en) 2016-11-22 2020-11-20 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device

Publications (1)

Publication Number Publication Date
US20190273944A1 true US20190273944A1 (en) 2019-09-05

Family

ID=62195530

Family Applications (4)

Application Number Title Priority Date Filing Date
US16/418,738 Abandoned US20190273944A1 (en) 2016-11-22 2019-05-21 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US17/100,117 Active US11218725B2 (en) 2016-11-22 2020-11-20 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US17/552,678 Active US11711540B2 (en) 2016-11-22 2021-12-16 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US18/332,136 Pending US20230328273A1 (en) 2016-11-22 2023-06-09 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device

Family Applications After (3)

Application Number Title Priority Date Filing Date
US17/100,117 Active US11218725B2 (en) 2016-11-22 2020-11-20 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US17/552,678 Active US11711540B2 (en) 2016-11-22 2021-12-16 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
US18/332,136 Pending US20230328273A1 (en) 2016-11-22 2023-06-09 Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device

Country Status (4)

Country Link
US (4) US20190273944A1 (ko)
KR (1) KR20180057789A (ko)
CN (1) CN109997362A (ko)
WO (1) WO2018097351A1 (ko)

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7627194B2 (en) * 2004-08-13 2009-12-01 Samsung Electronics Co., Ltd. Method and device for making virtual image region for motion estimation and compensation of panorama image
KR100677142B1 (ko) * 2004-08-13 2007-02-02 경희대학교 산학협력단 파노라마 영상의 움직임 추정 및 보상
KR100688383B1 (ko) * 2004-08-13 2007-03-02 경희대학교 산학협력단 파노라마 영상의 움직임 추정 및 보상
US7623682B2 (en) * 2004-08-13 2009-11-24 Samsung Electronics Co., Ltd. Method and device for motion estimation and compensation for panorama image
KR100966569B1 (ko) 2006-03-30 2010-06-30 엘지전자 주식회사 비디오 신호를 디코딩/인코딩하기 위한 방법 및 장치
KR20080008944A (ko) * 2006-07-21 2008-01-24 엘지전자 주식회사 비디오 신호 디코딩 방법 및 장치
KR20080015760A (ko) * 2006-08-16 2008-02-20 엘지전자 주식회사 비디오 신호 디코딩 방법 및 장치
KR101377527B1 (ko) * 2008-10-14 2014-03-25 에스케이 텔레콤주식회사 복수 개의 참조 픽처의 움직임 벡터 부호화/복호화 방법 및장치와 그를 이용한 영상 부호화/복호화 장치 및 방법
WO2010053332A2 (ko) * 2008-11-10 2010-05-14 엘지전자 주식회사 시점간 예측을 이용한 비디오 신호 처리 방법 및 장치
KR102005088B1 (ko) * 2011-03-21 2019-10-01 엘지전자 주식회사 움직임 벡터 예측자 선택 방법 및 이를 이용하는 장치
WO2012140821A1 (ja) * 2011-04-12 2012-10-18 パナソニック株式会社 動画像符号化方法、動画像符号化装置、動画像復号化方法、動画像復号化装置、および動画像符号化復号化装置
KR101646581B1 (ko) * 2012-02-01 2016-08-09 한국전자통신연구원 재구성 가능한 코덱에서 기능 유닛에 부가정보를 제공하는 장치 및 방법
US20130195185A1 (en) 2012-02-01 2013-08-01 Industry-University Cooperation Foundation Hanyang University Apparatus and method for providing additional information to functional unit in reconfigurable codec
CN104813662A (zh) * 2012-09-28 2015-07-29 诺基亚技术有限公司 用于视频编码和解码的装置、方法和计算机程序
CN104155765B (zh) 2013-05-15 2019-09-13 北京三星通信技术研究有限公司 在拼接式集成成像显示器中校正三维图像的方法和设备
CN105531999B (zh) * 2013-07-09 2019-08-09 诺基亚技术有限公司 涉及用于信号传输运动信息的语法的视频编码方法及装置

Also Published As

Publication number Publication date
US20230328273A1 (en) 2023-10-12
KR20180057789A (ko) 2018-05-31
US20210076064A1 (en) 2021-03-11
CN109997362A (zh) 2019-07-09
US11711540B2 (en) 2023-07-25
US11218725B2 (en) 2022-01-04
US20220109869A1 (en) 2022-04-07
WO2018097351A1 (ko) 2018-05-31

Similar Documents

Publication Publication Date Title
US11438506B2 (en) Method and apparatus for reconstructing 360-degree image according to projection format
CN110115037B (zh) 球面投影运动估计/补偿和模式决策
US20180131960A1 (en) Video coding method, video decoding method, video coding apparatus, and video decoding apparatus
KR101808327B1 (ko) 영상 코덱에서 패딩을 이용한 영상 부/복호화 방법 및 장치
CN110268716B (zh) 由球面投影处理等量矩形对象数据以补偿畸变
US12010308B2 (en) Method and device for transmitting block division information in image codec for security camera
US11457240B2 (en) Encoder, decoder, encoding method, and decoding method
US20230199189A1 (en) Image coding method and device on basis of wide-angle intra prediction and transform
CN110741638A (zh) 使用残差块能量分布的运动矢量代码化
CN112673627A (zh) 在图像编码***中使用仿射合并候选列表的基于仿射运动预测的图像解码方法和装置
US20210144365A1 (en) Method and apparatus of reference sample interpolation for bidirectional intra prediction
US11218725B2 (en) Method for encoding video using effective differential motion vector transmission method in omnidirectional camera, and method and device
KR102469039B1 (ko) 전방향 카메라에서 효과적인 영상 패딩 방법 및 장치
KR102646642B1 (ko) 확장된 Intra Block Copy 방법을 통한 비디오 복호화 방법 및 장치
KR102235314B1 (ko) 영상 코덱에서 패딩을 이용한 영상 부/복호화 방법 및 장치
KR102377482B1 (ko) 효과적인 영상 부/복호화를 위한 인루프 필터 정보 전송 방법 및 장치
CN116800985A (zh) 编解码方法和装置
CN116800984A (zh) 编解码方法和装置
KR20170113947A (ko) 글로벌 움직임 보상 방법 및 장치

Legal Events

Date Code Title Description
AS Assignment

Owner name: KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATI

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SIM, DONGGYU;PARK, SEANAE;REEL/FRAME:049550/0580

Effective date: 20190521

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: GOLDENWAVEPARTNERS CO., LTD., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KWANGWOON UNIVERSITY INDUSTRY-ACADEMIC COLLABORATION FOUNDATION;REEL/FRAME:050818/0571

Effective date: 20190726

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION