US10235795B2 - Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof - Google Patents
Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof Download PDFInfo
- Publication number
- US10235795B2 US10235795B2 US15/599,838 US201715599838A US10235795B2 US 10235795 B2 US10235795 B2 US 10235795B2 US 201715599838 A US201715599838 A US 201715599838A US 10235795 B2 US10235795 B2 US 10235795B2
- Authority
- US
- United States
- Prior art keywords
- pixel
- line
- texture image
- value
- index value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012545 processing Methods 0.000 title claims description 45
- 230000006835 compression Effects 0.000 claims abstract description 64
- 238000007906 compression Methods 0.000 claims abstract description 64
- 238000013507 mapping Methods 0.000 claims description 18
- 238000010586 diagram Methods 0.000 description 15
- 230000005540 biological transmission Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009429 electrical wiring Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
- G06T7/41—Analysis of texture based on statistical description of texture
- G06T7/44—Analysis of texture based on statistical description of texture using image operators, e.g. filters, edge density metrics or local histograms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/529—Depth or shape recovery from texture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/261—Image signal generators with monoscopic-to-stereoscopic image conversion
Definitions
- the invention generally relates to image processing, and more particularly, to an image data processing system for processing a texture image of a panoramic video and related methods of compressing a texture image and methods of generating a 360 degree panoramic video thereof.
- a panorama image where a plurality of images may be combined or stitched together to increase the field of view (FOV) without compromising resolution is an image with an unusually large field of view, an exaggerated aspect ratio, or both.
- a panorama image sometimes also called simply a “panorama”, can provide a 360 degree view of a scene.
- a 360 degree panoramic video is a set of video collections with 360 degree view in both horizontal and vertical directions, and a viewer's viewing angle is in a central perspective so that an active viewing can be provided to the viewers in response to the viewer perspective viewpoint and viewing angle to achieve a goal for where to see where it is viewed.
- different texture images can be constructed primarily by three models, such as a sphere model, a cylindrical model, or a cube model.
- the sphere model is often used to construct different texture images because a surface area of the sphere model is smaller than that of the cylindrical model and the cube model.
- the texture image is a digital image with a rectangular shape and has a coordinate space on which coordinates are labeled as (U,V).
- the texture image can be assigned to a set of coordinates (U, V) according to its location.
- a texture image mapping when a 3D image model is rendered to generate a target image, a texture is mapped onto the surface of the model being used.
- the spherical texture image utilized by the prior art is a specially processed image, which is generated by capturing a plurality of images by multiple video cameras at the same time, for example, by a plurality of cameras simultaneously photographing a plurality of pictures of a room in various angles, dynamically recording captured images in all directions, stitching them through the late professional stitching software to obtain a stitched image, and specially processing the stitched image to generate the spherical texture image suitable for applying to the surface of the sphere.
- the flat video is sequentially transmitted by one of frames of a two-dimensional image, and each frame of the transmitted two-dimensional image is then sequentially attached to the sphere model frame by frame, wherein the texture image of each frame transmitted within the video stream is used as the texture image of the sphere model to perform a conversion operation from the two-dimensional image to a three-dimensional space, and render scene changes of the three-dimensional space in real-time to form dynamic effects of the three-dimensional space, so as to generate a 360 degree panoramic video and generate a playback effect for playing a 360 degree panoramic video.
- texture image mapping often consumes a large amount of transmission bandwidth.
- texture images will be first compressed to reduce the required bandwidth.
- a rectangular image is typically used as the texture image of the sphere model.
- the texture image being used may be an image with an aspect ratio of 2:1, in which its width is being mapped to a circumference of a sphere at the equator and its height is being mapped to a length of connecting the longitude line at the North pole and the longitude line at the South pole on the sphere model.
- color values are sampled within the width range of the rectangular texture image on 0 degree latitude at the equator of the sphere, and color values are also sampled within the width range of the rectangular texture image on 90 degrees latitude at the North and South poles of the sphere (i.e., a (circle) ring of the 0 degree latitude and a (circle) ring of 90 degrees latitude are mapped into the same area size of the texture image).
- an area of the sampled area on the texture image corresponding to a viewport which is located close to the equator of the sphere is smaller than the area of the sampled area on the texture image corresponding to a viewport which is located close to the South pole or North pole of the sphere, so that color values of points located close to the South pole or North pole of the sphere may be mapped to the color values within the texture image width range on the texture image.
- a method of compressing a texture image for use in generating a 360 degree panoramic video comprises the steps of: receiving an original texture image for a sphere model, wherein the original texture image is an image with a rectangular shape and includes a plurality of pixel lines and each of the pixels lines has a corresponding spherical position on the sphere model; determining a corresponding compression ratio of each of the pixel lines according to the corresponding spherical position of each of the pixel lines; and compressing the pixel lines with the compression ratios corresponding thereto to generate a compressed texture image with a non-rectangular shape, wherein the compressed texture image is further being mapped to the sphere model to generate the 360 degree panoramic video.
- an image data processing system at least comprises a storage unit and a processor.
- the storage unit is configured to store at least one original texture image for a sphere model, wherein the original texture image is an image with a rectangular shape and includes a plurality of pixel lines and each of the pixels lines has a corresponding spherical position on the sphere model.
- the processor is coupled to the storage unit and configured to receive the original texture image, determine a corresponding compression ratio of each of the pixel lines according to the corresponding spherical position of each of the pixel lines, and compress the pixel lines with the compression ratios corresponding thereto to generate a compressed texture image with a non-rectangular shape and store the compressed texture image into the storage unit.
- a method of generating a 360 degree panoramic video comprising the steps of: receiving a compressed texture image, wherein the compressed texture image has a plurality of pixel lines i, each of the pixel lines i comprising a first number of pixels and each of the pixel lines i having a plurality of parameter values, wherein the parameter values of each pixel line i comprise the number of pixels being sampled, a first index value, and a second index value for the pixel line i; performing a texture coordinate mapping according to the parameter values of each pixel line i to obtain mapping texture coordinates of the pixel line i; configuring attributes of texture coordinates of each subdivided triangle on a sphere model based on the parameter values for each pixel line i; and mapping each pixel line i of the compressed texture image to a corresponding position on the sphere model based on the parameter values and the mapping texture coordinates for the pixel line i to generate the 360 degree panoramic video.
- Methods may be practiced by the disclosed apparatuses or systems which are suitable firmware or hardware components capable of performing specific functions. Methods may also take the form of a program code embodied in a tangible media. When the program code is loaded into and executed by an electronic device, a processor, a computer or a machine, the electronic device, the processor, the computer or the machine becomes an apparatus for practicing the disclosed method.
- FIG. 1 is a diagram of an image data processing system in accordance with an embodiment of the invention
- FIG. 2 is a diagram of an original texture image in accordance with an embodiment of the invention.
- FIG. 3 is a diagram illustrating a mapping of pixel lines of the texture image to a sphere in accordance with an embodiment of the invention
- FIG. 4 is a flow chart of a method of compression a texture image in accordance with an embodiment of the invention.
- FIG. 5 is a diagram illustrating the results of grouping the original texture image in accordance with an embodiment of the invention.
- FIG. 6 is a flowchart of a method of compression a texture image in accordance with another embodiment of the invention.
- FIG. 7 is a diagram of a compressed texture image in accordance with an embodiment of the invention.
- FIGS. 8A, 8B, and 8C are diagrams illustrating the construction of a sphere model in accordance with an embodiment of the invention.
- FIG. 9 is a flow chart of a method of generating a 360-degree panoramic video in accordance with an embodiment of the invention.
- FIG. 10 is a flowchart of a playback method for a 360 degree panoramic video in accordance with an embodiment of the invention.
- Embodiments of the invention provide a method of compressing a texture image and associated image data processing system and method of generating a 360 degree panoramic video thereof, which can determine a compression ratio of each of pixel lines on the texture image according to the different circumferences of rings of latitude line of the respective pixels on the sphere to compress original texture image which is a texture image with a rectangular shape utilized in the prior art into a texture image with an non-rectangular shape (e.g., an approximate elliptical texture image) so as to achieve the isotropic of the resolution on the sphere in all orientations, thus effectively solving the problem of excessive redundancy information sampled at the North and South poles of the texture image in the prior art.
- FIG. 1 is a diagram of an image data processing system in accordance with an embodiment of the invention.
- the image data processing system 100 can be a desktop computer, an All-In-One (AIO) computer, a notebook, a notebook with a touch panel, or a portable device or a handheld device (e.g., a PDA, a smartphone, a tablet computer, a mobile phone, an Mobile internet device (MID), a laptop computer, a car computer, a digital camera, a multimedia player or a game device, or any other type of mobile computational device) capable of processing image or video data or can be provided by more than one device.
- AIO All-In-One
- a notebook e.g., a notebook, a notebook with a touch panel
- a portable device or a handheld device e.g., a PDA, a smartphone, a tablet computer, a mobile phone, an Mobile internet device (MID), a laptop computer, a car computer, a digital camera, a multimedia player or a game
- the image data processing system 100 can also be implemented as multiple chips or a single ship such as a system on chip (SOC) or a mobile processor disposed in a mobile device.
- the image data processing system 100 comprises at least one of a processor 110 , an interface 120 , a graphics processing unit (GPU) 130 , a storage unit 140 and a display device 150 .
- the processor 110 , the GPU 130 , the storage unit 140 and the display device 150 can be coupled to each other through the interface 120 .
- the interface 120 may be a system bus that couples a variety of system components such as the GPU 130 , the storage unit 140 and the display device 150 to the processor 110 .
- the processor 110 may be a Central Processing Unit (CPU), a general-purpose processor, a Micro-Control Unit (MCU), a Digital Signal Processor (DSP), or any equivalent circuitry for providing the functionalities of data analysis, processing and computing, but the disclosure is not limited thereto.
- the storage unit 140 may include a volatile memory 141 and a non-volatile memory 142 for storing related data, such as intermediate data generated during the calculation process and the execution result information and so on.
- the volatile memory 141 may be a dynamic random access memory (DRAM) or a static random access memory (SRAM), and the non-volatile memory 142 may be a Read-Only Memory (ROM), a flash memory, a hard disk, a solid-state disk (SSD), etc.
- the storage unit 140 may also store instruction sets and/or program code modules that can be executed by the processor 110 and the GPU 130 .
- program code modules contain routines, programs, objects, components, and so on.
- the program codes of the applications for use on the image data processing system 100 can be pre-stored in the non-volatile memory 142 .
- the processor 110 may load program codes of applications from the non-volatile memory 142 to the volatile memory 141 , and execute the program code of the applications.
- the processor 110 may also transmit the graphics data to the GPU 130 , and the GPU 130 may determine the graphics data to be rendered on the display device 150 .
- the volatile memory 141 and the non-volatile memory 142 are illustrated as a memory unit, they can be implemented separately as different memory units.
- the display device 150 can be a display circuit or hardware that can be coupled for controlling a display (not shown).
- the display device 150 may include either or both of a driving circuit and a display panel and can be disposed internal or external to the image data processing system 100 .
- the display device 150 can display related data, such as texts, figures, interfaces, and/or information such as displaying or presenting operation results.
- the display device 150 may be configured to present a screen of the results, such as a Liquid-Crystal Display (LCD). It should be understood that, in some embodiments, the display device 150 may be a screen integrated with a touch-sensitive device (not shown).
- LCD Liquid-Crystal Display
- the touch-sensitive device has a touch-sensitive surface comprising sensors in at least one dimension to detect contact and movement of at least one input tool, such as a pen/stylus or finger near or on the touch-sensitive surface. Accordingly, users are able to input commands or signals via the display device 150 .
- the image data processing system 100 may further include an image input interface (not shown) for receiving source images, such as original texture image data or video data.
- the image input interface can be equipped with image capture devices for capturing the source images.
- the image capture devices may comprise imaging sensors which may be a single sensor or a sensor array including a plurality of individual or separate sensor units.
- the image capture devices can be multiple cameras with a fisheye lens.
- the image input interface can receive the source images from external image capture devices.
- the image input interface can obtain source images (e.g., fisheye images) and provide the source images to the processor 110 .
- the processor 110 may further include an encoder (not shown) to obtain the source images and encode the source images to generate encoded image, such as encoded video bitstream, in any suitable media format compatible with current video standards such as the H.264 (MPEG-4 AVC) or H.265 standard.
- the encoder may be, for example, a standard image/video encoder or an image/encoder with pre-warping function, but the disclosure is not limited thereto.
- the encoder is the image/video encoder with pre-warping function, it may further perform a remapping or warping operation on the encoded video bitstream during encoding to remove distortion on the original source images or video data.
- the processor 110 may further include a decoder (not shown) to decode the encoded video bitstream to obtain the source images using a suitable media format compatible with the video standard used by the encoded video bitstream such as the H.264 (MPEG-4 AVC) or H.265 standard.
- a decoder not shown to decode the encoded video bitstream to obtain the source images using a suitable media format compatible with the video standard used by the encoded video bitstream such as the H.264 (MPEG-4 AVC) or H.265 standard.
- the image data processing system 100 may comprise other functional units, such as a keyboard/keypad, a mouse, a touchpad, or a communication unit, such as an Ethernet card/chipset, a Wireless-Fidelity (WiFi) card/chipset, a baseband chipset and a Radio Frequency (RF) chipset for cellular communications.
- a keyboard/keypad such as a keyboard/keypad, a mouse, a touchpad
- a communication unit such as an Ethernet card/chipset, a Wireless-Fidelity (WiFi) card/chipset, a baseband chipset and a Radio Frequency (RF) chipset for cellular communications.
- WiFi Wireless-Fidelity
- RF Radio Frequency
- the processor 110 may be used to load and execute a series of instructions and/or program codes from the storage unit 140 to control the operations of the storage unit 110 and the display device 150 to perform the method of compressing a texture image of the invention, including the steps of: receiving an original texture image for a sphere model, wherein the original texture image is an image with a rectangular shape and includes a plurality of pixel lines and each of the pixels lines has a corresponding spherical position on the sphere model; determining a compression ratio of each of the pixel lines according to the corresponding spherical position of each of the pixel lines; and compressing the pixel lines with the compression ratios corresponding thereto to generate a compressed texture image with a non-rectangular shape, wherein the compressed texture image is further being mapped to the sphere model to generate a 360 degree panoramic video, the details of which will be described more in the following paragraphs.
- each original texture image is a rectangular image (an image with a rectangular shape) with a height value and a width value and the original texture image can then be utilized to generate a 360 degree panoramic video, as shown in FIG. 2 .
- FIG. 2 is a diagram of an original texture image 200 in accordance with an embodiment of the invention.
- the original texture image 200 may be a rectangular image with an aspect ratio of 2:1, which is a specially processed texture image suitable for applying to the sphere surface and its width is being mapped to a circumference of a sphere at the equator and its height is being mapped to a length of connecting the longitude line at the North pole and the longitude line at the South pole on the sphere model.
- Each original texture image can be divided in the same way.
- the original texture image may be stored in the storage unit 140 , for example, stored in the volatile memory 141 .
- the volatile memory 141 may store corresponding information for each pixel of the original texture image 200 , such as a pixel value of each pixel and the pixel line and the corresponding pixel position in the pixel line on which the pixel located and so on.
- the original texture image 200 may be divided into a plurality of pixel lines by latitude and each pixel line has a corresponding spherical position on the sphere model.
- color values are sampled within the width range of a first pixel line of the original texture image 200 on 0 degree latitude at the equator of the sphere, and color values are also sampled within the width range of a second pixel line of the original texture image 200 on 90 degrees latitude at the North and South poles of the sphere (i.e., a ring of the 0 degree latitude and a ring of 90 degrees latitude are mapped into the same area size of the original texture image 200 ).
- the 360 degree panoramic video is a video set of video collections with 360 degree view in both horizontal and vertical directions, and the viewer's viewing angle is in a central perspective so that an active viewing can be provided to the viewers in response to the viewer perspective viewpoint and viewing angle to achieve a goal for where to see where it is viewed.
- Such immersive experience surrounded by the environment allows the viewers to experience an immersive experience personally.
- the original texture image is mapped to a sphere model to generate the 360 degree panoramic video.
- the processor 110 of the present invention can obtain the corresponding information for each pixel of the original texture image from the volatile memory 141 and determine a compression ratio of each pixel line on the texture image according to the different circumferences of rings of latitude line of the respective pixels on the sphere, so as to generate a compressed texture image.
- the original texture image which is a texture image with a rectangular shape can be compressed into a texture image with an non-rectangular shape (e.g., an approximate elliptical texture image) to effectively solve the problem of excessive redundancy information sampled at the North and South poles of the texture image in the prior art.
- FIG. 3 is a diagram illustrating a mapping of pixel lines of the texture image to a sphere in accordance with an embodiment of the invention.
- the width value of the original texture image is “width”, the height value of which is “height”, and each pixel line (denoted as i in FIG. 3 ) on the original texture image is mapped to a respective ring of latitude line on the sphere, wherein the innermost pixel line of the original texture image (denoted as j in FIG. 3 ) is mapped onto the ring of latitude line of the sphere at the equator.
- the processor 110 determines the number of pixels on each pixel line of the compressed texture image according to the circumference of ring of the latitude line, and the size of the ring of latitude line is linearly related to the height of the latitude line. For example, the size of the ring of latitude line is directly determined by an angle value of the ⁇ angle as shown in FIG. 3 . Therefore, by using the angle value of the angle ⁇ and the width value “width” of the original texture image, the processor 110 may calculate the number of pixels for the i-th pixel line of the compressed texture image, PixelOfLine[i], by the following formulas (1), (2), and (3):
- width and height represent the width and height values of the original texture image, respectively
- offset represents an offset set for preserving the information at the North and South poles, which value is ranged from 0 to 1 (e.g., the offset is set to 0.05), wherein the value of the offset being set determines the effect of compression.
- the processor 110 calculates a start index value StartPixel[i] and an end index value EndPixel[i] of effective pixels stored in the i-th pixel line of the compressed texture image by the following formulas (4) and (5), respectively, using the number of pixels for the i-th pixel line of the compressed texture image, PixelOfLine[i], calculated by the formula (3):
- StartPixel ⁇ [ i ] ( width - PixelOfLine ⁇ [ i ] ) 2 ; ⁇ ⁇ and ( 4 )
- EndPixel ⁇ [ i ] StartPixel ⁇ [ i ] + PixelOfLine ⁇ [ i ] - 1 , ( 5 )
- start index value StartPixel[i] and the end index value EndPixel[i] of the respective pixel line represent a pixel position of the starting pixel and a pixel position of the ending pixel within the corresponding pixel line on which the effective pixels are stored, respectively. That is, the start index value StartPixel [i] and the end index value EndPixel[i] represent a sampling range of the effective pixels for the i-th pixel line.
- the processor 110 calculates a compression ratio ratio[i] of each pixel line i by the following formula (6) using the number of pixels for the i-th pixel line of the compressed texture image, PixelOfLine[i], calculated by the formula (3) and the width value width:
- FIG. 4 is a flow chart of a method of compression a texture image in accordance with an embodiment of the invention.
- the method of compression the texture image of the invention may be applied to an image data processing system, such as the image data processing system 100 as shown in FIG. 1 and performed by the processor 110 .
- step S 402 the processor 110 receives an original texture image for a sphere model, wherein the original texture image (such as the original texture image 200 as shown in FIG. 2 ) is a rectangular image which is an image with a rectangular shape and includes a plurality of pixels and each pixel line has a corresponding spherical position on the sphere model.
- the original texture image such as the original texture image 200 as shown in FIG. 2
- the processor 110 receives an original texture image for a sphere model, wherein the original texture image (such as the original texture image 200 as shown in FIG. 2 ) is a rectangular image which is an image with a rectangular shape and includes a plurality of pixels and each pixel line has a corresponding spherical position on the sphere model.
- step S 404 the processor 110 determines a corresponding compression ratio of each of the pixel lines according to the corresponding spherical position of each of the pixel lines.
- the compression ratio indicates a difference amount of pixels between the original texture image and the compressed texture image, and the larger the compression ratio is, the greater the difference in the number of pixels is.
- the processor 110 may determine parameter values of each pixel line of the compressed texture image by the aforementioned formulas (1) to (6), such as the number of pixels PixelOfLine[i] for the i-th pixel line, the start index value StartPixel[i] and the end index value EndPixel[i] of the respective pixel line on the compressed texture image on which the effective pixels are stored, so as to determine the corresponding compression ratio ratio[i] of each pixel line accordingly.
- the original texture image may have a height value and a width value
- the sphere model may be divided into a plurality of rings of latitude line along the latitude line direction, wherein all the pixels at the same pixel line correspond to the location of a corresponding one of the rings of latitude line located on the sphere model.
- the processor 110 may determine a corresponding ring of latitude line among the rings of latitude line according to the corresponding spherical position of each pixel line, determine the number of pixels based on the height value, the width value, and a circumference of the corresponding ring of latitude line, and determine the corresponding compression ratio of each pixel line according to the number of pixels of each pixel line and the width value, so as to determine the corresponding compression ratio of each pixel line according to the corresponding spherical position of each of the pixel lines, wherein the corresponding compression ratio of each pixel line is inversely proportional to the number of pixels of each pixel line.
- the processor 110 may determine the number of pixels PixelOfLine[i] in the i-th pixel line of the compressed texture image by the formulas (1) to (3), obtain the start index value StartPixel[i] and the end index value EndPixel[i] for storing the effective pixels of the i-th pixel line in the original texture image by the formulas (4) and (5), and then determine the compression ratio of the pixel line i by the formula (6).
- the pixel lines comprise at least one first pixel line whose corresponding spherical position is close to one of two poles of the sphere model and at least one second pixel line whose corresponding spherical position is far away from the two poles of the sphere model, and the corresponding compression ratio of the first pixel line whose corresponding spherical position is close to one of two poles of the sphere model is larger than the corresponding compression ratio of the second pixel line whose corresponding spherical position is far away from the two poles of the sphere model.
- color values may be sampled within the width range of the first pixel line of the original texture image 200 on 0 degree latitude at the equator of the sphere, while color value sampling are performed only within a few portion of the width range of the second pixel line of the original texture image 200 on 90 degrees latitude at the North and South poles of the sphere.
- each pixel line of the original texture image has a corresponding pixel line in the compressed texture image, and the start index value and the end index value of each pixel line represent the starting pixel position and the ending pixel position for the effective pixels stored on the corresponding pixel line, respectively.
- each pixel line of the original texture image includes a plurality of pixels in the corresponding pixel line of the compressed texture image and each pixel has a respective pixel position, and the processor 110 may further determine a pixel value for the respective pixel position of each pixel in the corresponding pixel lines based on the start index value and the end index value of each pixel line and the respective pixel position of the pixel.
- step S 406 the processor 110 then compresses each pixel line with the compression ratio corresponding thereto to generate a compressed texture image with a non-rectangular shape, wherein the compressed texture image is further being mapped to the sphere model to generate the 360 degree panoramic video. For example, assuming that the original texture image has N pixel lines and the first pixel line has a first compression ratio, the second pixel line has a second compression ratio, and the third pixel line has a third compression ratio, then the processor 110 samples the first pixel line of the original texture image based on the first compression ratio and the corresponding parameter values to generate a first pixel line of the compressed texture image.
- the processor 110 samples the second and third pixel lines of the original texture image based on the second and third compression ratios and the corresponding parameter values to generate second and third pixel lines of the compressed texture image, respectively, and so on, until the N pixel lines of the original texture image are all sampled by their corresponding compression ratio to get the corresponding pixel lines of the compressed texture image.
- a compressed texture image with a non-rectangular shape will be generated and finally the new compressed texture image will further be mapped to the sphere model to generate the 360 degree panoramic video.
- the processor 110 sequentially traverses all pixel positions of the respective pixel lines of the original texture image from left to right and determines a pixel value of the j-th pixel position New[i][j] in the i-th pixel line according to the corresponding parameter value of each pixel line and the corresponding pixel position being traversed.
- the determination of the pixel value of the j-th pixel position New[i][j] in the i-th pixel line according to the corresponding parameter value of each pixel line and the corresponding pixel position being traversed may comprise the following steps: the processor 110 sets the pixel value of New[i][j] as a black pixel value when j ⁇ StartPixel[i] or j>EndPixel[i]; and the processor 110 performs a grouping procedure to divide the i-th pixel line in the original texture image into a plurality of groups based on the corresponding compression ratio of the i-th pixel line in the original texture image and sets the pixel value of New[i][j] as a pixel value of a specified pixel position in the (j-Start [i])-th group among the groups of the i-th pixel line in the original texture image when StartPixel[i] ⁇ j ⁇ EndPixel[i].
- the division of the groups is determined
- FIG. 5 is a diagram illustrating a result of grouping the original texture image in accordance with an embodiment of the invention.
- the i-th line 510 of the original texture image has a total of 100 pixels and the compression ratio of which is 5
- the i-th line of the original texture image may be divided into 20 groups by dividing every 5 pixels as a group and the i-th line 520 of the compressed texture image corresponding to the i-th line 510 of the original texture image may be generated based on the result of the grouping, wherein the pixel value of the pixel position StartPixel[i] of the i-th line 520 of the compressed texture image is set as the first pixel value of the zero group (group0 as shown in FIG.
- FIG. 6 is a flowchart of a method of compression a texture image in accordance with another embodiment of the invention.
- the method of compressing the texture image of the invention may be applied to an image data processing system, such as the image data processing system 100 as shown in FIG. 1 and performed by the processor 110 , for example.
- the processor 110 may sequentially traverse all the pixel positions of the pixel lines from left to right and determine the pixel value of the j-th pixel position New[i][j] in the i-th pixel line of the compressed texture image based on the corresponding parameter values and the corresponding pixel position being traversed of each respective pixel line in the original texture image.
- the processor 110 prepares to start compressing the original texture image, and thus, in step S 602 , creates a storage space with a size New[height][width] of the compressed texture image New.
- the processor 110 calculates the parameters of each pixel line of the compressed texture image, wherein the parameters at least include the number of pixels PixelOfLine[i], the start index value StartPixel[i], and the end index value EndPixel[i] for each pixel line i.
- the number of pixels PixelOfLine[i] of each pixel line i may be determined by the height value of each pixel line, the width value, and a circumference of the ring of latitude line corresponding to the pixel line i, as shown in formula (3).
- the start index value StartPixel[i] may be determined based on the number of pixels of each pixel line i and the width value
- the end index value EndPixel[i] may be calculated based on the number of pixels and the start index value of each pixel line i, as shown in formulas (4) and (5).
- the processor 110 After determining the corresponding parameter value for the pixel lines of the compressed texture image, the processor 110 starts traversing each of the pixel positions of each of the respective pixel lines from left to right, and determines the pixel value of the j-th pixel position New [i][j] of the i-th pixel line according to the corresponding parameter values of the respective pixel line and the respective pixel position being traversed. In step S 606 , the processor 110 determines whether or not all the pixel lines i have been traversed. If so, it indicates that all the pixels of the original texture image have been compressed, and the processor 110 then proceeds to step S 612 to store the compressed texture image New into a storage device (e.g., the storage unit 140 ).
- a storage device e.g., the storage unit 140
- the processor 110 After calculating the compression ratio ratio[i] corresponding to the current pixel line i, the processor 110 then determines whether all the pixel positions j have been traversed (step S 610 ). If all the pixel positions j of the current pixel line i have been traversed (Yes in step S 610 ), it indicates that the current pixel line i has been completely compressed and processed, so that the processor 110 returns to step S 606 to continue to determine whether there are other pixel lines i needed to be processed.
- step S 610 If the current pixel line i still has at least one pixel position j has not yet traversed (No in step S 610 ), it indicates that the current pixel line i still has an uncompressed pixel position j, and the processor 110 then performs steps S 614 to S 618 to determine the pixel value of each pixel position j in the corresponding pixel line according to the start index value StartPixel[i] and the end index value EndPixel[i] of each line i and the pixel position j of each pixel.
- step S 614 the processor 110 determines whether the value of the pixel position j is smaller than the value of the pixel position corresponding to the start index value StartPixel[i] or the value of the pixel position j is greater than the pixel value of the pixel position corresponding to the end index value EndPixel[i]. If it is determined that the condition of step S 614 is satisfied (i.e., “j ⁇ StartPixel[i] ⁇ j>EndPixel[i]”), the processor 110 performs step S 618 , and the processor 110 performs step S 616 if it is determined that the condition of step S 614 is not satisfied.
- step S 618 the processor 110 sets the pixel value of New[i][j] as the black pixel value.
- step S 616 the processor 110 performs a grouping procedure as shown in FIG.
- Steps S 614 , S 616 , and S 618 are repeated until all the pixel positions j of the pixel line i have been completely traversed.
- the width value width of the original texture image is set as 360 and the value of the number of pixels PixelOfLine[i] for the i-th pixel line of the compressed texture image calculated by the formula (3) is set as 180
- end index value EndPixel[i] and compression ratio ratio[i] can be calculated as 90, 269, and 2 by the formulas (4), (5) and (6), respectively.
- 360 pixels in the i-th line of the original texture image are grouped into 180 groups by dividing each two pixels as a group, wherein the number of pixels for the i-th pixel line of the compressed texture image corresponding to the i-th pixel line of the original texture image is 180 pixels and, in the i-th pixel line of the compressed texture image, the pixel value of the pixel position represented by the start index value of the i-th pixel line of the compressed texture image (i.e., the 90th pixel position) is set to be the first pixel value of the 0th group in the i-th line of the original texture image, the pixel value of the 91th pixel position is set to be the first pixel value of the first group in the i-th line of the original texture image, the pixel value of the 92th pixel position is set to be the first pixel value of the second group in the i-th line of the original texture image and so on.
- the pixel values for the pixel positions located before the 89th pixel position are all set to be the black pixel values, and the pixel values for the pixel positions located after the 270th pixel position are also set to the black pixel values.
- the number of effective pixels required to be stored in the i-th line of the compressed texture image is only half of the number of effective pixels within the i-th line of the original texture image, thus effectively reducing the number of pixels within the effective texture range.
- FIG. 7 is a diagram of a compressed texture image 700 in accordance with an embodiment of the invention, wherein the compressed texture image 700 represents the compression result of the original texture image 200 .
- the compression ratio for each pixel line is determined according to a size of the circumference for a ring of latitude line on the sphere corresponding to the corresponding spherical position on the sphere for a respective pixel line of the original texture image 200 , wherein the corresponding compression ratio of the first pixel line whose corresponding spherical position is close to one of two poles of the sphere model is greater than the corresponding compression ratio of the second pixel line whose corresponding spherical position is far away from the two poles of the sphere model, thereby compressing the originally used rectangular texture image into an approximately elliptical texture image.
- the number of pixels in the effective texture range for the compressed texture image 700 that is generated by compressing the original texture image 200 is about 68% of that of the original texture image 200 .
- the method of compressing the texture image of the invention can determine the number of pixels on each pixel line of a compressed texture image with a non-rectangular shape based on the circumference of the ring of latitude line corresponding thereto on the sphere, such that only a few necessary information is stored for points of the texture image which located near the North and South poles of the sphere model, thus effectively solving the problem that too much redundant information of the texture image may be obtained at the North and South poles of the sphere model in the prior art.
- Another embodiment of the present invention may further provide a method of generating a 360 degree panoramic video by using the aforementioned compressed texture image.
- the compressed texture image is mapped to a corresponding position of a sphere model to construct a sphere model using the compressed texture image with the non-rectangular shape to generate the 360 degree panoramic video for 360 degree panoramic video playback.
- FIGS. 8A, 8B , and 8 C are diagrams illustrating the construction of a sphere model in accordance with some embodiments of the invention.
- FIG. 8A is a schematic diagram of a coordinate system of the sphere model.
- FIG. 8B is a schematic diagram illustrating a viewing result of the spherical model of FIG. 8A viewed vertically from the positive direction of the y-axis, and
- FIG. 8C is a schematic diagram of vertices of subdivision triangles generated by the intersections of the latitude and longitude lines on the sphere model.
- the GPU 130 first divides the spheres into iRings pieces along the direction of latitude line and divides the spheres into iSectors pieces along the direction of longitude line, so that there are a total of iRings latitude lines and a total of iSectors longitude lines crossed each other on the sphere.
- parameters of the sphere model which includes the intersection coordinates (x, y, z) and the intersection texture coordinates (u, v), can be calculated using the following formula for the construction of the sphere model and the sampling of the texture image.
- PixelLineOfRing[i] represents a pixel line of the texture image corresponding to the i-th latitude line
- fRadius is the radius of the sphere
- frRing[i] is the radius of the i-th ring of latitude line, as shown in FIG. 8A and FIG. 8B .
- Each iRing intersects iSector to form a subdivision triangle whose vertex coordinates are the coordinates pVertices [i][j] of the intersection point, and the spherical model shown in FIG. 8C can be constructed using all of the intersection coordinates, and attributes of texture coordinates of the subdivided triangles on the sphere can be set based on the pre-calculated parameter values StartPixel[i] and PixelOfLine[i] of each respective pixel line on the compressed texture image.
- the processor 110 calculates a first angle ⁇ as shown in the FIG. 8A based on the number of divided longitude lines iSector and the number of divided latitude lines by the following formula (7):
- the processor 110 determines a pixel line PixelLineOfiRing[i] on the texture image corresponding to the i-th latitude line based on the height value height and the number of divided latitude lines iRings by the following formula (8), in which pixel values of pixel points on the i-th latitude line are sampled from this pixel line:
- PixelLineOfiRing ⁇ [ i ] i * height iRings . ( 8 )
- the processor 110 calculates a second angle ⁇ by the following formula (9) according to the pixel line PixelLineOfiRing[i] on the texture image corresponding to the i-th latitude and the height value height:
- the processor 110 further calculates a pixel index value (PixelIndexInLine[j]) within the pixel line on the texture image corresponding to the i-th latitude line for the intersection point pVertices[i][j] based on the start index value (StartPixel[i]) of the corresponding pixel line and the number of pixels (PixelOfLine[i]) for the pixel line on the texture image corresponding to the i-th latitude line by the following formula (14):
- PixelOfLine ⁇ [ PixelLineOfiRing ⁇ [ i ] ] ( iSectors - 1 ) represents that the number of effective pixels within this line are being divided into (iSectors-1) pieces;
- the processor 110 calculates the value u of the texture coordinates (u,v) based on the pixel index value (PixelIndexInLine[j]) within the pixel line on the texture image corresponding to the i-th latitude line for the intersection point pVertices[i][j] and the width value width by the following formula (15) and calculates the value v of the texture coordinates (u,v) based on the pixel line (PixelLineOfiRing [i]) on the texture image corresponding to the i-th latitude line for the intersection point pVertices[i][j] and the height value height by the following formula (16):
- the GPU 130 may determine a correspondence relationship between the spherical coordinates (x,y,z) of the respective intersection point calculated by the formulas (11) to (13) and the mapping texture coordinates (u,v) calculated by the formulas (15) to (16) and utilize the compressed texture image as the texture image for the sphere model to map each point of the compressed texture image into a corresponding position of the sphere model based on the determined correspondence relationship, so as to generate the 360 degree panoramic video.
- FIG. 9 is a flow chart of a method of generating a 360-degree panoramic video in accordance with an embodiment of the invention. Please refer to FIG. 1 , FIG. 7 , FIGS. 8A, 8B and 8C and FIG. 9 .
- the method of generating the 360-degree panoramic video of the invention may be applied to an image data processing system, such as the image data processing system 100 as shown in FIG. 1 and may be performed by the processor 110 or the GPU 130 , for example.
- image data processing system 100 of FIG. 1 is utilized here for explanation of the flow chart, which however, is not limited to only being applied in the image data processing system 100 .
- the processor 110 or the GPU 130 may obtain a non-rectangular compressed texture image (e.g., the compressed texture image 700 as shown in FIG. 7 ) from a storage unit (e.g., the storage unit 140 ) and map the compressed texture image to the sphere model to generate the 360 degree panoramic video.
- a non-rectangular compressed texture image e.g., the compressed texture image 700 as shown in FIG. 7
- a storage unit e.g., the storage unit 140
- the GPU 130 receives a compressed texture image, wherein the compressed texture image has a plurality of pixel lines i, each of the pixel lines i comprising a first number of pixels and each of the pixel lines i having a plurality of parameter values, wherein the parameter values of each pixel line i comprise the number of pixels being sampled, a first index value, and a second index value for the pixel line i.
- the compressed texture image is a non-rectangular texture image (e.g., the compressed texture image 700 as shown in FIG. 7 ), in which each pixel line i of the compressed texture image may have a different number of pixels and it may also have different storage position of the effective pixels.
- each pixel line i has the number of pixels being sampled, a start index value (i.e., a first index value) and an end index value (i.e., a second index value) corresponding thereto, wherein the start index value and the end index value of each pixel line represent the starting pixel position and the ending pixel position of the effective pixels stored in the corresponding pixel line, respectively. That is, the start index value and the end index value represent an effective pixel sampling range of the i-th pixel line.
- These parameter values may be recorded together in the storage unit 140 during generating the compressed texture image so that the processor 110 may later read the compressed texture image from the storage unit 140 to receive the compressed texture image and read the relevant records directly to obtain the aforementioned parameter values.
- the processor 110 may analyze the parameter values of the pixel lines of the compressed texture image received and filter out the pixels with a predetermined fixed pixel value (e.g., black pixel values) to obtain the aforementioned parameter values.
- step S 904 the processor 110 performs a texture coordinate mapping based on the parameter values of each pixel line i to obtain mapping texture coordinates of each pixel line i.
- step S 906 the processor 110 configures attributes of texture coordinates of each subdivided triangle on a sphere model based on the parameter values for each pixel line i.
- the processor 110 may calculate the value u of the respective mapping texture coordinates (u,v) based on the pixel index value PixelIndexInLine[j] within the pixel line on the texture image corresponding to the i-th latitude line for the intersection point pVertices[i][j] and the width value width and calculates the value v of the respective texture coordinates (u,v) based on the pixel line PixelLineOfiRing [i] on the texture image corresponding to the i-th latitude line for the intersection point pVertices[i][j] and the height value height according to aforementioned formulas (7)-(14).
- the processor 110 may obtain the mapping texture coordinates (u, v) by aforementioned formulas (15)-(16), but the invention is not limited thereto.
- step S 908 the GPU 130 maps each point of the compressed texture image to a corresponding position on the sphere model based on the parameter values and the mapping texture coordinates for each pixel line i to generate the 360 degree panoramic video.
- the GPU 130 may determine a correspondence relationship between the spherical coordinates (x, y, z) and the respective mapping texture coordinates (u,v) for each intersection point and utilize the compressed texture image as the texture image for the sphere model to map each point of the compressed texture image into a corresponding position of the sphere model based on the determined correspondence relationship, so as to generate the 360 degree panoramic video.
- FIG. 10 is a flowchart of a playback method for a 360 degree panoramic video in accordance with an embodiment of the invention.
- the playback method for the 360 degree panoramic video of the invention may be applied to an image data processing system, such as the image data processing system 100 as shown in FIG. 1 and may be performed by a GPU such as the GPU 130 , for example.
- image data processing system 100 of FIG. 1 is utilized here for explanation of the flow chart, which however, is not limited to only being applied in the image data processing system 100 .
- the storage unit 140 may further include a system memory (not shown) and the system memory stores a compressed texture image with a non-rectangular shape generated by the processor 110 (such as the compressed texture image 700 as shown in FIG.
- the GPU may read the compressed texture image with a non-rectangular shape from the system memory, load it into the built-in local memory of the GPU, map the compressed texture image to a sphere model of the corresponding sphere parameters to generate a 360 degree panoramic video and then select a current viewing image from the 360 degree panorama video based on a user's viewing angle selection and display the current viewing image on the display device 150 .
- step S 1002 the GPU loads the compressed texture image in the system memory into its local memory.
- step S 1004 the GPU loads the corresponding sphere model parameters stored in the system memory and constructs a sphere model (as shown in FIG. 8C ) according to the sphere model parameters.
- step S 1006 the GPU maps the compressed texture image to the sphere model frame by frame and generates a 360 degree video stream as a 360 degree panoramic video, wherein the 360 degree video stream is a set of 360 degree video data in both horizontal and vertical directions.
- the viewing angle of the user is preset at the center position and the current viewing image is a default image corresponding to the center position.
- a user's viewing angle selection will be generated.
- the GPU receives the user's viewing angle selection, it indicates that the user wants to change the currently viewing image. Accordingly, in step S 1008 , the GPU determines a current viewing image of the 360 degree video stream according to the user's viewing angle selection and displays the determined viewing image on the display device 150 .
- the GPU can map between the compressed texture image generated by the method of compressing the texture image of the invention and the sphere model to generate the 360 degree video stream and change the current viewing image being displayed as the user's viewing angle changes, thus achieving a 360-degree panoramic video playback.
- a compression ratio of each of pixel lines on the texture image can be determined according to the different circumferences of rings of latitude line of the respective pixels on the sphere to compress original texture image which is a texture image with a rectangular shape utilized in the prior art into a texture image with an non-rectangular shape such that the number of effective pixels stored in the compressed texture image can be significantly reduced, thus effectively solving the problem of excessive redundancy information sampled at the North and South poles of the texture image in the prior art, obtaining unity in the texture pixel resolution on the sphere, and achieving a goal for reducing the information redundancy on the texture image and reducing the bandwidth required for transmission.
- the method of generating the 360 degree panoramic video of the present invention can further apply a compressed non-rectangular texture image to construct a sphere model and thereby generate a 360 degree panoramic video and change current viewing image being displayed as the user's viewing angle changes, thus achieving a 360-degree panoramic video playback.
- Systems and method thereof, or certain aspects or portions thereof, may take the form of a program code (i.e., executable instructions) embodied in tangible media, such as floppy diskettes, CD-ROMS, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine thereby becomes an apparatus for practicing the methods.
- the methods may also be embodied in the form of a program code transmitted over some transmission medium, such as electrical wiring or cabling, through fiber optics, or via any other form of transmission, wherein, when the program code is received and loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the disclosed methods.
- the program code When implemented on a general-purpose processor, the program code combines with the processor to provide a unique apparatus that operates analogously to application specific logic circuits.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Image Generation (AREA)
- Image Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
Abstract
Description
frRing[i]=fRadius*sin(δ) (10).
pVertices[i][j]·x=frRing*sin(θ) (11);
pVertices[i][j]·y=fRadius*cos(δ) (12); and
pVertices[i][j]·z=frRing*cos(θ) (13).
Wherein, PixelLineOfiRing[i] represents the line number (i.e., line index value) of this point; PixelOfLine[PixelLineOfiRing[i]] represents the number of pixels in the pixel line PixelLineOfiRing[i] (also referred to as the number of effective pixels);
represents that the number of effective pixels within this line are being divided into (iSectors-1) pieces;
is calculated by first obtaining an index value within the effective pixels of the corresponding point (in which the index value of the first effective pixel value is set to be 0) and then calculating a coordinate index value PixelIndexInLine[j] in the pixel line (including invalid pixels) on which the corresponding point located based on the coordinates StartPixel[PixelLineOfiRing[i]] of the first effective pixel point, and the intersection point pVertices[i][j] can be mapped to a point on the texture image. It should be understood that formulas illustrated above are only an example method for calculating the coordinates of the corresponding point on the texture image, but the invention is not limited thereto.
Claims (14)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710311245 | 2017-05-05 | ||
CN201710311245.XA CN107146274B (en) | 2017-05-05 | 2017-05-05 | Image data processing system, texture mapping compression method and method for generating panoramic video |
CN201710311245.X | 2017-05-05 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180322685A1 US20180322685A1 (en) | 2018-11-08 |
US10235795B2 true US10235795B2 (en) | 2019-03-19 |
Family
ID=59776859
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/599,838 Active 2037-06-03 US10235795B2 (en) | 2017-05-05 | 2017-05-19 | Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof |
Country Status (3)
Country | Link |
---|---|
US (1) | US10235795B2 (en) |
CN (1) | CN107146274B (en) |
TW (1) | TWI637355B (en) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106651764B (en) * | 2016-12-29 | 2019-10-15 | 北京奇艺世纪科技有限公司 | A kind of panorama sketch compression method and device |
US11397511B1 (en) * | 2017-10-18 | 2022-07-26 | Nationwide Mutual Insurance Company | System and method for implementing improved user interface |
CA3028794A1 (en) * | 2018-01-04 | 2019-07-04 | 8259402 Canada Inc. | Immersive environment with digital environment to enhance depth sensation |
CN110505470B (en) * | 2018-05-18 | 2021-04-09 | 深圳裸眼威阿科技有限公司 | Image playing system and method based on spherical display screen |
CN108921778B (en) * | 2018-07-06 | 2022-12-30 | 成都品果科技有限公司 | Method for generating star effect map |
TWI735237B (en) | 2020-05-22 | 2021-08-01 | 茂傑國際股份有限公司 | Camera and method for selecting and displaying 360-degree panoramic images |
US11210840B1 (en) * | 2020-08-27 | 2021-12-28 | Aspeed Technology Inc. | Transform method for rendering post-rotation panoramic images |
CN112406706B (en) * | 2020-11-20 | 2022-07-22 | 上海华兴数字科技有限公司 | Vehicle scene display method and device, readable storage medium and electronic equipment |
CN112675545B (en) * | 2021-01-07 | 2022-12-13 | 腾讯科技(深圳)有限公司 | Method and device for displaying surface simulation picture, storage medium and electronic equipment |
CN115482147B (en) * | 2022-09-14 | 2023-04-28 | 中国人民大学 | Efficient parallel graph processing method and system based on compressed data direct calculation |
CN116721196B (en) * | 2023-06-26 | 2023-11-07 | 北京麟卓信息科技有限公司 | Dynamic conversion method for compressed textures |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030095131A1 (en) * | 2001-11-08 | 2003-05-22 | Michael Rondinelli | Method and apparatus for processing photographic images |
US20040169726A1 (en) | 2001-07-20 | 2004-09-02 | 6115187 Canada Inc. | Method for capturing a panoramic image by means of an image sensor rectangular in shape |
US20080303814A1 (en) * | 2006-03-17 | 2008-12-11 | Nec Corporation | Three-dimensional data processing system |
US7483590B2 (en) * | 2001-10-29 | 2009-01-27 | Sony Corporation | Image processing apparatus and image processing method for non-planar image, storage medium, and computer program |
CN102426703A (en) | 2011-08-11 | 2012-04-25 | 重庆市勘测院 | Course and pitch angle speed acquisition method of vehicle panoramic image acquisition platform |
EP2518686A1 (en) | 2007-05-25 | 2012-10-31 | Google Inc. | Rendering, viewing and annotating panoramic images, and applications thereof |
CN102982516A (en) | 2012-10-25 | 2013-03-20 | 西安理工大学 | Panoramic picture method based on hemisphere annular panoramic camera |
CN103493105A (en) | 2011-04-25 | 2014-01-01 | 林光雄 | Omnidirectional image editing program and omnidirectional image editing device |
CN103905761A (en) | 2012-12-26 | 2014-07-02 | 株式会社理光 | Image-processing system, image-processing method and program |
TW201601115A (en) | 2014-04-05 | 2016-01-01 | 新力電腦娛樂美國有限責任公司 | Varying effective resolution by screen location in graphics processing by approximating projection of vertices onto curved viewport |
CN106131540A (en) | 2016-07-29 | 2016-11-16 | 暴风集团股份有限公司 | Method and system based on D3D playing panoramic video |
CN106412582A (en) | 2016-10-21 | 2017-02-15 | 北京大学深圳研究生院 | Panoramic video region of interest description method and coding method |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9007432B2 (en) * | 2010-12-16 | 2015-04-14 | The Massachusetts Institute Of Technology | Imaging systems and methods for immersive surveillance |
WO2014043814A1 (en) * | 2012-09-21 | 2014-03-27 | Tamaggo Inc. | Methods and apparatus for displaying and manipulating a panoramic image by tiles |
CN105245838A (en) * | 2015-09-29 | 2016-01-13 | 成都虚拟世界科技有限公司 | Panoramic video playing method and player |
CN105872353A (en) * | 2015-12-15 | 2016-08-17 | 乐视网信息技术(北京)股份有限公司 | System and method for implementing playback of panoramic video on mobile device |
CN105631809B (en) * | 2015-12-31 | 2018-11-06 | 北京理工大学 | A kind of non-uniform resolution spherical panorama drawing generating method |
CN105761296B (en) * | 2016-01-15 | 2018-09-18 | 网易(杭州)网络有限公司 | A kind of assembling system and method, terminal of compressed textures |
CN105915907B (en) * | 2016-06-07 | 2019-07-26 | 北京圣威特科技有限公司 | Compression method, the apparatus and system of panorama sketch |
CN105933707B (en) * | 2016-06-30 | 2019-02-22 | 北京奇艺世纪科技有限公司 | A kind of compression method and device of panoramic video |
CN106527857A (en) * | 2016-10-10 | 2017-03-22 | 成都斯斐德科技有限公司 | Virtual reality-based panoramic video interaction method |
-
2017
- 2017-05-05 CN CN201710311245.XA patent/CN107146274B/en active Active
- 2017-05-19 US US15/599,838 patent/US10235795B2/en active Active
- 2017-11-20 TW TW106140077A patent/TWI637355B/en active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040169726A1 (en) | 2001-07-20 | 2004-09-02 | 6115187 Canada Inc. | Method for capturing a panoramic image by means of an image sensor rectangular in shape |
US7483590B2 (en) * | 2001-10-29 | 2009-01-27 | Sony Corporation | Image processing apparatus and image processing method for non-planar image, storage medium, and computer program |
US20030095131A1 (en) * | 2001-11-08 | 2003-05-22 | Michael Rondinelli | Method and apparatus for processing photographic images |
US20080303814A1 (en) * | 2006-03-17 | 2008-12-11 | Nec Corporation | Three-dimensional data processing system |
EP2518686A1 (en) | 2007-05-25 | 2012-10-31 | Google Inc. | Rendering, viewing and annotating panoramic images, and applications thereof |
CN103493105A (en) | 2011-04-25 | 2014-01-01 | 林光雄 | Omnidirectional image editing program and omnidirectional image editing device |
CN102426703A (en) | 2011-08-11 | 2012-04-25 | 重庆市勘测院 | Course and pitch angle speed acquisition method of vehicle panoramic image acquisition platform |
CN102982516A (en) | 2012-10-25 | 2013-03-20 | 西安理工大学 | Panoramic picture method based on hemisphere annular panoramic camera |
CN103905761A (en) | 2012-12-26 | 2014-07-02 | 株式会社理光 | Image-processing system, image-processing method and program |
TW201601115A (en) | 2014-04-05 | 2016-01-01 | 新力電腦娛樂美國有限責任公司 | Varying effective resolution by screen location in graphics processing by approximating projection of vertices onto curved viewport |
CN106131540A (en) | 2016-07-29 | 2016-11-16 | 暴风集团股份有限公司 | Method and system based on D3D playing panoramic video |
CN106412582A (en) | 2016-10-21 | 2017-02-15 | 北京大学深圳研究生院 | Panoramic video region of interest description method and coding method |
Non-Patent Citations (1)
Title |
---|
Pekofsky et al. , WO 2014/043814 A1, Mar. 27, 2014, pp. 1-63 (Year: 2014). * |
Also Published As
Publication number | Publication date |
---|---|
TW201843651A (en) | 2018-12-16 |
TWI637355B (en) | 2018-10-01 |
CN107146274B (en) | 2021-06-22 |
CN107146274A (en) | 2017-09-08 |
US20180322685A1 (en) | 2018-11-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10235795B2 (en) | Methods of compressing a texture image and image data processing system and methods of generating a 360 degree panoramic video thereof | |
TWI619088B (en) | Image data processing system and associated methods for processing panorama images and image blending using the same | |
US11477395B2 (en) | Apparatus and methods for the storage of overlapping regions of imaging data for the generation of optimized stitched images | |
US11756152B2 (en) | Apparatus and methods for the optimal stitch zone calculation of a generated projection of a spherical image | |
CN109618090B (en) | Method and system for image distortion correction of images captured using wide angle lenses | |
US10595004B2 (en) | Electronic device for generating 360-degree three-dimensional image and method therefor | |
US10521468B2 (en) | Animated seek preview for panoramic videos | |
CN109891465B (en) | Method and device for processing virtual reality image | |
WO2017088491A1 (en) | Video playing method and device | |
CN109997167B (en) | Directional image stitching for spherical image content | |
CN108282694B (en) | Panoramic video rendering method and device and electronic equipment | |
US10650592B2 (en) | Methods and apparatus for providing rotated spherical viewpoints | |
KR20190039118A (en) | Panorama image compression method and apparatus | |
US11218676B2 (en) | Methods and apparatus for projection conversion decoding for applications eco-systems | |
KR20190011212A (en) | Method of and data processing system for providing an output surface | |
WO2023280266A1 (en) | Fisheye image compression method, fisheye video stream compression method and panoramic video generation method | |
Liu et al. | A 360-degree 4K× 2K pan oramic video processing Over Smart-phones | |
US20240257297A1 (en) | Apparatus and methods for the optimal stitch zone calculation of a generated projection of a spherical image | |
US11043022B2 (en) | Viewability metrics of a multidimensional object in a multidimensional digital environment | |
CN107040792A (en) | Panoramic video player method, panoramic video playing device and player | |
CN110062225B (en) | Picture filtering method and device | |
CN118413745A (en) | Method, device, equipment and storage medium for playing panoramic video | |
WO2020016350A1 (en) | A method and apparatus for generating an immersive image from images captured by a plurality of cameras |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: VIA ALLIANCE SEMICONDUCTOR CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YI, YITING;CHEN, GONG;XIE, RONG;REEL/FRAME:042436/0369 Effective date: 20170510 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: GLENFLY TECH CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHANGHAI ZHAOXIN SEMICONDUCTOR CO., LTD.;REEL/FRAME:058965/0042 Effective date: 20211101 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |