WO2023040932A1 - Panoramic video frame interpolation method and apparatus, and computer-readable storage medium - Google Patents
Panoramic video frame interpolation method and apparatus, and computer-readable storage medium Download PDFInfo
- Publication number
- WO2023040932A1 WO2023040932A1 PCT/CN2022/118916 CN2022118916W WO2023040932A1 WO 2023040932 A1 WO2023040932 A1 WO 2023040932A1 CN 2022118916 W CN2022118916 W CN 2022118916W WO 2023040932 A1 WO2023040932 A1 WO 2023040932A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- images
- panoramic
- panoramic video
- video frame
- spherical
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000003709 image segmentation Methods 0.000 claims abstract description 9
- 230000011218 segmentation Effects 0.000 claims abstract description 6
- 238000012966 insertion method Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 6
- 238000003780 insertion Methods 0.000 claims description 6
- 230000037431 insertion Effects 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 4
- 230000015654 memory Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005294 ferromagnetic effect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000005291 magnetic effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
Definitions
- the present application relates to the technical field of image processing, and in particular to a panoramic video frame insertion method, device, and computer-readable storage medium.
- Video frame interpolation is a common video processing technology. Generally, by analyzing the content motion information of two or more frames before and after, estimating the motion process at the middle moment, so as to complement the picture at the middle moment.
- the existing video frame interpolation scheme does not do special processing for the panoramic video, and can only interpolate the flat picture obtained after the panoramic video is projected through the panoramic plane as a normal video.
- the most commonly used method of frame interpolation for existing panoramic video is: under equidistant cylindrical projection, the frame interpolation is performed by using the common plane video frame interpolation method, but there are two problems as follows: first, the area near the north and south poles of the panoramic spherical Severe stretching and deformation, the movement of objects under this stretched picture is seriously distorted, it is difficult to analyze and obtain the correct motion track with the existing video frame interpolation method, resulting in poor frame interpolation effect; second, the continuous north and south poles The picture is stretched to become the upper and lower boundaries of the projection plane, and the continuous picture at a certain meridian is divided and expanded into the left and right boundaries of the projection plane, while the ordinary plane interpolation method analyzes the motion independently at each boundary, As a result, the original continuous movement at the boundary is divided into independent movements, the original movement information is lost, and the movement analysis at the boundary is wrong.
- the Chinese patent publication number is CN111372087A, and the title of the invention is "Panoramic Video Frame Interpolation Method, Device and Corresponding Storage Medium” discloses a panoramic video frame interpolation method, which expands the The boundary is a mirrored or repeated picture, thus retaining the continuous picture information divided by the equidistant cylindrical projection, which can solve the problem of loss of continuous motion information at the boundary after planar projection to a certain extent.
- the optical flow calculation steps used to estimate the motion are still carried out under the equidistant cylindrical projection screen, so there is still the problem of motion estimation failure caused by the severe stretching and deformation of the north and south poles.
- the purpose of the present invention is to provide a panoramic video frame insertion method, device and computer-readable storage medium, aiming to solve the defects existing in the existing panoramic video frame insertion method.
- the present invention provides a panoramic video frame insertion method, the method comprising:
- the present invention provides a panoramic video frame insertion device, which includes:
- the projection module is used to project the first and second panoramic video frames to the spherical surface to form the first and second panoramic spherical images respectively;
- the segmentation module is used to divide the first and second panoramic spherical images into pairs by the spherical image segmentation method The N groups of first images and second images;
- the intermediate interpolation module is used to interpolate the paired N groups of first images and second images to obtain N frames of the third image at the intermediate moment;
- the splicing module is used to combine
- the third image of N frames is spliced into an intermediate moment panoramic video frame;
- the panoramic frame interpolation module is used to insert the intermediate moment panoramic video frame between the first and second panoramic video frames; wherein, N is a natural number not less than 2, and any Two adjacent first images have overlapping areas and any adjacent two second images have overlapping areas.
- the present invention provides a computer-readable storage medium on which computer programs/instructions are stored, wherein when the computer programs/instructions are executed by a processor, the steps of the above-mentioned panoramic video frame insertion method are implemented.
- the present invention improves the motion estimation accuracy of panoramic video frame interpolation, and at the same time avoids the loss of boundary continuous motion information caused by boundary segmentation of panoramic plane projection, thereby improving the frame interpolation effect of panoramic video.
- FIG. 1 is a flow chart of a panoramic video frame insertion method in Embodiment 1 of the present invention.
- FIG. 2 is a schematic diagram of implementing panoramic video frame interpolation through N virtual fisheye lenses in Embodiment 1 of the present invention.
- FIG. 3 is a schematic diagram of segmenting a panoramic spherical image using two virtual fisheye lenses in FIG. 2 .
- Fig. 4 is a schematic diagram of implementing panoramic video frame interpolation through extended cube projection in Embodiment 1 of the present invention.
- FIG. 5 is a schematic diagram of segmenting a panoramic spherical image by the over-extended cube projection in FIG. 4 .
- FIG. 6 is another perspective view of FIG. 5 .
- Fig. 7 is a structural block diagram of a panoramic video frame insertion device in Embodiment 2 of the present invention.
- this embodiment discloses a frame insertion method for a panoramic video, which includes the following steps.
- S1 Projecting the first and second panoramic video frames onto a spherical surface respectively to form first and second panoramic spherical images.
- the first panoramic video frame and the second panoramic video frame are generally two consecutive panoramic video frames, and in some special cases, they may also be two panoramic video frames separated by one or more frames, For example, when some panoramic video frames have occluders or poor shooting effects, these panoramic video frames need to be discarded. It is an existing technology to project panoramic video frames onto a spherical surface to form a panoramic spherical image. Through this step, the picture information in the first and second panoramic video frames can be converted into pixel values at each latitude and longitude on the panoramic spherical surface.
- S2 Segment the first and second panoramic spherical images into N sets of paired first images and second images by using a spherical image segmentation method.
- multiple virtual fisheye lenses are used to segment the first and second panoramic spherical images.
- N is a natural number not less than 2
- the internal parameters of each virtual fisheye lens are small Hole imaging model or other camera parameter model settings.
- the field of view of each virtual lens covers a part of the panoramic spherical image, so that any point in the panoramic spherical image is within a certain virtual fisheye lens.
- any two adjacent fisheye lenses must have a sufficient overlapping area (generally exceeding 10°), so that two adjacent virtual fisheye lenses have overlapping areas.
- two virtual fisheye lenses are taken as an example for illustration.
- the first virtual fisheye lens and the second virtual fisheye lens are located at the center of the panoramic spherical image and are arranged in a back-to-back manner.
- the first virtual fisheye lens The fisheye lens faces upward, the second virtual fisheye lens faces downward, and the field of view of each virtual fisheye lens ⁇ 190°, so that the angle ⁇ of the overlapping area of the field of view of the two virtual fisheye lenses ⁇ 10° , thus, the two virtual fisheye lenses divide the first panoramic spherical image into two first fisheye images with overlapping regions, and divide the second panoramic spherical image into two second fisheye images with overlapping regions, Thereby forming two sets of paired fisheye images, that is, the first fisheye image corresponding to the first virtual fisheye lens in the first panoramic spherical image and the second fisheye image corresponding to the first virtual fisheye lens in the second panoramic spherical image
- the eye images are a paired set of fisheye images.
- the specific method of segmentation is: according to the set internal and external parameters of the virtual lens, calculate the coordinate mapping from the panoramic sphere to the fisheye image, so as to project the panoramic image into two fisheye images.
- three or more virtual fisheye lenses can also be used to project the first and second panoramic spherical images, thereby dividing them into a corresponding number of paired first fisheye images and second fisheye images.
- Any two adjacent fisheye images also have overlapping regions.
- the first and second panoramic spherical images are divided into six pairs of paired first, upper, lower, left, right, front, front, front and back pairs by extended cube projection.
- the extended cube projection in this solution refers to a planar projection projected onto a cube whose field angles of the six faces in the horizontal and vertical directions are all greater than 90°.
- the center point O of the cube is used for plane projection in six mutually perpendicular directions of the cube's up, down, left, right, front, and back, and a total of six square plane images are obtained.
- the field angle ⁇ towards each face of the cube needs to exceed 90°, and generally requires a field angle of more than 100°, so as to ensure that the plane images of adjacent faces have sufficient overlapping areas, so that The motion of the object at the original boundary retains continuity.
- the angle of view of the plane image of the standard cube is 90°
- the angle of view ⁇ of the plane image of the extended cube (including the horizontal angle of view and vertical viewing angle) greater than 90°.
- the plane projection above the cube and the plane projection on the right side of the cube have an overlapping area at the connection between the top and the right, and the angle ⁇ ⁇ 10° corresponding to the overlapping area.
- the first and second panoramic spherical images can be divided into six groups of paired first plane images and second plane images in total, up, down, left, right, front and back, wherein the first plane images and the second plane images with the same direction for a set of paired images.
- S3 Perform frame interpolation on the paired N groups of first images and second images to obtain N frames of third images at intermediate moments.
- the paired first fisheye image and the second fisheye image formed by N virtual fisheye lenses in step S2 are executed to perform a plane video frame interpolation operation to obtain the third fisheye picture of N frames at the intermediate moment; for step S2
- the first and second panoramic spherical images are divided into six pairs of pairs of up, down, left, right, front, back, front and back through extended cube projection.
- the plane interpolation operation is performed to obtain six frames of the third plane at the middle moment. video.
- the frame insertion method of the planar video is an existing technology, such as optical flow frame insertion, which will not be repeated here.
- Splicing multiple frames of fisheye lens images into panoramic video frames belongs to the prior art. You can refer to the applicant's patent application number 201610417580.3 filed in 2016. Multiple N frames of third images can be spliced into an intermediate moment panoramic video frames.
- the inverse transformation of the extended cube projection is first performed on the third plane image, and the panoramic image on the spherical surface is restored. Specifically, the plane of the extended cube projection is executed.
- the playback timing of the panoramic video frame at the intermediate moment may be placed between the first panoramic video frame and the second panoramic video frame.
- This embodiment discloses a panoramic video frame insertion device, which includes: a projection module, used to project the first and second panoramic video frames onto a spherical surface to form the first and second panoramic spherical images; a segmentation module, used to The first and second panoramic spherical images are divided into paired N groups of first images and second images by a spherical image segmentation method; the intermediate interpolation module is used to interpolate the paired N groups of first images and second images To obtain the third image of N frames at the intermediate moment; the splicing module is used to splice the third image of the N frame into an intermediate moment panoramic video frame; the panoramic frame interpolation module is used to insert the intermediate moment panoramic video frame into the first and second panoramic views Between video frames; wherein, N is a natural number not less than 2, and any adjacent two of the first images have an overlapping area and any adjacent two of the second images have an overlapping area.
- Embodiment 1 discloses that the present invention provides a computer-readable storage medium on which computer programs/instructions are stored, and it is characterized in that, when the computer program/instructions are executed by a processor, the panoramic video interpolation in Embodiment 1 is realized. The steps of the frame method.
- the storage medium can be a computer-readable storage medium, for example, a ferroelectric memory (FRAM , Ferromagnetic Random Access Memory), Read Only Memory (ROM, Read Only Memory), Programmable Read Only Memory (PROM, Programmable Read Only Memory), Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory, magnetic surface memory, optical disc, or CD-ROM (Compact Disk-Read Only Memory) and other memories; it can also be Various devices including one or any combination of the above memories.
- FRAM ferroelectric memory
- ROM Read Only Memory
- PROM Programmable Read Only Memory
- EPROM Erasable Programmable Read Only Memory
- EEPROM Electrically Erasable Programmable Read Only Memory
- flash memory magnetic surface memory, optical disc, or CD-ROM (Compact Disk-Read Only Memory) and other memories; it can also be Various devices including one or any combination of the above
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
Provided in the present invention is a panoramic video frame interpolation method. The method comprises: respectively projecting a first panoramic video frame and a second panoramic video frame onto a spherical surface to form a first panoramic spherical image and a second panoramic spherical image; segmenting, by means of a spherical image segmentation method, the first panoramic spherical image and the second panoramic spherical image into N groups of paired first images and second images; performing frame interpolation on the N groups of paired first images and second images, so as to obtain N frames of third images at an intermediate moment; splicing the N frames of third images into an intermediate-moment panoramic video frame; and interpolating the intermediate-moment panoramic video frame between the first panoramic video frame and the second panoramic video frame, wherein N is a natural number not less than 2, any two adjacent first images have an overlapping region, and any two adjacent second images also have an overlapping region. Compared with the prior art, by means of the present invention, the accuracy of motion estimation of panoramic video frame interpolation is improved, and the problem of boundary continuous motion information being lost due to boundary segmentation of panoramic plane projection is also avoided, thereby improving the frame interpolation effect of a panoramic video.
Description
本申请涉及图像处理技术领域，具体涉及一种全景视频插帧方法、装置及计算机可读存储介质。The present application relates to the technical field of image processing, and in particular to a panoramic video frame insertion method, device, and computer-readable storage medium.
视频插帧是常见的视频处理技术，一般是通过分析前后两帧或更多帧的内容运动信息，估算中间时刻的运动过程，从而补全中间时刻的画面。现有视频插帧方案未对全景视频做特殊处理，只能将全景视频经过全景平面投影后得到的平面画面作为普通视频进行插帧。Video frame interpolation is a common video processing technology. Generally, by analyzing the content motion information of two or more frames before and after, estimating the motion process at the middle moment, so as to complement the picture at the middle moment. The existing video frame interpolation scheme does not do special processing for the panoramic video, and can only interpolate the flat picture obtained after the panoramic video is projected through the panoramic plane as a normal video.
现有的全景视频插帧最常用的方式为：在等距圆柱投影下，采用普通的平面视频插帧方法进行插帧，但存在以下两个问题：第一，全景球面的南北极附近区域被严重拉伸变形，在此拉伸的画面下物体运动严重失真，现有的视频插帧方法很难分析得到正确的运动轨迹，导致插帧效果不好；第二，北极点和南极点连续的画面分别被拉伸成为投影平面的上、下边界，某一经线处连续的画面被分割、展开为投影平面的左、右边界，而普通的平面插帧方法在每个边界处独自分析运动，导致边界处原本连续的运动被切分为各自独立的运动，丢失了原本运动信息，导致边界处运动分析出错。The most commonly used method of frame interpolation for existing panoramic video is: under equidistant cylindrical projection, the frame interpolation is performed by using the common plane video frame interpolation method, but there are two problems as follows: first, the area near the north and south poles of the panoramic spherical Severe stretching and deformation, the movement of objects under this stretched picture is seriously distorted, it is difficult to analyze and obtain the correct motion track with the existing video frame interpolation method, resulting in poor frame interpolation effect; second, the continuous north and south poles The picture is stretched to become the upper and lower boundaries of the projection plane, and the continuous picture at a certain meridian is divided and expanded into the left and right boundaries of the projection plane, while the ordinary plane interpolation method analyzes the motion independently at each boundary, As a result, the original continuous movement at the boundary is divided into independent movements, the original movement information is lost, and the movement analysis at the boundary is wrong.
中国专利公开号为CN111372087A、发明名称为“全景视频插帧方法、装置及对应的存储介质”公开了一种全景视频插帧方法，通过对等距圆柱投影的四个边界区域进行拼接操作，拓展边界为镜像或重复的画面，从而保留了被等距圆柱投影所分割开的连续画面信息，在一定程度上可以解决平面投影后边界处连续运动信息丢失的问题。但估计运动所用的光流计算步骤，依然是在等距圆柱投影画面下进行，因此仍然存在因南北极区域被严重拉伸变形而导致的运动估计失败的问题。The Chinese patent publication number is CN111372087A, and the title of the invention is "Panoramic Video Frame Interpolation Method, Device and Corresponding Storage Medium" discloses a panoramic video frame interpolation method, which expands the The boundary is a mirrored or repeated picture, thus retaining the continuous picture information divided by the equidistant cylindrical projection, which can solve the problem of loss of continuous motion information at the boundary after planar projection to a certain extent. However, the optical flow calculation steps used to estimate the motion are still carried out under the equidistant cylindrical projection screen, so there is still the problem of motion estimation failure caused by the severe stretching and deformation of the north and south poles.
因此，有必要对现有的全景视频插帧方法进行改进。Therefore, it is necessary to improve the existing panoramic video frame interpolation method.
发明内容Contents of the invention
本发明的目的在于提供一种全景视频插帧方法、装置及计算机可读存储介质，旨在解决现有全景视频插帧方法方法存在的缺陷。The purpose of the present invention is to provide a panoramic video frame insertion method, device and computer-readable storage medium, aiming to solve the defects existing in the existing panoramic video frame insertion method.
第一方面，本发明提供了一种全景视频插帧方法，该方法包括：In a first aspect, the present invention provides a panoramic video frame insertion method, the method comprising:
将第一、第二全景视频帧分别投影至球面形成第一、第二全景球形图像；将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像；对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像；将N帧第三图像拼接成中间时刻全景视频帧；将中间时刻全景视频帧***第一、第二全景视频帧之间；其中，N为不小于2的自然数，且任意相邻的两个所述第一图像具有重叠区域以及任意相邻的两个所述第二图像具有重叠区域。Projecting the first and second panoramic video frames to the spherical surface to form the first and second panoramic spherical images respectively; dividing the first and second panoramic spherical images into paired N groups of first images and second images by a spherical image segmentation method ; The paired N groups of first images and second images are interpolated to obtain the third image of N frames at the intermediate moment; the third image of the N frames is stitched into an intermediate moment panoramic video frame; the intermediate moment panoramic video frame is inserted into the first , Between the second panoramic video frames; wherein, N is a natural number not less than 2, and any two adjacent first images have overlapping areas and any adjacent two second images have overlapping areas.
第二方面，本发明提供了一种全景视频插帧装置，该装置包括：In a second aspect, the present invention provides a panoramic video frame insertion device, which includes:
投影模块，用于将第一、第二全景视频帧分别投影至球面形成第一、第二全景球形图像；分割模块，用于将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像；中间插帧模块，用于对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像；拼接模块，用于将N帧第三图像拼接成中间时刻全景视频帧；全景插帧模块，用于将中间时刻全景视频帧***第一、第二全景视频帧之间；其中，N为不小于2的自然数，且任意相邻的两个所述第一图像具有重叠区域以及任意相邻的两个所述第二图像具有重叠区域。The projection module is used to project the first and second panoramic video frames to the spherical surface to form the first and second panoramic spherical images respectively; the segmentation module is used to divide the first and second panoramic spherical images into pairs by the spherical image segmentation method The N groups of first images and second images; the intermediate interpolation module is used to interpolate the paired N groups of first images and second images to obtain N frames of the third image at the intermediate moment; the splicing module is used to combine The third image of N frames is spliced into an intermediate moment panoramic video frame; the panoramic frame interpolation module is used to insert the intermediate moment panoramic video frame between the first and second panoramic video frames; wherein, N is a natural number not less than 2, and any Two adjacent first images have overlapping areas and any adjacent two second images have overlapping areas.
第三方面，本发明提供了一种计算机可读存储介质，其上存储有计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现上述全景视频插帧方法的步骤。In a third aspect, the present invention provides a computer-readable storage medium on which computer programs/instructions are stored, wherein when the computer programs/instructions are executed by a processor, the steps of the above-mentioned panoramic video frame insertion method are implemented.
与现有技术相比，本发明提升了全景视频插帧的运动估算准确度，同时避免了全景平面投影的边界分割导致的边界连续运动信息丢失的问题，从而提升全景视频的插帧效果。Compared with the prior art, the present invention improves the motion estimation accuracy of panoramic video frame interpolation, and at the same time avoids the loss of boundary continuous motion information caused by boundary segmentation of panoramic plane projection, thereby improving the frame interpolation effect of panoramic video.
图1是本发明实施例1中的全景视频插帧方法的流程图。FIG. 1 is a flow chart of a panoramic video frame insertion method in Embodiment 1 of the present invention.
图2是本发明实施例1中的通过N个虚拟鱼眼镜头实现全景视频插帧的示意图。FIG. 2 is a schematic diagram of implementing panoramic video frame interpolation through N virtual fisheye lenses in Embodiment 1 of the present invention.
图3是图2中采用两个虚拟鱼眼镜头分割全景球形图像的示意图。FIG. 3 is a schematic diagram of segmenting a panoramic spherical image using two virtual fisheye lenses in FIG. 2 .
图4是本发明实施例1中的通过扩展立方体投影实现全景视频插帧的示意图。Fig. 4 is a schematic diagram of implementing panoramic video frame interpolation through extended cube projection in Embodiment 1 of the present invention.
图5是图4中的过扩展立方体投影分割全景球形图像的示意图。FIG. 5 is a schematic diagram of segmenting a panoramic spherical image by the over-extended cube projection in FIG. 4 .
图6是图5的另一视角图。FIG. 6 is another perspective view of FIG. 5 .
图7是本发明实施例2中的全景视频插帧装置的结构框图。Fig. 7 is a structural block diagram of a panoramic video frame insertion device in Embodiment 2 of the present invention.
为了使本发明的目的、技术方案及有益效果更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。In order to make the object, technical solution and beneficial effects of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
为了说明本发明所述的技术方案，下面通过具体实施例来进行说明。In order to illustrate the technical solutions of the present invention, specific examples are used below to illustrate.
如图1所示，本实施例揭示了一种全景视频插帧方法，包括以下步骤。As shown in FIG. 1 , this embodiment discloses a frame insertion method for a panoramic video, which includes the following steps.
S1：将第一、第二全景视频帧分别投影至球面形成第一、第二全景球形图像。S1: Projecting the first and second panoramic video frames onto a spherical surface respectively to form first and second panoramic spherical images.
在本实施例中，第一全景视频帧和第二全景视频帧一般为连续的两帧全景视频帧，在一些特殊的情况下，也可以为间隔一帧或多帧的两帧全景视频帧，例如，当某些全景视频帧出现遮挡物或拍摄效果不好时，需舍弃这些全景视频帧。将全景视频帧投影至球面形成全景球形图像为现有技术，通过此步骤，可以将第一、第二全景视频帧中的画面信息在转化为全景球面上的在各个经纬度像素值。In this embodiment, the first panoramic video frame and the second panoramic video frame are generally two consecutive panoramic video frames, and in some special cases, they may also be two panoramic video frames separated by one or more frames, For example, when some panoramic video frames have occluders or poor shooting effects, these panoramic video frames need to be discarded. It is an existing technology to project panoramic video frames onto a spherical surface to form a panoramic spherical image. Through this step, the picture information in the first and second panoramic video frames can be converted into pixel values at each latitude and longitude on the panoramic spherical surface.
S2：将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像。S2: Segment the first and second panoramic spherical images into N sets of paired first images and second images by using a spherical image segmentation method.
在本步骤的优选方案中，采用多个虚拟鱼眼镜头对第一、第二全景球形图像进行分割。如图2、图3所示，仿照多路全景相机的组合方式，设定N个(N为不小于2的自然数) 朝向不同方向的虚拟鱼眼镜头，每个虚拟鱼眼镜头内参数采用小孔成像模型或其他相机参数模型设置。将虚拟鱼眼镜头镜头位于全景球形图像的球心处，朝四周各个方向放置，每个虚拟镜头的视场覆盖全景球形图像的画面一部分，使得全景球面图像中的任意一点都在某一虚拟鱼眼镜头的视场中，同时任意相邻的两个鱼眼镜头视场需有足够重合区域(一般超过10°)，使得相邻的两个虚拟鱼眼镜头的具有重叠区域。如图3所示，以两个虚拟鱼眼镜头为例进行说明，第一虚拟鱼眼镜头和第二虚拟鱼眼镜头位于全景球形图像的中心位置，且以背靠背的方式进行布局，第一虚拟鱼眼镜头朝向上方，第二虚拟鱼眼镜头朝向下方，每个虚拟鱼眼镜头的视场角α≥190°，使得两个虚拟鱼眼镜头的视场角的重叠区域的角度β≥10°，由此，两个虚拟鱼眼镜头将第一全景球形图像分割成具有重叠区域的两个第一鱼眼图像、将第二全景球形图像分割成具有重叠区域的两个第二鱼眼图像，从而形成两组配对的鱼眼图像，即第一虚拟鱼眼镜头在第一全景球形图像中对应的第一鱼眼图像和第一虚拟鱼眼镜头在第二全景球形图像中对应的第二鱼眼图像为配对的一组鱼眼图像。分割的具体方式为：根据设置的虚拟镜头内外参，计算全景球面到鱼眼图像的坐标映射，从而将全景画面投影为两个鱼眼图像。当然，也可以采用3个或以上的虚拟鱼眼镜头对第一、第二全景球形图像进行投影，从而将其分割成对应数目的配对的第一鱼眼图像和第二鱼眼图像，当然，任意两个相邻的鱼眼图像也具有重叠区域。In a preferred solution of this step, multiple virtual fisheye lenses are used to segment the first and second panoramic spherical images. As shown in Figure 2 and Figure 3, imitating the combination of multi-channel panoramic cameras, set N (N is a natural number not less than 2) virtual fisheye lenses facing different directions, and the internal parameters of each virtual fisheye lens are small Hole imaging model or other camera parameter model settings. Place the virtual fisheye lens at the spherical center of the panoramic spherical image and place it in all directions. The field of view of each virtual lens covers a part of the panoramic spherical image, so that any point in the panoramic spherical image is within a certain virtual fisheye lens. In the field of view of the lens, at the same time, the fields of view of any two adjacent fisheye lenses must have a sufficient overlapping area (generally exceeding 10°), so that two adjacent virtual fisheye lenses have overlapping areas. As shown in Figure 3, two virtual fisheye lenses are taken as an example for illustration. The first virtual fisheye lens and the second virtual fisheye lens are located at the center of the panoramic spherical image and are arranged in a back-to-back manner. The first virtual fisheye lens The fisheye lens faces upward, the second virtual fisheye lens faces downward, and the field of view of each virtual fisheye lens α≥190°, so that the angle β of the overlapping area of the field of view of the two virtual fisheye lenses ≥10° , thus, the two virtual fisheye lenses divide the first panoramic spherical image into two first fisheye images with overlapping regions, and divide the second panoramic spherical image into two second fisheye images with overlapping regions, Thereby forming two sets of paired fisheye images, that is, the first fisheye image corresponding to the first virtual fisheye lens in the first panoramic spherical image and the second fisheye image corresponding to the first virtual fisheye lens in the second panoramic spherical image The eye images are a paired set of fisheye images. The specific method of segmentation is: according to the set internal and external parameters of the virtual lens, calculate the coordinate mapping from the panoramic sphere to the fisheye image, so as to project the panoramic image into two fisheye images. Of course, three or more virtual fisheye lenses can also be used to project the first and second panoramic spherical images, thereby dividing them into a corresponding number of paired first fisheye images and second fisheye images. Of course, Any two adjacent fisheye images also have overlapping regions.
在本步骤的另一方案中，通过扩展立方体投影将第一、第二全景球形图像分割成配对的上下左右前后共六组配对的第一平面图像和第二平面图像。本方案中的扩展立方体投影是指投影到立方体的六个面在水平方向和垂直方向的视场角都大于90°的平面投影。具体地，如图5所示，以立方体中心点O朝立方体的上、下、左、右、前、后六个互相垂直的方向进行平面投影，一共得到六个正方形平面图像。为保证运动经过边界处的连续性，朝向立方体每个面的视场角α需要超过90°，一般需要超过100°的视场角，以保证相邻面的平面图像具有足够的重叠区域，使得原本边界处的物体运动保留连续性，从图5中可以看出，标准立方 体的平面图像的视场角为90°，而扩展立方体的平面图像的视场角α(包括水平视场角和垂直视角)大于90°。如图6所示，在立方体上方的平面投影和立方体右方的平面投影在上方和右方的连接处有重叠区域，且重叠区域对应的角度β≥10°。因此，通过上述方式，可以将第一、第二全景球形图像分别分割成上下左右前后共六组配对的第一平面图像和第二平面图像，其中，方向相同的第一平面图像和第二图像为一组配对图像。In another solution of this step, the first and second panoramic spherical images are divided into six pairs of paired first, upper, lower, left, right, front, front, front and back pairs by extended cube projection. The extended cube projection in this solution refers to a planar projection projected onto a cube whose field angles of the six faces in the horizontal and vertical directions are all greater than 90°. Specifically, as shown in FIG. 5 , the center point O of the cube is used for plane projection in six mutually perpendicular directions of the cube's up, down, left, right, front, and back, and a total of six square plane images are obtained. In order to ensure the continuity of movement through the boundary, the field angle α towards each face of the cube needs to exceed 90°, and generally requires a field angle of more than 100°, so as to ensure that the plane images of adjacent faces have sufficient overlapping areas, so that The motion of the object at the original boundary retains continuity. It can be seen from Figure 5 that the angle of view of the plane image of the standard cube is 90°, while the angle of view α of the plane image of the extended cube (including the horizontal angle of view and vertical viewing angle) greater than 90°. As shown in Figure 6, the plane projection above the cube and the plane projection on the right side of the cube have an overlapping area at the connection between the top and the right, and the angle β ≥ 10° corresponding to the overlapping area. Therefore, through the above method, the first and second panoramic spherical images can be divided into six groups of paired first plane images and second plane images in total, up, down, left, right, front and back, wherein the first plane images and the second plane images with the same direction for a set of paired images.
S3：对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像。S3: Perform frame interpolation on the paired N groups of first images and second images to obtain N frames of third images at intermediate moments.
对步骤S2中的通过N个虚拟鱼眼镜头形成的配对的第一鱼眼图像和第二鱼眼图像执行平面视频插帧操作，得到中间时刻的N帧第三鱼眼画面；对步骤S2中的通过扩展立方体投影将第一、第二全景球形图像分割成配对的上下左右前后共六组配对的第一平面图像和第二平面图像执行平面插帧操作，得到中间时刻的六帧第三平面视频。其中，平面视频插帧方法为现有技术，如光流插帧，这里不再赘述。The paired first fisheye image and the second fisheye image formed by N virtual fisheye lenses in step S2 are executed to perform a plane video frame interpolation operation to obtain the third fisheye picture of N frames at the intermediate moment; for step S2 The first and second panoramic spherical images are divided into six pairs of pairs of up, down, left, right, front, back, front and back through extended cube projection. The plane interpolation operation is performed to obtain six frames of the third plane at the middle moment. video. Among them, the frame insertion method of the planar video is an existing technology, such as optical flow frame insertion, which will not be repeated here.
S4：将N帧第三图像拼接成中间时刻全景视频帧。S4: Stitching N frames of third images into a panoramic video frame at an intermediate moment.
将多帧鱼眼镜头画面拼接成全景视频帧属于现有技术，可参考申请人的在2016年申请的、专利申请号为201610417580.3的专利文献，可以将多个N帧第三图像拼接成中间时刻的全景视频帧。对于通过立方体投影后的六帧第三平面图像的拼接，先对第三平面图像执行扩展立方体投影的逆变换，恢复为球面上的全景画面，具体地，将扩展立方体投影的平面执行裁剪操作，去除超过90°视场角的部分，得到水平视场角和垂直视场角皆为90°的投影平面，从而将中间时刻的平面图像的扩展立方体投影的画面变换为中间时刻帧的普通立方体投影，再将中间时刻帧的普通立方体投影画面转换为球面上的中间时刻全景视频帧。Splicing multiple frames of fisheye lens images into panoramic video frames belongs to the prior art. You can refer to the applicant's patent application number 201610417580.3 filed in 2016. Multiple N frames of third images can be spliced into an intermediate moment panoramic video frames. For the splicing of the six frames of the third plane image after the cube projection, the inverse transformation of the extended cube projection is first performed on the third plane image, and the panoramic image on the spherical surface is restored. Specifically, the plane of the extended cube projection is executed. Remove the part with an angle of view exceeding 90°, and obtain a projection plane with both a horizontal angle of view and a vertical angle of view of 90°, so that the image of the extended cube projection of the plane image at the intermediate moment is transformed into an ordinary cube projection of the frame at the intermediate moment , and then convert the ordinary cube projection picture of the intermediate moment frame into the intermediate moment panoramic video frame on the spherical surface.
S5：将中间时刻全景视频帧***第一、第二全景视频帧之间。S5: Insert the panoramic video frame at the middle moment between the first and second panoramic video frames.
将中间时刻全景视频帧的播放时序放在第一全景视频帧和第二全景视频帧之间即可。The playback timing of the panoramic video frame at the intermediate moment may be placed between the first panoramic video frame and the second panoramic video frame.
实施例2Example 2
本实施例揭示了一种全景视频插帧装置，该装置包括：投影模块，用于将第一、第二全 景视频帧分别投影至球面形成第一、第二全景球形图像；分割模块，用于将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像；中间插帧模块，用于对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像；拼接模块，用于将N帧第三图像拼接成中间时刻全景视频帧；全景插帧模块，用于将中间时刻全景视频帧***第一、第二全景视频帧之间；其中，N为不小于2的自然数，且任意相邻的两个所述第一图像具有重叠区域以及任意相邻的两个所述第二图像具有重叠区域。This embodiment discloses a panoramic video frame insertion device, which includes: a projection module, used to project the first and second panoramic video frames onto a spherical surface to form the first and second panoramic spherical images; a segmentation module, used to The first and second panoramic spherical images are divided into paired N groups of first images and second images by a spherical image segmentation method; the intermediate interpolation module is used to interpolate the paired N groups of first images and second images To obtain the third image of N frames at the intermediate moment; the splicing module is used to splice the third image of the N frame into an intermediate moment panoramic video frame; the panoramic frame interpolation module is used to insert the intermediate moment panoramic video frame into the first and second panoramic views Between video frames; wherein, N is a natural number not less than 2, and any adjacent two of the first images have an overlapping area and any adjacent two of the second images have an overlapping area.
实施例3Example 3
本实施例揭示了本发明提供了一种计算机可读存储介质，其上存储有计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现实施例1中的全景视频插帧方法的步骤。This embodiment discloses that the present invention provides a computer-readable storage medium on which computer programs/instructions are stored, and it is characterized in that, when the computer program/instructions are executed by a processor, the panoramic video interpolation in Embodiment 1 is realized. The steps of the frame method.
本领域普通技术人员可以理解上述各个实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，存储介质可以是计算机可读存储介质，例如，铁电存储器(FRAM，Ferromagnetic Random Access Memory)、只读存储器(ROM，Read Only Memory)、可编程只读存储器(PROM，Programmable Read Only Memory)、可擦除可编程只读存储器(EPROM，Erasable Programmable Read Only Memory)、带电可擦可编程只读存储器(EEPROM，Electrically Erasable Programmable Read Only Memory)、闪存、磁表面存储器、光盘、或光盘只读存储器(CD-ROM，Compact Disk-Read Only Memory)等存储器；也可以是包括上述存储器之一或任意组合的各种设备。Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the storage medium can be a computer-readable storage medium, for example, a ferroelectric memory (FRAM , Ferromagnetic Random Access Memory), Read Only Memory (ROM, Read Only Memory), Programmable Read Only Memory (PROM, Programmable Read Only Memory), Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory, magnetic surface memory, optical disc, or CD-ROM (Compact Disk-Read Only Memory) and other memories; it can also be Various devices including one or any combination of the above memories.
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.
Claims (10)
- 一种全景视频插帧方法，其特征在于，包括：A panoramic video frame insertion method, characterized in that, comprising:将第一、第二全景视频帧分别投影至球面形成第一、第二全景球形图像；Projecting the first and second panoramic video frames onto a spherical surface to form first and second panoramic spherical images respectively;将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像；The first and second panoramic spherical images are divided into paired N groups of first images and second images by a spherical image segmentation method;对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像；Interpolating the paired N groups of first images and second images to obtain N frames of third images at intermediate moments;将N帧第三图像拼接成中间时刻全景视频帧；The third image of N frames is spliced into an intermediate moment panoramic video frame;将中间时刻全景视频帧***第一、第二全景视频帧之间；Inserting the panoramic video frame at the intermediate moment between the first and second panoramic video frames;其中，N为不小于2的自然数，且任意相邻的两个所述第一图像具有重叠区域以及任意相邻的两个所述第二图像具有重叠区域。Wherein, N is a natural number not less than 2, and any adjacent two of the first images have an overlapping area and any adjacent two of the second images have an overlapping area.
- 根据权利要求1所述的全景视频插帧方法，其特征在于，所述球形图像分割方法为：The panoramic video frame interpolation method according to claim 1, wherein the spherical image segmentation method is:通过N个虚拟鱼眼镜头将第一、第二全景球形图像分割成配对的N组第一鱼眼图像和第二鱼眼图像。The first and second panoramic spherical images are divided into N pairs of first fisheye images and second fisheye images by using N virtual fisheye lenses.
- 根据权利要求2所述的全景视频插帧方法，其特征在于，The panoramic video frame insertion method according to claim 2, wherein:所述N个虚拟鱼眼镜头位于第一、第二全景球形图像的球心位置且朝向不同方向。The N virtual fisheye lenses are located at the centers of the first and second panoramic spherical images and face in different directions.
- 根据权利要求3所述的全景视频插帧方法，其特征在于，The panoramic video frame insertion method according to claim 3, characterized in that,所述N个虚拟鱼眼镜头朝水平方向均匀排布且各个重叠区域的视场角相等。The N virtual fisheye lenses are uniformly arranged in the horizontal direction, and the viewing angles of each overlapping area are equal.
- 根据权利要求2所述的全景视频插帧方法，其特征在于，The panoramic video frame insertion method according to claim 2, wherein:所述虚拟鱼眼镜头包括两个，两个所述虚拟鱼眼镜头背靠背放置且水平方向和垂直方向的视场角大于190°。The virtual fisheye lens includes two, and the two virtual fisheye lenses are placed back to back, and the field angles in the horizontal direction and the vertical direction are greater than 190°.
- 根据权利要求1所述全景视频插帧方法，其特征在于，所述球形图像分割方法为：通过扩展立方体投影将第一、第二全景球形图像分割成配对的上下左右前后共六组配对的第一平面图像和第二平面图像。According to the panoramic video frame insertion method according to claim 1, it is characterized in that the spherical image segmentation method is: by expanding the cube projection, the first and second panoramic spherical images are divided into six pairs of pairs of up, down, left, right, front, back, front, and back. A plane image and a second plane image.
- 根据权利要求1所述的全景视频插帧方法，其特征在于，The panoramic video frame insertion method according to claim 1, characterized in that,所述重叠区域对应的视场角不小于10°。The angle of view corresponding to the overlapping area is not less than 10°.
- 根据权利要求1所述的全景视频插帧方法，其特征在于，The panoramic video frame insertion method according to claim 1, characterized in that,第一、第二全景视频帧为连续的两帧全景视频帧。The first and second panoramic video frames are two consecutive panoramic video frames.
- 一种全景视频插帧装置，其特征在于，包括：A panoramic video frame insertion device, characterized in that it comprises:投影模块，用于将第一、第二全景视频帧分别投影至球面形成第一、第二全景球形图像；The projection module is used to project the first and second panoramic video frames to the spherical surface to form the first and second panoramic spherical images respectively;分割模块，用于将第一、第二全景球形图像通过球形图像分割方法分割成配对的N组第一图像和第二图像；Segmentation module, for dividing the first and second panoramic spherical images into paired N groups of first images and second images by spherical image segmentation method;中间插帧模块，用于对配对的N组第一图像和第二图像进行插帧以得到中间时刻的N帧第三图像；An intermediate frame interpolation module, configured to interpolate the paired N groups of first images and second images to obtain N frames of the third image at the intermediate moment;拼接模块，用于将N帧第三图像拼接成中间时刻全景视频帧；Stitching module, for splicing the 3rd image of N frame into intermediate moment panorama video frame;全景插帧模块，用于将中间时刻全景视频帧***第一、第二全景视频帧之间；Panorama frame insertion module, for inserting the panorama video frame at the intermediate moment between the first and second panorama video frames;其中，N为不小于2的自然数，且任意相邻的两个所述第一图像以及任意相邻的两个所述第二图像具有重叠区域。Wherein, N is a natural number not less than 2, and any two adjacent first images and any adjacent two second images have an overlapping area.
- 一种计算机可读存储介质，其上存储有计算机程序/指令，其特征在于，所述计算机程序/指令被处理器执行时实现权利要求1至8任意一项所述全景视频插帧方法的步骤。A computer-readable storage medium, on which computer programs/instructions are stored, characterized in that, when the computer program/instructions are executed by a processor, the steps of the panoramic video frame insertion method described in any one of claims 1 to 8 are implemented .
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111095045.8A CN113873259A (en) | 2021-09-17 | 2021-09-17 | Panoramic video frame insertion method and device and computer readable storage medium |
| CN202111095045.8 | 2021-09-17 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023040932A1 true WO2023040932A1 (en) | 2023-03-23 |
Family
ID=78996550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2022/118916 WO2023040932A1 (en) | 2021-09-17 | 2022-09-15 | Panoramic video frame interpolation method and apparatus, and computer-readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113873259A (en) |
| WO (1) | WO2023040932A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113873259A (en) * | 2021-09-17 | 2021-12-31 | 影石创新科技股份有限公司 | Panoramic video frame insertion method and device and computer readable storage medium |
| WO2024103366A1 (en) * | 2022-11-18 | 2024-05-23 | 影石创新科技股份有限公司 | Panoramic unmanned aerial vehicle |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080291201A1 (en) * | 2007-05-25 | 2008-11-27 | Google, Inc. | Efficient rendering of panoramic images, and applications thereof |
| CN106023070A (en) * | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| CN109246477A (en) * | 2018-08-17 | 2019-01-18 | 南京泓众电子科技有限公司 | A kind of panoramic video frame interpolation method and device |
| CN110351494A (en) * | 2018-04-02 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of panoramic video synthetic method, device and electronic equipment |
| CN112351287A (en) * | 2020-09-30 | 2021-02-09 | 深圳市为汉科技有限公司 | Multimedia data processing method and related product thereof |
| CN113873259A (en) * | 2021-09-17 | 2021-12-31 | 影石创新科技股份有限公司 | Panoramic video frame insertion method and device and computer readable storage medium |
-
2021
- 2021-09-17 CN CN202111095045.8A patent/CN113873259A/en active Pending
-
2022
- 2022-09-15 WO PCT/CN2022/118916 patent/WO2023040932A1/en unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080291201A1 (en) * | 2007-05-25 | 2008-11-27 | Google, Inc. | Efficient rendering of panoramic images, and applications thereof |
| CN106023070A (en) * | 2016-06-14 | 2016-10-12 | 北京岚锋创视网络科技有限公司 | Real-time panoramic splicing method and device |
| CN110351494A (en) * | 2018-04-02 | 2019-10-18 | 杭州海康威视数字技术股份有限公司 | A kind of panoramic video synthetic method, device and electronic equipment |
| CN109246477A (en) * | 2018-08-17 | 2019-01-18 | 南京泓众电子科技有限公司 | A kind of panoramic video frame interpolation method and device |
| CN112351287A (en) * | 2020-09-30 | 2021-02-09 | 深圳市为汉科技有限公司 | Multimedia data processing method and related product thereof |
| CN113873259A (en) * | 2021-09-17 | 2021-12-31 | 影石创新科技股份有限公司 | Panoramic video frame insertion method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113873259A (en) | 2021-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023040932A1 (en) | Panoramic video frame interpolation method and apparatus, and computer-readable storage medium | |
| TWI712918B (en) | Method, device and equipment for displaying images of augmented reality | |
| WO2021012856A1 (en) | Method for photographing panoramic image | |
| JP4351996B2 (en) | Method for generating a stereoscopic image from a monoscope image | |
| US6133943A (en) | Method and apparatus for producing a composite image | |
| CN108122191B (en) | Method and device for splicing fisheye images into panoramic image and panoramic video | |
| US20110085027A1 (en) | Image processing device and method, and program | |
| CN110764613A (en) | Eye movement tracking calibration method based on head-mounted eye movement module | |
| CN112085659B (en) | Panorama splicing and fusing method and system based on dome camera and storage medium | |
| CN105072314A (en) | Virtual studio implementation method capable of automatically tracking objects | |
| CN108038820B (en) | Method and device for achieving bullet time shooting effect and panoramic camera | |
| US20120242780A1 (en) | Image processing apparatus and method, and program | |
| CN111866523B (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| WO2013120308A1 (en) | Three dimensions display method and system | |
| WO2024002023A1 (en) | Method and apparatus for generating panoramic stereoscopic image, and electronic device | |
| WO2022121963A1 (en) | Image occlusion detection method and apparatus, photographing device and medium | |
| CN108282650B (en) | Naked eye three-dimensional display method, device and system and storage medium | |
| JP2004186922A (en) | Wide range photographing method using a plurality of cameras | |
| CN108205236B (en) | Panoramic camera and lens thereof | |
| WO2017092261A1 (en) | Camera module, mobile terminal, and image shooting method and apparatus therefor | |
| CN108234904B (en) | Multi-video fusion method, device and system | |
| JPH0865715A (en) | Method and device for stereoscopic video display | |
| CN115065782A (en) | Scene acquisition method, acquisition device, camera equipment and storage medium | |
| Swaminathan et al. | Polycameras: Camera clusters for wide angle imaging | |
| WO2022036512A1 (en) | Data processing method and device, terminal, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22869301 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |