TWI698117B - Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus - Google Patents

Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus Download PDF

Info

Publication number
TWI698117B
TWI698117B TW107127500A TW107127500A TWI698117B TW I698117 B TWI698117 B TW I698117B TW 107127500 A TW107127500 A TW 107127500A TW 107127500 A TW107127500 A TW 107127500A TW I698117 B TWI698117 B TW I698117B
Authority
TW
Taiwan
Prior art keywords
multimedia file
data track
positions
file
image object
Prior art date
Application number
TW107127500A
Other languages
Chinese (zh)
Other versions
TW202008781A (en
Inventor
袁嘉尚
Original Assignee
宏碁股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 宏碁股份有限公司 filed Critical 宏碁股份有限公司
Priority to TW107127500A priority Critical patent/TWI698117B/en
Priority to US16/532,511 priority patent/US11211097B2/en
Publication of TW202008781A publication Critical patent/TW202008781A/en
Application granted granted Critical
Publication of TWI698117B publication Critical patent/TWI698117B/en

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/102—Programmed access in sequence to addressed parts of tracks of operating record carriers

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Television Signal Processing For Recording (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Abstract

A generating method and a playing method of multimedia file, a multimedia file generation apparatus and a multimedia file playback apparatus are provided. The playing method of multimedia file includes following steps. A multimedia file including a panoramic video associated with a time axis is received. A first data track of the multimedia file is fetched to obtain object positions of the first image object in the panoramic video with respect to the time axis. When playing the panoramic video, an icon corresponding to the first image object is displayed in a display frame of a screen. In response to detecting a selection operation performed on the icon, a playback field of view (FoV) for playing the panoramic video is determined according to a first object position recorded in a first data track, and display frames including the first image object are played based on the playback FoV.

Description

多媒體檔案的產生方法與播放方法、多媒體檔案產生裝置與多媒體檔案播放裝置Multimedia file generating method and playing method, multimedia file generating device and multimedia file playing device

本發明是有關於一種影片播放技術，且特別是有關於一種多媒體檔案的產生方法與多媒體檔案產生裝置，以及多媒體檔案的播放方法與多媒體檔案播放裝置。The invention relates to a video playback technology, and in particular to a multimedia file generation method and multimedia file generation device, as well as a multimedia file playback method and multimedia file playback device.

現今360度影片以及全景相機越來越流行，使用者已經可透過電腦螢幕觀看360度影片（亦稱為全景影片）。透過多個鏡頭所拍攝的不同角度的影像並經過後製影像縫合處理，可產生無視野死角的360度影片，其能夠提供使用者身歷其境的效果。Nowadays, 360-degree videos and panoramic cameras are becoming more and more popular, and users can already watch 360-degree videos (also known as panoramic videos) through the computer screen. Images of different angles shot through multiple lenses and processed by post-production image stitching can produce 360-degree videos with no blind spots in the field of view, which can provide the user with an immersive effect.

當使用者利用電腦螢幕觀賞360度影片時，使用者事實上只能看到整個360度場景中的一隅。使用者需要調整360度影片的播放視角，才有辦法觀賞到360度場景中的不同角落。因此，當使用者觀賞360度影片時，使用者需要手動調整播放視角來搜尋感興趣物件，此舉大幅降低使用者觀看360度影片的便利性。再者，受限於一般消費性電子產品的效能，一般消費性電子裝置也難以針對360度影片進行即時的影像物件辨識與追蹤。因此，當使用者想要鎖定某一個感興趣物件進行觀看時，使用者需要隨著感興趣物件的移動而隨時隨地手動控制播放視角。When a user watches a 360-degree video on a computer screen, the user can actually only see a corner of the entire 360-degree scene. The user needs to adjust the viewing angle of the 360-degree video to be able to watch different corners of the 360-degree scene. Therefore, when a user watches a 360-degree video, the user needs to manually adjust the viewing angle to search for objects of interest, which greatly reduces the convenience for the user to watch the 360-degree video. Furthermore, limited by the performance of general consumer electronic products, it is also difficult for general consumer electronic devices to identify and track real-time image objects for 360-degree videos. Therefore, when the user wants to lock a certain object of interest for viewing, the user needs to manually control the playback angle of view anytime and anywhere as the object of interest moves.

有鑑於此，本發明提供一種多媒體檔案的產生方法與多媒體檔案產生裝置，可基於影像物件的位置資訊建立多媒體檔案的特定資料軌，以產生包括全景影片且記錄有位置資訊的該多媒體檔案。In view of this, the present invention provides a multimedia file generating method and a multimedia file generating device, which can create a specific data track of a multimedia file based on the location information of an image object to generate the multimedia file including a panoramic video and recorded with location information.

並且，本發明提供一種多媒體檔案的播放方法與多媒體檔案播放裝置，可依據多媒體檔案中的特定資料軌獲取影像物件的位置資訊，以依據使用者感興趣的影像物件的位置資訊動態調整播放視角。In addition, the present invention provides a multimedia file playback method and multimedia file playback device, which can obtain position information of image objects according to a specific data track in the multimedia file, and dynamically adjust the playback angle of view according to the position information of the image object that the user is interested in.

本發明的實施例提供一種多媒體檔案的產生方法，適用於多媒體檔案產生裝置，所述方法包括下列步驟。取得關聯於時間軸的全景影片，其中此全景影片包括至少一影像物件。取得此影像物件相對於時間軸的多個物件位置。將這些物件位置製作成物件位置檔案。依據此物件位置檔案產生多媒體檔案的至少一資料軌，以生成包括全景影片且記錄有物件位置的多媒體檔案。The embodiment of the present invention provides a method for generating multimedia files, which is suitable for a device for generating multimedia files. The method includes the following steps. Obtain a panoramic video related to the time axis, where the panoramic video includes at least one image object. Obtain multiple object positions of this image object relative to the timeline. Make these object locations into object location files. At least one data track of the multimedia file is generated based on the object location file to generate a multimedia file including a panoramic video and recording the object location.

本發明的實施例提供一種多媒體檔案產生裝置，包括儲存裝置與處理器。儲存裝置儲存有多個模組。處理器耦接儲存裝置，載入並執行儲存裝置中的模組。所述模組包括影片取得模組、位置取得模組、檔案製作模組，以及檔案嵌入模組。影片取得模組取得關聯於時間軸的全景影片，其中此全景影片包括至少一影像物件。位置取得模組取得影像物件相對於時間軸的多個物件位置。檔案製作模組將這些物件位置製作成物件位置檔案。檔案嵌入模組依據物件位置檔案產生多媒體檔案的至少一資料軌，以生成包括全景影片且記錄有物件位置的多媒體檔案。An embodiment of the present invention provides a multimedia file generating device, including a storage device and a processor. The storage device stores multiple modules. The processor is coupled to the storage device, and loads and executes the modules in the storage device. The modules include a video acquisition module, a location acquisition module, a file production module, and a file embedding module. The video obtaining module obtains a panoramic video associated with the time axis, where the panoramic video includes at least one image object. The position obtaining module obtains the position of multiple objects of the image object relative to the time axis. The file creation module creates these object locations into object location files. The file embedding module generates at least one data track of the multimedia file according to the object location file to generate the multimedia file including the panoramic video and recording the object location.

對應的，本發明的實施例提供一種多媒體檔案的播放方法，適用於多媒體檔案播放裝置，所述方法包括下列步驟。接收包括關聯於時間軸的全景影片的多媒體檔案。提取多媒體檔案的第一資料軌，以取得全景影片中第一影像物件相對於時間軸的多個第一物件位置。當播放全景影片，顯示對應於第一影像物件的圖示於螢幕的畫面。響應於偵測到施於圖示的選擇操作，依據第一資料軌所記錄的第一物件位置決定用以播放全景影片的播放視角，並基於播放視角播放包括第一影像物件的畫面。Correspondingly, an embodiment of the present invention provides a multimedia file playback method, which is suitable for a multimedia file playback device, and the method includes the following steps. Receive multimedia files including panoramic videos associated with the time axis. The first data track of the multimedia file is extracted to obtain the positions of the first image objects relative to the time axis in the panoramic video. When the panoramic video is played, an icon corresponding to the first image object is displayed on the screen. In response to detecting the selection operation performed on the icon, the playback angle of view for playing the panoramic video is determined according to the position of the first object recorded in the first data track, and the screen including the first image object is played based on the playback angle.

對應的，本發明的實施例提供一種多媒體檔案播放裝置，包括螢幕、儲存有多個模組的儲存裝置，以及處理器。處理器耦接儲存裝置與螢幕，載入並執行儲存裝置中的所述模組。所述模組包括影片接收模組、資料軌提取模組、介面提供模組，以及影片播放模組。影片接收模組接收包括關聯於時間軸的全景影片的多媒體檔案。資料軌提取模組提取多媒體檔案的第一資料軌，以取得全景影片中第一影像物件相對於時間軸的多個第一物件位置。當播放全景影片，介面提供模組顯示對應於第一影像物件的圖示於螢幕的畫面。影片播放模組響應於偵測到施於圖示的選擇操作，依據第一資料軌所記錄的第一物件位置決定用以播放全景影片的播放視角，並基於播放視角播放包括第一影像物件的畫面。Correspondingly, an embodiment of the present invention provides a multimedia file playback device, including a screen, a storage device storing a plurality of modules, and a processor. The processor is coupled to the storage device and the screen, and loads and executes the module in the storage device. The modules include a video receiving module, a data track extraction module, an interface providing module, and a video playback module. The video receiving module receives a multimedia file including a panoramic video related to the time axis. The data track extraction module extracts the first data track of the multimedia file to obtain a plurality of first object positions of the first image object relative to the time axis in the panoramic video. When the panoramic video is played, the interface providing module displays the icon corresponding to the first image object on the screen. In response to detecting the selection operation performed on the icon, the video playback module determines the playback angle of view for playing the panoramic video based on the position of the first object recorded in the first data track, and based on the playback angle of view, plays the first image object Picture.

基於上述，多媒體檔案產生裝置可依據出現於全景影片中之影像物件的物件位置建立多媒體檔案的特定資料軌。基此，當播放全景影片時，多媒體檔案播放裝置可從多媒體檔案的特定資料軌中獲取影像物件於全景影片中的物件位置，進而依據影像物件的物件位置動態決定播放視角。藉此，使用者不須隨時手動調整全景影片的播放視角，就可以鎖定全景影片中的特定影像物件進行觀看。Based on the above, the multimedia file generating device can create a specific data track of the multimedia file according to the object position of the image object appearing in the panoramic video. Based on this, when the panoramic video is played, the multimedia file playback device can obtain the object position of the image object in the panoramic video from the specific data track of the multimedia file, and then dynamically determine the playback angle of view according to the object position of the image object. In this way, the user does not need to manually adjust the viewing angle of the panoramic video at any time, and can lock a specific image object in the panoramic video for viewing.

為讓本發明的上述特徵和優點能更明顯易懂，下文特舉實施例，並配合所附圖式作詳細說明如下。In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

本發明的部份實施例接下來將會配合附圖來詳細描述，以下的描述所引用的元件符號，當不同附圖出現相同的元件符號將視為相同或相似的元件。這些實施例只是本發明的一部份，並未揭示所有本發明的可實施方式。更確切的說，這些實施例只是本發明的專利申請範圍中的方法以及頭戴顯示裝置的範例。Part of the embodiments of the present invention will be described in detail in conjunction with the accompanying drawings. The reference symbols in the following description will be regarded as the same or similar elements when the same symbol appears in different drawings. These embodiments are only a part of the present invention, and do not disclose all the possible implementation modes of the present invention. More precisely, these embodiments are only examples of methods and head-mounted display devices within the scope of the patent application of the present invention.

圖1是依照本發明實施例所繪示之多媒體檔案產生裝置的方塊圖，但此僅是為了方便說明，並不用以限制本發明。首先圖1先介紹多媒體檔案產生裝置之所有構件以及配置關係，詳細功能將配合圖2一併揭露。FIG. 1 is a block diagram of a multimedia file generating device according to an embodiment of the present invention, but this is only for convenience of description and is not intended to limit the present invention. First, Figure 1 first introduces all the components and configuration relationships of the multimedia file generating device, and detailed functions will be disclosed in conjunction with Figure 2.

請參照圖1，多媒體檔案產生裝置10可以是具備運算能力的各式電子裝置，例如桌上型電腦、筆記型電腦或伺服器等等，本發明對此並不限制。多媒體檔案產生裝置10包括處理器110與儲存裝置120，其功能分述如下：Please refer to FIG. 1, the multimedia file generating device 10 may be various electronic devices with computing capabilities, such as a desktop computer, a notebook computer, or a server, etc. The present invention is not limited thereto. The multimedia file generating device 10 includes a processor 110 and a storage device 120, and its functions are described as follows:

儲存裝置120例如是任何型態的固定式或可移動式隨機存取記憶體（random access memory，RAM）、唯讀記憶體（read-only memory，ROM）、快閃記憶體（flash memory）或類似元件或上述元件的組合。在本實施例中，儲存裝置120用以記錄影片取得模組121、位置取得模組122、檔案製作模組123，以及檔案嵌入模組123。The storage device 120 is, for example, any type of fixed or removable random access memory (random access memory, RAM), read-only memory (read-only memory, ROM), flash memory (flash memory) or Similar elements or combinations of the above elements. In this embodiment, the storage device 120 is used to record the video acquisition module 121, the position acquisition module 122, the file production module 123, and the file embedding module 123.

處理器110例如是中央處理單元（Central Processing Unit，CPU），或是其他可程式化之一般用途或特殊用途的微處理器（Microprocessor）、數位訊號處理器（Digital Signal Processor，DSP）、可程式化控制器、特殊應用積體電路（Application Specific Integrated Circuits，ASIC）、可程式化邏輯裝置（Programmable Logic Device，PLD）或其他類似裝置或這些裝置的組合，其與儲存裝置120連接。The processor 110 is, for example, a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessor), digital signal processor (Digital Signal Processor, DSP), programmable A controller, a special application integrated circuit (Application Specific Integrated Circuits, ASIC), a programmable logic device (Programmable Logic Device, PLD) or other similar devices or a combination of these devices are connected to the storage device 120.

在本實施例中，儲存在儲存裝置120中的模組例如是電腦程式，而可由處理器110載入，據以執行本實施例的多媒體檔案的產生方法。In this embodiment, the modules stored in the storage device 120 are, for example, computer programs, which can be loaded by the processor 110 to execute the multimedia file generation method of this embodiment.

圖2是依照本發明實施例所繪示之多媒體檔案的產生方法的流程圖，而圖2的方法流程可以圖1的多媒體檔案產生裝置10的各元件實現。請同時參照圖1及圖2，以下即搭配圖1中多媒體檔案產生裝置10的各項元件與裝置，說明本實施例之多媒體檔案的產生方法的詳細步驟。FIG. 2 is a flowchart of a method for generating a multimedia file according to an embodiment of the present invention, and the method flow of FIG. 2 can be implemented by the components of the multimedia file generating device 10 of FIG. 1. Please refer to FIGS. 1 and 2 at the same time. The following is a description of the detailed steps of the multimedia file generating method of this embodiment in conjunction with the various components and devices of the multimedia file generating device 10 in FIG. 1.

於步驟S201，影片取得模組121取得關聯於時間軸的全景影片，其中此全景影片包括至少一影像物件。在此，影片取得模組121可以是自多媒體檔案產生裝置10本身的影像擷取模組（未繪示）或者是自其它電子裝置取得全景影片。全景影片也可稱為360度影片，其由對應至一時間軸上不同時間戳記（timestamp）的視頻幀所構成，而這些視頻幀為以特定格式保存的360度影像。上述特定格式例如是Equiangular格式等等。需說明的是，於本發明的實施例中，全景影片包括拍攝至少一被攝物件而產生的至少一影像物件，亦即，所述影像物件呈現於全景影片的視頻幀之中。全景影片中的影像物件例如是人臉，但本發明對此並不限制，其也可能是其他種類的影像物件。In step S201, the video obtaining module 121 obtains a panoramic video associated with the time axis, wherein the panoramic video includes at least one image object. Here, the video obtaining module 121 may be an image capturing module (not shown) of the multimedia file generating device 10 itself or a panoramic video obtained from other electronic devices. A panoramic movie can also be called a 360-degree movie, which is composed of video frames corresponding to different timestamps on a time axis, and these video frames are 360-degree images saved in a specific format. The above-mentioned specific format is, for example, the Equiangular format. It should be noted that, in the embodiment of the present invention, the panoramic video includes at least one image object generated by shooting at least one object, that is, the image object is presented in a video frame of the panoramic video. The image object in the panoramic video is, for example, a human face, but the present invention is not limited to this, and it may also be other types of image objects.

於步驟S202，位置取得模組122取得影像物件相對於時間軸的多個物件位置。於一實施例中，影像物件的多個物件位置可由影片編輯者事先透過肉眼觀看並經由手動編輯而產生。換言之，透過影片編輯者肉眼觀賞全景影片並標註影像物件的多個物件位置，位置取得模組122可取得影像物件於一立體座標系下的多個物件位置。或者，於一實施例中，影像物件相對於時間軸的多個物件位置可由影像處理技術的物件偵測與辨識演算法而自動產生。換言之，透過利用物件偵測與辨識演算法追蹤全景影片中的特定影像物件，位置取得模組122可取得影像物件於一立體座標系下相對於不同時間區間的多個物件位置。影像物件的物件位置可例如由球座標系的球座標來表示。In step S202, the position obtaining module 122 obtains a plurality of object positions of the image object relative to the time axis. In one embodiment, the multiple object positions of the image object can be generated by the video editor through visual observation in advance and manual editing. In other words, by viewing the panoramic video with the naked eye of the video editor and marking the multiple object positions of the image object, the position obtaining module 122 can obtain multiple object positions of the image object in a three-dimensional coordinate system. Alternatively, in one embodiment, the multiple object positions of the image object relative to the time axis can be automatically generated by an object detection and recognition algorithm of image processing technology. In other words, by using an object detection and recognition algorithm to track a specific image object in the panoramic video, the position obtaining module 122 can obtain multiple object positions of the image object in a three-dimensional coordinate system with respect to different time intervals. The object position of the image object can be represented by the spherical coordinates of the spherical coordinate system, for example.

於一實施例中，此影像物件的多個物件位置分別對應至時間軸上的多個時間區間。也就是說，影像物件的這些物件位置是可依據固定或不固定的時間間隔而被取樣出來。請參照圖3A，圖3A是依據本發明實施例所繪示之對應至多個時間區間的多個物件位置的示意圖。對於一影像物件而言，位置取得模組122可取得對應於時間區間P1的物件位置(r1,θ1,ψ1)、對應於時間區間P2的物件位置(r2,θ2,ψ2)，以及對應於時間區間P3的物件位置(r3,θ3,ψ3)。需說明的是，時間區間P1~P3的時間長度可相同或相異，本發明對此並不限制。In one embodiment, the multiple object positions of the image object correspond to multiple time intervals on the time axis, respectively. In other words, the object positions of the image objects can be sampled according to fixed or non-fixed time intervals. Please refer to FIG. 3A. FIG. 3A is a schematic diagram illustrating the positions of multiple objects corresponding to multiple time intervals according to an embodiment of the present invention. For an image object, the position obtaining module 122 can obtain the object position (r1, θ1, ψ1) corresponding to the time interval P1, the object position (r2, θ2, ψ2) corresponding to the time interval P2, and the corresponding time Object position (r3, θ3, ψ3) in interval P3. It should be noted that the time lengths of the time intervals P1 to P3 may be the same or different, which is not limited by the present invention.

此外，於一實施例中，全景影片中影像物件的數量可以是兩個以上。像是，全景影片中的至少一影像物件可包括第一影像物件與第二影像物件。對應的，相對於時間軸的物件位置將包括第一影像物件的多個第一物件位置與第二影像物件的多個第二物件位置。請參照圖3B，圖3B是依據本發明實施例所繪示之對應至多個時間區間的多個物件位置的示意圖。對於第一影像物件而言，位置取得模組122可取得對應於時間區間P1的物件位置(r4,θ4,ψ4)與對應於時間區間P2的物件位置(r5,θ5,ψ5)。對於第二影像物件而言，位置取得模組122可取得對應於時間區間P1的物件位置(r6,θ6,ψ6)以及對應於時間區間P2的物件位置(r7,θ7,ψ7)。In addition, in one embodiment, the number of image objects in the panoramic video may be more than two. For example, the at least one image object in the panoramic video may include a first image object and a second image object. Correspondingly, the position of the object relative to the time axis will include a plurality of first object positions of the first image object and a plurality of second object positions of the second image object. Please refer to FIG. 3B. FIG. 3B is a schematic diagram of multiple object positions corresponding to multiple time intervals according to an embodiment of the present invention. For the first image object, the position obtaining module 122 can obtain the object position (r4, θ4, ψ4) corresponding to the time interval P1 and the object position (r5, θ5, ψ5) corresponding to the time interval P2. For the second image object, the position obtaining module 122 can obtain the object position (r6, θ6, ψ6) corresponding to the time interval P1 and the object position (r7, θ7, ψ7) corresponding to the time interval P2.

接著，回到圖2的流程，於步驟S203，檔案製作模組123將這些物件位置製作成物件位置檔案。具體而言，檔案製作模組123可以預設的檔案格式將對應至時間軸上各時間區間的物件位置編寫為一物件位置檔案。於一實施例中，產生物件位置檔案的方式可類似於產生影片字幕檔的方式。請參照圖4，圖4是依據本發明實施例所繪示之物件位置檔案的範例。物件位置檔案40記錄有分別命名為“物件名稱A”與“物件名稱B”的兩個影像物件於全景影片中的物件位置，而這些物件位置是以固定的時間間隔被記錄下來。圖4所示範例係以時間間隔為1秒為例，但本發明並不以此為限。例如，於時間點00:01.000，命名為“物件名稱A”的影像物件的物件位置為(r6,θ6,ψ6)，而命名為“物件名稱B”的影像物件的物件位置為(r7,θ7,ψ7)。於時間點00:02.000，命名為“物件名稱A”的影像物件的物件位置為(r8,θ8,ψ8)，而命名為“物件名稱B”的影像物件的物件位置為(r9,θ9,ψ9)。Then, returning to the flow of FIG. 2, in step S203, the file creation module 123 creates these object locations into object location files. Specifically, the file production module 123 can compile the object positions corresponding to each time interval on the time axis as an object position file in a preset file format. In one embodiment, the method of generating the object location file may be similar to the method of generating the video subtitle file. Please refer to FIG. 4, which is an example of an object location file drawn according to an embodiment of the present invention. The object position file 40 records the object positions of two image objects named "object name A" and "object name B" in the panoramic video, and these object positions are recorded at fixed time intervals. The example shown in FIG. 4 takes the time interval of 1 second as an example, but the present invention is not limited to this. For example, at time 00:01.000, the object position of the image object named "object name A" is (r6,θ6,ψ6), and the object position of the image object named "object name B" is (r7,θ7 ,ψ7). At 00:02.000, the object position of the image object named "Object name A" is (r8,θ8,ψ8), and the object position of the image object named "Object name B" is (r9,θ9,ψ9 ).

此外，於一實施例中，檔案製作模組123可將記錄為多個立體位置座標的物件位置映射成多個二維位置座標，並將這些二維位置座標記錄於物件位置檔案之中。一般而言，全景影片中的各張視頻幀系透過將全景影像映射為二維影像的方式來進行保存，像是以Equiangular格式來進行保存。這些記錄為多個立體位置座標（例如球座標)的物件位置也可被映射為二維座標系統下的二維位置座標並予以儲存，以降低物件位置檔案的資料量。In addition, in one embodiment, the file creation module 123 can map the object positions recorded as a plurality of three-dimensional position coordinates into a plurality of two-dimensional position coordinates, and record these two-dimensional position coordinates in the object position file. Generally speaking, each video frame in a panoramic video is saved by mapping the panoramic image into a two-dimensional image, such as saving in Equiangular format. These object positions recorded as multiple three-dimensional position coordinates (such as spherical coordinates) can also be mapped to two-dimensional position coordinates under a two-dimensional coordinate system and stored, so as to reduce the amount of data in the object position file.

之後，於步驟S204，檔案嵌入模組124依據物件位置檔產生多媒體檔案的至少一資料軌，以生成包括全景影片且記錄有物件位置的多媒體檔案。具體而言，圖5是依據本發明實施例所繪示之多媒體檔案架構的示意圖。多媒體檔案50包括標頭51與多媒體資料52，而多媒體資料52包括了可分類為多個資料軌的多媒體資料。換言之，多媒體檔案50可包括多個資料軌。標頭51中記錄有這些資料軌之特性的描述與這些資料軌的數目，這些資料軌可包含視訊資料軌（video track）521、音訊資料軌（audio track）522、字幕資料軌（sub-picture track）523以及物件位置資料軌（object position track）524。其中，視訊資料軌是用來對視訊資料分類；音訊資料軌則是用來對音訊資料分類，不同的音訊資料軌可代表不同的語言；字幕資料軌是用來對字幕資料分類，不同的字幕資料軌可代表不同的語言的字幕。Then, in step S204, the file embedding module 124 generates at least one data track of the multimedia file according to the object position file to generate a multimedia file including a panoramic video and recording the object position. Specifically, FIG. 5 is a schematic diagram of a multimedia file structure according to an embodiment of the present invention. The multimedia file 50 includes a header 51 and multimedia data 52, and the multimedia data 52 includes multimedia data that can be classified into multiple data tracks. In other words, the multimedia file 50 may include multiple data tracks. The header 51 records the description of the characteristics of these data tracks and the number of these data tracks. These data tracks can include a video track 521, an audio track 522, and a sub-picture track. track) 523 and object position track 524. Among them, the video data track is used to classify video data; the audio data track is used to classify audio data, and different audio data tracks can represent different languages; the subtitle data track is used to classify subtitle data, and different subtitles The data track can represent subtitles in different languages.

於一實施例中，當物件位置檔案包括第一影像物件的多個第一物件位置與第二影像物件的多個第二物件位置時（例如圖4的範例所示），檔案嵌入模組124可產生對應至第一影像物件的第一資料軌，並將物件位置檔案中的第一物件位置（例如圖4的(r4,θ4,ψ4)、(r6,θ6,ψ6)、(r6,θ6,ψ6)）嵌入至第一資料軌。另一方面，檔案嵌入模組124可產生對應至第二影像物件的第二資料軌，並將物件位置檔案中的第二物件位置（例如圖4的(r5,θ5,ψ5)、(r7,θ7,ψ7)、(r9,θ9,ψ9)）嵌入至第二資料軌。也就是說，物件位置資料軌的數目將被標註之視影像物件的數目而決定，而每一個影像物件的物件位置係由對應的物件位置資料軌來記錄。亦即不同的物件位置資料軌可代表不同的影像物件的位置資訊。In one embodiment, when the object position file includes a plurality of first object positions of a first image object and a plurality of second object positions of a second image object (for example, as shown in the example of FIG. 4), the file embedding module 124 The first data track corresponding to the first image object can be generated, and the position of the first object in the object location file (for example, (r4,θ4,ψ4), (r6,θ6,ψ6), (r6,θ6) ,ψ6)) embedded in the first data track. On the other hand, the file embedding module 124 can generate a second data track corresponding to the second image object, and combine the position of the second object in the object position file (for example, (r5,θ5,ψ5), (r7, θ7,ψ7), (r9,θ9,ψ9)) are embedded in the second data track. In other words, the number of object position data tracks will be determined by the number of image objects marked, and the object position of each image object is recorded by the corresponding object position data track. That is, different object location data tracks can represent the location information of different image objects.

值得注意的是，相較於習知的多媒體檔案，本實施例之多媒體檔案50更包括用以記錄物件位置的物件位置資料軌524。檔案嵌入模組124可依據物件位置檔案建立多媒體檔案50的至少一資料軌（即物件位置資料軌524），例如將圖4所示的物件位置檔案40中的資料嵌入至多媒體檔案50的物件位置資料軌524。於此，將特定資料嵌入至多媒體檔案50的至少一資料軌代表將特定資料嵌入至多媒體檔案50中所述資料軌的資料塊中。再者，標頭51更記錄有物件位置資料軌之特性的描述與物件位置資料軌的數目。如此一來，用以播放多媒體檔案50的播放器除了可播放多媒體檔案50中的全景影片之外，更可從物件位置資料軌524得知一或多個影像物件於全景影片中的位置資訊。It is worth noting that, compared with the conventional multimedia file, the multimedia file 50 of this embodiment further includes an object position data track 524 for recording the position of the object. The file embedding module 124 can create at least one data track (ie, the object location data track 524) of the multimedia file 50 according to the object location file, for example, embed the data in the object location file 40 shown in FIG. 4 into the object location of the multimedia file 50 Data track 524. Here, embedding the specific data into at least one data track of the multimedia file 50 means embedding the specific data into the data block of the data track in the multimedia file 50. Furthermore, the header 51 further records the description of the characteristics of the object location data track and the number of the object location data track. In this way, the player used to play the multimedia file 50 can not only play the panoramic video in the multimedia file 50, but also obtain the position information of one or more image objects in the panoramic video from the object location data track 524.

在說明完如何產生記錄有影像物件於全景影片中的物件位置的多媒體檔案之後，以下將列舉實施例說明如何依據本揭露的多媒體檔案播放全景影片。After explaining how to generate a multimedia file recording the object position of the image object in the panoramic video, the following examples illustrate how to play the panoramic video according to the disclosed multimedia file.

圖6是依照本發明實施例所繪示之多媒體檔案播放裝置的方塊圖，但此僅是為了方便說明，並不用以限制本發明。首先圖6先介紹多媒體檔案播放裝置之所有構件以及配置關係，詳細功能將配合圖7一併揭露。FIG. 6 is a block diagram of a multimedia file playback device according to an embodiment of the present invention, but this is only for convenience of description and is not intended to limit the present invention. First, Figure 6 first introduces all the components and configuration relationships of the multimedia file playback device. The detailed functions will be disclosed together with Figure 7.

請參照圖6，多媒體檔案播放裝置60可以是具備運算能力與畫面顯示能力的各式電子裝置，例如桌上型電腦、筆記型電腦、智慧型手機、平板等等，本發明對此並不限制。多媒體檔案播放裝置60包括處理器610、儲存裝置620與螢幕630。Please refer to FIG. 6, the multimedia file playback device 60 can be various electronic devices with computing capabilities and screen display capabilities, such as desktop computers, notebook computers, smart phones, tablets, etc., which is not limited by the present invention. . The multimedia file playback device 60 includes a processor 610, a storage device 620, and a screen 630.

儲存裝置620例如是任何型態的固定式或可移動式隨機存取記憶體、唯讀記憶體、快閃記憶體或類似元件或上述元件的組合。在本實施例中，儲存裝置620用以記錄影片接收模組621、資料軌提取模組622、介面提供模組623，以及影片播放模組624。於一實施例中，上述模組可實施為一軟體播放器。The storage device 620 is, for example, any type of fixed or removable random access memory, read-only memory, flash memory or similar components or a combination of the above components. In this embodiment, the storage device 620 is used to record the video receiving module 621, the data track extraction module 622, the interface providing module 623, and the video playback module 624. In one embodiment, the above-mentioned module can be implemented as a software player.

處理器910例如是中央處理單元，或是其他可程式化之一般用途或特殊用途的微處理器、數位訊號處理器、可程式化控制器、特殊應用積體電路、可程式化邏輯裝置或其他類似裝置或這些裝置的組合，其與儲存裝置620連接。The processor 910 is, for example, a central processing unit, or other programmable general-purpose or special-purpose microprocessors, digital signal processors, programmable controllers, special application integrated circuits, programmable logic devices, or other A similar device or a combination of these devices is connected to the storage device 620.

螢幕630其係用以顯示多媒體檔案播放裝置60輸出的畫面而提供給使用者觀看。在本實施例中，多媒體檔案播放裝置60例如是液晶顯示器（Liquid Crystal Display，LCD）、發光二極體（Light-Emitting Diode，LED）顯示器、場發射顯示器（Field Emission Display，FED）或其他種類的顯示器。The screen 630 is used to display the screen output by the multimedia file playback device 60 for the user to watch. In this embodiment, the multimedia file playback device 60 is, for example, a Liquid Crystal Display (LCD), a Light-Emitting Diode (LED) display, a Field Emission Display (FED) or other types. Monitor.

在本實施例中，儲存在儲存裝置620中的模組例如是電腦程式，而可由處理器610載入，據以執行本實施例的多媒體檔案的播放方法。In this embodiment, the module stored in the storage device 620 is, for example, a computer program, which can be loaded by the processor 610 to execute the multimedia file playback method of this embodiment.

圖7是依照本發明實施例所繪示之多媒體檔案的播放方法的流程圖，而圖7的方法流程可以圖6的多媒體檔案播放裝置60的各元件實現。請同時參照圖6及圖7，以下即搭配圖6中多媒體檔案產生裝置60的各項元件與裝置，說明本實施例之多媒體檔案的播放方法的詳細步驟。FIG. 7 is a flowchart of a multimedia file playback method according to an embodiment of the present invention, and the method flow of FIG. 7 can be implemented by the components of the multimedia file playback device 60 of FIG. 6. Please refer to FIG. 6 and FIG. 7 at the same time. The following is a description of the detailed steps of the multimedia file playback method of this embodiment in conjunction with the components and devices of the multimedia file generating device 60 in FIG. 6.

於步驟S701，影片接收模組621接收包括關聯於時間軸的全景影片的多媒體檔案。影片接收模組621可經由有線或無線網路接收包括全景影片的多媒體檔案，也可讀取儲存於儲存裝置620或其他外接儲存裝置中的多媒體檔案。於步驟S702，資料軌提取模組622提取多媒體檔案的第一資料軌，以取得全景影片中第一影像物件相對於時間軸的多個第一物件位置。具體而言，資料軌提取模組622可解多工（demux）多媒體檔案，以獲取對應至各個資料軌的多媒體資料。於一實施例中，多媒體檔案的資料軌可包括視頻資料軌、音訊資料軌、字幕資料軌以及物件位置資料軌。資料軌提取模組622可從多媒體檔案提取出分類至物件位置資料軌的多媒體資料，分類至物件位置資料軌的多媒體資料即為全景影片中第一影像物件相對於時間軸的多個第一物件位置。關於物件位置資料軌之中的物件位置已於前述實施例詳加描述，於此不再贅述。相似的，資料軌提取模組622也可從多媒體檔案提取出分類至視頻資料軌的視頻資料，並解碼視頻資料而獲取全景影片的多個視頻幀。In step S701, the video receiving module 621 receives the multimedia file including the panoramic video related to the time axis. The video receiving module 621 can receive multimedia files including panoramic videos via a wired or wireless network, and can also read multimedia files stored in the storage device 620 or other external storage devices. In step S702, the data track extraction module 622 extracts the first data track of the multimedia file to obtain a plurality of first object positions of the first image object relative to the time axis in the panoramic video. Specifically, the data track extraction module 622 can demux multimedia files to obtain multimedia data corresponding to each data track. In one embodiment, the data track of the multimedia file may include a video data track, an audio data track, a subtitle data track, and an object position data track. The data track extraction module 622 can extract the multimedia data classified into the object location data track from the multimedia file. The multimedia data classified into the object location data track are the multiple first objects of the first image object relative to the time axis in the panoramic video. position. The object location in the object location data track has been described in detail in the foregoing embodiment, and will not be repeated here. Similarly, the data track extraction module 622 can also extract video data classified into a video data track from the multimedia file, and decode the video data to obtain multiple video frames of the panoramic video.

之後，於步驟S703，當播放全景影片時，介面提供模組623顯示對應於第一影像物件的圖示於螢幕610的畫面。具體而言，介面提供模組623可提供播放器的使用者介面，其可包括畫面播放區與播放控制列。值得注意的是，藉由解析多媒體檔案的標頭（例如圖5所示的標頭51）中物件位置資料軌的數目，介面提供模組623可得知全景影片的影片內容包括多少個事先被標註之影像物件。於是，於播放全景影片的同時，介面提供模組623可顯示事先被標註之影像物件的圖示於螢幕610的畫面。在此的圖示可以是任何形狀的互動式物件，並且各個圖示中將會呈現所對應的影像物件的名稱或代表圖案，以快速導覽使用者有關於全景影像的重點。此外，各個圖示可顯示於播放畫面的邊緣或是播放器的播放控制列中，以避免影響使用者觀看全景影片。After that, in step S703, when the panoramic video is played, the interface providing module 623 displays an icon corresponding to the first image object on the screen 610. Specifically, the interface providing module 623 can provide a user interface of the player, which can include a screen play area and a play control bar. It is worth noting that by parsing the number of object location data tracks in the header of the multimedia file (such as header 51 shown in Figure 5), the interface providing module 623 can know how many pre-recorded video content of the panoramic video includes The marked image object. Thus, while the panoramic video is being played, the interface providing module 623 can display the image of the previously marked image object on the screen 610. The icons here can be interactive objects of any shape, and the names or representative patterns of the corresponding image objects will be displayed in each icon, so as to quickly guide the user to the key points of the panoramic image. In addition, each icon can be displayed on the edge of the playback screen or in the playback control bar of the player to avoid affecting the user to watch the panoramic video.

接著，處理器610將會不斷地偵測使用者是否針對任何圖示進行選擇，而響應於偵測到使用者施於某一圖示的選擇操作。因此，於步驟S704，影片播放模組624響應於偵測到施於圖示的選擇操作，依據第一資料軌所記錄的第一物件位置決定用以播放全景影片的播放視角，並基於播放視角播放包括第一影像物件的畫面。也就是說，當使用者選擇對應至第一影像物件的圖示時，影片播放模組624可從物件位置資料軌得知第一影像物件於全景影片中當下的物件位置。接著，影片播放模組624可依據第一影像物件當下的物件位置決定播放視角，而播放畫面將會從全景影片的預設區域位移至第一影像物件所在的第一區域，以讓使用者可快速地觀看所選擇的重點對象。Then, the processor 610 will continuously detect whether the user selects any icon, and respond to detecting the user's selection operation on a certain icon. Therefore, in step S704, the video playback module 624, in response to detecting the selection operation performed on the icon, determines the playback angle of view for playing the panoramic video according to the position of the first object recorded in the first data track, and based on the playback angle of view Play the screen including the first image object. That is, when the user selects the icon corresponding to the first image object, the video playback module 624 can learn the current object position of the first image object in the panoramic video from the object position data track. Then, the video playback module 624 can determine the playback angle of view according to the current object position of the first image object, and the playback screen will shift from the default area of the panoramic video to the first area where the first image object is located, so that the user can Quickly view the selected key objects.

值得注意的是，被選定的第一影像物件的第一物件位置可能改變。以圖3B為例，第一影像物件的第一物件位置可能從(r4,θ4,ψ4)轉變為(r6,θ6,ψ6)。若未調整播放視角，第一影像物件可能消失於播放畫面之中。於一實施例中，影片播放模組624可響應於識別到第一物件位置的改變，重新依據改變後的第一物件位置來切換播放視角。以圖3B為例，響應於第一影像物件的第一物件位置從(r4,θ4,ψ4)改變為(r6,θ6,ψ6)，影片播放模組624將播放視角從第一視角切換為第二視角。對應的，播放畫面將會從原本的第一區域調整為第一影像物件所在的第二區域。亦即，影片播放模組624於時間區間P1中以第一視角播放全景影片的第一區域，接著於時間區間P2中以第二視角播放全景影片的第二區域。如此，使用者可在沒有手動調整播放視角的情況下持續觀看到所選擇的重點對象。It is worth noting that the position of the first object of the selected first image object may change. Taking FIG. 3B as an example, the position of the first object of the first image object may be changed from (r4, θ4, ψ4) to (r6, θ6, ψ6). If the playback angle is not adjusted, the first image object may disappear in the playback screen. In one embodiment, the video playback module 624 may, in response to recognizing a change in the position of the first object, switch the playback perspective again according to the changed position of the first object. Taking FIG. 3B as an example, in response to the first object position of the first image object being changed from (r4,θ4,ψ4) to (r6,θ6,ψ6), the video playback module 624 switches the playback angle of view from the first angle of view to the first angle of view. Two perspectives. Correspondingly, the playback screen will be adjusted from the original first area to the second area where the first image object is located. That is, the video playing module 624 plays the first area of the panoramic movie from the first perspective in the time interval P1, and then plays the second area of the panoramic movie from the second perspective in the time interval P2. In this way, the user can continuously watch the selected key object without manually adjusting the playback angle of view.

可知的，影像物件的數目可以為兩個以上。於一實施例中，除了提取多媒體檔案的第一資料軌，資料軌提取模組622更可提取多媒體檔案的第二資料軌，以取得全景影片中第二影像物件相對於該時間軸的多個第二物件位置。於是，當播放全景影片，介面提供模組623也將顯示對應於第二影像物件的另一圖示於螢幕610的畫面。於是，影片播放模組624響應於偵測到施於另一圖示的選擇操作，依據第二資料軌所記錄的第二物件位置切換播放視角，並基於所切換的播放視角播放包括第二影像物件的畫面。It can be seen that the number of image objects can be more than two. In one embodiment, in addition to extracting the first data track of the multimedia file, the data track extraction module 622 can also extract the second data track of the multimedia file to obtain a plurality of second image objects in the panoramic video relative to the time axis. The location of the second object. Therefore, when the panoramic video is played, the interface providing module 623 will also display another icon on the screen 610 corresponding to the second image object. Therefore, the video playback module 624, in response to detecting the selection operation performed on another icon, switches the playback angle of view according to the position of the second object recorded in the second data track, and plays the second image based on the switched playback angle of view. The screen of the object.

舉例而言，圖8A與8B是依照本發明實施例所繪示之多媒體檔案的播放範例示意圖。請先參照圖8A，當多媒體檔案播放裝置60播放本揭露所產生的多媒體檔案時，全景影片隨時間軸播放。使用者可藉由對虛擬控制鈕82的操作來調整全景影片的播放視角。多媒體檔案播放裝置60可依據多媒體檔案的標頭而獲取物件位置資料軌之描述的特性與物件位置資料軌的數目，從而得知事先被標註之影像物件的數目與物件名稱等等。於本範例中，假設事先被標註之影像物件的數目為3，因此多媒體檔案播放裝置60將顯示三個圖示I1～I3於畫面80，且這三個圖示I1～I3分別呈現有此三個影像物件的代表名稱‘A’、‘B’、‘C’。For example, FIGS. 8A and 8B are schematic diagrams showing examples of playing multimedia files according to an embodiment of the present invention. Please refer to FIG. 8A first. When the multimedia file playback device 60 plays the multimedia file generated in this disclosure, the panoramic video is played along the time axis. The user can adjust the viewing angle of the panoramic video by operating the virtual control button 82. The multimedia file playback device 60 can obtain the characteristics of the description of the object location data track and the number of the object location data track according to the header of the multimedia file, thereby knowing the number of image objects that are marked in advance, the name of the object, and so on. In this example, assuming that the number of previously marked image objects is 3, the multimedia file playback device 60 will display three icons I1 ~ I3 on the screen 80, and the three icons I1 ~ I3 are presented with these three The representative names of each image object are'A','B', and'C'.

假設使用者欲觀看對應於圖示I1的影像物件83（即關鍵人物A），響應於偵測到使用者針對圖示I1進行選擇操作，多媒體檔案播放裝置60將依據物件位置資料軌所記錄之影像物件83的物件位置決定播放全景影片的播放視角，以依據剛決定的播放視角來播放包括影像物件83的畫面80。於本範例中，被選定的影像物件83將位於畫面80的中間。之後，假設使用者欲觀看對應於圖示I2的影像物件84（即關鍵人物B），響應於偵測到使用者針對圖示I2進行選擇操作，多媒體檔案播放裝置60將依據物件位置資料軌所記錄之影像物件84的物件位置切換播放視角，並基於所切換的播放視角播放包括影像物件84的畫面86。於本範例中，切換播放視角之後，被選定的影像物件84將位於畫面80的中間。Assuming that the user wants to view the image object 83 corresponding to the icon I1 (that is, the key person A), in response to detecting that the user selects the icon I1, the multimedia file playback device 60 will follow the object location data track recorded The object position of the image object 83 determines the viewing angle of the panoramic video, and the screen 80 including the image object 83 is played according to the newly determined viewing angle. In this example, the selected image object 83 will be located in the middle of the screen 80. After that, suppose that the user wants to watch the image object 84 (ie key person B) corresponding to the icon I2, and in response to detecting that the user selects the icon I2, the multimedia file playback device 60 will follow the object location data track. The object position of the recorded video object 84 is switched from the perspective of playback, and the screen 86 including the video object 84 is played based on the switched perspective of playback. In this example, after switching the viewing angle, the selected image object 84 will be located in the middle of the screen 80.

綜上所述，於本發明實施例中，包括全景影片的多媒體檔案更包括記錄有影像物件之位置資訊的物件位置資料軌。多媒體檔案產生裝置將影像物件的物件位置嵌入至多媒體檔案之中，致使多媒體檔案播放裝置於播放全景影片期間可依據物件位置資料軌即時得知特定影像物件的物件位置。基此，使用者的多媒體播放裝置不須具備強大的運算能力來進行影像物件的辨識與追蹤。此外，當使用者選擇感興趣的影像物件之後，多媒體檔案播放裝置可依據此影像物件的物件位置動態調整全景影片的播放視角，來達到追蹤特定影像物件的播放功能。因此，使用者無須手動調整播放視角來確保有觀看到感興趣的影像物件，從而大幅提昇觀賞360度影片的便利性。本發明更可讓使用者快速地瀏覽全景影片中的重點，以讓使用者觀看全景影片時可擁有直覺快速的操作以及觀看體驗。In summary, in the embodiment of the present invention, the multimedia file including the panoramic video further includes an object location data track recording the location information of the image object. The multimedia file generating device embeds the object position of the image object into the multimedia file, so that the multimedia file playback device can instantly know the object position of the specific image object according to the object position data track during the playback of the panoramic video. Based on this, the user's multimedia playback device does not need to have powerful computing capabilities to identify and track image objects. In addition, after the user selects the image object of interest, the multimedia file playback device can dynamically adjust the viewing angle of the panoramic video according to the object position of the image object to achieve the playback function of tracking the specific image object. Therefore, the user does not need to manually adjust the playback angle of view to ensure that the image object of interest is viewed, thereby greatly improving the convenience of viewing 360-degree videos. The present invention allows the user to quickly browse the key points in the panoramic video, so that the user can have intuitive and fast operation and viewing experience when watching the panoramic video.

雖然本發明已以實施例揭露如上，然其並非用以限定本發明，任何所屬技術領域中具有通常知識者，在不脫離本發明的精神和範圍內，當可作些許的更動與潤飾，故本發明的保護範圍當視後附的申請專利範圍所界定者為準。Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make slight changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to those defined by the attached patent scope.

10：多媒體檔案產生裝置 60：多媒體檔案播放裝置 110、610：處理器 120、620：儲存裝置 630：螢幕 121：影片取得模組 122：位置取得模組 123：檔案製作模組 124：檔案嵌入模組 621：影片接收模組 622：資料軌提取模組 623：介面提供模組 624：影片播放模組 P1～P3：時間區間 40：物件位置檔案 50：多媒體檔案 51：標頭 52：多媒體資料 521：視訊資料軌 522：音訊資料軌 523：字幕資料軌 524：物件位置資料軌 80、86：畫面 82：虛擬控制鈕 I1～I3：圖示 83、84：影像物件 S201～S204、S701～S704：步驟10: Multimedia file generating device 60: Multimedia file playback device 110, 610: Processor 120, 620: Storage device 630: Screen 121: Video acquisition module 122: Location acquisition module 123: File production module 124: File embedding module Group 621: Video receiving module 622: Data track extraction module 623: Interface providing module 624: Video playback module P1～P3: Time interval 40: Object location file 50: Multimedia file 51: Header 52: Multimedia data 521 : Video data track 522: Audio data track 523: Subtitle data track 524: Object position data track 80, 86: Screen 82: Virtual control buttons I1～I3: Icons 83, 84: Image objects S201～S204, S701～S704: step

圖1是依照本發明實施例所繪示之多媒體檔案產生裝置的方塊圖。 圖2是依照本發明實施例所繪示之多媒體檔案的產生方法的流程圖。 圖3A與圖3B是依據本發明實施例所繪示之對應至多個時間區間的多個物件位置的示意圖。 圖4是依據本發明實施例所繪示之物件位置檔案的範例。 圖5是依據本發明實施例所繪示之多媒體檔案架構的示意圖。 圖6是依照本發明實施例所繪示之多媒體檔案播放裝置的方塊圖。 圖7是依照本發明實施例所繪示之多媒體檔案的播放方法的流程圖。 圖8A與8B是依照本發明實施例所繪示之多媒體檔案的播放範例示意圖。FIG. 1 is a block diagram of a multimedia file generating device according to an embodiment of the invention. FIG. 2 is a flowchart of a method for generating a multimedia file according to an embodiment of the present invention. 3A and 3B are schematic diagrams illustrating the positions of multiple objects corresponding to multiple time intervals according to an embodiment of the present invention. FIG. 4 is an example of an object location file drawn according to an embodiment of the present invention. FIG. 5 is a schematic diagram of a multimedia file structure according to an embodiment of the present invention. FIG. 6 is a block diagram of a multimedia file playback device according to an embodiment of the invention. FIG. 7 is a flowchart of a method for playing multimedia files according to an embodiment of the present invention. 8A and 8B are schematic diagrams showing examples of playing multimedia files according to an embodiment of the present invention.

S701～S704：步驟S701～S704: steps

Claims (20)

一種多媒體檔案的產生方法，適用於一多媒體檔案產生裝置，所述方法包括： 取得關聯於一時間軸的一全景影片，其中該全景影片包括至少一影像物件； 取得該至少一影像物件相對於該時間軸的多個物件位置； 將該些物件位置製作成一物件位置檔案；以及 依據該物件位置檔案產生一多媒體檔案的至少一資料軌，以生成包括該全景影片且記錄有該些物件位置的該多媒體檔案。A method for generating a multimedia file is suitable for a multimedia file generating device. The method includes: obtaining a panoramic video associated with a time axis, wherein the panoramic video includes at least one image object; obtaining the at least one image object relative to the Multiple object positions in the time axis; creating the object positions into an object position file; and generating at least one data track of a multimedia file based on the object position file to generate the panoramic video and recording the object positions Multimedia files. 如申請專利範圍第1項所述的方法，其中該至少一影像物件包括第一影像物件與第二影像物件，相對於該時間軸的該些物件位置包括該第一影像物件的多個第一物件位置與該第二影像物件的多個第二物件位置，而依據該物件位置檔案產生該多媒體檔案的該至少一資料軌的步驟包括： 將物件位置檔案中的該些第一物件位置嵌入至一第一資料軌；以及 將物件位置檔案中的該些第二物件位置嵌入至一第二資料軌。The method according to claim 1, wherein the at least one image object includes a first image object and a second image object, and the positions of the objects relative to the time axis include a plurality of first image objects of the first image object. The object position and a plurality of second object positions of the second image object, and the step of generating the at least one data track of the multimedia file according to the object position file includes: embedding the first object positions in the object position file into A first data track; and embedding the second object positions in the object position file into a second data track. 如申請專利範圍第1項所述的方法，其中該多媒體檔案包括一標頭與多個資料軌，該些資料軌包括視訊資料軌與用以記錄該些物件位置的至少一物件位置資料軌，且該標頭記錄有該至少一物件位置資料軌之特性的描述與該至少一物件位置資料軌的數目。For the method described in claim 1, wherein the multimedia file includes a header and a plurality of data tracks, and the data tracks include a video data track and at least one object position data track for recording the positions of the objects, And the header records the description of the characteristics of the at least one object location data track and the number of the at least one object location data track. 如申請專利範圍第1項所述的方法，其中該至少一影像物件的該些物件位置分別對應至該時間軸上的多個時間區間。According to the method described in claim 1, wherein the object positions of the at least one image object respectively correspond to a plurality of time intervals on the time axis. 如申請專利範圍第4項所述的方法，其中將該些物件位置製作成該物件位置檔案的步驟包括： 將記錄為多個立體位置座標的該些物件位置映射成多個二維位置座標，並將該些二維位置座標記錄於該物件位置檔案之中。For the method described in item 4 of the scope of patent application, the step of making the positions of the objects into the object position file includes: mapping the positions of the objects recorded as a plurality of three-dimensional position coordinates into a plurality of two-dimensional position coordinates, And record the two-dimensional position coordinates in the object position file. 一種多媒體檔案產生裝置，包括： 一儲存裝置，儲存有多個模組； 一處理器，耦接該儲存裝置，載入並執行該儲存裝置中的所述模組，所述模組包括：       影片取得模組，取得關聯於一時間軸的一全景影片，其中該全景影片包括至少一影像物件；       位置取得模組，取得該至少一影像物件相對於該時間軸的多個物件位置；       檔案製作模組，將該些物件位置製作成一物件位置檔案；以及       檔案嵌入模組，依據該物件位置檔案產生一多媒體檔案的至少一資料軌，以生成包括該全景影片且記錄有該些物件位置的該多媒體檔案。A multimedia file generating device includes: a storage device storing a plurality of modules; a processor coupled to the storage device to load and execute the modules in the storage device, and the modules include: a video Obtain the module to obtain a panoramic video related to a time axis, where the panoramic video includes at least one image object; The position obtain module, to obtain the position of multiple objects of the at least one image object relative to the time axis; File production model Group, make the position of these objects into an object position file; and the file embedding module generates at least one data track of a multimedia file based on the object position file to generate the multimedia including the panoramic video and recording the position of the objects file. 如申請專利範圍第6項所述的多媒體檔案產生裝置，其中該至少一影像物件包括第一影像物件與第二影像物件，相對於該時間軸的該些物件位置包括該第一影像物件的多個第一物件位置與該第二影像物件的多個第二物件位置， 其中該檔案嵌入模組將物件位置檔案中的該些第一物件位置嵌入至一第一資料軌，以及將物件位置檔案中的該些第二物件位置嵌入至一第二資料軌。According to the multimedia file generating device described in claim 6, wherein the at least one image object includes a first image object and a second image object, and the position of the objects relative to the time axis includes a multiple of the first image object A first object position and a plurality of second object positions of the second image object, wherein the file embedding module embeds the first object positions in the object position file into a first data track, and the object position file The positions of the second objects in are embedded in a second data track. 如申請專利範圍第6項所述的多媒體檔案產生裝置，其中該多媒體檔案包括一標頭與多個資料軌，該些資料軌包括視訊資料軌與用以記錄該些物件位置的至少一物件位置資料軌，且該標頭記錄有該至少一物件位置資料軌之特性的描述與該至少一物件位置資料軌的數目。For example, the multimedia file generating device described in claim 6, wherein the multimedia file includes a header and a plurality of data tracks, and the data tracks include a video data track and at least one object location for recording the location of the objects A data track, and the header records a description of the characteristics of the at least one object location data track and the number of the at least one object location data track. 如申請專利範圍第6項所述的多媒體檔案產生裝置，其中該至少一影像物件的該些物件位置分別對應至該時間軸上的多個時間區間。According to the multimedia file generating device described in claim 6, wherein the object positions of the at least one image object respectively correspond to multiple time intervals on the time axis. 如申請專利範圍第6項所述的多媒體檔案產生裝置，其中該檔案製作模組更將記錄為多個立體位置座標的該些物件位置映射成多個二維位置座標，並將該些二維位置座標記錄於該物件位置檔案之中。For example, in the multimedia file generating device described in item 6 of the scope of patent application, the file production module further maps the positions of the objects recorded as a plurality of three-dimensional position coordinates into a plurality of two-dimensional position coordinates, and then The location coordinates are recorded in the object location file. 一種多媒體檔案的播放方法，適用於一多媒體檔案播放裝置，所述方法包括： 接收包括關聯於一時間軸的一全景影片的一多媒體檔案； 提取該多媒體檔案的一第一資料軌，以取得該全景影片中一第一影像物件相對於該時間軸的多個第一物件位置； 當播放該全景影片，顯示對應於該第一影像物件的一圖示於一螢幕的畫面；以及 響應於偵測到施於該圖示的選擇操作，依據該第一資料軌所記錄的該些第一物件位置決定用以播放該全景影片的一播放視角，並基於該播放視角播放包括該第一影像物件的畫面。A method for playing multimedia files is suitable for a multimedia file playing device. The method includes: receiving a multimedia file including a panoramic video associated with a time axis; extracting a first data track of the multimedia file to obtain the multimedia file The position of a first image object in the panoramic video relative to the first object positions of the time axis; when the panoramic video is played, an icon corresponding to the first image object is displayed on a screen; and in response to detection After the selection operation performed on the icon, a playback angle of view for playing the panoramic video is determined according to the positions of the first objects recorded in the first data track, and based on the playback angle, the first image object is played. Picture. 如申請專利範圍第11項所述的方法，其中響應於偵測到施於該圖示的選擇操作，依據該資料軌所記錄的該些第一物件位置決定用以播放該全景影片的該播放視角的步驟包括： 響應於識別到該些第一物件位置的改變，切換該播放視角。The method according to claim 11, wherein in response to detecting the selection operation applied to the icon, the playback for playing the panoramic video is determined according to the positions of the first objects recorded in the data track The step of viewing angle includes: in response to recognizing a change in the position of the first objects, switching the playback viewing angle. 如申請專利範圍第11項所述的方法，更包括： 提取該多媒體檔案的一第二資料軌，以取得該全景影片中一第二影像物件相對於該時間軸的多個第二物件位置；以及 當播放該全景影片，顯示對應於該第二影像物件的另一圖示於該螢幕的畫面。For example, the method described in claim 11 further includes: extracting a second data track of the multimedia file to obtain positions of a second image object in the panoramic video relative to the time axis; And when the panoramic video is played, another icon corresponding to the second image object is displayed on the screen. 如申請專利範圍第11項所述的方法，其中在響應於偵測到施於該圖示的選擇操作，依據該第一資料軌所記錄的該些第一物件位置決定用以播放該全景影片的該播放視角，並基於該播放視角播放包括該第一影像物件的畫面的步驟之後，所述方法更包括： 響應於偵測到施於該另一圖示的選擇操作，依據該第二資料軌所記錄的該些第二物件位置切換該播放視角，並基於所切換的該播放視角播放包括該第二影像物件的畫面。The method described in claim 11, wherein, in response to detecting the selection operation applied to the icon, it is determined to play the panoramic video according to the positions of the first objects recorded in the first data track After the step of playing the screen including the first image object based on the playback angle of view, the method further includes: in response to detecting a selection operation performed on the other icon, according to the second data The playback angle of view is switched over the positions of the second objects recorded in the track, and a screen including the second image object is played based on the switched playback angle of view. 如申請專利範圍第11項所述的方法，其中該多媒體檔案包括一標頭與多個資料軌，該些資料軌包括視訊資料軌與用以記錄該些物件位置的至少一物件位置資料軌，且該標頭記錄有該至少一物件位置資料軌之特性的描述與該至少一物件位置資料軌的數目。Such as the method described in claim 11, wherein the multimedia file includes a header and a plurality of data tracks, and the data tracks include a video data track and at least one object position data track for recording the positions of the objects, And the header records the description of the characteristics of the at least one object location data track and the number of the at least one object location data track. 一種多媒體檔案播放裝置，包括： 一螢幕； 一儲存裝置，儲存有多個模組； 一處理器，耦接該儲存裝置與該螢幕，載入並執行該儲存裝置中的所述模組，所述模組包括：       影片接收模組，接收包括關聯於一時間軸的一全景影片的一多媒體檔案；       資料軌提取模組，提取該多媒體檔案的一第一資料軌，以取得該全景影片中一第一影像物件相對於該時間軸的多個第一物件位置；       介面提供模組，當播放該全景影片，顯示對應於該第一影像物件的一圖示於該螢幕的畫面；以及       影片播放模組，響應於偵測到施於該圖示的選擇操作，依據該第一資料軌所記錄的該些第一物件位置決定用以播放該全景影片的一播放視角，並基於該播放視角播放包括該第一影像物件的畫面。A multimedia file playback device, comprising: a screen; a storage device storing a plurality of modules; a processor, coupled to the storage device and the screen, loads and executes the modules in the storage device, The modules include: a video receiving module, which receives a multimedia file including a panoramic video associated with a time axis; a data track extraction module, which extracts a first data track of the multimedia file to obtain one of the panoramic videos The position of the first image object relative to the multiple first objects in the timeline; the interface providing module, when the panoramic video is played, an icon corresponding to the first image object is displayed on the screen; and the video playback mode Group, in response to detecting the selection operation performed on the icon, determining a playback angle for playing the panoramic video according to the positions of the first objects recorded in the first data track, and playing based on the playback angle includes The frame of the first image object. 如申請專利範圍第16項所述的多媒體檔案播放裝置，其中該影片播放模組響應於識別到該些第一物件位置的改變，切換該播放視角。In the multimedia file playback device described in claim 16, wherein the video playback module switches the playback angle of view in response to recognizing changes in the positions of the first objects. 如申請專利範圍第16項所述的多媒體檔案播放裝置，其中該資料軌提取模組提取該多媒體檔案的一第二資料軌，以取得該全景影片中一第二影像物件相對於該時間軸的多個第二物件位置；以及當播放該全景影片，該介面提供模組顯示對應於該第二影像物件的另一圖示於該螢幕的畫面。For example, the multimedia file playback device described in the scope of patent application, wherein the data track extraction module extracts a second data track of the multimedia file to obtain a second image object in the panoramic video relative to the time axis A plurality of second object positions; and when the panoramic video is played, the interface providing module displays another icon corresponding to the second image object on the screen. 如申請專利範圍第18項所述的多媒體檔案播放裝置，其中該影片播放模組響應於偵測到施於該另一圖示的選擇操作，依據該第二資料軌所記錄的該些第二物件位置切換該播放視角，並基於所切換的該播放視角播放包括該第二影像物件的畫面。According to the multimedia file playback device described in claim 18, the video playback module responds to detecting the selection operation performed on the other icon according to the second data tracks recorded in the second data track. The object position switches the playback perspective, and based on the switched playback perspective, the screen including the second image object is played. 如申請專利範圍第16項所述的多媒體檔案播放裝置，其中該多媒體檔案包括一標頭與多個資料軌，該些資料軌包括視訊資料軌與用以記錄該些物件位置的至少一物件位置資料軌，且該標頭記錄有該至少一物件位置資料軌之特性的描述與該至少一物件位置資料軌的數目。Such as the multimedia file playback device described in claim 16, wherein the multimedia file includes a header and a plurality of data tracks, and the data tracks include a video data track and at least one object location for recording the location of the objects A data track, and the header records a description of the characteristics of the at least one object location data track and the number of the at least one object location data track.
TW107127500A 2018-08-07 2018-08-07 Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus TWI698117B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
TW107127500A TWI698117B (en) 2018-08-07 2018-08-07 Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus
US16/532,511 US11211097B2 (en) 2018-08-07 2019-08-06 Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW107127500A TWI698117B (en) 2018-08-07 2018-08-07 Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus

Publications (2)

Publication Number Publication Date
TW202008781A TW202008781A (en) 2020-02-16
TWI698117B true TWI698117B (en) 2020-07-01

Family

ID=69406348

Family Applications (1)

Application Number Title Priority Date Filing Date
TW107127500A TWI698117B (en) 2018-08-07 2018-08-07 Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus

Country Status (2)

Country Link
US (1) US11211097B2 (en)
TW (1) TWI698117B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111163267B (en) * 2020-01-07 2021-12-21 影石创新科技股份有限公司 Panoramic video editing method, device, equipment and storage medium
CN114026595A (en) * 2020-06-29 2022-02-08 深圳市大疆创新科技有限公司 Image processing method, device and storage medium
CN111917979B (en) * 2020-07-27 2022-09-23 维沃移动通信有限公司 Multimedia file output method and device, electronic equipment and readable storage medium
CN111954012A (en) * 2020-08-12 2020-11-17 上海遥知信息技术有限公司 Multimedia resource file playing method and device, computer equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI478581B (en) * 2008-06-10 2015-03-21 Sunplus Technology Co Ltd Method for seamless playback of multiple multimedia files
US9620168B1 (en) * 2015-12-21 2017-04-11 Amazon Technologies, Inc. Cataloging video and creating video summaries
WO2017103065A1 (en) * 2015-12-17 2017-06-22 Koninklijke Kpn N.V. Generating output video from video streams
US20170195576A1 (en) * 2016-01-05 2017-07-06 360fly, Inc. Dynamic field of view adjustment for panoramic video content

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080294691A1 (en) * 2007-05-22 2008-11-27 Sunplus Technology Co., Ltd. Methods for generating and playing multimedia file and recording medium storing multimedia file
WO2011114610A1 (en) * 2010-03-18 2011-09-22 パナソニック株式会社 Omnidirectional image processing device and omnidirectional image processing method
JP6167703B2 (en) * 2013-07-08 2017-07-26 株式会社リコー Display control device, program, and recording medium
TW201738649A (en) 2016-04-28 2017-11-01 Yun-Hong Shen Corresponding method between panorama video and message and system thereof having cloud software calculator to obtain an angle value and a first time point from the cloud database, and calculate and search a picture of panorama video corresponding to the message
US11272160B2 (en) * 2017-06-15 2022-03-08 Lenovo (Singapore) Pte. Ltd. Tracking a point of interest in a panoramic video

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI478581B (en) * 2008-06-10 2015-03-21 Sunplus Technology Co Ltd Method for seamless playback of multiple multimedia files
WO2017103065A1 (en) * 2015-12-17 2017-06-22 Koninklijke Kpn N.V. Generating output video from video streams
US9620168B1 (en) * 2015-12-21 2017-04-11 Amazon Technologies, Inc. Cataloging video and creating video summaries
US20170195576A1 (en) * 2016-01-05 2017-07-06 360fly, Inc. Dynamic field of view adjustment for panoramic video content

Also Published As

Publication number Publication date
TW202008781A (en) 2020-02-16
US11211097B2 (en) 2021-12-28
US20200051594A1 (en) 2020-02-13

Similar Documents

Publication Publication Date Title
TWI698117B (en) Generating method and playing method of multimedia file, multimedia file generation apparatus and multimedia file playback apparatus
US8875212B2 (en) Systems and methods for remote control of interactive video
US8656282B2 (en) Authoring tool for providing tags associated with items in a video playback
US8559683B2 (en) Electronic apparatus and scene-type display method
US9484065B2 (en) Intelligent determination of replays based on event identification
Chen et al. An autonomous framework to produce and distribute personalized team-sport video summaries: A basketball case study
KR20160097870A (en) System and method for browsing summary image
ES2914124T3 (en) Media targeting
TW201520827A (en) System and method of providing augmented reality effect for multi-media data
US20190130193A1 (en) Virtual Reality Causal Summary Content
US8244005B2 (en) Electronic apparatus and image display method
CN104023181A (en) Information processing method and device
JP6203188B2 (en) Similar image search device
TWM506428U (en) Display system for video stream on augmented reality
EP2942949A1 (en) System for providing complex-dimensional content service using complex 2d-3d content file, method for providing said service, and complex-dimensional content file therefor
KR102138835B1 (en) Apparatus and method for providing information exposure protecting image
JP5850188B2 (en) Image display system
US20120128325A1 (en) Authoring device and authoring method
CN110929056B (en) Multimedia file generating method, multimedia file playing method, multimedia file generating device and multimedia file playing device
WO2021087773A1 (en) Recognition method and apparatus, electronic device, and storage medium
US20110231763A1 (en) Electronic apparatus and image processing method
TWI762830B (en) System for displaying hint in augmented reality to play continuing film and method thereof
US11962743B2 (en) 3D display system and 3D display method
US20230326094A1 (en) Integrating overlaid content into displayed data via graphics processing circuitry and processing circuitry using a computing memory and an operating system memory
JP2010157960A (en) Imaging apparatus