TWI822473B

TWI822473B - Method, processing control device and system of providing movement improvement information

Info

Publication number: TWI822473B
Application number: TW111144172A
Authority: TW
Inventors: 陳弘恩; 張曉珍; 邱璟明
Original assignee: 財團法人資訊工業策進會
Priority date: 2022-11-18
Filing date: 2022-11-18
Publication date: 2023-11-11
Also published as: CN118071971A

Abstract

A method of providing movement improvement information, performed by a processing control device, includes: obtaining a plurality of first images and a plurality of second images, wherein the first images are generated by photographing a first user at a first angle, and correspond to different time points of the first user performing a to-be-evaluated movement, and the second images are generated by photographing the first user at a second angle, and correspond to different time points of the first user performing the to-be-evaluated movement, and the second angle is different from the first angle, generating to-be-evaluated movement 3D information corresponding to the first user according to the first images and the second images, generating movement improvement information using the to-be-evaluated movement 3D information and a predetermined movement 3D model, and outputting the movement improvement information through an interface.

Description

提供動作改善資訊的方法、處理控制裝置以及系統Methods, processing control devices and systems for providing motion improvement information

本發明係關於一種提供動作改善資訊的方法、處理控制裝置以及系統。The present invention relates to a method, processing control device and system for providing action improvement information.

近年來，物聯網、人工智慧等科技趨勢帶動了數位分身技術的發展。數位分身可以作為現實世界中各種物件的虛擬複製品，所述物件包含工廠設備、自駕車、機器人、真人等。在人物相關的數位分身方面，現有之數位分身建置或應用系統多是用來進行人物或裝置的模擬和分析，後來開始發展元宇宙系統，則是在虛擬的三維空間中創建出對應人物的虛擬分身，目前也僅能提供人物姿態虛實同步的功能，或是模擬一虛擬對手和對應人物的虛擬分身來進行互動，無法提供動作改善建議，況且對於講求人物動作之精細度的專業領域（例如運動）而言，若想要經由影像來精準分析和評估人物的動作，進而能提供動作改善建議，系統需搭載多個高速攝影機及能夠進行大量運算的處理器，造成極高的建置成本而難以被實際建置及應用。In recent years, technological trends such as the Internet of Things and artificial intelligence have driven the development of digital avatar technology. Digital avatars can serve as virtual replicas of various objects in the real world, including factory equipment, self-driving cars, robots, real people, etc. In terms of digital avatars related to characters, most of the existing digital avatar construction or application systems are used to simulate and analyze characters or devices. Later, the metaverse system began to be developed, which creates corresponding characters in a virtual three-dimensional space. Virtual avatars currently only provide the function of synchronizing virtual and real character postures, or simulate a virtual opponent and a virtual avatar of the corresponding character to interact, and cannot provide suggestions for action improvement. Moreover, for professional fields that emphasize the precision of character movements (such as For sports), if you want to accurately analyze and evaluate the actions of characters through images, and then provide action improvement suggestions, the system needs to be equipped with multiple high-speed cameras and a processor capable of performing large amounts of calculations, resulting in extremely high construction costs. It is difficult to be actually constructed and applied.

鑒於上述問題，本發明提供一種提供動作改善資訊的方法、處理控制裝置以及系統。In view of the above problems, the present invention provides a method, processing control device and system for providing action improvement information.

依據本發明一實施例的一種提供動作改善資訊的方法，包括以處理控制裝置執行：取得多張第一影像，所述多張第一影像係以第一角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點；取得多張第二影像，所述多張第二影像係以第二角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點，其中第二角度不同於第一角度；根據所述多張第一影像及所述多張第二影像產生對應第一使用者之待評估動作三維資訊；利用待評估動作三維資訊及預設動作三維模型產生動作改善資訊，動作改善資訊包含對應待評估動作在三維空間中的角度校正資訊及速度校正資訊；以及經由介面輸出動作改善資訊。According to an embodiment of the present invention, a method for providing motion improvement information includes executing with a processing control device: obtaining a plurality of first images, the plurality of first images being generated by photographing the first user at a first angle and corresponding to At different time points when the first user performs the action to be evaluated, obtain a plurality of second images. The plurality of second images are generated by photographing the first user at a second angle and correspond to the first user's actions. At different time points when performing the action to be evaluated, the second angle is different from the first angle; generating three-dimensional information corresponding to the first user's action to be evaluated based on the plurality of first images and the plurality of second images; using The three-dimensional information of the movement to be evaluated and the preset movement three-dimensional model generate movement improvement information. The movement improvement information includes angle correction information and speed correction information corresponding to the movement to be evaluated in the three-dimensional space; and the movement improvement information is output through the interface.

依據本發明一實施例的一種處理控制裝置包括記憶體以及處理器，其中處理器連接於記憶體。記憶體儲存用於提供動作改善資訊的多個指令。處理器用於讀取所述多個指令以執行：取得多張第一影像，所述多張第一影像係以第一角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點；取得多張第二影像，所述多張第二影像係以第二角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點，其中第二角度不同於第一角度；根據所述多張第一影像及所述多張第二影像產生對應第一使用者之待評估動作三維資訊；利用待評估動作三維資訊及預設動作三維模型產生動作改善資訊，動作改善資訊包含對應待評估動作在三維空間中的角度校正資訊及速度校正資訊；以及經由介面輸出動作改善資訊。A processing control device according to an embodiment of the present invention includes a memory and a processor, wherein the processor is connected to the memory. The memory stores multiple instructions for providing motion improvement information. The processor is used to read the plurality of instructions to execute: obtain a plurality of first images, the plurality of first images are generated by photographing the first user at a first angle and correspond to the first user's evaluation to be performed. Different time points during the action; obtain a plurality of second images, the plurality of second images are generated by photographing the first user at a second angle and correspond to different time points when the first user performs the action to be evaluated. , wherein the second angle is different from the first angle; generating three-dimensional information corresponding to the first user's action to be evaluated based on the plurality of first images and the plurality of second images; using the three-dimensional information of the action to be evaluated and the preset action The three-dimensional model generates motion improvement information, which includes angle correction information and speed correction information corresponding to the motion to be evaluated in the three-dimensional space; and the motion improvement information is output through the interface.

依據本發明一實施例的一種提供動作改善資訊的系統，包括：第一攝像裝置，用於以第一角度拍攝第一使用者以產生多張第一影像，其中所述多張第一影像對應於第一使用者在進行待評估動作時的不同時間點；第二攝像裝置，用於以第二角度拍攝第一使用者以產生多張第二影像，其中所述多張第二影像對應於第一使用者在進行待評估動作時的不同時間點，且第二角度不同於第一角度；一輸出介面；以及處理控制裝置，連接於第一攝像裝置、第二攝像裝置及輸出介面，且用於根據所述多張第一影像及所述多張第二影像產生待評估動作三維資訊，利用待評估動作三維資訊及預設動作三維模型產生動作改善資訊，並經由輸出介面輸出動作改善資訊，其中動作改善資訊包含對應待評估動作在三維空間中的角度校正資訊及速度校正資訊。A system for providing motion improvement information according to an embodiment of the present invention includes: a first camera device for photographing a first user at a first angle to generate a plurality of first images, wherein the plurality of first images correspond to At different time points when the first user performs the action to be evaluated; the second camera device is used to photograph the first user at a second angle to generate a plurality of second images, wherein the plurality of second images correspond to At different time points when the first user performs the action to be evaluated, and the second angle is different from the first angle; an output interface; and a processing control device connected to the first camera device, the second camera device and the output interface, and Used to generate three-dimensional motion information to be evaluated based on the plurality of first images and the plurality of second images, generate motion improvement information using the three-dimensional motion information to be evaluated and a preset motion three-dimensional model, and output the motion improvement information through an output interface , where the action improvement information includes angle correction information and speed correction information corresponding to the action to be evaluated in the three-dimensional space.

透過上述架構，本案所揭示的提供動作改善資訊方法、處理控制裝置以及系統可以非接觸式的方式即時分析使用者的動作並提供改善資訊，藉此，在運動訓練或是其他精緻動作訓練的應用上，可以達到訓練中分析及事後提供改善建議的目的。另外，相較於一般在運動訓練的系統中所使用多個高速攝影機的配置，本案所揭示的提供動作改善方法、處理控制裝置以及系統可基於兩台攝像裝置所拍攝的影像分析得到動作改善資訊，可以具有較低的影像資料處理量，且可以大幅減少整體系統的配置成本。Through the above structure, the method for providing movement improvement information, the processing control device and the system disclosed in this case can analyze the user's movements in real time and provide improvement information in a non-contact manner, thereby being used in sports training or other delicate movement training applications. It can achieve the purpose of analyzing during training and providing improvement suggestions afterwards. In addition, compared with the configuration of multiple high-speed cameras commonly used in sports training systems, the action improvement method, processing control device and system disclosed in this case can obtain action improvement information based on image analysis captured by two camera devices. , can have lower image data processing volume, and can significantly reduce the overall system configuration cost.

以上之關於本揭露內容之說明及以下之實施方式之說明係用以示範與解釋本發明之精神與原理，並且提供本發明之專利申請範圍更進一步之解釋。The above description of the present disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principles of the present invention, and to provide further explanation of the patent application scope of the present invention.

以下在實施方式中詳細敘述本發明之詳細特徵以及優點，其內容足以使任何熟習相關技藝者了解本發明之技術內容並據以實施，且根據本說明書所揭露之內容、申請專利範圍及圖式，任何熟習相關技藝者可輕易地理解本發明相關之目的及優點。以下之實施例係進一步詳細說明本發明之觀點，但非以任何觀點限制本發明之範疇。The detailed features and advantages of the present invention are described in detail below in the implementation mode. The content is sufficient to enable anyone skilled in the relevant art to understand the technical content of the present invention and implement it according to the content disclosed in this specification, the patent scope and the drawings. , anyone familiar with the relevant art can easily understand the relevant objectives and advantages of the present invention. The following examples further illustrate the aspects of the present invention in detail, but do not limit the scope of the present invention in any way.

請參考圖1，圖1係依據本發明一實施例所繪示的提供動作改善資訊的系統1的方塊圖。如圖1所示，提供動作改善資訊的系統1包括第一攝像裝置11、第二攝像裝置12、處理控制裝置13以及輸出介面14。處理控制裝置13電性或通訊連接於第一攝像裝置11、第二攝像裝置12及輸出介面14。Please refer to FIG. 1 , which is a block diagram of a system 1 for providing action improvement information according to an embodiment of the present invention. As shown in FIG. 1 , a system 1 for providing motion improvement information includes a first camera device 11 , a second camera device 12 , a processing control device 13 and an output interface 14 . The processing control device 13 is electrically or communicatively connected to the first camera device 11 , the second camera device 12 and the output interface 14 .

第一攝像裝置11及第二攝像裝置12可為任何能夠取像的攝影機，且較佳是相同型號的攝像裝置。第一使用者在進行一待評估動作的一段時間內，第一攝像裝置11用於以第一角度拍攝第一使用者（例如受訓者）以產生多張第一影像，所述多張第一影像分別對應於第一使用者在進行一待評估動作時的多個不同時間點。第二攝像裝置12用於以不同於第一角度的第二角度拍攝第一使用者以產生多張第二影像，所述多張第二影像分別對應於第一使用者在進行所述待評估動作時的多個不同時間點。所述多張第一影像所對應的多個時間點及所述多張第二影像所對應的多個時間點至少部分相同。換句話說，第一攝像裝置11及第二攝像裝置12可以具有不同或相同的取像幀率，本案不予限制。特別來說，第一攝像裝置11的拍攝方向（對應於第一角度）與第二攝像裝置12的拍攝方向（對應於第二角度）之間的夾角較佳為90度，然本發明不以上述夾角為限。The first camera device 11 and the second camera device 12 can be any cameras capable of capturing images, and preferably are camera devices of the same model. During a period of time when the first user performs an action to be evaluated, the first camera device 11 is used to photograph the first user (such as a trainee) at a first angle to generate a plurality of first images. The plurality of first images are The images respectively correspond to multiple different time points when the first user performs an action to be evaluated. The second camera device 12 is used to photograph the first user at a second angle different from the first angle to generate a plurality of second images. The plurality of second images respectively correspond to the first user performing the evaluation to be evaluated. Multiple different points in time during action. The plurality of time points corresponding to the plurality of first images and the plurality of time points corresponding to the plurality of second images are at least partially the same. In other words, the first camera device 11 and the second camera device 12 may have different or the same imaging frame rate, which is not limited in this case. In particular, the angle between the shooting direction of the first camera device 11 (corresponding to the first angle) and the shooting direction of the second camera device 12 (corresponding to the second angle) is preferably 90 degrees. However, the present invention does not use The above angle is limited.

處理控制裝置13用於利用上述第一影像及第二影像取得第一使用者的待評估動作，分析第一使用者的待評估動作與預設動作的差異以產生動作改善資訊，並透過輸出介面14輸出對應的動作改善資訊。上述處理控制裝置13所執行之提供動作改善資訊的具體實施內容將於後描述。處理控制裝置13可以包含一或多個處理器，所述處理器例如為中央處理器、繪圖處理器、微控制器、可程式化邏輯控制器、其他具有訊號處理功能的處理器或其結合。輸出介面14可以是顯示裝置（例如但不限於螢幕）、各類擴增實境或虛擬實境裝置（例如但不限於頭戴顯示器、AR/VR/MR裝置、浮空投影裝置或其結合）、輸出聲音的裝置（例如但不限於喇叭）或其結合等。The processing control device 13 is used to obtain the first user's action to be evaluated by using the first image and the second image, analyze the difference between the first user's action to be evaluated and the default action to generate action improvement information, and output the action through the output interface. 14 Output the corresponding action improvement information. The specific implementation content of providing action improvement information executed by the above-mentioned processing control device 13 will be described later. The processing control device 13 may include one or more processors, such as a central processing unit, a graphics processor, a microcontroller, a programmable logic controller, other processors with signal processing functions, or a combination thereof. The output interface 14 may be a display device (such as but not limited to a screen), various augmented reality or virtual reality devices (such as but not limited to a head-mounted display, an AR/VR/MR device, a floating projection device or a combination thereof) , devices that output sound (such as but not limited to speakers) or combinations thereof, etc.

請參考圖2，圖2係依據本發明一實施例所繪示的處理控制裝置13的方塊圖。如圖2所示，處理控制裝置13包括記憶體131及處理器132，記憶體131電性或通訊連接於處理器132。Please refer to FIG. 2 , which is a block diagram of the processing control device 13 according to an embodiment of the present invention. As shown in FIG. 2 , the processing control device 13 includes a memory 131 and a processor 132 . The memory 131 is electrically or communicatively connected to the processor 132 .

記憶體131用於儲存提供動作改善資訊的多個指令，處理器132用於讀取所述多個指令以執行如後所述之提供動作改善資訊的方法及／或流程。記憶體131可為非揮發性記憶體（NVM），例如唯讀記憶體（ROM）、快閃記憶體及/或非揮發性隨機存取記憶體（NVRM）等。The memory 131 is used to store multiple instructions for providing action improvement information, and the processor 132 is used to read the multiple instructions to execute the method and/or process of providing action improvement information as described below. The memory 131 may be a non-volatile memory (NVM), such as a read-only memory (ROM), flash memory and/or non-volatile random access memory (NVRM).

請一併參考圖1-3、圖4(a)及圖4(b)以說明提供動作改善資訊的方法，其中圖3係依據本發明一實施例所繪示的提供動作改善資訊的方法的流程圖，可被編譯為指令，圖4(a)示例性地呈現以第一角度拍攝第一使用者的第一影像 IMG1，圖4(b)示例性地呈現以第二角度拍攝第一使用者的第二影像IMG2。如圖3所示，提供動作改善資訊的方法包括步驟S101：取得多張第一影像，該些第一影像係以第一角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點；步驟S103：取得多張第二影像，該些第二影像係以第二角度拍攝第一使用者而產生且對應於第一使用者在進行待評估動作時的不同時間點，其中第二角度不同於第一角度；步驟S105：根據該些第一影像及該些第二影像產生對應第一使用者之待評估動作三維資訊；步驟S107：利用待評估動作三維資訊及預設動作三維模型產生動作改善資訊，動作改善資訊包括對應待評估動作在三維空間中的角度校正資訊及速度校正資訊；以及步驟S109：經由介面輸出動作改善資訊。也就是說，根據本方法，僅需透過低成本的硬體裝置配備即可取得第一角度及第二角度的影像來輸出動作改善資訊以供第一使用者S獲得如何改善自身的動作。如此一來，先前技術的問題可被改善。Please refer to Figures 1-3, Figure 4(a) and Figure 4(b) to illustrate a method of providing action improvement information. Figure 3 illustrates a method of providing action improvement information according to an embodiment of the present invention. The flow chart can be compiled into instructions. Figure 4(a) exemplarily presents the first image IMG1 of the first user taken at a first angle. Figure 4(b) exemplarily presents the first image IMG1 taken of the first user at a second angle. The user's second image IMG2. As shown in Figure 3, the method for providing motion improvement information includes step S101: Obtaining a plurality of first images. These first images are generated by photographing the first user at a first angle and correspond to the first user's actions. Different time points when the action is evaluated; Step S103: Obtain a plurality of second images. These second images are generated by photographing the first user at a second angle and correspond to the different characteristics of the first user when performing the action to be evaluated. time point, where the second angle is different from the first angle; Step S105: Generate three-dimensional information of the action to be evaluated corresponding to the first user based on the first images and the second images; Step S107: Use the three-dimensional information of the action to be evaluated And the preset motion three-dimensional model generates motion improvement information, the motion improvement information includes angle correction information and speed correction information corresponding to the motion to be evaluated in the three-dimensional space; and step S109: output the motion improvement information through the interface. That is to say, according to this method, only low-cost hardware equipment can be used to obtain the images of the first angle and the second angle and output the action improvement information so that the first user S can learn how to improve his own action. In this way, the problems of the prior art can be improved.

於步驟S101，處理控制裝置13可以從第一攝像裝置11取得多張第一影像。第一影像是由第一攝像裝置11以第一角度拍攝第一使用者S而產生，且所述多張第一影像對應於第一使用者S在進行待評估動作時的不同時間。換言之，一張第一影像IMG1對應於一個取像時間點。In step S101, the processing control device 13 can obtain a plurality of first images from the first camera device 11. The first image is generated by the first camera device 11 photographing the first user S at a first angle, and the plurality of first images correspond to different times when the first user S performs the action to be evaluated. In other words, one first image IMG1 corresponds to one imaging time point.

於步驟S103，處理控制裝置13可以從第二攝像裝置12取得多張第二影像。第二影像是由第二攝像裝置12以第二角度拍攝第一使用者S而產生，且所述多張第二影像對應於第一使用者S在進行所述待評估動作時的不同時間，其中第二角度不同於第一角度。換言之，一張第二影像IMG2對應於一個取像時間點。第一攝像裝置11及第二攝像裝置12是同時對進行待評估動作的第一使用者S取像以分別得到第一影像IMG1及第二影像IMG2。In step S103, the processing control device 13 may obtain a plurality of second images from the second camera device 12. The second image is generated by the second camera device 12 photographing the first user S at a second angle, and the plurality of second images correspond to different times when the first user S performs the action to be evaluated, The second angle is different from the first angle. In other words, one second image IMG2 corresponds to one imaging time point. The first camera device 11 and the second camera device 12 capture images of the first user S performing the action to be evaluated at the same time to obtain the first image IMG1 and the second image IMG2 respectively.

以圖4(a)及圖4(b)為例，所述待評估動作是羽球的擊球動作，第一角度是指第一攝像裝置11的拍攝方向垂直於羽球場地的發球線，第二角度是指第二攝像裝置12的拍攝方向垂直於羽球場地的中線，然本發明不以上述為限。第一角度和第二角度是不同的角度，在一較佳的實施例中，第一角度和第二角度之間的夾角是90度，然本發明不以上述夾角為限。Taking Figure 4(a) and Figure 4(b) as an example, the action to be evaluated is the batting action of badminton. The first angle means that the shooting direction of the first camera device 11 is perpendicular to the service line of the badminton court. The angle means that the shooting direction of the second camera device 12 is perpendicular to the center line of the badminton court, but the present invention is not limited to the above. The first angle and the second angle are different angles. In a preferred embodiment, the included angle between the first angle and the second angle is 90 degrees. However, the present invention is not limited to the above included angle.

於步驟S105，處理控制裝置13根據第一影像IMG1及第二影像IMG2中的一或多個特徵點產生對應第一使用者S的待評估動作三維資訊。舉例而言，處理控制裝置13可以將第一影像IMG1中的第一使用者S的關節點作為一組特徵點，及將第二影像IMG2中的第一使用者S的關節點作為另一組特徵點，再根據這兩組特徵點分別在第一影像IMG1及第二影像IMG2中的座標產生待評估動作三維資訊。In step S105, the processing control device 13 generates three-dimensional information corresponding to the action to be evaluated of the first user S based on one or more feature points in the first image IMG1 and the second image IMG2. For example, the processing control device 13 may use the joint points of the first user S in the first image IMG1 as one set of feature points, and the joint points of the first user S in the second image IMG2 as another set of feature points. feature points, and then generate three-dimensional information of the action to be evaluated based on the coordinates of these two sets of feature points in the first image IMG1 and the second image IMG2 respectively.

於步驟S107，處理控制裝置13比對待評估動作三維資訊與預設動作三維模型以產生動作改善資訊。預設動作三維模型例如是預先建立的動作模型，且可以是預存在處理控制裝置13的記憶體131中或由處理控制裝置13從外部資料庫（例如雲端資料庫）取得。以前述的羽球為例，預設動作三維模型例如是經由羽球訓練員、羽球教練等專業人員，事先進行對應各種待評估動作的標準動作所建立的動作模型。進一步而言，記憶體131或外部資料庫可儲存多個候選動作三維模型，例如，羽球的發球動作可以有正手發長球、正手發短球、反手發長球、反手發短球等，分別建立不同的候選動作模型，在一些實施例中，也可以將多個動作整合在一個模型中。處理器132可以接收第一使用者S提供的輸入資訊，所述輸入資訊可以是第一使用者S想要進行訓練的待評估動作，例如正手發長球，處理器132即可從這些候選動作三維模型中選取對應待評估動作的一者，例如以建立好的正手發長球的動作模型，作為預設動作三維模型。In step S107, the processing control device 13 compares the three-dimensional motion information to be evaluated and the preset motion three-dimensional model to generate motion improvement information. The preset action three-dimensional model is, for example, a pre-established action model, and may be pre-stored in the memory 131 of the processing control device 13 or obtained by the processing control device 13 from an external database (such as a cloud database). Taking the aforementioned badminton as an example, the preset action three-dimensional model is, for example, an action model established by badminton trainers, badminton coaches and other professionals who perform standard actions corresponding to various actions to be evaluated in advance. Furthermore, the memory 131 or an external database can store multiple candidate action three-dimensional models. For example, badminton serving actions can include a long forehand serve, a short forehand serve, a long backhand serve, a short backhand serve, etc. , build different candidate action models respectively. In some embodiments, multiple actions can also be integrated into one model. The processor 132 can receive input information provided by the first user S. The input information can be an action to be evaluated that the first user S wants to train, such as a long forehand serve. The processor 132 can then select from these candidates. Select one of the action three-dimensional models that corresponds to the action to be evaluated, for example, use the established action model of a long forehand serve as the preset action three-dimensional model.

動作改善資訊包括對應待評估動作在三維空間中的角度校正資訊及速度校正資訊。處理控制裝置13可判斷待評估動作三維資訊與預設動作三維模型之間的角度差及速度差，以產生角度校正資訊及速度校正資訊。以圖4(a)及圖4(b)為例，角度校正資訊可以指示揮動球拍時手腕的角度要再向哪個方向旋轉多少角度，速度校正資訊可以指示揮動球拍時手腕的速度要再加快多少。Action improvement information includes angle correction information and speed correction information corresponding to the action to be evaluated in the three-dimensional space. The processing control device 13 can determine the angle difference and speed difference between the three-dimensional motion information to be evaluated and the preset motion three-dimensional model to generate angle correction information and speed correction information. Taking Figure 4(a) and Figure 4(b) as an example, the angle correction information can indicate how much the wrist angle needs to be rotated in which direction when swinging the racket, and the speed correction information can indicate how much the wrist speed needs to be accelerated when swinging the racket. .

接著，於步驟S109，處理控制裝置13透過輸出介面14輸出動作改善資訊給第一使用者S（例如以供第一使用者S獲得如何改善自身的動作）。Next, in step S109, the processing control device 13 outputs action improvement information to the first user S through the output interface 14 (for example, for the first user S to obtain how to improve his or her actions).

除了上述實施例之流程，在判得動作改善資訊後，處理控制裝置13可更透過輸出介面14輸出預設動作三維模型中對應動作改善資訊的多張參考影像。參考影像可為預設動作三維模型的多幀影像等。舉例而言，羽球受訓員（第一使用者S）在進行正手發長球的練習時，可依據羽球教練所建立的正手發長球的動作模型來比對，且依據其差異來提供改善建議，例如當其差異是擊球之前受訓員的手肘角度過大時，依據預設動作三維模型中羽球教練的標準正手發長球的動作模擬出標準的手肘角度的軸參考影像，以提供給羽球受訓員參考。據此，可以讓第一使用者更清楚知道如何改善自身的動作。In addition to the process of the above embodiment, after determining the motion improvement information, the processing control device 13 can further output multiple reference images corresponding to the motion improvement information in the default motion three-dimensional model through the output interface 14 . The reference image may be a multi-frame image of a preset motion 3D model, etc. For example, when the badminton trainee (the first user S) practices a long forehand serve, he or she can compare it with the action model of a long forehand serve established by the badminton coach, and provide information based on the difference. Suggestions for improvement, for example, when the difference is that the trainee's elbow angle is too large before hitting the ball, simulate the axis reference image of the standard elbow angle based on the badminton coach's standard forehand long ball action in the preset action 3D model. To provide reference for badminton trainees. Accordingly, the first user can know more clearly how to improve his/her movements.

請接著一併參考圖1、圖5及圖6，其中圖5係依據本發明一實施例所繪示的產生待評估動作三維資訊的流程圖，可被編譯為指令；圖6係繪示在三維空間XYZ中使用第一攝像裝置11以第一角度拍攝第一使用者S的第一影像及使用第二攝像裝置12以第二角度拍攝第一使用者S的第二影像的示意圖，其中三維空間XYZ可為X軸、Y軸及Z軸所形成的空間，以及X軸、Y軸及Z軸中任二者互相垂直且分別形成X-Y平面、X-Z平面及Y-Z平面。圖5可視為圖3之步驟S105的一實施例的細部流程圖。如圖5所示，產生待評估動作三維資訊的方法包括：步驟S201：從該些第一影像分別取得多個第一二維座標組；步驟S203：從該些第二影像分別取得多個第二二維座標組；步驟S205：對該些第一二維座標組及該些第二二維座標組進行深度補償以取得多個原始三維座標組；以及步驟S207：根據該些原始三維座標組產生角度原始資訊及速度原始資訊，以作為待評估動作三維資訊。在圖5中，步驟S201係繪示為執行在步驟S203之前，但步驟S201亦可執行在步驟S203之後，或是步驟S201可與步驟S203被同時執行。Please refer to Figure 1, Figure 5 and Figure 6 together. Figure 5 is a flow chart of generating three-dimensional information of an action to be evaluated according to an embodiment of the present invention, which can be compiled into instructions; Figure 6 is a flow chart of A schematic diagram of using the first camera device 11 to capture the first image of the first user S at a first angle and using the second camera device 12 to capture the second image of the first user S at a second angle in the three-dimensional space XYZ. The three-dimensional The space XYZ can be the space formed by the X-axis, the Y-axis and the Z-axis, and any two of the X-axis, Y-axis and Z-axis are perpendicular to each other and form the X-Y plane, the X-Z plane and the Y-Z plane respectively. FIG. 5 can be viewed as a detailed flow chart of an embodiment of step S105 in FIG. 3 . As shown in Figure 5, the method of generating three-dimensional information of the action to be evaluated includes: Step S201: Obtaining a plurality of first two-dimensional coordinate groups from the first images; Step S203: Obtaining a plurality of second two-dimensional coordinate groups from the second images. Two two-dimensional coordinate groups; Step S205: Perform depth compensation on the first two-dimensional coordinate groups and the second two-dimensional coordinate groups to obtain multiple original three-dimensional coordinate groups; and Step S207: According to the original three-dimensional coordinate groups The original angle information and the original speed information are generated as the three-dimensional information of the action to be evaluated. In FIG. 5 , step S201 is shown as being executed before step S203 , but step S201 may also be executed after step S203 , or step S201 may be executed simultaneously with step S203 .

於步驟S201，處理控制裝置13從每張第一影像取得對應第一使用者的第一二維座標組。於步驟S203，處理控制裝置13從每張第二影像取得對應第一使用者的第二二維座標組。第一二維座標組及第二二維座標組可包括分別對應於第一使用者S之待評估動作的關節點的多個二維座標。以圖6為例，第一二維座標組包括第一使用者S的多個關節點在二維平面Y-Z中的多個第一二維座標，第二二維座標組包括第一使用者S的該些關節點在二維平面X-Z中的多個第二二維座標。具體地，一幀第一影像對應於一個取像時間點，處理控制裝置13基於此第一影像產生一個第一二維座標組。換言之，該些第一二維座標組對應於不同的時間點，且所述不同的時間點分別與該些第一影像的取像時間點相同。同理，一幀第二影像對應於一個取像時間點，處理控制裝置13基於此第二影像產生一個第二二維座標組。換言之，該些第二二維座標組對應於不同的時間點，且所述不同的時間點分別與該些第二影像的取像時間點相同。In step S201, the processing control device 13 obtains the first two-dimensional coordinate set corresponding to the first user from each first image. In step S203, the processing control device 13 obtains the second two-dimensional coordinate set corresponding to the first user from each second image. The first two-dimensional coordinate set and the second two-dimensional coordinate set may include a plurality of two-dimensional coordinates respectively corresponding to joint points of the first user S's motion to be evaluated. Taking Figure 6 as an example, the first two-dimensional coordinate group includes a plurality of first two-dimensional coordinates of the joint points of the first user S in the two-dimensional plane Y-Z, and the second two-dimensional coordinate group includes the first user S A plurality of second two-dimensional coordinates of the joint points in the two-dimensional plane X-Z. Specifically, one frame of the first image corresponds to one imaging time point, and the processing control device 13 generates a first two-dimensional coordinate group based on the first image. In other words, the first two-dimensional coordinate groups correspond to different time points, and the different time points are respectively the same as the imaging time points of the first images. Similarly, one frame of the second image corresponds to one imaging time point, and the processing control device 13 generates a second two-dimensional coordinate group based on the second image. In other words, the second two-dimensional coordinate groups correspond to different time points, and the different time points are respectively the same as the imaging time points of the second images.

於步驟S205，處理控制裝置13對第一二維座標組中的每個二維座標與第二二維座標組中對應的二維座標進行深度補償，以降低或消除第一二維座標組的各二維座標與第二二維座標組中對應的二維座標之間的差異，進而取得由第一二維座標組及第二二維座標組形成的原始三維座標組。具體地，由於第一二維座標組及第二二維座標組係分別由第一攝像裝置11及第二攝像裝置12從不同角度拍攝的影像而得，即使第一二維座標組的二維座標及第二二維座標組的二維座標係代表相同的關節點，這兩個二維座標之間仍可能因攝像裝置的座標轉換系統有差異等原因而有不一致的狀況。因此，透過執行深度補償，可以降低或消除第一二維座標組與第二二維座標組之間的差異，進而形成原始三維座標組。換言之，一個原始三維座標組可包括對應於第一使用者S在三維空間XYZ中的多個關節點的三維座標。In step S205, the processing control device 13 performs depth compensation on each two-dimensional coordinate in the first two-dimensional coordinate group and the corresponding two-dimensional coordinate in the second two-dimensional coordinate group, so as to reduce or eliminate the distortion of the first two-dimensional coordinate group. The difference between each two-dimensional coordinate and the corresponding two-dimensional coordinate in the second two-dimensional coordinate group is used to obtain the original three-dimensional coordinate group formed by the first two-dimensional coordinate group and the second two-dimensional coordinate group. Specifically, since the first two-dimensional coordinate set and the second two-dimensional coordinate set are respectively obtained from images captured by the first camera 11 and the second camera 12 from different angles, even if the two-dimensional coordinates of the first two-dimensional coordinate set are The coordinates and the two-dimensional coordinate system of the second two-dimensional coordinate group represent the same joint point. There may still be inconsistencies between the two two-dimensional coordinates due to differences in the coordinate conversion system of the camera device and other reasons. Therefore, by performing depth compensation, the difference between the first two-dimensional coordinate set and the second two-dimensional coordinate set can be reduced or eliminated, thereby forming an original three-dimensional coordinate set. In other words, an original three-dimensional coordinate set may include three-dimensional coordinates corresponding to multiple joint points of the first user S in the three-dimensional space XYZ.

於步驟S207，處理控制裝置13可根據原始三維座標組判斷各關節點的角度以作為角度原始資訊，及根據不同時間點的多個原始三維座標組判斷各關節點的移動速度以作為速度原始資訊，並將角度原始資訊及速度原始資訊作為待評估動作三維資訊。In step S207, the processing control device 13 can determine the angle of each joint point according to the original three-dimensional coordinate group as the original angle information, and determine the moving speed of each joint point according to multiple original three-dimensional coordinate groups at different time points as the original speed information. , and use the original angle information and the original speed information as the three-dimensional information of the action to be evaluated.

請接著一併參考圖1、圖6、圖7(a)、圖7(b)及圖8，其中圖7(a)係繪示第一影像IMG1的一第一二維座標組的示意圖，圖7(a)所示的第一影像IMG1包括關節點D11到D15；圖7(b)係繪示第二影像IMG2的一第二二維座標組的示意圖，圖7(b)所示的第二影像IMG2包括關節點D21到D25；圖8係依據本發明一實施例所繪示的深度補償的流程圖，可被編譯為指令。圖8可視為圖5之步驟S205的一實施例的細部流程圖。如圖8所示，深度補償的方法包括：步驟S301：將每一第一二維座標組作為第一目標座標組，將對應於第一目標座標組所對應的時間點的第二二維座標組作為第二目標座標組；步驟S303：判斷第一目標座標組的該些二維座標中的每一個及其於第二目標座標組的該些二維座標中對應於相同關節點的二維座標；步驟S305：判斷第二軸座標與三軸座標是否相同；若步驟S305的判斷結果為「是」，執行步驟S307：以第二軸座標及第三軸座標中的一者、第一軸座標以及第四軸座標組成三維座標；若步驟S305的判斷結果為「否」，執行步驟S309：根據第二軸座標與第三軸座標之間的差距調整第四軸座標，並以第一軸座標、第二軸座標及經調整的第四軸座標組成三維座標；以及步驟S311：以第一目標座標組的該些二維座標各自對應的三維座標組成該些第一三維座標組中的一個。Please refer to Figure 1, Figure 6, Figure 7(a), Figure 7(b) and Figure 8 together. Figure 7(a) is a schematic diagram of a first two-dimensional coordinate group of the first image IMG1. The first image IMG1 shown in Figure 7(a) includes joint points D11 to D15; Figure 7(b) is a schematic diagram showing a second two-dimensional coordinate group of the second image IMG2. Figure 7(b) The second image IMG2 includes joint points D21 to D25; FIG. 8 is a flow chart of depth compensation according to an embodiment of the present invention, which can be compiled into instructions. FIG. 8 can be viewed as a detailed flow chart of an embodiment of step S205 in FIG. 5 . As shown in Figure 8, the depth compensation method includes: Step S301: Use each first two-dimensional coordinate group as a first target coordinate group, and use the second two-dimensional coordinate corresponding to the time point corresponding to the first target coordinate group. group as the second target coordinate group; Step S303: Determine each of the two-dimensional coordinates of the first target coordinate group and its two-dimensional coordinates corresponding to the same joint point among the two-dimensional coordinates of the second target coordinate group. coordinates; Step S305: Determine whether the second-axis coordinate and the three-axis coordinate are the same; if the judgment result of step S305 is "Yes", execute step S307: Use one of the second-axis coordinate and the third-axis coordinate, the first axis The coordinates and the fourth-axis coordinates form a three-dimensional coordinate; if the judgment result in step S305 is "No", perform step S309: adjust the fourth-axis coordinate according to the difference between the second-axis coordinate and the third-axis coordinate, and use the first-axis coordinate The coordinates, the second axis coordinates and the adjusted fourth axis coordinates form a three-dimensional coordinate; and step S311: use the three-dimensional coordinates corresponding to the two-dimensional coordinates of the first target coordinate group to form one of the first three-dimensional coordinate groups. .

需先說明的是，第一影像IMG1的第一二維座標組各自的該些二維座標各包括第一軸座標及第二軸座標，第二影像IMG2的第二二維座標組各自的該些二維座標各包含第三軸座標及第四軸座標，第二軸座標與第三軸座標對應於三維空間中的同一軸。以圖6為例，第一二維座標組的每個二維座標包括的第一軸座標為Y軸座標，第二軸座標為Z軸座標；第二二維座標組的每個二維座標包括的第三軸座標為Z軸座標，第四軸座標為X軸座標。It should be noted that each of the two-dimensional coordinates of the first two-dimensional coordinate group of the first image IMG1 includes a first axis coordinate and a second axis coordinate, and the respective two-dimensional coordinates of the second two-dimensional coordinate group of the second image IMG2 are Each of the two-dimensional coordinates includes a third-axis coordinate and a fourth-axis coordinate, and the second-axis coordinate and the third-axis coordinate correspond to the same axis in the three-dimensional space. Taking Figure 6 as an example, each two-dimensional coordinate of the first two-dimensional coordinate group includes the first axis coordinate as the Y-axis coordinate and the second axis coordinate as the Z-axis coordinate; each two-dimensional coordinate of the second two-dimensional coordinate group includes The included third axis coordinate is the Z axis coordinate, and the fourth axis coordinate is the X axis coordinate.

於步驟S301，處理控制裝置13將該些第一二維座標組中的每一個作為第一目標座標組，及將該些第二二維座標組中對應於第一目標座標組所對應的時間點的第二二維座標組作為第二目標座標組，以執行後續步驟。換言之，於步驟S301，處理控制裝置13將第一影像IMG1的第一二維座標組作為第一目標座標組，及將具有相同或相近的取像時間點的第二影像IMG2的第二二維座標組作為第二目標座標組，以執行後續步驟。以圖7(a)為例，第一影像IMG1包括的關節點D11到D15的二維座標可作為一個第一目標座標組。相似地，以圖7(b)為例，第二影像IMG2包括的關節點D21到D25的二維座標可作為一個第二目標座標組。以第一影像IMG1及第二影像IMG2為例，本文中所述「相近的時間點的第二影像」是指多張第二影像中，取像時間點最接近當前第一影像IMG1的取像時間點的一者。In step S301, the processing control device 13 uses each of the first two-dimensional coordinate groups as a first target coordinate group, and uses the time corresponding to the first target coordinate group in the second two-dimensional coordinate groups. The second two-dimensional coordinate group of the point is used as the second target coordinate group to perform subsequent steps. In other words, in step S301, the processing control device 13 uses the first two-dimensional coordinate set of the first image IMG1 as the first target coordinate set, and uses the second two-dimensional coordinate set of the second image IMG2 with the same or similar imaging time point. The coordinate group serves as the second target coordinate group to perform subsequent steps. Taking Figure 7(a) as an example, the two-dimensional coordinates of joint points D11 to D15 included in the first image IMG1 can be used as a first target coordinate group. Similarly, taking FIG. 7(b) as an example, the two-dimensional coordinates of joint points D21 to D25 included in the second image IMG2 can be used as a second target coordinate group. Taking the first image IMG1 and the second image IMG2 as an example, the "second image at a similar time point" mentioned in this article refers to the image capture time point closest to the current first image IMG1 among multiple second images. One of the points in time.

於步驟S303，處理控制裝置13判斷第一目標座標組的二維座標中的每一個及其於第二目標座標組的該些二維座標中對應於相同關節點的二維座標，以執行後續步驟。以圖7(a)及圖7(b)中的一個關節點為例，處理控制裝置13判斷關節點D11的二維座標，及關節點D11在第二影像IMG2中所對應的關節點D21的二維座標，並根據關節點D11的二維座標及關節點D21的二維座標執行後續步驟。In step S303, the processing control device 13 determines each of the two-dimensional coordinates of the first target coordinate group and the two-dimensional coordinates corresponding to the same joint point among the two-dimensional coordinates of the second target coordinate group to perform subsequent steps. Taking a joint point in Figure 7(a) and Figure 7(b) as an example, the processing control device 13 determines the two-dimensional coordinates of the joint point D11 and the coordinates of the joint point D21 corresponding to the joint point D11 in the second image IMG2. two-dimensional coordinates, and perform subsequent steps based on the two-dimensional coordinates of joint point D11 and the two-dimensional coordinates of joint point D21.

於步驟S305，處理控制裝置13判斷對應第一目標座標組的二維座標的第二軸座標與第二目標座標組中對應的二維座標的第三軸座標是否彼此相同。以關節點D11及D21為例，關節點D11的二維座標包括的第一軸座標為Y1，第二軸座標為Z1；關節點D21的二維座標包括的第三軸座標為Z2，第四軸座標為X2，處理控制裝置13於步驟S305判斷第二軸座標Z1與第三軸座標Z2是否相同。In step S305, the processing control device 13 determines whether the second axis coordinate corresponding to the two-dimensional coordinate in the first target coordinate group and the third axis coordinate corresponding to the two-dimensional coordinate in the second target coordinate group are the same as each other. Taking joint points D11 and D21 as an example, the two-dimensional coordinates of joint point D11 include the first axis coordinate Y1 and the second axis coordinate Z1; the two-dimensional coordinates of joint point D21 include the third axis coordinate Z2 and the fourth axis coordinate. The axis coordinate is X2, and the processing control device 13 determines whether the second axis coordinate Z1 and the third axis coordinate Z2 are the same in step S305.

若處理控制裝置13判斷對應關節點D11的第二軸座標Z1與對應關節點D21的第三軸座標Z2彼此相同，則於步驟S307，處理控制裝置13以第二軸座標Z1或第三軸座標Z2中的一者、第一軸座標Y1及第四軸座標X2組成一個三維座標，即（Y1，Z1或Z2，X2）。If the processing control device 13 determines that the second axis coordinate Z1 corresponding to the joint point D11 and the third axis coordinate Z2 corresponding to the joint point D21 are the same as each other, then in step S307, the processing control device 13 uses the second axis coordinate Z1 or the third axis coordinate One of Z2, the first axis coordinate Y1 and the fourth axis coordinate X2 form a three-dimensional coordinate, namely (Y1, Z1 or Z2, X2).

若處理控制裝置13判斷對應關節點D11的第二軸座標Z1與對應關節點D21的第三軸座標Z2彼此不同，則於步驟S309，處理控制裝置13根據第二軸座標Z1與第三軸座標Z2之間的差距等比例地調整第四軸座標X2，並以第一軸座標Y1、第二軸座標Z1及經調整後的第四軸座標組成一個三維座標，即（Y1，Z1， *X2）。 If the processing control device 13 determines that the second axis coordinate Z1 corresponding to the joint point D11 and the third axis coordinate Z2 corresponding to the joint point D21 are different from each other, then in step S309, the processing control device 13 determines according to the second axis coordinate Z1 and the third axis coordinate The fourth axis coordinate X2 is adjusted proportionally to the difference between Z2, and a three-dimensional coordinate is composed of the first axis coordinate Y1, the second axis coordinate Z1 and the adjusted fourth axis coordinate, that is, (Y1, Z1, *X2).

在透過步驟S307或步驟S309取得一個三維座標後，於步驟S311，處理控制裝置13以第一目標座標組的各二維座標對應的三維座標組成一個第一三維座標組，以產生經深度補償後的第一三維座標組。簡言之，於步驟S311，處理控制裝置13將基於第一目標座標組的各二維座標產生的三維座標組成第一三維座標組。After obtaining a three-dimensional coordinate through step S307 or step S309, in step S311, the processing control device 13 uses the three-dimensional coordinates corresponding to each two-dimensional coordinate of the first target coordinate group to form a first three-dimensional coordinate group to generate a depth-compensated The first three-dimensional coordinate group. In short, in step S311, the processing control device 13 forms the first three-dimensional coordinate group with three-dimensional coordinates generated based on each two-dimensional coordinate of the first target coordinate group.

在圖8的實施例中，對於每張第一影像IMG1中的多個關節點的二維座標與對應相同或相近時間點的第二影像IMG2中的該些關節點的二維座標，處理控制裝置13執行步驟S305、S307/S309及S311，以對第一影像IMG1及第二影像IMG2中代表同一關節點的兩個二維座標進行補償，以產生該關節點的一個三維座標，並將第一影像IMG1中的各關節點對應的三維座標組成第一三維座標組。以圖7(a)及圖7(b)為例，處理控制裝置13對關節點D11的二維座標與關節點D21的二維座標執行步驟S305、S307/S309及S311，以產生關節點D11/關節點D21在三維空間中的第一三維座標組。In the embodiment of FIG. 8 , for the two-dimensional coordinates of multiple joint points in each first image IMG1 and the two-dimensional coordinates of the joint points in the second image IMG2 corresponding to the same or similar time point, the processing control The device 13 executes steps S305, S307/S309 and S311 to compensate the two two-dimensional coordinates representing the same joint point in the first image IMG1 and the second image IMG2 to generate a three-dimensional coordinate of the joint point, and convert the first image IMG1 to the second image IMG2. The three-dimensional coordinates corresponding to each joint point in an image IMG1 form a first three-dimensional coordinate group. Taking Figure 7(a) and Figure 7(b) as an example, the processing control device 13 executes steps S305, S307/S309 and S311 on the two-dimensional coordinates of the joint point D11 and the two-dimensional coordinates of the joint point D21 to generate the joint point D11. /The first three-dimensional coordinate group of joint point D21 in three-dimensional space.

請接著一併參考圖1及圖9，其中圖9係依據本發明另一實施例所繪示的產生待評估動作三維資訊的流程圖，可被編譯為指令。圖9可視為圖3之步驟S105的另一實施例的細部流程圖。如圖9所示，產生待評估動作三維資訊的方法包括：步驟S401：分別對該些第一影像及該些第二影像進行姿態辨識以取得對應第一使用者的多個人體骨架；步驟S403：從該些人體骨架分別取得對應待評估動作的多個關節點的多個二維座標組；以及步驟S405：利用該些二維座標組產生待評估動作三維資訊。Please refer to FIG. 1 and FIG. 9 together. FIG. 9 is a flow chart of generating three-dimensional information of an action to be evaluated according to another embodiment of the present invention, which can be compiled into instructions. FIG. 9 can be viewed as a detailed flow chart of another embodiment of step S105 in FIG. 3 . As shown in Figure 9, the method of generating three-dimensional information of the action to be evaluated includes: Step S401: Perform posture recognition on the first images and the second images to obtain multiple human skeletons corresponding to the first user; Step S403 : Obtain multiple two-dimensional coordinate sets corresponding to multiple joint points of the action to be evaluated from the human skeletons; and step S405: Use the two-dimensional coordinate sets to generate three-dimensional information of the action to be evaluated.

於步驟S401，處理控制裝置13對每張第一影像進行姿態辨識以取得對應第一使用者的一第一人體骨架，及對每張第二影像進行姿態辨識以取得對應第一使用者的一第二人體骨架。因此，處理控制裝置1可以從多張第一影像取得多個第一人體骨架，從多張第二影像取得多個第二人體骨架。舉例來說，所述姿態辨識可以OpenPose演算法來執行。In step S401, the processing control device 13 performs posture recognition on each first image to obtain a first human skeleton corresponding to the first user, and performs posture recognition on each second image to obtain a first human skeleton corresponding to the first user. Second human skeleton. Therefore, the processing control device 1 can obtain a plurality of first human skeletons from a plurality of first images, and obtain a plurality of second human skeletons from a plurality of second images. For example, the gesture recognition can be performed using the OpenPose algorithm.

於步驟S403，處理控制裝置13從每個第一人體骨架取得對應待評估動作的多個關節點的二維座標作為一第一二維座標組，及從每個第二人體骨架取得對應待評估動作的該些關節點的二維座標作為一第二二維座標組。換言之，一張第一影像對應於一個第一二維座標組，此第一二維座標組包括對應的第一人體骨架的多個關節點的二維座標；一張第二影像包括一個第二二維座標組，此第二二維座標組包括對應的第二人體骨架的該些關節點的二維座標。特別來說，上述步驟S401及S403的執行內容可為圖5所示之S201及S203的具體實現方式。In step S403, the processing control device 13 obtains the two-dimensional coordinates of multiple joint points corresponding to the motion to be evaluated from each first human skeleton as a first two-dimensional coordinate group, and obtains the corresponding two-dimensional coordinates to be evaluated from each second human skeleton. The two-dimensional coordinates of the joint points of the action are used as a second two-dimensional coordinate group. In other words, a first image corresponds to a first two-dimensional coordinate group, and the first two-dimensional coordinate group includes corresponding two-dimensional coordinates of multiple joint points of the first human skeleton; a second image includes a second The second two-dimensional coordinate group includes the two-dimensional coordinates of the corresponding joint points of the second human skeleton. Specifically, the execution content of the above-mentioned steps S401 and S403 may be the specific implementation of S201 and S203 shown in FIG. 5 .

於步驟S405，處理控制裝置13利用第一影像的二維座標組與對應的第二影像的二維座標組產生待評估動作三維資訊。舉例而言，如前所述，第一影像的第一二維座標組中分別對應於不同關節點的多個二維座標可各為（Y，Z），對應於相同或相近時間點的第二影像的第二二維座標組中分別對應於不同關節點的多個二維座標可為（X，Z），處理控制裝置13可基於每個關節點所對應的二維座標（Y，Z）與二維座標（X，Z）建立一個三維座標，而評估動作三維資訊可包括以上述方式針對多個不同時間點及多個不同關節點而建立之多個三維座標。以圖6為例，此處示例的二維座標（X，Z）是指在X軸上的座標及Z軸上的座標；二維座標（Y，Z）是指在Y軸上的座標及Z軸上的座標。In step S405, the processing control device 13 uses the two-dimensional coordinate set of the first image and the corresponding two-dimensional coordinate set of the second image to generate three-dimensional information of the action to be evaluated. For example, as mentioned above, the plurality of two-dimensional coordinates corresponding to different joint points in the first two-dimensional coordinate group of the first image may each be (Y, Z), corresponding to the first coordinate at the same or similar time point. The multiple two-dimensional coordinates corresponding to different joint points in the second two-dimensional coordinate group of the two images can be (X, Z), and the processing control device 13 can be based on the two-dimensional coordinates (Y, Z) corresponding to each joint point. ) and two-dimensional coordinates (X, Z) to establish a three-dimensional coordinate, and the three-dimensional information of the evaluation action may include multiple three-dimensional coordinates established for multiple different time points and multiple different joint points in the above manner. Take Figure 6 as an example. The two-dimensional coordinates (X, Z) in the example here refer to the coordinates on the X-axis and the coordinates on the Z-axis; the two-dimensional coordinates (Y, Z) refer to the coordinates on the Y-axis and The coordinate on the Z axis.

請接著一併參考圖1及圖10，其中圖10係依據本發明一實施例所繪示的產生動作改善資訊的流程圖，可被編譯為指令。圖10可視為圖3之步驟S107的一實施例的細部流程圖。如圖10所示，產生動作改善資訊的方法包括：步驟S501：利用該些原始三維座標組產生角度原始資訊及速度原始資訊；步驟S503：利用多個標準三維座標組產生角度標準資訊及速度標準資訊；步驟S505：利用角度原始資訊與角度標準資訊的差異產生角度校正資訊；以及步驟S507：利用速度原始資訊與速度標準資訊的差異產生速度校正資訊。在圖10中，步驟S501係繪示為執行在步驟S503之前，但步驟S501亦可執行在步驟S503之後，或是步驟S501可與步驟S503被同時執行。相似地，在圖10中，步驟S505係繪示為執行在步驟S507之前，但步驟S505亦可執行在步驟S507之後，或是步驟S505可與步驟S507被同時執行。Please refer to FIG. 1 and FIG. 10 together. FIG. 10 is a flow chart of generating action improvement information according to an embodiment of the present invention, which can be compiled into instructions. FIG. 10 can be viewed as a detailed flow chart of an embodiment of step S107 in FIG. 3 . As shown in Figure 10, the method of generating motion improvement information includes: step S501: using these original three-dimensional coordinate groups to generate angle original information and speed original information; step S503: using multiple standard three-dimensional coordinate groups to generate angle standard information and speed standard information; step S505: generate angle correction information using the difference between the original angle information and the angle standard information; and step S507: generate speed correction information using the difference between the original speed information and the speed standard information. In FIG. 10 , step S501 is shown as being executed before step S503, but step S501 may also be executed after step S503, or step S501 may be executed simultaneously with step S503. Similarly, in FIG. 10 , step S505 is shown as being executed before step S507, but step S505 may also be executed after step S507, or step S505 may be executed simultaneously with step S507.

需先說明的是，待評估動作三維資訊包括對應於待評估動作的該些關節點在不同時間點的該些原始三維座標組，預設動作三維模型包括對應於待評估動作的該些關節點在不同時間點的多個標準三維座標組。換言之，待評估動作三維資訊代表第一使用者動作，包括透過第一影像及第二影像取得的多個關節點的多個原始三維座標組；預設動作三維模型代表訓練員動作，包括訓練員在執行待評估動作時的該些關節點的標準三維座標組。It should be noted that the three-dimensional information of the action to be evaluated includes the original three-dimensional coordinate groups at different time points corresponding to the joint points of the action to be evaluated, and the preset action three-dimensional model includes the joint points corresponding to the action to be evaluated. Multiple sets of standard 3D coordinates at different points in time. In other words, the three-dimensional information of the action to be evaluated represents the first user's action, including multiple original three-dimensional coordinate groups of multiple joint points obtained through the first image and the second image; the default action three-dimensional model represents the trainer's action, including the trainer's action. The standard three-dimensional coordinate set of the joint points when performing the action to be evaluated.

於步驟S501，處理控制裝置13利用該些原始三維座標組產生指示各關節點的彎曲/旋轉角度的角度原始資訊，及利用該些原始三維座標組產生指示各關節點的移動速度的速度原始資訊。In step S501, the processing control device 13 uses the original three-dimensional coordinate groups to generate original angle information indicating the bending/rotation angle of each joint point, and uses the original three-dimensional coordinate groups to generate original speed information indicating the moving speed of each joint point. .

於步驟S503，處理控制裝置13利用該些標準三維座標組產生指示訓練員的各關節點的彎曲/旋轉角度的角度標準資訊，及利用該些原始三維座標組產生指示訓練員的各關節點的移動速度的速度標準資訊。In step S503, the processing control device 13 uses the standard three-dimensional coordinate sets to generate angle standard information indicating the bending/rotation angle of each joint point of the trainer, and uses the original three-dimensional coordinate sets to generate angle standard information indicating the bending/rotation angle of each joint point of the trainer. Speed standard information for movement speed.

於步驟S505及步驟S507，處理控制裝置13利用角度原始資訊與角度標準資訊的差異產生角度校正資訊，其中角度校正資訊可以指示角度原始資訊到角度標準資訊所需的角度變化量；及利用速度原始資訊與速度標準資訊的差異產生速度校正資訊，其中速度校正資訊可以指示速度原始資訊到速度標準資訊所需的速度變化量。In steps S505 and S507, the processing control device 13 generates angle correction information using the difference between the angle original information and the angle standard information, where the angle correction information can indicate the angle change required from the angle original information to the angle standard information; and uses the speed original information. The difference between the information and the speed standard information generates speed correction information, where the speed correction information can indicate the amount of speed change required from the original speed information to the speed standard information.

換言之，在圖10的實施例中，處理控制裝置13將第一使用者的角度原始資訊及速度原始資訊分別與訓練員的角度標準資訊及速度標準資訊進行比對，以產生角度校正資訊及速度校正資訊。據此，第一使用者即可根據角度校正資訊及速度校正資訊判斷自身的動作與訓練員的標準動作之間的差異，進而改善自身在進行待評估動作時的姿態。In other words, in the embodiment of FIG. 10 , the processing control device 13 compares the first user's original angle information and the original speed information with the trainer's angle standard information and speed standard information respectively to generate angle correction information and speed. Calibration information. Accordingly, the first user can determine the difference between his own movements and the trainer's standard movements based on the angle correction information and the speed correction information, thereby improving his posture when performing the movements to be evaluated.

於一例子中，上述步驟S501可以包含：從該些原始三維座標組取得指定關節點所對應的多個原始座標；以該些原始座標中的最高座標所對應的時間點作為起始時間點；以該些原始座標中的最低座標所對應的時間點作為終止時間點；以及利用該些原始座標組中對應於起始時間點至終止時間點的時間區間的部分，計算角度原始資訊及速度原始資訊。舉例而言，假設待評估動作為羽球的發球動作，指定關節點為手腕，處理控制裝置13可取得手腕在三維空間中的對應不同時間點的多個原始座標，將這些原始座標中具最大Z軸座標的一者（視為發球的起點座標）所對應的時間點作為起始時間點，及將這些原始座標中具最小Z軸座標的一者（視為發球的終點座標）所對應的時間點作為終止時間點，並利用在起始時間點至終止時間點的時間區間的原始座標，計算角度原始資訊及速度原始資訊。In one example, the above step S501 may include: obtaining multiple original coordinates corresponding to the specified joint point from the original three-dimensional coordinate groups; using the time point corresponding to the highest coordinate among the original coordinates as the starting time point; Use the time point corresponding to the lowest coordinate among the original coordinates as the end time point; and use the part of the original coordinate group corresponding to the time interval from the starting time point to the ending time point to calculate the original angle information and the original speed information. For example, assuming that the action to be evaluated is a badminton serving action, and the designated joint point is the wrist, the processing control device 13 can obtain multiple original coordinates of the wrist corresponding to different time points in the three-dimensional space, and assign the maximum Z among these original coordinates. The time point corresponding to one of the axis coordinates (regarded as the starting point coordinate of the serve) is used as the starting time point, and the time corresponding to the one with the smallest Z-axis coordinate among these original coordinates (regarded as the end coordinate of the serve) point as the end time point, and use the original coordinates of the time interval from the start time point to the end time point to calculate the original angle information and the original speed information.

於另一例子中，上述步驟S501可以包含：從該些原始三維座標組取得指定關節點所對應的多個原始座標；當該些原始座標中的一者位於預設座標範圍內時，以該些原始座標中的該者所對應的時間點作為起始時間點；以及利用該些原始三維座標組中對應該起始時間點之後的時間的部分，計算角度原始資訊及速度原始資訊。同樣假設待評估動作為羽球的發球動作，指定關節點為手腕，預設座標範圍為Z軸座標高於第一使用者的頭頂的Z軸座標的範圍。在此例子中，處理控制裝置13可取得手腕在三維空間中的對應不同時間點的多個原始座標，並於原始座標的Z軸座標高於第一使用者的頭頂的Z軸座標時，以此原始座標所對應的時間點作為起始時間點，並以此起始時間點之後的原始座標計算角度原始資訊及速度原始資訊。In another example, the above step S501 may include: obtaining a plurality of original coordinates corresponding to the specified joint point from the original three-dimensional coordinate groups; when one of the original coordinates is within the preset coordinate range, use the The time point corresponding to the original coordinates is used as the starting time point; and the angle original information and the velocity original information are calculated using the part of the original three-dimensional coordinate group corresponding to the time after the starting time point. It is also assumed that the action to be evaluated is a badminton serving action, the specified joint point is the wrist, and the preset coordinate range is the range where the Z-axis coordinate is higher than the Z-axis coordinate of the first user's head. In this example, the processing control device 13 can obtain multiple original coordinates of the wrist in the three-dimensional space corresponding to different time points, and when the Z-axis coordinate of the original coordinates is higher than the Z-axis coordinate of the first user's head, The time point corresponding to this original coordinate is used as the starting time point, and the original angle information and speed original information are calculated from the original coordinates after this starting time point.

請接著參考圖11，圖11係依據本發明另一實施例所繪示的提供動作改善資訊的系統2的方塊圖。如圖11所示，提供動作改善資訊的系統2包括第一攝像裝置21、第二攝像裝置22、處理控制裝置23、第三攝像裝置24、第四攝像裝置25、輸出介面26以及慣性量測裝置27及28。第一攝像裝置21、第二攝像裝置22以及輸出介面26可分別與圖1的第一攝像裝置11、第二攝像裝置12以及輸出介面14相同；處理控制裝置23可與圖1及圖2的處理控制裝置13相同，故相同之處不再於此贅述。Please refer next to FIG. 11 , which is a block diagram of a system 2 for providing action improvement information according to another embodiment of the present invention. As shown in FIG. 11 , the system 2 for providing motion improvement information includes a first camera device 21 , a second camera device 22 , a processing control device 23 , a third camera device 24 , a fourth camera device 25 , an output interface 26 and an inertia measurement device. Devices 27 and 28. The first camera device 21 , the second camera device 22 and the output interface 26 may be the same as the first camera device 11 , the second camera device 12 and the output interface 14 of FIG. 1 respectively; the processing control device 23 may be the same as those of FIGS. 1 and 2 The processing control device 13 is the same, so the similarities will not be repeated here.

相較於圖1的提供動作改善資訊的系統1，提供動作改善資訊的系統2更包括第三攝像裝置24、第四攝像裝置25以及慣性量測裝置27及28。第三攝像裝置24及第四攝像裝置25可為任何能夠取像的攝影機，且較佳是相同型號的攝像裝置。第三攝像裝置24及第四攝像裝置25分別用於取得多張第三影像及多張第四影像，第三影像及第四影像係用於建立前述的預設動作三維模型。慣性量測裝置27及28連接於處理控制裝置23，且用於設置於第二使用者（例如，前述的訓練員）的多個關節點以測得對應於第二使用者在進行待評估動作時的不同時間點的多筆慣性量測資訊。慣性量測裝置27及28例如為六軸感測器，慣性量測資訊可包括加速度及角速度。另外，圖11示例性地繪示兩個慣性量測裝置27及28，但慣性量測裝置的數量可依需量測的關節點的數量設置，本發明不對慣性量測裝置的數量予以限制。Compared with the system 1 for providing motion improvement information in FIG. 1 , the system 2 for providing motion improvement information further includes a third camera device 24 , a fourth camera device 25 and inertial measurement devices 27 and 28 . The third camera device 24 and the fourth camera device 25 can be any camera capable of capturing images, and preferably are camera devices of the same model. The third camera device 24 and the fourth camera device 25 are respectively used to obtain a plurality of third images and a plurality of fourth images. The third images and the fourth images are used to establish the aforementioned three-dimensional model of the preset action. The inertial measurement devices 27 and 28 are connected to the processing control device 23 and are used to be arranged at multiple joint points of the second user (for example, the aforementioned trainer) to measure the motion corresponding to the second user's actions to be evaluated. Multiple pieces of inertial measurement information at different time points. The inertial measurement devices 27 and 28 are, for example, six-axis sensors, and the inertial measurement information may include acceleration and angular velocity. In addition, FIG. 11 illustrates two inertial measurement devices 27 and 28 as an example. However, the number of inertial measurement devices can be set according to the number of joint points that need to be measured. The present invention does not limit the number of inertial measurement devices.

請接著參考圖12，圖12係依據本發明再一實施例所繪示的提供動作改善資訊的系統2’的方塊圖。如圖12所示，提供動作改善資訊的系統2’與圖11所示的提供動作改善資訊的系統2相似，故相同之處不再於此贅述。相較於提供動作改善資訊的系統2，動作改善資訊的系統2’更包括另一處理控制裝置29，電性或通訊連接於處理控制裝置23、第三攝像裝置24、第四攝像裝置25以及慣性量測裝置27及28。處理控制裝置29可以包含一或多個處理器，所述處理器例如為中央處理器、繪圖處理器、微控制器、可程式化邏輯控制器、其他具有訊號處理功能的處理器或其結合。Please refer next to Figure 12, which is a block diagram of a system 2' for providing action improvement information according to yet another embodiment of the present invention. As shown in Figure 12, the system 2' for providing action improvement information is similar to the system 2 for providing action improvement information shown in Figure 11, so the similarities will not be repeated here. Compared with the system 2 that provides action improvement information, the system 2' that provides action improvement information further includes another processing control device 29, which is electrically or communicatively connected to the processing control device 23, the third camera device 24, the fourth camera device 25 and Inertial measurement devices 27 and 28. The processing control device 29 may include one or more processors, such as a central processing unit, a graphics processor, a microcontroller, a programmable logic controller, other processors with signal processing functions, or a combination thereof.

在圖11的實施例中，處理控制裝置23的處理器用於根據第一影像及第二影像執行圖3、圖5及圖8到圖10的步驟；以及基於第三影像及第四影像產生前述的預設動作三維模型。在圖12的實施例中，處理控制裝置23的處理器用於根據第一影像及第二影像執行圖3、圖5及圖8到圖10的步驟；處理控制裝置29的處理器用於基於第三影像及第四影像產生預設動作三維模型。以下以處理控制裝置29為例說明產生預設動作三維模型的方式，惟須注意的是，下述產生預設動作三維模型的方式或可由處理控制裝置23執行。也就是說，處理控制裝置29及處理控制裝置23可為（例如被整合為）相同的處理控制裝置。In the embodiment of FIG. 11 , the processor of the processing control device 23 is configured to perform the steps of FIG. 3 , FIG. 5 , and FIG. 8 to FIG. 10 based on the first image and the second image; and generate the aforementioned steps based on the third image and the fourth image. 3D model of preset actions. In the embodiment of FIG. 12 , the processor of the processing control device 23 is configured to execute the steps of FIG. 3 , FIG. 5 and FIG. 8 to FIG. 10 based on the first image and the second image; the processor of the processing control device 29 is configured to execute the steps of FIG. 3 , FIG. 5 , and FIG. 8 to FIG. The image and the fourth image generate a preset action three-dimensional model. The following uses the processing control device 29 as an example to illustrate the method of generating a preset action three-dimensional model. However, it should be noted that the following method of generating a preset action three-dimensional model may be executed by the processing control device 23 . That is, process control device 29 and process control device 23 may be (eg, integrated into) the same process control device.

具體地，處理控制裝置29從第三攝像裝置24取得多張第三影像，該些第三影像係以第一角度拍攝第二使用者而產生且對應於第二使用者在進行待評估動作時的不同時間點；處理控制裝置29從第四攝像裝置25取得多張第四影像，該些第四影像係以第二角度拍攝第二使用者而產生且對應於第二使用者在進行待評估動作時的不同時間點。第三攝像裝置24及第四攝像裝置25是同時對進行待評估動作的第二使用者取像以分別得到第三影像及第四影像。此述取得第三影像及第四影像的方式同理於前列實施例所述之取得第一影像及第二影像的方式，故於此不予贅述。進一步來說，第三影像及第四影像的取像時間點不同於第一影像及第二影像的取像時間點，特別係早於第一影像及第二影像的取像時間點。Specifically, the processing control device 29 obtains a plurality of third images from the third camera device 24. These third images are generated by photographing the second user at a first angle and correspond to when the second user performs an action to be evaluated. at different time points; the processing control device 29 obtains a plurality of fourth images from the fourth camera device 25. These fourth images are generated by photographing the second user at a second angle and correspond to the second user's process to be evaluated. different time points during action. The third camera device 24 and the fourth camera device 25 capture images of the second user performing the action to be evaluated at the same time to obtain the third image and the fourth image respectively. The method of obtaining the third image and the fourth image is the same as the method of obtaining the first image and the second image described in the previous embodiments, and therefore will not be described again here. Furthermore, the imaging time point of the third image and the fourth image is different from the imaging time point of the first image and the second image, especially earlier than the imaging time point of the first image and the second image.

在第三攝像裝置24及第四攝像裝置25進行取像的同時，設置於第二使用者的多個關節點上的慣性量測裝置27及28進行量測。處理控制裝置29可從慣性量測裝置27及28取得第二使用者在進行待評估動作時的不同時間點的多筆慣性量測資訊。換言之，一筆慣性量測資訊可包括不同關節點的上的慣性量測裝置在同一個時間點產生的資料。While the third camera device 24 and the fourth camera device 25 capture images, the inertial measurement devices 27 and 28 disposed on multiple joint points of the second user perform measurements. The processing control device 29 can obtain multiple pieces of inertial measurement information from the inertial measurement devices 27 and 28 at different time points when the second user performs the action to be evaluated. In other words, a piece of inertial measurement information may include data generated by inertial measurement devices on different joint points at the same point in time.

具體而言，待評估動作是由一連串的連續動作組合而成，為了建立更精準的預設動作三維模型，除了第三影像及第四影像，慣性量測資訊可作為進一步調整預設動作三維模型的三維資訊。因此，在取得第三影像、第四影像及慣性量測資訊後，處理控制裝置29可根據第三影像、第四影像及慣性量測資訊產生預設動作三維模型。Specifically, the action to be evaluated is composed of a series of continuous actions. In order to establish a more accurate 3D model of the preset action, in addition to the third image and the fourth image, the inertial measurement information can be used to further adjust the 3D model of the preset action. three-dimensional information. Therefore, after obtaining the third image, the fourth image and the inertial measurement information, the processing control device 29 can generate a preset motion three-dimensional model based on the third image, the fourth image and the inertial measurement information.

為了更詳細說明根據第三影像、第四影像及慣性量測資訊產生預設動作三維模型的方法，請接著一併參考圖12及圖13，圖13 係依據本發明一實施例所繪示的產生預設動作三維模型的流程圖，可被編譯為指令。如圖13所示，產生預設動作三維模型的方法包括由處理控制裝置29執行：步驟S601：從該些第三影像分別取得多個第三二維座標組；步驟S603：從該些第四影像分別取得多個第四二維座標組；步驟S605：對該些第三二維座標組及該些第四二維座標組進行深度補償以取得多個第一三維座標組；步驟S607：根據該些慣性量測資訊取得多個第二三維座標組；步驟S609：對該些第一三維座標組及該些第二三維座標組進行對準校正以產生多個經校正三維座標組；以及步驟S611：對該些經校正三維座標組進行渲染以產生預設動作三維模型。In order to explain in more detail the method of generating a preset action three-dimensional model based on the third image, the fourth image and the inertial measurement information, please refer to Figure 12 and Figure 13 together. Figure 13 is shown according to an embodiment of the present invention. A flowchart that generates a 3D model of preset actions can be compiled into instructions. As shown in Figure 13, the method of generating a preset action three-dimensional model includes executing by the processing control device 29: Step S601: Obtain a plurality of third two-dimensional coordinate groups from the third images; Step S603: Obtain a plurality of third two-dimensional coordinate groups from the fourth images. The images respectively obtain multiple fourth two-dimensional coordinate groups; Step S605: Perform depth compensation on the third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups to obtain multiple first three-dimensional coordinate groups; Step S607: According to The inertial measurement information obtains a plurality of second three-dimensional coordinate groups; step S609: perform alignment correction on the first three-dimensional coordinate groups and the second three-dimensional coordinate groups to generate a plurality of corrected three-dimensional coordinate groups; and step S611: Render these corrected three-dimensional coordinate groups to generate a preset action three-dimensional model.

步驟S601、S603及S605的實現方式可分別與圖5的步驟S201、S203及S205的實現方式相同，其中步驟S605亦可以是以圖8的方法實現。簡言之，該些第三二維座標組及該些第四二維座標組各包括分別對應於該些關節點的多個二維座標；該些第一三維座標組包括分別對應於該些關節點的多個三維座標；預設動作三維模型可包括角度標準資訊及速度標準資訊，且角度標準資訊及速度標準資訊可分別與前述的角度原始資訊及速度原始資訊以同樣方式產生。因此，步驟S601、S603及S605的內容不再於此贅述。The implementation manner of steps S601, S603 and S605 can be the same as the implementation manner of steps S201, S203 and S205 in Figure 5 respectively, wherein step S605 can also be implemented by the method in Figure 8. In short, the third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups each include a plurality of two-dimensional coordinates respectively corresponding to the joint points; the first three-dimensional coordinate groups include respectively corresponding to the joint points. Multiple three-dimensional coordinates of joint points; the preset action three-dimensional model can include angle standard information and speed standard information, and the angle standard information and speed standard information can be generated in the same manner as the aforementioned angle original information and speed original information respectively. Therefore, the contents of steps S601, S603 and S605 will not be described again here.

在取得第一三維座標組後，對於與每一幀第三影像對應相同或相近時間點的慣性量測資訊，於步驟S607，處理控制裝置29將慣性量測資訊轉換為對應各關節點的三維座標，並將這些三維座標作為一個第二三維座標組。該些第二三維座標組包括分別對應於該些關節點的多個三維座標。After obtaining the first three-dimensional coordinate group, for the inertial measurement information corresponding to the same or similar time point as each frame of the third image, in step S607, the processing control device 29 converts the inertial measurement information into three-dimensional information corresponding to each joint point. coordinates, and use these three-dimensional coordinates as a second three-dimensional coordinate group. The second three-dimensional coordinate groups include a plurality of three-dimensional coordinates respectively corresponding to the joint points.

於步驟S609，處理控制裝置29對對應相同或相近時間點的第一三維座標組及第二三維座標組進行對準校正，以產生多個經校正三維座標組，其中經校正三維座標組各包含分別對應於該些關節點的多個三維座標。以第二使用者的手腕作為一個關節點為例，假設基於步驟S601到S607產生了分別對應三幀第三影像的三個第一三維座標組，每個第一三維座標組對應一個時間點，則處理控制裝置29可基於內插法，利用兩個第一三維座標組之間的時間點對應的慣性量測資訊補足無對應的第三影像及第四影像的時間點的三維座標組，以產生經校正三維座標組。In step S609, the processing control device 29 performs alignment correction on the first three-dimensional coordinate group and the second three-dimensional coordinate group corresponding to the same or similar time points to generate a plurality of corrected three-dimensional coordinate groups, wherein the corrected three-dimensional coordinate groups each include Multiple three-dimensional coordinates respectively corresponding to the joint points. Taking the second user's wrist as a joint point as an example, assume that three first three-dimensional coordinate groups corresponding to three frames of the third image are generated based on steps S601 to S607, and each first three-dimensional coordinate group corresponds to a time point. Then the processing control device 29 can use the inertial measurement information corresponding to the time points between the two first three-dimensional coordinate groups to supplement the uncorresponding three-dimensional coordinate groups of the time points of the third image and the fourth image based on the interpolation method, so as to Produces a corrected three-dimensional coordinate set.

於步驟S611，處理控制裝置29對該些經校正三維座標組進行渲染以產生預設動作三維模型。具體地，處理控制裝置29將經校正三維座標組渲染為以立體影像呈現的預設動作三維模型。此外，處理控制裝置29可將預設動作三維模型輸出至處理控制裝置23。In step S611, the processing control device 29 renders these corrected three-dimensional coordinate sets to generate a preset motion three-dimensional model. Specifically, the processing control device 29 renders the corrected three-dimensional coordinate set into a preset action three-dimensional model presented in a stereoscopic image. In addition, the processing control device 29 can output the preset action three-dimensional model to the processing control device 23 .

請接著一併參考圖12及圖14，圖14係依據本發明另一實施例所繪示的產生預設動作三維模型的流程圖，可被編譯為指令。如圖14所示，產生預設動作三維模型的方法包括由處理控制裝置29執行：步驟S701：分別對該些第三影像及該些第四影像進行姿態辨識以取得對應第二使用者的多個人體骨架；步驟S703：從該些人體骨架分別取得對應待評估動作的多個關節點的多個二維座標組；以及步驟S705：利用二維座標組及慣性量測資訊產生預設動作三維模型。步驟S701及步驟S703的實現方式分別與圖9的步驟S401及步驟S403的實現方式相同，故不於此贅述。Please refer to FIG. 12 and FIG. 14 together. FIG. 14 is a flow chart of generating a three-dimensional model of a preset action according to another embodiment of the present invention, which can be compiled into instructions. As shown in Figure 14, the method of generating a preset action three-dimensional model includes executing by the processing control device 29: Step S701: Perform posture recognition on the third images and the fourth images respectively to obtain multiple images corresponding to the second user. Personal human skeleton; Step S703: Obtain multiple two-dimensional coordinate sets corresponding to multiple joint points of the motion to be evaluated from the human skeleton; and Step S705: Use the two-dimensional coordinate set and inertial measurement information to generate a preset motion three-dimensional Model. The implementation manner of step S701 and step S703 is the same as the implementation manner of step S401 and step S403 in FIG. 9 respectively, and therefore will not be described again here.

於步驟S705，處理控制裝置29利用第三影像的二維座標組、與第三影像對應相同或相近時間點的第四影像的二維座標組以及慣性量測資訊，產生待評估動作三維資訊。舉例而言，產生第三影像、第四影像以及慣性量測資訊的時間點可彼此相同或相近，處理控制裝置29可以圖9的步驟S405的方法基於第三影像及第四影像建立三維座標，以及利用慣性量測資訊調整三維座標，而預設動作三維模型即可包括經調整後的三維座標。「第三影像、第四影像以及慣性量測資訊的時間點彼此相近」是指多張第四影像中，取像時間點最接近當前第三影像的取像時間點的一者，以及在多筆的慣性量測資訊中，產生的時間點最接近當前第三影像的一者。In step S705, the processing control device 29 uses the two-dimensional coordinate set of the third image, the two-dimensional coordinate set of the fourth image corresponding to the same or similar time point as the third image, and the inertial measurement information to generate three-dimensional information of the action to be evaluated. For example, the time points at which the third image, the fourth image and the inertial measurement information are generated can be the same or close to each other, and the processing control device 29 can establish three-dimensional coordinates based on the third image and the fourth image in the method of step S405 in FIG. 9 , And use the inertial measurement information to adjust the three-dimensional coordinates, and the preset action three-dimensional model can include the adjusted three-dimensional coordinates. "The time points of the third image, the fourth image and the inertial measurement information are close to each other" means that among the multiple fourth images, the imaging time point is closest to the imaging time point of the current third image, and among the multiple fourth images, the imaging time point is closest to the imaging time point of the current third image. Among the pen's inertial measurement information, the time point generated is closest to the current third image.

請接著參考圖12及圖15，其中圖15係依據本發明另一實施例所繪示的深度補償的流程圖，可被編譯為指令。如圖15所示，對第三二維座標組及第四二維座標組進行深度補償以取得第一三維座標組的方法包括由處理控制裝置29執行：步驟S801：將每一第三二維座標組作為第一目標座標組，將對應於第一目標座標組所對應的時間點的第四二維座標組作為第二目標座標組；步驟S803：判斷第一目標座標組的該些二維座標中的每一個及其於第二目標座標組的該些二維座標中對應於相同關節點的二維座標；步驟S805：判斷第二軸座標與三軸座標是否相同；若步驟S805的判斷結果為「是」，執行步驟S807：以第二軸座標及第三軸座標中的一者、第一軸座標以及第四軸座標組成三維座標；若步驟S805的判斷結果為「否」，執行步驟S809：根據第二軸座標與第三軸座標之間的差距調整第四軸座標，並以第一軸座標、第二軸座標及經調整的第四軸座標組成三維座標；以及步驟S811：以第一目標座標組的該些二維座標各自對應的三維座標組成該些第一三維座標組中的一個。Please refer to FIG. 12 and FIG. 15 . FIG. 15 is a flow chart of depth compensation according to another embodiment of the present invention, which can be compiled into instructions. As shown in FIG. 15 , the method of performing depth compensation on the third two-dimensional coordinate group and the fourth two-dimensional coordinate group to obtain the first three-dimensional coordinate group includes executing by the processing control device 29: Step S801: The coordinate group is used as the first target coordinate group, and the fourth two-dimensional coordinate group corresponding to the time point corresponding to the first target coordinate group is used as the second target coordinate group; Step S803: Determine the two-dimensional coordinates of the first target coordinate group. Each of the coordinates and the two-dimensional coordinates corresponding to the same joint point among the two-dimensional coordinates of the second target coordinate group; Step S805: Determine whether the second-axis coordinates and the three-axis coordinates are the same; if the judgment of step S805 The result is "Yes", execute step S807: use one of the second axis coordinate and the third axis coordinate, the first axis coordinate and the fourth axis coordinate to form a three-dimensional coordinate; if the judgment result of step S805 is "No", execute Step S809: Adjust the fourth axis coordinate according to the difference between the second axis coordinate and the third axis coordinate, and form a three-dimensional coordinate with the first axis coordinate, the second axis coordinate and the adjusted fourth axis coordinate; and step S811: One of the first three-dimensional coordinate groups is composed of three-dimensional coordinates corresponding to the two-dimensional coordinates of the first target coordinate group.

圖15的實施例與圖8的實施例相似，圖15的步驟S801、S803、S805、S807、S809及S811的執行內容同理於圖8所示之步驟S301、S303、S305、S307、S309及S311，故不再於此贅述。簡言之，圖15的實施例是對第三影像的第三二維座標組及第四影像的第四二維座標組進行深度補償，以取得用於產生前述經校正三維座標組的多個三維座標組。The embodiment of Fig. 15 is similar to the embodiment of Fig. 8. The execution contents of steps S801, S803, S805, S807, S809 and S811 of Fig. 15 are the same as those of steps S301, S303, S305, S307, S309 and S309 shown in Fig. 8 S311, so I won’t repeat it here. In short, the embodiment of FIG. 15 performs depth compensation on the third two-dimensional coordinate set of the third image and the fourth two-dimensional coordinate set of the fourth image to obtain multiple coordinates for generating the aforementioned corrected three-dimensional coordinate set. Three-dimensional coordinate set.

特別來說，上述一或多個實施例所提供之提供動作改善資訊的方法、處理控制裝置以及系統可應用但不限於元宇宙。具體來說，上述利用兩個不同拍攝角度的影像產生使用者之待評估動作三維資訊的步驟猶如於元宇宙中建置使用者的數位分身並從中取得動作資訊，利用待評估動作三維資訊及預設動作三維模型產生動作改善資訊的步驟猶如於元宇宙中將使用者的數位分身執行之動作與預設數位分身執行之動作進行比較。使用者可以透過虛擬實境眼鏡、擴增實境眼鏡、手機、個人電腦或其他個人裝置進入元宇宙，以自己的數位分身和系統預設的數位分身進行互動，甚至是和另一使用者的數位分身進行互動。In particular, the method, processing control device and system for providing action improvement information provided by one or more of the above embodiments can be applied to, but are not limited to, the Metaverse. Specifically, the above-mentioned steps of using images from two different shooting angles to generate three-dimensional information about the user's movements to be evaluated are like building a digital clone of the user in the metaverse and obtaining movement information therefrom. Using the three-dimensional information about the movements to be evaluated and prediction The step of setting up a three-dimensional motion model to generate motion improvement information is like comparing the actions performed by the user's digital avatar with the actions performed by the preset digital avatar in the metaverse. Users can enter the metaverse through virtual reality glasses, augmented reality glasses, mobile phones, personal computers or other personal devices, and interact with their own digital avatars with the system's preset digital avatars, or even with another user's Interact with digital avatars.

透過上述架構，本案所揭示的提供動作改善資訊方法、處理控制裝置以及系統可以非接觸式的方式即時分析使用者的動作並提供改善資訊，藉此，在運動訓練的應用上，可以達到訓練中分析及事後檢討的目的。另外，相較於一般使用多個高速攝影機的配置，本案所揭示的提供動作改善方法、處理控制裝置以及系統可基於兩台攝像裝置所拍攝的影像分析得到動作改善資訊，可以具有較低的影像資料處理量，且可以減少整體系統的配置成本。此外，透過慣性量測資訊的使用，可以產生精準的各關節點的三維座標，進而建立更精準的預設動作三維模型。Through the above structure, the method of providing movement improvement information, the processing control device and the system disclosed in this case can real-time analyze the user's movements and provide improvement information in a non-contact manner, thereby, in the application of sports training, it can achieve the goal of training. The purpose of analysis and post-mortem review. In addition, compared with the general configuration of using multiple high-speed cameras, the motion improvement method, processing control device and system disclosed in this case can obtain motion improvement information based on image analysis captured by two camera devices, and can have lower image quality. data processing volume, and can reduce the overall system configuration cost. In addition, through the use of inertial measurement information, accurate three-dimensional coordinates of each joint point can be generated, thereby establishing a more accurate three-dimensional model of preset actions.

上述的「訓練員」指的是可協助「受訓者」達成特定運動項目專業的人員。上述「第一」、「第二」、「第三」或「第四」僅是用來區別相同的陳述（例如影像、攝像裝置或使用者），而非用來限制該些陳述之間具有任何順序，也非用來限制該些陳述所涉及的步驟之間具有任何順序。The above-mentioned "trainers" refer to those who can assist "trainees" in achieving expertise in specific sports. The above "first", "second", "third" or "fourth" are only used to distinguish the same statements (such as images, camera devices or users), but are not used to limit the differences between these statements. Any order is not intended to limit any order between the steps involved in these statements.

雖然本發明以前述之實施例揭露如上，然其並非用以限定本發明。在不脫離本發明之精神和範圍內，所為之更動與潤飾，均屬本發明之專利保護範圍。關於本發明所界定之保護範圍請參考所附之申請專利範圍。Although the present invention is disclosed in the foregoing embodiments, they are not intended to limit the present invention. All changes and modifications made without departing from the spirit and scope of the present invention shall fall within the scope of patent protection of the present invention. Regarding the protection scope defined by the present invention, please refer to the attached patent application scope.

1,2,2’:提供動作改善資訊的系統 11,21:第一攝像裝置 12,22:第二攝像裝置 24:第三攝像裝置 25:第四攝像裝置 13,23,29:處理控制裝置 14,26:輸出介面 27,28:慣性量測裝置 131:記憶體 132:處理器 IMG1:第一影像 IMG2:第二影像 S:第一使用者 D11,D12,D13,D14,D15,D21,D22,D23,D24,D25:關節點 S101,S103,S105,S107,S109,S201,S203,S205,S207,S301,S303,S305,S307,S309,S311,S401,S403,S405,S501,S503,S505,S507,S601,S603,S605,S607,S609,S611,S701,S703,S705,S801,S803,S805,S807,S809,S811:步驟 1,2,2’: System that provides action improvement information 11,21: First camera device 12,22: Second camera device 24:Third camera device 25: The fourth camera device 13,23,29: Processing control device 14,26:Output interface 27,28:Inertial measurement device 131:Memory 132: Processor IMG1:First Image IMG2: Second image S: first user D11,D12,D13,D14,D15,D21,D22,D23,D24,D25: joint points S101,S103,S105,S107,S109,S201,S203,S205,S207,S301,S303,S305,S307,S309,S311,S401,S403,S405,S501,S503,S505,S507,S601,S603,S6 05, S607, S609, S611, S701, S703, S705, S801, S803, S805, S807, S809, S811: Steps

圖1係依據本發明一實施例所繪示的提供動作改善資訊的系統的方塊圖。圖2係依據本發明一實施例所繪示的處理控制裝置的方塊圖。圖3係依據本發明一實施例所繪示的提供動作改善資訊的方法的流程圖。圖4(a)示例性地呈現以第一角度拍攝第一使用者的第一影像。圖4(b)示例性地呈現以第二角度拍攝第一使用者的第二影像。圖5係依據本發明一實施例所繪示的產生待評估動作三維資訊的流程圖。圖6係繪示以第一角度拍攝第一使用者的第一影像及以第二角度拍攝第一使用者的第二影像的示意圖。圖7(a)係繪示第一影像的一第一二維座標組的示意圖。圖7(b)係繪示繪示第二影像的一第二二維座標組的示意圖。圖8係依據本發明一實施例所繪示的深度補償的流程圖。圖9係依據本發明另一實施例所繪示的產生待評估動作三維資訊的流程圖。圖10係依據本發明一實施例所繪示的產生動作改善資訊的流程圖。圖11係依據本發明另一實施例所繪示的提供動作改善資訊的系統的方塊圖。圖12係依據本發明再一實施例所繪示的提供動作改善資訊的系統的方塊圖。圖13係依據本發明一實施例所繪示的產生預設動作三維模型的流程圖。圖14係依據本發明另一實施例所繪示的產生預設動作三維模型的流程圖。圖15係依據本發明另一實施例所繪示的深度補償的流程圖。 FIG. 1 is a block diagram of a system for providing action improvement information according to an embodiment of the present invention. FIG. 2 is a block diagram of a processing control device according to an embodiment of the present invention. FIG. 3 is a flowchart of a method for providing action improvement information according to an embodiment of the present invention. FIG. 4(a) exemplarily presents a first image of the first user taken at a first angle. FIG. 4(b) exemplarily presents a second image of the first user taken at a second angle. FIG. 5 is a flow chart for generating three-dimensional information of actions to be evaluated according to an embodiment of the present invention. FIG. 6 is a schematic diagram of capturing a first image of the first user at a first angle and capturing a second image of the first user at a second angle. FIG. 7(a) is a schematic diagram illustrating a first two-dimensional coordinate set of the first image. FIG. 7(b) is a schematic diagram illustrating a second two-dimensional coordinate set representing the second image. FIG. 8 is a flow chart of depth compensation according to an embodiment of the present invention. FIG. 9 is a flow chart for generating three-dimensional information of actions to be evaluated according to another embodiment of the present invention. FIG. 10 is a flowchart of generating action improvement information according to an embodiment of the present invention. FIG. 11 is a block diagram of a system for providing action improvement information according to another embodiment of the present invention. FIG. 12 is a block diagram of a system for providing action improvement information according to yet another embodiment of the present invention. Figure 13 is a flowchart of generating a three-dimensional model of preset actions according to an embodiment of the present invention. FIG. 14 is a flowchart of generating a three-dimensional model of a preset action according to another embodiment of the present invention. FIG. 15 is a flow chart of depth compensation according to another embodiment of the present invention.

S101,S103,S105,S107,S109:步驟 S101, S103, S105, S107, S109: Steps

Claims

一種提供動作改善資訊的方法，包含以一處理控制裝置執行：取得多張第一影像，該些第一影像係以一第一角度拍攝一第一使用者而產生且對應於該第一使用者在進行一待評估動作時的不同時間點；取得多張第二影像，該些第二影像係以一第二角度拍攝該第一使用者而產生且對應於該第一使用者在進行該待評估動作時的不同時間點，其中該第二角度不同於該第一角度；根據該些第一影像及該些第二影像產生對應該第一使用者之一待評估動作三維資訊；利用該待評估動作三維資訊及一預設動作三維模型產生一動作改善資訊，該動作改善資訊包含對應該待評估動作在一三維空間中的一角度校正資訊及一速度校正資訊；以及經由一介面輸出該動作改善資訊。 A method of providing action improvement information includes executing with a processing control device: Obtain a plurality of first images, which are generated by photographing a first user at a first angle and correspond to different time points when the first user performs an action to be evaluated; Obtain a plurality of second images, which are generated by photographing the first user at a second angle and correspond to different time points when the first user performs the action to be evaluated, wherein the second The angle is different from the first angle; Generate three-dimensional information corresponding to an action to be evaluated of the first user based on the first images and the second images; Using the three-dimensional movement information to be evaluated and a preset movement three-dimensional model to generate movement improvement information, the movement improvement information includes an angle correction information and a speed correction information corresponding to the movement to be evaluated in a three-dimensional space; and The action improvement information is output through an interface.

如請求項1所述的提供動作改善資訊的方法，更包含經由該介面輸出於該預設動作三維模型中對應該動作改善資訊的多張參考影像。The method of providing motion improvement information as described in claim 1 further includes outputting, through the interface, a plurality of reference images corresponding to the motion improvement information in the preset motion three-dimensional model.

如請求項1所述的提供動作改善資訊的方法，其中根據該些第一影像及該些第二影像產生該待評估動作三維資訊包含：從該些第一影像分別取得多個第一二維座標組；從該些第二影像分別取得多個第二二維座標組；對該些第一二維座標組及該些第二二維座標組進行深度補償以取得多個原始三維座標組；以及根據該些原始三維座標組產生一角度原始資訊及一速度原始資訊，以作為該待評估動作三維資訊；其中該些第一二維座標組及該些第二二維座標組各包含分別對應於該第一使用者之該待評估動作的多個關節點的多個二維座標，且該些原始三維座標組各包含分別對應於該些關節點的多個三維座標。 The method for providing motion improvement information as described in claim 1, wherein generating the three-dimensional motion information to be evaluated based on the first images and the second images includes: Obtain a plurality of first two-dimensional coordinate groups from the first images respectively; Obtain a plurality of second two-dimensional coordinate groups from the second images respectively; Perform depth compensation on the first two-dimensional coordinate groups and the second two-dimensional coordinate groups to obtain a plurality of original three-dimensional coordinate groups; and Generate an angle original information and a speed original information based on the original three-dimensional coordinate groups as the three-dimensional information of the action to be evaluated; The first two-dimensional coordinate groups and the second two-dimensional coordinate groups each include a plurality of two-dimensional coordinates respectively corresponding to a plurality of joint points of the first user's action to be evaluated, and the original three-dimensional Each coordinate group includes a plurality of three-dimensional coordinates respectively corresponding to the joint points.

如請求項1所述的提供動作改善資訊的方法，其中根據該些第一影像及該些第二影像產生該待評估動作三維資訊包含：分別對該些第一影像及該些第二影像進行姿態辨識以取得對應該第一使用者的多個人體骨架；從該些人體骨架分別取得對應該待評估動作的多個關節點的多個二維座標組；以及利用該些二維座標組產生該待評估動作三維資訊。 The method for providing motion improvement information as described in claim 1, wherein generating the three-dimensional motion information to be evaluated based on the first images and the second images includes: Perform posture recognition on the first images and the second images respectively to obtain multiple human skeletons corresponding to the first user; Obtain multiple two-dimensional coordinate groups corresponding to multiple joint points of the action to be evaluated from the human skeletons; and The two-dimensional coordinate groups are used to generate three-dimensional information of the action to be evaluated.

如請求項1所述的提供動作改善資訊的方法，其中該待評估動作三維資訊包含對應於該待評估動作的多個關節點在不同時間點的多個原始三維座標組，該預設動作三維模型包含對應於該待評估動作的該些關節點在不同時間點的多個標準三維座標組，且利用該待評估動作三維資訊及該預設動作三維模型產生該動作改善資訊包含：利用該些原始三維座標組產生一角度原始資訊及一速度原始資訊；利用該些標準三維座標組產生一角度標準資訊及一速度標準資訊；利用該角度原始資訊與該角度標準資訊的差異產生該角度校正資訊；以及利用該速度原始資訊與該速度標準資訊的差異產生該速度校正資訊。 The method for providing action improvement information as described in claim 1, wherein the three-dimensional action information to be evaluated includes multiple original three-dimensional coordinate groups corresponding to multiple joint points of the action to be evaluated at different time points, and the preset action three-dimensional The model includes a plurality of standard three-dimensional coordinate groups corresponding to the joint points of the action to be evaluated at different time points, and the action improvement information generated using the three-dimensional information of the action to be evaluated and the preset action three-dimensional model includes: Using the original three-dimensional coordinate groups to generate an angle original information and a speed original information; Using these standard three-dimensional coordinate sets to generate an angle standard information and a speed standard information; Using the difference between the angle original information and the angle standard information to generate the angle correction information; and The speed correction information is generated using the difference between the speed original information and the speed standard information.

如請求項1所述的提供動作改善資訊的方法，更包含在利用該預設動作三維模型之前，以該處理控制裝置或另一處理控制裝置執行：取得多張第三影像，該些第三影像係以該第一角度拍攝一第二使用者而產生且對應於該第二使用者在進行該待評估動作時的不同時間點；取得多張第四影像，該些第四影像係以該第二角度拍攝該第二使用者而產生且對應於該第二使用者在進行該待評估動作時的不同時間點；從設置於該第二使用者的多個關節點的多個慣性量測裝置取得對應於該第二使用者在進行該待評估動作時的不同時間點的多筆慣性量測資訊；以及根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型。 The method of providing motion improvement information as described in claim 1 further includes, before using the preset motion three-dimensional model, executing with the processing control device or another processing control device: Obtain a plurality of third images, which are generated by photographing a second user at the first angle and correspond to different time points when the second user performs the action to be evaluated; Obtain a plurality of fourth images, which are generated by photographing the second user at the second angle and correspond to different time points when the second user performs the action to be evaluated; Obtain multiple pieces of inertial measurement information corresponding to different time points when the second user performs the action to be evaluated from multiple inertial measurement devices disposed at multiple joint points of the second user; and The preset action three-dimensional model is generated based on the third images, the fourth images and the inertial measurement information.

如請求項6所述的提供動作改善資訊的方法，其中根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型包含：從該些第三影像分別取得多個第三二維座標組；從該些第四影像分別取得多個第四二維座標組；對該些第三二維座標組及該些第四二維座標組進行深度補償以取得多個第一三維座標組；根據該些慣性量測資訊取得多個第二三維座標組；對該些第一三維座標組及該些第二三維座標組進行對準校正以產生多個經校正三維座標組；以及對該些經校正三維座標組進行渲染以產生該預設動作三維模型；其中該些第三二維座標組及該些第四二維座標組各包含分別對應於該些關節點的多個二維座標，且該些第一三維座標組、該些第二三維座標組及該些經校正三維座標組各包含分別對應於該些關節點的多個三維座標。 The method for providing motion improvement information as described in claim 6, wherein generating the preset motion three-dimensional model based on the third images, the fourth images and the inertial measurement information includes: Obtain multiple third two-dimensional coordinate groups from the third images respectively; Obtain a plurality of fourth two-dimensional coordinate groups from the fourth images respectively; Perform depth compensation on the third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups to obtain a plurality of first three-dimensional coordinate groups; Obtain a plurality of second three-dimensional coordinate groups based on the inertial measurement information; performing alignment correction on the first three-dimensional coordinate groups and the second three-dimensional coordinate groups to generate a plurality of corrected three-dimensional coordinate groups; and Rendering the corrected three-dimensional coordinate sets to generate the preset action three-dimensional model; The third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups each include a plurality of two-dimensional coordinates respectively corresponding to the joint points, and the first three-dimensional coordinate groups and the second three-dimensional coordinate groups And each of the corrected three-dimensional coordinate groups includes a plurality of three-dimensional coordinates respectively corresponding to the joint points.

如請求項6所述的提供動作改善資訊的方法，其中根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型包含：分別對該些第三影像及該些第四影像進行姿態辨識以取得對應該第二使用者的多個人體骨架；從該些人體骨架分別取得對應該待評估動作的該些關節點的多個二維座標組；以及利用該些二維座標組及該些慣性量測資訊產生該預設動作三維模型。 The method for providing motion improvement information as described in claim 6, wherein generating the preset motion three-dimensional model based on the third images, the fourth images and the inertial measurement information includes: Perform posture recognition on the third images and the fourth images respectively to obtain multiple human skeletons corresponding to the second user; Obtain multiple two-dimensional coordinate groups corresponding to the joint points of the action to be evaluated from the human skeletons; and The preset motion three-dimensional model is generated using the two-dimensional coordinate sets and the inertial measurement information.

如請求項7所述的提供動作改善資訊的方法，其中該些第三二維座標組各自的多個二維座標各包含一第一軸座標及一第二軸座標，該些第四二維座標組各自的多個二維座標各包含一第三軸座標及一第四軸座標，該二軸座標與該第三軸座標對應於該三維空間中的同一軸，且對該些第三二維座標組及該些第四二維座標組進行該深度補償以取得該些第一三維座標組包含：將該些第三二維座標組中的每一個作為一第一目標座標組，將該些第四二維座標組中對應於該第一目標座標組所對應的時間點的第四二維座標組作為一第二目標座標組，執行：對該第一目標座標組的該些二維座標中的每一個及其於該第二目標座標組的該些二維座標中對應於相同關節點的二維座標，執行：當該第二軸座標與該三軸座標相同時，以該第二軸座標及該第三軸座標中的一者、該第一軸座標以及該第四軸座標組成一三維座標；以及當該第二軸座標與該三軸座標不相同時，根據該第二軸座標與該第三軸座標之間的差距調整該第四軸座標，並以該第一軸座標、該第二軸座標及經調整的該第四軸座標組成該三維座標；以及以該第一目標座標組的該些二維座標各自對應的該三維座標組成該些第一三維座標組中的一個。 The method for providing motion improvement information as described in claim 7, wherein the plurality of two-dimensional coordinates of each of the third two-dimensional coordinate groups each include a first axis coordinate and a second axis coordinate, and the fourth two-dimensional coordinates Each of the plurality of two-dimensional coordinates of the coordinate group includes a third-axis coordinate and a fourth-axis coordinate. The two-axis coordinate and the third-axis coordinate correspond to the same axis in the three-dimensional space, and for these third-axis coordinates, The depth compensation is performed on the dimensional coordinate group and the fourth 2D coordinate group to obtain the first 3D coordinate group including: Each of the third two-dimensional coordinate groups is used as a first target coordinate group, and the fourth two-dimensional coordinates of the fourth two-dimensional coordinate groups corresponding to the time point corresponding to the first target coordinate group are Group as a second target coordinate group, execute: For each of the two-dimensional coordinates of the first target coordinate group and the two-dimensional coordinates corresponding to the same joint point among the two-dimensional coordinates of the second target coordinate group, execute: When the second axis coordinate is the same as the three-axis coordinate, one of the second axis coordinate and the third axis coordinate, the first axis coordinate and the fourth axis coordinate form a three-dimensional coordinate; and When the second axis coordinate is different from the three-axis coordinate, the fourth axis coordinate is adjusted according to the difference between the second axis coordinate and the third axis coordinate, and the first axis coordinate, the second axis coordinate coordinates and the adjusted fourth-axis coordinates to form the three-dimensional coordinates; and The three-dimensional coordinates corresponding to the two-dimensional coordinates of the first target coordinate group form one of the first three-dimensional coordinate groups.

一種處理控制裝置，包含：一記憶體，儲存用於提供動作改善資訊的多個指令；以及一處理器，連接於該記憶體，用於讀取該些指令以執行：取得多張第一影像，該些第一影像係以一第一角度拍攝一第一使用者而產生且對應於該第一使用者在進行一待評估動作時的不同時間點；取得多張第二影像，該些第二影像係以一第二角度拍攝該第一使用者而產生且對應於該第一使用者在進行該待評估動作時的不同時間點，其中該第二角度不同於該第一角度；根據該些第一影像及該些第二影像產生對應該第一使用者之一待評估動作三維資訊；利用該待評估動作三維資訊及一預設動作三維模型產生一動作改善資訊，該動作改善資訊包含對應該待評估動作在一三維空間中的一角度校正資訊及一速度校正資訊；以及經由一介面輸出該動作改善資訊。 A processing control device comprising: a memory storing a plurality of instructions for providing movement improvement information; and A processor, connected to the memory, is used to read the instructions to execute: Obtain a plurality of first images, which are generated by photographing a first user at a first angle and correspond to different time points when the first user performs an action to be evaluated; Obtain a plurality of second images, which are generated by photographing the first user at a second angle and correspond to different time points when the first user performs the action to be evaluated, wherein the second The angle is different from the first angle; Generate three-dimensional information corresponding to an action to be evaluated of the first user based on the first images and the second images; Using the three-dimensional movement information to be evaluated and a preset movement three-dimensional model to generate movement improvement information, the movement improvement information includes an angle correction information and a speed correction information corresponding to the movement to be evaluated in a three-dimensional space; and The action improvement information is output through an interface.

一種提供動作改善資訊的系統，包含：一第一攝像裝置，用於以一第一角度拍攝一第一使用者以產生多張第一影像，其中該些第一影像對應於該第一使用者在進行一待評估動作時的不同時間點；一第二攝像裝置，用於以一第二角度拍攝該第一使用者以產生多張第二影像，其中該些第二影像對應於該第一使用者在進行該待評估動作時的不同時間點，且該第二角度不同於該第一角度；一輸出介面；以及一處理控制裝置，連接於該第一攝像裝置、該第二攝像裝置及該輸出介面，且用於根據該些第一影像及該些第二影像產生一待評估姿態三維資訊，利用該待評估姿態三維資訊及一預設姿態三維模型產生一動作改善資訊，並經由該輸出介面輸出該動作改善資訊，其中該動作改善資訊包含對應該待評估動作在一三維空間中的一角度校正資訊及一速度校正資訊。 A system that provides movement improvement information, including: A first camera device for photographing a first user at a first angle to generate a plurality of first images, wherein the first images correspond to different times when the first user performs an action to be evaluated. point; A second camera device for photographing the first user at a second angle to generate a plurality of second images, wherein the second images correspond to different times when the first user performs the action to be evaluated. point, and the second angle is different from the first angle; an output interface; and A processing control device connected to the first camera device, the second camera device and the output interface, and used to generate three-dimensional information of a posture to be evaluated based on the first images and the second images, using the posture to be evaluated The posture three-dimensional information and a preset posture three-dimensional model generate a movement improvement information, and the movement improvement information is output through the output interface, wherein the movement improvement information includes an angle correction information corresponding to the movement to be evaluated in a three-dimensional space and an Speed correction information.

如請求項11所述的提供動作改善資訊的系統，其中該輸出介面更用於輸出於該預設動作三維模型中對應該動作改善資訊的多張參考影像。The system for providing motion improvement information as described in claim 11, wherein the output interface is further used to output a plurality of reference images corresponding to the motion improvement information in the preset motion three-dimensional model.

如請求項11所述的提供動作改善資訊的系統，其中該處理控制裝置執行根據該些第一影像及該些第二影像產生該待評估動作三維資訊包含：從該些第一影像分別取得多個第一二維座標組；從該些第二影像分別取得多個第二二維座標組；對該些第一二維座標組及該些第二二維座標組進行深度補償以取得多個原始三維座標組；以及根據該些原始三維座標組產生一角度原始資訊及一速度原始資訊，以作為該待評估動作三維資訊；其中該些第一二維座標組及該些第二二維座標組各包含分別對應於該第一使用者之該待評估動作的多個關節點的多個二維座標，且該些原始三維座標組各包含分別對應於該些關節點的多個三維座標。 The system for providing motion improvement information as described in claim 11, wherein the processing control device generates the three-dimensional motion information to be evaluated based on the first images and the second images including: Obtain a plurality of first two-dimensional coordinate groups from the first images respectively; Obtain a plurality of second two-dimensional coordinate groups from the second images respectively; Perform depth compensation on the first two-dimensional coordinate groups and the second two-dimensional coordinate groups to obtain a plurality of original three-dimensional coordinate groups; and Generate an angle original information and a speed original information based on the original three-dimensional coordinate groups as the three-dimensional information of the action to be evaluated; The first two-dimensional coordinate groups and the second two-dimensional coordinate groups each include a plurality of two-dimensional coordinates respectively corresponding to a plurality of joint points of the first user's action to be evaluated, and the original three-dimensional Each coordinate group includes a plurality of three-dimensional coordinates respectively corresponding to the joint points.

如請求項11所述的提供動作改善資訊的系統，其中該處理控制裝置執行根據該些第一影像及該些第二影像產生該待評估動作三維資訊包含：分別對該些第一影像及該些第二影像進行姿態辨識以取得對應該第一使用者的多個人體骨架；從該些人體骨架分別取得對應該待評估動作的多個關節點的多個二維座標組；以及利用該些二維座標組產生該待評估動作三維資訊。 The system for providing motion improvement information as described in claim 11, wherein the processing control device generates the three-dimensional motion information to be evaluated based on the first images and the second images including: Perform posture recognition on the first images and the second images respectively to obtain multiple human skeletons corresponding to the first user; Obtain multiple two-dimensional coordinate groups corresponding to multiple joint points of the action to be evaluated from the human skeletons; and The two-dimensional coordinate groups are used to generate three-dimensional information of the action to be evaluated.

如請求項11所述的提供動作改善資訊的系統，其中該待評估動作三維資訊包含對應於該待評估動作的多個關節點在不同時間點的多個原始三維座標組，該預設動作三維模型包含對應於該待評估動作的該些關節點在不同時間點的多個標準三維座標組，且該處理控制裝置執行利用該待評估動作三維資訊及該預設動作三維模型產生該動作改善資訊包含：利用該些原始三維座標組產生一角度原始資訊及一速度原始資訊；利用該些標準三維座標組產生一角度標準資訊及一速度標準資訊；利用該角度原始資訊與該角度標準資訊的差異產生該角度校正資訊；以及利用該速度原始資訊與該速度標準資訊的差異產生該速度校正資訊。 The system for providing action improvement information as described in claim 11, wherein the three-dimensional information of the action to be evaluated includes a plurality of original three-dimensional coordinate groups corresponding to multiple joint points of the action to be evaluated at different time points, and the preset action three-dimensional The model includes a plurality of standard three-dimensional coordinate groups corresponding to the joint points of the action to be evaluated at different time points, and the processing control device executes using the three-dimensional information of the action to be evaluated and the preset action three-dimensional model to generate the action improvement information. Include: Using the original three-dimensional coordinate groups to generate an angle original information and a speed original information; Using these standard three-dimensional coordinate sets to generate an angle standard information and a speed standard information; Using the difference between the angle original information and the angle standard information to generate the angle correction information; and The speed correction information is generated using the difference between the speed original information and the speed standard information.

如請求項11所述的提供動作改善資訊的系統，更包含：一第三攝像裝置，連接於該處理控制裝置，且用於以該第一角度拍攝一第二使用者以產生對應於該第二使用者在進行該待評估動作時的不同時間點的多張第三影像；一第四攝像裝置，連接於該處理控制裝置，且用於以該第二角度拍攝該第二使用者以產生對應於該第二使用者在進行該待評估動作時的不同時間點的多張第四影像；以及多個慣性量測裝置，連接於該處理控制裝置，且用於設置於該第二使用者的多個關節點以測得對應於該第二使用者在進行該待評估動作時的不同時間點的多筆慣性量測資訊；其中該處理控制裝置更用於在利用該預設動作三維模型之前執行：取得該些第三影像；取得該些第四影像；取得該些慣性量測資訊；以及根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型。 The system for providing action improvement information as described in request 11 further includes: A third camera device connected to the processing control device and used to photograph a second user at the first angle to generate a plurality of pictures corresponding to different time points when the second user performs the action to be evaluated. third image; A fourth camera device connected to the processing control device and used to photograph the second user at the second angle to generate a plurality of pictures corresponding to different time points when the second user performs the action to be evaluated. fourth image; and A plurality of inertial measurement devices are connected to the processing control device and used to be arranged at multiple joint points of the second user to measure different time points corresponding to the second user when performing the action to be evaluated. Multiple pieces of inertial measurement information; The processing control device is further configured to execute before utilizing the preset action three-dimensional model: Obtain the third images; Obtain the fourth images; Obtain the inertial measurement information; and The preset action three-dimensional model is generated based on the third images, the fourth images and the inertial measurement information.

如請求項11所述的提供動作改善資訊的系統，更包含：一第三攝像裝置，用於以該第一角度拍攝一第二使用者以產生對應於該第二使用者在進行該待評估動作時的不同時間點的多張第三影像；一第四攝像裝置，用於以該第二角度拍攝該第二使用者以產生對應於該第二使用者在進行該待評估動作時的不同時間點的多張第四影像；多個慣性量測裝置，用於設置於該第二使用者的多個關節點以測得對應於該第二使用者在進行該待評估動作時的不同時間點的多筆慣性量測資訊；以及另一處理控制裝置，連接於該第三攝像裝置、該第四攝像裝置、該些慣性量測裝置及該處理控制裝置，且用於執行：取得該些第三影像；取得該些第四影像；取得該些慣性量測資訊；以及根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型，並提供至該處理控制裝置。 The system for providing action improvement information as described in request 11 further includes: a third camera device for photographing a second user at the first angle to generate a plurality of third images corresponding to different time points when the second user performs the action to be evaluated; a fourth camera device for photographing the second user at the second angle to generate a plurality of fourth images corresponding to different time points when the second user performs the action to be evaluated; A plurality of inertia measurement devices arranged at multiple joint points of the second user to measure a plurality of inertia measurement information corresponding to different time points when the second user performs the action to be evaluated; as well as Another processing control device is connected to the third camera device, the fourth camera device, the inertial measurement devices and the processing control device, and is used to perform: Obtain the third images; Obtain the fourth images; Obtain the inertial measurement information; and The preset action three-dimensional model is generated based on the third images, the fourth images and the inertial measurement information, and is provided to the processing control device.

如請求項16或17所述的提供動作改善資訊的系統，其中該處理控制裝置或該另一處理控制裝置執行根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型包含：從該些第三影像分別取得多個第三二維座標組；從該些第四影像分別取得多個第四二維座標組；對該些第三二維座標組及該些第四二維座標組進行深度補償以取得多個第一三維座標組；根據該些慣性量測資訊取得多個第二三維座標組；對該些第一三維座標組及該些第二三維座標組進行對準校正以產生多個經校正三維座標組；以及對該些經校正三維座標組進行渲染以產生該預設動作三維模型；其中該些第三二維座標組及該些第四二維座標組各包含分別對應於該些關節點的多個二維座標，且該些第一三維座標組、該些第二三維座標組及該些經校正三維座標組各包含分別對應於該些關節點的多個三維座標。 The system for providing motion improvement information as described in claim 16 or 17, wherein the processing control device or the other processing control device executes generating the process based on the third images, the fourth images and the inertial measurement information. Preset action 3D models include: Obtain multiple third two-dimensional coordinate groups from the third images respectively; Obtain a plurality of fourth two-dimensional coordinate groups from the fourth images respectively; Perform depth compensation on the third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups to obtain a plurality of first three-dimensional coordinate groups; Obtain a plurality of second three-dimensional coordinate groups based on the inertial measurement information; performing alignment correction on the first three-dimensional coordinate groups and the second three-dimensional coordinate groups to generate a plurality of corrected three-dimensional coordinate groups; and Rendering the corrected three-dimensional coordinate sets to generate the preset action three-dimensional model; The third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups each include a plurality of two-dimensional coordinates respectively corresponding to the joint points, and the first three-dimensional coordinate groups and the second three-dimensional coordinate groups And each of the corrected three-dimensional coordinate groups includes a plurality of three-dimensional coordinates respectively corresponding to the joint points.

如請求項16或17所述的提供動作改善資訊的系統，其中該處理控制裝置或該另一處理控制裝置執行根據該些第三影像、該些第四影像及該些慣性量測資訊產生該預設動作三維模型包含：分別對該些第三影像及該些第四影像進行姿態辨識以取得對應該第二使用者的多個人體骨架；從該些人體骨架分別取得對應該待評估動作的該些關節點的多個二維座標組；以及利用該些二維座標組及該些慣性量測資訊產生該預設動作三維模型。 The system for providing motion improvement information as described in claim 16 or 17, wherein the processing control device or the other processing control device executes generating the process based on the third images, the fourth images and the inertial measurement information. Preset action 3D models include: Perform posture recognition on the third images and the fourth images respectively to obtain multiple human skeletons corresponding to the second user; Obtain multiple two-dimensional coordinate groups corresponding to the joint points of the action to be evaluated from the human skeletons; and The preset motion three-dimensional model is generated using the two-dimensional coordinate sets and the inertial measurement information.

如請求項18所述的提供動作改善資訊的系統，其中該些第三二維座標組各自的多個二維座標各包含一第一軸座標及一第二軸座標，該些第四二維座標組各自的多個二維座標各包含一第三軸座標及一第四軸座標，該第二軸座標與該第三軸座標對應於該三維空間中的同一軸，且該處理控制裝置或該另一處理控制裝置執行對該些第三二維座標組及該些第四二維座標組進行該深度補償以取得該些第一三維座標組包含：將該些第三二維座標組中的每一個作為一第一目標座標組，將該些第四二維座標組中對應於該第一目標座標組所對應的時間點的第四二維座標組作為一第二目標座標組，執行：對該第一目標座標組的該些二維座標中的每一個及其於該第二目標座標組的該些二維座標中對應於相同關節點的二維座標，執行：當該第二軸座標與該三軸座標相同時，以該第二軸座標及該第三軸座標中的一者、該第一軸座標以及該第四軸座標組成一三維座標；以及當該第二軸座標與該三軸座標不相同時，根據該第二軸座標與該第三軸座標之間的差距調整該第四軸座標，並以該第一軸座標、該第二軸座標及經調整的該第四軸座標組成該三維座標；以及以該第一目標座標組的該些二維座標各自對應的該三維座標組成該些第一三維座標組中的一個。 The system for providing motion improvement information as described in claim 18, wherein the plurality of two-dimensional coordinates of each of the third two-dimensional coordinate groups each include a first axis coordinate and a second axis coordinate, and the fourth two-dimensional coordinates Each of the plurality of two-dimensional coordinates of the coordinate group includes a third-axis coordinate and a fourth-axis coordinate, the second-axis coordinate and the third-axis coordinate correspond to the same axis in the three-dimensional space, and the processing control device or The other processing control device performs the depth compensation on the third two-dimensional coordinate groups and the fourth two-dimensional coordinate groups to obtain the first three-dimensional coordinate groups including: Each of the third two-dimensional coordinate groups is used as a first target coordinate group, and the fourth two-dimensional coordinates of the fourth two-dimensional coordinate groups corresponding to the time point corresponding to the first target coordinate group are Group as a second target coordinate group, execute: For each of the two-dimensional coordinates of the first target coordinate group and the two-dimensional coordinates corresponding to the same joint point among the two-dimensional coordinates of the second target coordinate group, execute: When the second axis coordinate is the same as the three-axis coordinate, one of the second axis coordinate and the third axis coordinate, the first axis coordinate and the fourth axis coordinate form a three-dimensional coordinate; and When the second axis coordinate is different from the three-axis coordinate, the fourth axis coordinate is adjusted according to the difference between the second axis coordinate and the third axis coordinate, and the first axis coordinate, the second axis coordinate coordinates and the adjusted fourth-axis coordinates to form the three-dimensional coordinates; and The three-dimensional coordinates corresponding to the two-dimensional coordinates of the first target coordinate group form one of the first three-dimensional coordinate groups.