TWI779696B - Generation system and generation method for perspective images - Google Patents
Generation system and generation method for perspective images Download PDFInfo
- Publication number
- TWI779696B TWI779696B TW110124351A TW110124351A TWI779696B TW I779696 B TWI779696 B TW I779696B TW 110124351 A TW110124351 A TW 110124351A TW 110124351 A TW110124351 A TW 110124351A TW I779696 B TWI779696 B TW I779696B
- Authority
- TW
- Taiwan
- Prior art keywords
- target object
- data set
- image
- module
- projection
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Processing (AREA)
- Image Generation (AREA)
- Container Filling Or Packaging Operations (AREA)
Abstract
Description
本發明係與生成視角影像有關,特別有關於視角影像的生成系統與生成方法。 The present invention is related to generating perspective images, in particular to a generation system and method for perspective images.
為了進行人工智慧或專業檢查人員(如進行違禁品檢查的海關人員)的物件辨識訓練,需要生成大量的影像(如X光影像)來做為訓練影像,除了可對多元的物件進行各種角度的掃描來獲得造影影像,或是再將造影影像進行各種形變來生成大量的虛擬影像,然前者會耗費許多時間與人力來進行掃描;後者亦有可能會有失真的結果。 In order to conduct object recognition training for artificial intelligence or professional inspectors (such as customs officers conducting contraband inspections), it is necessary to generate a large number of images (such as X-ray images) as training images, in addition to various angles for multiple objects Scanning to obtain contrast images, or performing various deformations on contrast images to generate a large number of virtual images, but the former will consume a lot of time and manpower to scan; the latter may also have distortion results.
因此,現有訓練影像的生成方法存在前述問題,而亟待更有效的方案被提出。 Therefore, the existing training image generation methods have the aforementioned problems, and more effective solutions are urgently needed to be proposed.
本發明之主要目的,係在於提供一種視角影像的生成系統與生成方法,不需要執行3D建模,也能有效取得目標物件大量的不同視角的二維影像。 The main purpose of the present invention is to provide a system and method for generating perspective images, which can effectively obtain a large number of 2D images of different perspectives of a target object without performing 3D modeling.
於一實施例中,一種視角影像的生成方法,包括以下步驟:a)取得對一目標物件執行斷層掃描所獲得的一造影資料集,其中該造影資料集包括連續的多個切片影像;b)決定該目標物件的一視角所對應的一旋轉資訊;c)基於該旋轉資訊旋轉該造影資料集與一投影面的至少其中之一,以使該造影資料集的該視角朝向一投影面;及,d)對該造影資料集朝該投影面執行一壓合處理來組合該多個切片影像為該視角的一二維影像。 In one embodiment, a method for generating a viewing angle image includes the following steps: a) obtaining a contrast data set obtained by performing tomographic scanning on a target object, wherein the contrast data set includes a plurality of continuous slice images; b) determining a rotation information corresponding to a viewing angle of the target object; c) rotating at least one of the imaging data set and a projection surface based on the rotation information, so that the viewing angle of the imaging data set faces a projection surface; and , d) performing a lamination process on the imaging data set toward the projection surface to combine the plurality of slice images into a two-dimensional image of the viewing angle.
於一實施例中,一種視角影像的生成系統,包括一儲存模組、一輸出模組及電性連接該儲存裝置及該輸出裝置的一處理模組;該儲存模組用以儲存對一目標物件執行斷層掃描所獲得的一造影資料集,其中該造影資料集包括連續的多個切片影像;該輸出模組用以輸出該目標物件的一視角的一二維影像;該處理模組被配置來決定該目標物件的該視角所對應的一旋轉資訊,基於該旋轉資訊旋轉該造影資料集與一投影面的至少其中之一,以使該造影資料集的該視角朝向一投影面,並對旋轉後的該造影資料集朝該投影面執行一壓合處理來組合該多個切片影像為該視角的該二維影像平面。 In one embodiment, a system for generating perspective images includes a storage module, an output module, and a processing module electrically connected to the storage device and the output device; A contrasting data set obtained by performing tomographic scanning on the object, wherein the contrasting data set includes a plurality of continuous slice images; the output module is used to output a two-dimensional image of a viewing angle of the target object; the processing module is configured determining a rotation information corresponding to the viewing angle of the target object, rotating at least one of the imaging data set and a projection surface based on the rotation information, so that the viewing angle of the imaging data set faces a projection surface, and The rotated imaging data set performs a lamination process toward the projection surface to combine the plurality of slice images into the 2D image plane of the viewing angle.
本發明可快速地生成目標物件的指定視角的二維影像。 The invention can quickly generate a two-dimensional image of a specified viewing angle of a target object.
2:生成系統 2: Generate system
20:處理模組 20: Processing modules
21:儲存模組 21: Storage module
210:造影資料集 210: Angiographic data set
211:電腦程式 211: Computer program
22:輸出模組 22: Output module
23:輸入模組 23: Input module
30:造影設備 30: Imaging equipment
31:應用端 31: Application side
40:顯示模組 40:Display module
41:訓練模組 41:Training module
42:機器學習模型 42:Machine Learning Models
43:辨識模組 43: Identification module
50:視角選擇模組 50: Angle selection module
51:旋轉模組 51:Rotary module
52:壓合模組 52: Pressing module
520:投影模組 520: Projection module
521:組合模組 521: combination module
53:過濾模組 53: Filter module
6:目標物件 6: Target object
60:造影資料集 60: Angiographic data set
61-6n:二維影像 61-6n: Two-dimensional image
7:目標物件 7: Target object
71-73:二維影像 71-73: Two-dimensional images
80、83、85:投影面 80, 83, 85: projection surface
81、82、84、86:二維影像 81, 82, 84, 86: 2D image
91、92、94:造影資料集 91, 92, 94: Angiographic data set
X、Y、Z、X1-X5、Y1-Y5、Z1-Z5:軸 X, Y, Z, X1-X5, Y1-Y5, Z1-Z5: axes
P1、P2:位置 P1, P2: position
α、β、γ:角度 α, β, γ: angle
S10-S13:第一生成步驟 S10-S13: first generation step
S20-S25:第二生成步驟 S20-S25: second generation step
S30-S34:第三生成步驟 S30-S34: third generation step
S40-S42:壓合步驟 S40-S42: Pressing step
S50-S52:訓練步驟 S50-S52: training steps
S60-S62:辨識步驟 S60-S62: Identification steps
圖1為本發明一實施例的生成系統的架構圖。 FIG. 1 is an architecture diagram of a generating system according to an embodiment of the present invention.
圖2為本發明一實施例的生成系統的應用示意圖。 Fig. 2 is a schematic diagram of the application of the generating system according to an embodiment of the present invention.
圖3為本發明一實施例的應用端的架構圖。 FIG. 3 is an architecture diagram of an application end according to an embodiment of the present invention.
圖4為本發明一實施例的機器學習模型的應用示意圖。 FIG. 4 is a schematic diagram of the application of a machine learning model according to an embodiment of the present invention.
圖5為本發明一實施例的處理模組的架構圖。 FIG. 5 is a structural diagram of a processing module according to an embodiment of the present invention.
圖6為本發明一實施例的生成方法的流程圖。 Fig. 6 is a flowchart of a generating method according to an embodiment of the present invention.
圖7為本發明一實施例的生成方法的流程圖。 Fig. 7 is a flowchart of a generating method according to an embodiment of the present invention.
圖8為本發明一實施例的生成方法的流程圖。 Fig. 8 is a flowchart of a generating method according to an embodiment of the present invention.
圖9為本發明一實施例的訓練處理的流程圖。 FIG. 9 is a flowchart of training processing according to an embodiment of the present invention.
圖10為本發明一實施例的辨識處理的流程圖。 FIG. 10 is a flowchart of identification processing according to an embodiment of the present invention.
圖11為本發明一實施例的生成視角影像的示意圖。 FIG. 11 is a schematic diagram of generating perspective images according to an embodiment of the present invention.
圖12為本發明一實施例的造影資料集的示意圖。 FIG. 12 is a schematic diagram of a contrast data set according to an embodiment of the present invention.
圖13為本發明一實施例的旋轉造影資料集的示意圖。 FIG. 13 is a schematic diagram of a rotational imaging data set according to an embodiment of the present invention.
圖14為本發明一實施例的投影的示意圖。 Fig. 14 is a schematic diagram of projection according to an embodiment of the present invention.
圖15為本發明一實施例的旋轉造影資料集後投影的示意圖。 FIG. 15 is a schematic diagram of post-projection of a rotational imaging data set according to an embodiment of the present invention.
圖16為本發明一實施例的投影獲得的二維影像的示意圖。 FIG. 16 is a schematic diagram of a two-dimensional image obtained by projection according to an embodiment of the present invention.
圖17為本發明一實施例的旋轉後投影獲得的二維影像的示意圖。 FIG. 17 is a schematic diagram of a 2D image obtained by projection after rotation according to an embodiment of the present invention.
圖18為本發明一實施例的投影的示意圖。 Fig. 18 is a schematic diagram of projection according to an embodiment of the present invention.
圖19為本發明一實施例的旋轉投影面後投影的示意圖。 FIG. 19 is a schematic diagram of post-projection after rotating the projection plane according to an embodiment of the present invention.
茲就本發明之一較佳實施例,配合圖式,詳細說明如後。 A preferred embodiment of the present invention will be described in detail below with reference to the drawings.
本發明提出一種視角影像的生成系統與生成方法,能基於目標物件的造影資料集的多張切片影像直接生成此目標物件的大量的不同視角的二維影像,而不需執行會耗費大量的運算資源的3D建模。並且,前述任意視角的二維影像,其視覺效果是極為近似對目標物件的該視角進行造影所獲得的透視影像。 The present invention proposes a system and method for generating perspective images, which can directly generate a large number of two-dimensional images of different perspectives of the target object based on multiple slice images of the imaging data set of the target object, without the need to perform a large number of calculations 3D modeling of resources. Moreover, the visual effect of the aforementioned two-dimensional image at any viewing angle is very similar to the perspective image obtained by imaging the target object at the viewing angle.
值得一提的是,為了生成目標物件的指定視角的二維影像,目標物件的造影資料集是必須的。具體而言,造影資料集是包括目標物件的多層且基於同一軸向的連續切片影像(即斷層影像),這使得本發明透過以不同方向壓合這些連續切片影像可以生成對應視角的具有透視效果的二維影像。 It is worth mentioning that in order to generate a 2D image of a target object at a specified viewing angle, a radiography dataset of the target object is necessary. Specifically, the imaging data set includes multiple layers of the target object and is based on the same axial continuous slice images (ie, tomographic images), which allows the present invention to generate perspective effects with corresponding viewing angles by laminating these continuous slice images in different directions. two-dimensional image.
因此,當使用一般光學攝影機(如不具有透視攝影效果的RGB攝影機或3D掃描器)時,並無法生成具有透視效果的該視角的二維影像。 Therefore, when a general optical camera (such as an RGB camera without a perspective photography effect or a 3D scanner) is used, the 2D image with the perspective effect cannot be generated.
此外,當所使用的多張切片影像是基於不同軸向所生成時,由於這些切片影像無法或難以組合,同樣難以生成任意視角的二維影像。 In addition, when the multiple slice images used are generated based on different axes, it is also difficult to generate a 2D image of any viewing angle because these slice images cannot or are difficult to combine.
請參閱圖1,本發明之生成系統2可包括儲存模組21、輸出模組22、輸入模組23與電性連接上述模組的處理模組20。
Please refer to FIG. 1 , the
儲存模組21(例如是RAM、EEPROM、固態硬碟、磁碟硬碟、快閃記憶體等儲存裝置或其任意組合)用以儲存資料。於一實施例中,儲存模組21可儲存一或多個目標物件的造影資料集210。此處造影資料集210是事先透過造影設備30(如電腦斷層掃描機(CT)、磁力共振成像機(MRI)、正電子斷層造影機(PET)或其他造影設備)對目標物件執行斷層造影所獲得,並儲存於儲存模組21以用於本發明。
The storage module 21 (for example, storage devices such as RAM, EEPROM, solid state hard disk, magnetic disk hard disk, flash memory, or any combination thereof) is used for storing data. In one embodiment, the
輸出模組22,用以輸出資訊。於一實施例中,輸出模組22可包括顯示器、觸控螢幕、或投影模組等顯示設備,而可於本地端實現資料顯示,如顯示所生成的二維影像或所匯入的造影資料集。
The
輸入模組23,用以接受用戶操作。於一實施例中,輸入模組23可包括滑鼠、鍵盤、各式按鍵、或觸控板等輸入設備。
The
處理模組20(如CPU、GPU、TPU、MCU等處理器或其任意組合),用以控制生成系統2並實現本發明所提出之功能。
The processing module 20 (such as CPU, GPU, TPU, MCU and other processors or any combination thereof) is used to control the
於一實施例中,生成系統2可包括電性連接處理模組20的網路模組(圖未標示),如乙太網路卡、Wi-Fi網卡、Bluetooth網卡、蜂巢網路模組等,網路模組可透過網路(如無線網路、有線網路、蜂巢網路、區域網路或網際網路等)與外部設備(如後述之造影射設備30及/或應用端31)通訊。
In one embodiment, the
請一併參閱圖5,於一實施例中,處理模組20可包括模組50-53與模組520-521。這些模組分別被設定來實做不同的功能。
Please refer to FIG. 5 together. In one embodiment, the
前述模組是相互連接(可為電性連接與資訊鏈結),並可為硬體模組(例如是電子電路模組、積體電路模組、SoC等等)、軟體模組(例如是韌體、作業系統或應用程式)或軟硬體模組混搭,不加以限定。 The aforementioned modules are connected to each other (which may be electrical connection and information link), and may be hardware modules (such as electronic circuit modules, integrated circuit modules, SoC, etc.), software modules (such as firmware, operating system, or application) or a mix of hardware and software modules, without limitation.
值得一提的是,當前述模組為軟體模組(例如是韌體、作業系統或應用程式)時,儲存模組21可包括非暫態電腦可讀取記錄媒體(圖未標示),前述非暫態電腦可讀取記錄媒體儲存有電腦程式211,電腦程式211記錄有電腦可執行之程式碼,當處理模組20執行前述程式碼後,可實做對應模組之功能。
It is worth mentioning that when the above-mentioned module is a software module (such as firmware, operating system or application program), the
接著對本發明之生成方法進行說明,本發明各實施例的生成方法可應用於圖1-5中任一實施例的生成系統2。
Next, the generation method of the present invention will be described. The generation method of each embodiment of the present invention can be applied to the
請一併參閱圖6,本實施例的生成方法包括以下步驟。 Please also refer to FIG. 6 , the generation method in this embodiment includes the following steps.
步驟S10:處理模組20自儲存模組21讀取指定的目標物件的造影資料集210(即所讀取的造影資料集210是對目標物件執行斷層掃描所獲得的)。造影資料集210包括目標物件的連續的多個具有透視效果的切片影像(斷層影像)。
Step S10: The processing
於一實施例中,儲存模組21是儲存多個物件的造影資料集210,用戶可透過輸入模組23選擇多個物件的其中之一作為目標物件,以讀取此物件的造影資料集210。
In one embodiment, the
步驟S11:處理模組20透過視角選擇模組50決定目標物件的指定視角所對應的旋轉資訊,如計算可使此視角朝向投影面的尤拉角或其他類型的旋轉座標。
Step S11: The processing
於一實施例中,用戶可透過輸入模組23設定要生成二維影像的視角(如輸入各軸旋轉角度)。 In one embodiment, the user can set the viewing angle of the 2D image to be generated through the input module 23 (for example, input the rotation angle of each axis).
於一實施例中,儲存模組21可儲存一設定檔,設定檔內可記錄有執行生成方法所需的各種參數,如需要擷取的視角(或其對應的旋轉資訊)、影像解析度、投影取值方式、及/或後述之顯示範圍等。處理模組20可讀取此設定檔來決定指定視角所對應的旋轉資訊。
In one embodiment, the
步驟S12:處理模組20透過旋轉模組51依據所決定的旋轉資訊進行旋轉處理,以使造影資料集210的所指定視角朝向預設的投影面。
Step S12: The processing
於一實施例中,本發明是旋轉造影資料集210。具體而言,本發明可依據旋轉資訊所指示的各軸的旋轉角度來對造影資料集210的所有切片影像的影像位置進行旋轉變換,藉以實現旋轉造影資料集210。
In one embodiment, the present invention rotates the
請參閱圖12,造影資料集60是對目標物件6執行斷層掃描所生成。由於造影資料集60的多張切片影像之間具有位置關聯,本發明可透過直接旋轉造影資料集60,如沿X軸翻滾(Roll)、沿Y軸俯仰(pitch)或沿Z軸偏擺(yaw),來改變所有切片影像的位置,來獲得所要擷取的視角。
Please refer to FIG. 12 , the
於一實施例中,本發明是旋轉投影面。具體而言,本發明可依據旋轉資訊所指示的旋轉角度(或旋轉後的3D空間的座標位置)來使投影面繞造影資料集60旋轉並移動至該視角所對應的3D空間的座標位置,藉以實現旋轉該投影面。
In one embodiment, the present invention is a rotating projection surface. Specifically, according to the rotation angle (or the coordinate position of the rotated 3D space) indicated by the rotation information, the present invention can make the projection plane rotate around the
復請參閱圖6,接著執行步驟S13:處理模組20透過壓合模組52對旋轉後的造影資料集朝投影面執行壓合處理來組合造影資料集的多個切片影像為指定視角的二維影像。
Referring again to FIG. 6 , step S13 is then performed: the
藉此,本發明可以最小的運算量與複雜度來生成目標物件的指定視角的二維影像,而不需執行運算量大且複雜的3D建模相關步驟。由於本發明運算量與複雜度的降低,除了可節省處理時間,還可適用於中低階的軟硬體設備(如使用時脈較低的CPU、運算速度較慢的GPU、存取速度較慢的硬碟,且不需3D建模軟體),而可以降低設置成本。 Thereby, the present invention can generate a 2D image of a target object at a specified viewing angle with the minimum amount of computation and complexity, without performing steps related to 3D modeling with a large amount of computation and complexity. Due to the reduction of the amount of calculation and the complexity of the present invention, in addition to saving processing time, it can also be applicable to low- and middle-level software and hardware equipment (such as using a lower CPU with a clock pulse, a slower GPU with a lower computing speed, and a slower GPU with a faster access speed). slow hard disk, and does not require 3D modeling software), which can reduce setup costs.
接著,以旋轉造影資料集為例,說明本發明之旋轉處理與投影處理。請參閱圖14,旋轉前的造影資料集91的三軸角度以X3、Y3、Z3標示。當以此視角對投影面80進行投影(如前述之壓合處理)時,可獲得此視角的二維影像81(如圖16所示第一視角)。
Next, the rotation processing and projection processing of the present invention will be described by taking the rotation imaging data set as an example. Please refer to FIG. 14 , the three-axis angles of the angiography data set 91 before rotation are indicated by X3, Y3, and Z3. When the
請參閱圖15,經過本發明旋轉後的造影資料集92的三軸角度以X4、Y4、Z4標示,即造影資料集92是以不同視角朝向投影面80。當以此不同視角對投影面80進行投影(如前述之壓合處理)時,可獲得此不同視角的二維影像82(如圖17所示的第二視角)。
Please refer to FIG. 15 , the three-axis angles of the
接著,以旋轉投影面為例,說明本發明之旋轉處理與投影處理。請參閱圖18,造影資料集94的三軸角度以X5、Y5、Z5標示,旋轉前的投影面83於3D空間的座標位置為P1。當以此視角對投影面83進行投影(如前述之壓合處理)時,可獲得此視角的二維影像84(如圖16所示第一視角)。
Next, the rotation processing and projection processing of the present invention will be described by taking the rotation of the projection plane as an example. Please refer to FIG. 18 , the three-axis angles of the
請參閱圖19,於旋轉處理(即投影面83繞造影資料集94移動來抵達投影面85的位置P2)後,造影資料集94由於沒有自轉,其三軸角度仍為X5、Y5、Z5,但旋轉後的投影面85於3D空間的座標位置變為P2,且是朝向造影資
料集94的不同視角。當以此不同視角對旋轉後的投影面85進行投影(如前述之壓合處理)時,可獲得此不同視角的二維影像86(如圖17所示的第二視角)。
Please refer to FIG. 19 , after the rotation process (that is, the
值得一提的是,前述第一視角的二維影像81、84(如圖16)與第二視角的二維影像82、86(如圖17)由於取自造影資料而具有透視效果,而可以生成目標物件(如瑞士刀)於不同視角下的透視二維影像。
It is worth mentioning that the aforementioned two-
接著,舉例說明本發明如何生成目標物件的多視角的二維影像。請參閱圖11,造影資料60是對目標物件6進行造影所生成的。透過對造影資料60的不同的多個視角分別執行壓合處理,可生成多視角的二維影像61-6n。
Next, an example is given to illustrate how the present invention generates a multi-view two-dimensional image of a target object. Please refer to FIG. 11 , the
如對側視角執行壓合處理可生成側視的二維影像61,對正視角執行壓合處理可生成正視的二維影像62,對俯視角執行壓合處理可生成俯視的二維影像63,以此類推。
For example, performing compression processing on a side view can generate a side-view two-
於另一例子中,當本發明應用於目標物件7(如瑞士刀,亦可替換為其他欲檢測的物件)的造影資料時,可生成瑞士刀的多視角的具有透視效果的二維影像71-73。藉此,於取得目標物件7的多視角影像後,這些多視角影像可用於訓練電腦(如後述之機器學習模型42)或人員來學習從不同視角辨識出目標物件7,以提升其辨識目標物件7的準確率與速度。
In another example, when the present invention is applied to imaging data of a target object 7 (such as a Swiss knife, which can also be replaced by other objects to be detected), a multi-view two-
請參閱圖7,本發明之生成方法可設定為自動生成同一目標物件的多視角影像(步驟S24)與自動生成多物件的多視角影像(步驟S24與步驟S25)。具體而言,本實施例的生成方法包括以下步驟。 Please refer to FIG. 7 , the generation method of the present invention can be set to automatically generate a multi-view image of the same target object (step S24) and automatically generate multi-object multi-view images (step S24 and step S25). Specifically, the generating method of this embodiment includes the following steps.
首先,第一次執行步驟S20-S23,以產生指定視角(第一視角)的二維影像(第一二維影像)。步驟S20-S23是與前述之步驟S10-S13相同或相似,於此不再贅述。 Firstly, steps S20-S23 are executed for the first time to generate a 2D image (first 2D image) of a specified viewing angle (first viewing angle). Steps S20-S23 are the same as or similar to the aforementioned steps S10-S13, and will not be repeated here.
於產生第一二維影像,執行步驟S24:處理模組20透過視角選擇模組50於預先設定的多個視角中,選擇未完成影像壓合的另一視角,並再次執
行步驟S21-S23以獲得目標物件的另一視角(第二視角)的二維影像(第二二維影像),以此類推,直到完成所有預設視角的壓合,而獲得此目標物件許多張不同視角的二維影像。
When the first two-dimensional image is generated, step S24 is executed: the
於完成壓合生成此目標物件(第一物件)的多視角影像後,執行步驟S25:當有多個物件時,處理模組20判斷是否繼續另一物件(第二物件)的多視角影像的擷取工作。
After the lamination is completed to generate the multi-view image of the target object (the first object), step S25 is executed: when there are multiple objects, the
若否,則結束生成方法。 If not, the generate method ends.
若是,則處理模組20選擇另一物件作為目標物件來執行步驟S20-S24,以載入另一造影資料集,並逐一視角進行擷取來獲得多視角影像,以此類推,直到完成所有物件的多視角影像。
If yes, the
值得一提的是,這些物件的多視角影像可用來訓練人員或電腦(如後述之機器學習模型42),以提升其辨識這些物件的準確率與速度。補充說明,本發明不用將造影資料集進行3D建模處理,可直接將多個切片影像進行旋轉與壓合,因此不需將造影資料集的各切片影像中提取目標物件的點雲數據(point clouds),再對所提取的點雲數據(point clouds)執行3D建模(polygon meshes)等步驟,因此本發明大幅降低運算複雜度。
It is worth mentioning that the multi-view images of these objects can be used to train personnel or computers (such as the
請參閱圖8,本實施例之生成方法可包括過濾功能(步驟S33與步驟S42),可排除沒有興趣的影像部分。具體而言,本實施例的生成方法包括以下步驟。 Please refer to FIG. 8 , the generating method of this embodiment may include a filtering function (step S33 and step S42 ), which can exclude uninteresting image parts. Specifically, the generating method of this embodiment includes the following steps.
步驟S30-S32是與前述之步驟S10-S12相同或相似,於此不再贅述。 Steps S30-S32 are the same as or similar to the above-mentioned steps S10-S12, and will not be repeated here.
步驟S33:處理模組20透過過濾模組53取得設定的顯示範圍。前述顯示範圍可由用戶透過輸入模組23進行設定,或由系統自動設定,如可依據
目標物件的不同材質(阻射性,即射線的可穿透性)設定不同的感興趣的韓森費爾德單位(HU)或灰階值範圍。
Step S33: The processing
舉例來說,顯示範圍可被設定為「HU為+1000~+2000之間」,以使HU不在此範圍內的影像部分可被視為雜訊濾除。 For example, the display range can be set as "the HU is between +1000~+2000", so that the part of the image whose HU is not within this range can be regarded as noise and filtered out.
於另一例子中,以8位元灰階為例,顯示範圍可被設定為「灰階值為100-200之間」,以使灰階值不在此範圍內的影像部分可被視為雜訊濾除。 In another example, taking 8-bit grayscale as an example, the display range can be set as "the grayscale value is between 100-200", so that the image part whose grayscale value is not within this range can be regarded as noise. Information filtering.
步驟S34:處理模組20透過壓合模組52基於顯示範圍對旋轉後的造影資料集執行壓合處理。
Step S34: The processing
於一實施例中,壓合模組52可包括投影模組520與組合模組521。步驟S34可包括步驟S40-S42。
In one embodiment, the
步驟S40:處理模組20透過投影模組520於旋轉後的造影資料集的多個切片影像中,基於被投影至各投影位置的一或多個切片像素決定各投影位置的值。
Step S40: The processing
於一實施例中,以旋轉造影資料集為例,請參閱圖13,旋轉前的造影資料集的座標軸為X1、Y1、Z1,旋轉後的造影資料集的座標軸為X2、Y2、Z2,N為法線。由圖可知,旋轉後的造影資料集的尤拉角為(α,β,γ)。 In one embodiment, taking the rotating radiography data set as an example, please refer to FIG. 13 , the coordinate axes of the radiography data set before rotation are X1, Y1, Z1, and the coordinate axes of the radiography data set after rotation are X2, Y2, Z2, N for the normal. It can be seen from the figure that the Euler angle of the rotated contrast data set is (α, β, γ).
並且,基於此尤拉角可以設定一組對應的旋轉變換矩陣M(α,β,γ)。前述旋轉變換矩陣M(α,β,γ)可用來變換各切片影像的3D空間位置,以模擬造影資料集的旋轉。 And, based on the Euler angle, a set of corresponding rotation transformation matrices M(α, β, γ) can be set. The aforementioned rotation transformation matrix M(α, β, γ) can be used to transform the 3D spatial position of each slice image to simulate the rotation of the imaging data set.
於一實施例中,本發明可透過下述方程式計算尤拉角(α,β,γ)的旋轉變換矩陣M(α,β,γ)。並且,將各切片影像的3D空間位置乘上旋轉變換矩陣M(α,β,γ)即可獲得旋轉後的切片影像的3D空間位置。 In one embodiment, the present invention can calculate the rotation transformation matrix M(α, β, γ) of the Euler angles (α, β, γ) through the following equation. Moreover, the 3D spatial position of the rotated slice image can be obtained by multiplying the 3D spatial position of each slice image by the rotation transformation matrix M(α, β, γ).
請參閱圖14與圖15,於一實施例中,本發明可透過下述方程式來計算投影,即旋轉後的二維影像82的各點像素值。
Please refer to FIG. 14 and FIG. 15 , in one embodiment, the present invention can calculate the projection through the following equation, that is, the pixel value of each point of the rotated two-
其中,[x',y',w']為二維影像82的投影位置的值;[u,v,w]為旋轉座標;
為各切片影像的像素值。
Wherein, [ x', y', w' ] are the values of the projected position of the two-
復請參閱圖8,接著執行步驟S41:處理模組20透過組合模組521基於多個投影位置的值生成投影面的二維影像(如圖14-17所示)。
Referring again to FIG. 8 , step S41 is then executed: the
於一實施例中,當多個切片像素被投影至二維影像的相同的投影位置時,本發明可對這些切片像素執行聯集運算,如設定這些切片像素的最大值、最小值或平均值作為此投影位置的最終值。 In one embodiment, when multiple slice pixels are projected to the same projection position of the 2D image, the present invention can perform union operation on these slice pixels, such as setting the maximum value, minimum value or average value of these slice pixels as the final value for this projected position.
值得一提的是,本發明設定這些切片像素的最大值、最小值或平均值(可視目標物件的性質來加以選擇)作為此投影位置的最終值,是為了使生成的二維影像具有最佳的透視效果,以利於辨識訓練。 It is worth mentioning that the present invention sets the maximum value, minimum value or average value (selected depending on the nature of the target object) of these slice pixels as the final value of the projection position, in order to make the generated two-dimensional image have the best The perspective effect is good for recognition training.
更進一步地,當累加這些切片像素的值作為此投影位置的最終值,最終值有極高的機率為像素範圍的最大值(以256色灰階為例,為255),而失去透視效果,因此,本發明不採用累加的運算方式。 Furthermore, when adding the values of these slice pixels as the final value of the projection position, the final value has a very high probability of being the maximum value of the pixel range (take 256-color grayscale as an example, it is 255), and loses the perspective effect. Therefore, the present invention does not use an accumulative calculation method.
於壓合處理中,還可執行步驟S42:處理模組20透過過濾模組53可採用以下兩種方式的至少其中之一來排除顯示範圍外的像素值,以減少雜訊。
During the pressing process, step S42 can also be executed: the
(1)於步驟S40執行過程中,對於各切片影像,排除不符顯示範圍的切片像素;(2)於步驟S41執行過程中,對於二維影像,排除不符顯示範圍的投影位置的值。 (1) During the execution of step S40, for each slice image, exclude the slice pixels that do not match the display range; (2) During the execution of step S41, for the two-dimensional image, exclude the value of the projection position that does not match the display range.
藉此,本發明可有效排除不感興趣的影像或是雜訊。 Therefore, the present invention can effectively eliminate uninteresting images or noises.
請參閱圖2,於一實施例中,生成系統2可連接造影設備30,並自造影設備30接收對物件執行斷層掃描所生成的造影資料集。
Please refer to FIG. 2 , in one embodiment, the
於一實施例中,本發明可透過下述方式來於造影設備30的斷層掃描造影過程中即時識別目標物件的位置與範圍,並基於此位置與範圍執行掃描,來獲得僅包括目標物件的造影資料集。
In one embodiment, the present invention can identify the position and range of the target object in real time during the tomography imaging process of the
具體而言,當目標物件經過造影設備30的初步掃描,例如生成至少兩張不同掃描角度的二維基準影像。
Specifically, when the target object is initially scanned by the
接著,將基準影像經由AI人工智慧演算法進行判讀後可得到基準影像中關鍵物體(目標物件)的關鍵位置與範圍,前述位置與範圍可以包圍盒(bounding box)呈現。 Then, the key position and range of the key object (target object) in the reference image can be obtained after interpreting the reference image through an AI algorithm, and the aforementioned position and range can be presented in a bounding box.
接著,造影設備30可對此關鍵位置與範圍開始進行斷層掃描造影,來得到目標物件的切片影像序列檔案(造影資料集)。前述切片影像序列檔案是根據目標物件的關鍵位置的起始點為編號0開始掃描至物體關鍵位置結束點N。如造影資料集可包括編號為C_0000.raw~C_NNNN.raw的多張連續切片影像。
Then, the
於一實施例中,生成系統2可連接應用端31(如遠端電腦、伺服器或雲端服務平台雲端服務平台,如Amazon Web Service、Google Cloud Platform或Microsoft Azure等),並將所生成的多視角影像提供至應用端31進行後續應用。
In one embodiment, the
請參閱圖3,於一實施例中,應用端31可包括顯示模組40,如顯示器、觸控螢幕、或投影模組等。顯示模組40可用來顯示所取得的多視角二維影像。
Please refer to FIG. 3 , in an embodiment, the
於一實施例中,應用端31可包括訓練模組41、機器學習模型42與辨識模組43。訓練模組41被配置來透過多視角影像訓練機器學習模型42。辨識模組43被配置來使用機器學習模型42執行物件辨識。
In one embodiment, the
訓練機器學習模型42可例如透過卷積神經網路(CNN)、深度學習(Deep Learning)等演算法實現,但不加以限定。
The training of the
本發明所生成的多視角影像即是用來建立並訓練機器學習模型42,以提升機器學習模型42對於目標物件的辨識精確度與速度。
The multi-view image generated by the present invention is used to establish and train the
與模組50-53、520-521相似的是,訓練模組41、機器學習模型42與辨識模組43可以為軟體模組、硬體模組或軟硬體模組混搭。
Similar to the modules 50-53, 520-521, the
請參閱圖4,於機器學習模型42訓練完成後,當將檢測影像(如行李X光機所拍攝的X光影像)輸入至機器學習模型42後,機器學習模型42可自動分析檢測影像,並對影像中物件進行辨識分類,來辨識影像中各物件的物件資訊(如物件名稱或類型,如***、刀片等)。
Please refer to FIG. 4, after the training of the
請一併參閱圖9,接著說明本發明如何使用多視角影像訓練機器學習模型42。本實施例的生成方式更包括用來執行訓練的以下步驟。
Please also refer to FIG. 9 , and then describe how the present invention uses multi-view images to train the
步驟S50:應用端31透過訓練模組41設定此組多視角影像(即分別對應同一目標物件的不同視角的多張二維影像)的物件資訊,即設定目標物件的物件資訊。物件資訊可包括但不限於物件名稱、類別、特徵、安全性等。
Step S50: The
於一實施例中,用戶可透過應用端31的人機介面(圖未標示)輸入物件資訊。
In one embodiment, the user can input object information through the man-machine interface (not shown) of the
步驟S51:應用端31透過訓練模組41載入生成系統2所生成的多視角影像。
Step S51: The
步驟S52:應用端31透過訓練模組41以所設定的物件資訊及多視角影像對機器學習模型42進行訓練,以提升機器學習模型42對於目標物件的辨識成功機率與辨識速度。
Step S52: The
於一實施例中,機器學習模型42可對多視角的二維影像分別擷取影像特徵,以取得此目標物件的各視角的影像特徵或分類規則,來做為後續比對辨識使用。
In one embodiment, the
於一實施例中,藉由將多個物件的多視角影像輸入至機器學習模型42進行訓練,機器學習模型42可以具備辨識多個物件的能力,且可以提升辨識這些物件的準確率與速度。
In one embodiment, by inputting multi-view images of multiple objects into the
值得一提的是,相較於與輸入3D模型進行訓練,本發明以2D的多視角影像作為訓練輸入,可大幅減少所需的資料量與訓練/辨識所需的時間。 It is worth mentioning that, compared with training with input 3D models, the present invention uses 2D multi-view images as training input, which can greatly reduce the amount of data required and the time required for training/recognition.
並且,本發明以2D影像作為訓練/辨識人工智慧模型的輸入可更佳靈活的調整訓練參數,而可避免分類器過擬合(overfitting)、進而提升辨識準確度。 Moreover, the present invention uses 2D images as the input of the training/recognition artificial intelligence model to adjust the training parameters more flexibly, thereby avoiding overfitting of the classifier, thereby improving the recognition accuracy.
請參閱圖10,接著說明本發明如何使用訓練完成的機器學習模型42執行物件辨識。本實施例的生成方式更包括用來執行物件辨識的以下步驟。
Please refer to FIG. 10 , and then describe how the present invention uses the trained
步驟S60:應用端31透過造影設備30掃描待檢測物件以取得待檢測物件的可透視的二維檢測影像,以行李檢查為例,可利用X光機掃描行李箱以取得X光檢測影像。
Step S60: The
步驟S61:應用端31透過辨識模組43使用機器學習模型42來對檢測影像執行物件辨識處理,來辨識檢測影像中的各物件的物件資訊,如辨識行李箱中的物品為何。
Step S61: The
步驟S62:應用端31透過辨識模組43輸出辨識結果,如輸出辨識到的物件的物件資訊,或者於辨識到的物件包括違禁品時輸出警示等,不加以限定。
Step S62: The
藉此,透過本發明所訓練的機器學習模型42具有極佳的辨識能力,而可以執行快準確的物件辨識。
Therefore, the
以上所述僅為本發明之較佳具體實例,非因此即侷限本發明之申請專利範圍,故舉凡運用本發明內容所為之等效變化,均同理皆包含於本發明之範圍內,合予陳明。 The above descriptions are only preferred specific examples of the present invention, and are not intended to limit the patent scope of the present invention. Therefore, all equivalent changes made by using the content of the present invention are all included in the scope of the present invention in the same way. Chen Ming.
S10-S13:第一生成步驟 S10-S13: first generation step
Claims (18)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/499,360 US20220114773A1 (en) | 2020-10-14 | 2021-10-12 | Generation system and generation method for perspective image |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202063091857P | 2020-10-14 | 2020-10-14 | |
US63/091,857 | 2020-10-14 |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202215374A TW202215374A (en) | 2022-04-16 |
TWI779696B true TWI779696B (en) | 2022-10-01 |
Family
ID=81308989
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109142187A TWI770699B (en) | 2020-10-14 | 2020-12-01 | System of automatically generating training images and method thereof |
TW110124351A TWI779696B (en) | 2020-10-14 | 2021-07-02 | Generation system and generation method for perspective images |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109142187A TWI770699B (en) | 2020-10-14 | 2020-12-01 | System of automatically generating training images and method thereof |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114429569A (en) |
TW (2) | TWI770699B (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW201827014A (en) * | 2017-01-20 | 2018-08-01 | 台達電子工業股份有限公司 | Imaging Method For Computer Tomographic System |
US20190236380A1 (en) * | 2016-10-06 | 2019-08-01 | Advanced Data Controls Corp. | Image generation system, program and method, and simulation system, program and method |
CN110335344A (en) * | 2019-06-20 | 2019-10-15 | 中国科学院自动化研究所 | Three-dimensional rebuilding method based on 2D-3D attention mechanism neural network model |
CN111373448A (en) * | 2017-09-22 | 2020-07-03 | 尼维医疗公司 | Image reconstruction using machine learning regularizer |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107067020B (en) * | 2016-12-30 | 2019-11-15 | 腾讯科技(上海)有限公司 | Image identification method and device |
TW202006608A (en) * | 2018-07-02 | 2020-02-01 | 由田新技股份有限公司 | Recursive training method and detection system for deep learning system |
CN114577812A (en) * | 2018-08-15 | 2022-06-03 | 心鉴智控(深圳)科技有限公司 | Method and system for training efficient quality inspection model based on ultra-small sample |
-
2020
- 2020-12-01 TW TW109142187A patent/TWI770699B/en active
- 2020-12-01 CN CN202011389588.6A patent/CN114429569A/en active Pending
-
2021
- 2021-07-02 TW TW110124351A patent/TWI779696B/en active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190236380A1 (en) * | 2016-10-06 | 2019-08-01 | Advanced Data Controls Corp. | Image generation system, program and method, and simulation system, program and method |
TW201827014A (en) * | 2017-01-20 | 2018-08-01 | 台達電子工業股份有限公司 | Imaging Method For Computer Tomographic System |
CN111373448A (en) * | 2017-09-22 | 2020-07-03 | 尼维医疗公司 | Image reconstruction using machine learning regularizer |
CN110335344A (en) * | 2019-06-20 | 2019-10-15 | 中国科学院自动化研究所 | Three-dimensional rebuilding method based on 2D-3D attention mechanism neural network model |
Also Published As
Publication number | Publication date |
---|---|
TW202215291A (en) | 2022-04-16 |
TW202215374A (en) | 2022-04-16 |
TWI770699B (en) | 2022-07-11 |
CN114429569A (en) | 2022-05-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11747898B2 (en) | Method and apparatus with gaze estimation | |
CN107403463B (en) | Human body representation with non-rigid parts in an imaging system | |
CN109003253A (en) | Neural network point cloud generates system | |
CN110728196B (en) | Face recognition method and device and terminal equipment | |
JP2019076699A (en) | Nodule detection with false positive reduction | |
US20200111234A1 (en) | Dual-view angle image calibration method and apparatus, storage medium and electronic device | |
CN111667464A (en) | Dangerous goods three-dimensional image detection method and device, computer equipment and storage medium | |
US11276490B2 (en) | Method and apparatus for classification of lesion based on learning data applying one or more augmentation methods in lesion information augmented patch of medical image | |
US20200057778A1 (en) | Depth image pose search with a bootstrapped-created database | |
US11170246B2 (en) | Recognition processing device, recognition processing method, and program | |
Han et al. | Automated monitoring of operation-level construction progress using 4D BIM and daily site photologs | |
JP6824845B2 (en) | Image processing systems, equipment, methods and programs | |
CN112802208B (en) | Three-dimensional visualization method and device in terminal building | |
Sengan et al. | Cost-effective and efficient 3D human model creation and re-identification application for human digital twins | |
JP2024515448A (en) | Systems and methods for dynamic identification of surgical trays and items contained thereon - Patents.com | |
US11816854B2 (en) | Image processing apparatus and image processing method | |
Zhao et al. | Region-based saliency estimation for 3D shape analysis and understanding | |
KR20240025683A (en) | Security inspection CT target identification method and device | |
Haiying et al. | False-positive reduction of pulmonary nodule detection based on deformable convolutional neural networks | |
US11816857B2 (en) | Methods and apparatus for generating point cloud histograms | |
JP2012123631A (en) | Attention area detection method, attention area detection device, and program | |
CN112883920A (en) | Point cloud deep learning-based three-dimensional face scanning feature point detection method and device | |
TWI779696B (en) | Generation system and generation method for perspective images | |
Bowyer et al. | Overview of work in empirical evaluation of computer vision algorithms | |
US20220114773A1 (en) | Generation system and generation method for perspective image |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
GD4A | Issue of patent certificate for granted invention patent |