TWI720640B

TWI720640B - A method and a device for displaying origin of workpiece coordinate system and tool path on a workpiece by augmented reality

Info

Publication number: TWI720640B
Application number: TW108136111A
Authority: TW
Inventors: 李維楨; 陳立昌; 韋經智; 項天瑞
Original assignee: 國立臺灣科技大學
Priority date: 2019-10-04
Filing date: 2019-10-04
Publication date: 2021-03-01
Also published as: TW202115562A

Abstract

Present invention is related to a method and a device for displaying origin of workpiece coordinate system and tool path on a workpiece by augmented reality. The said method can be achieved by firstly scanning identification markers on a workpiece by an augmented reality device for obtaining a work order number. Each work order number is corresponded to at least one tool path file. A server corresponded to each work order number will then obtain a matching tool path file and a relative distance and position between origin of workpiece coordinate system and the identification marker. The identification marker serves as a reference of position and scale. Images of the origin of workpiece coordinate system and the tool path are superimposed on the image of the workpiece taken by the device for reference by the worker. Therefore, the present invention can avoid errors from the workers miss setting and reduce the risk of defective products and machine crash.

Description

以擴增實境方式於加工件上顯示其加工程式原點及加工路徑的方法與裝置 Display the origin and processing of the processing program on the processed part in an augmented reality method Path method and device

本發明涉及一種擴增實境的應用方法與裝置，尤其涉及一種以擴增實境方式於加工件上顯示其加工程式原點及加工路徑的方法與裝置。 The present invention relates to an application method and device of augmented reality, in particular to a method and device for displaying the origin and processing path of a processing program on a processed part in an augmented reality manner.

目前使用CAM(computer-aided manufacturing)軟體規劃CNC(電腦數值控制)工具機加工路徑以生成加工程式的CAM工程師，和實際操作CNC工具機加工的工程師或技術員，往往不是同一人。CAM工程師在規劃加工路徑時，會設定加工程式原點，也就是工件坐標原點，也就是加工路徑上各點所參考的座標原點。 At present, the CAM engineer who uses CAM (computer-aided manufacturing) software to plan CNC (computer numerical control) tool machining paths to generate machining programs is often not the same person as the engineer or technician who actually operates CNC tool machining. When CAM engineers plan the machining path, they will set the origin of the machining program, that is, the origin of the workpiece coordinates, which is the origin of the coordinates referenced by each point on the machining path.

實際加工時，CNC加工機操作者(加工者)將加工機的機械坐標原點和工件坐標原點之間的關係，輸入到加工機的控制器中，以使加工機能正確地加工出所要的產品。如果機械坐標原點和工件坐標原點之間的關係輸入錯誤，則加工出來的產品也會是錯誤的，甚至可能造成機臺撞機損壞。因此，確認加工程式原點(工件座標原點)在加工件的位置是很重要的。 In actual processing, the operator of the CNC processing machine (processor) inputs the relationship between the origin of the machine's mechanical coordinates and the origin of the workpiece coordinates into the controller of the processing machine, so that the processing machine can correctly process the desired product. If the relationship between the origin of the machine coordinate and the origin of the workpiece coordinate is entered incorrectly, the processed product will also be wrong, and it may even cause the machine to collide and damage the machine. Therefore, it is important to confirm the position of the machining program origin (workpiece coordinate origin) on the workpiece.

然而，上述加工程式原點(工件座標原點)的資訊無法直接從加工者拿到的加工程式的文字檔案中直接獲得，因此加工者必須去開啟CAM軟體查看程式原點的位置，或是詢問規劃加工路徑的CAM工程師有關加工程式原點的位置，在機械加工作業上，是很麻煩且浪費時間的。 However, the information of the origin of the above-mentioned processing program (workpiece coordinate origin) cannot be directly obtained from the text file of the processing program obtained by the processor, so the processor must open the CAM software Checking the location of the program origin, or asking the CAM engineer planning the processing path about the location of the processing program origin, is very troublesome and time-consuming in machining operations.

由於現有的CNC加工機由加工程式無法直接看出加工程式原點的位置，因此造成需要開啟CAM軟體或向知情者詢問的加工麻煩。為此，本發明透過辨識工件表面的標記物，經由資料庫得到工件的加工程式原點及加工路徑，並以擴增實境的方式將此資訊顯示於待加工的工件上，供加工者參考，達到可避免加工者設定錯誤，減少不良品的產生及機臺撞機的可能。 Because the existing CNC processing machine cannot directly see the position of the origin of the processing program from the processing program, it causes the processing troubles of opening the CAM software or asking the insider. To this end, the present invention obtains the origin and processing path of the processing program of the workpiece through the database by identifying the markers on the surface of the workpiece, and displays this information on the workpiece to be processed in the form of augmented reality for the reference of the processor , It can avoid the setting error of the processor, reduce the production of defective products and the possibility of machine collision.

為達到上述創作目的，本發明提供一種以擴增實境方式於加工件上顯示其加工程式原點及加工路徑的方法，應用於一裝置，該裝置包括一通訊模組、一相機模組，以及一螢幕，該方法包括：控制該相機模組掃描一工件表面的標記物，辨識該標記物取得一工單編號，該相機模組持續拍攝該工件，在該螢幕上顯示一工件影像；該通訊模組與該擴增實境程式介面伺服器溝通，取得與該工單編號對應的加工程式原點與該標記物的相對距離與位置，將該相機模組拍攝到的該標記物的影像作為位置及比例尺的基準，在該螢幕的該工件影像上顯示該加工程式原點；以及該通訊模組與該擴增實境程式介面伺服器溝通，取得與該工單編號對應的一個以上的加工路徑檔案在一檔案傳輸協定伺服器的位置與數量，從該檔案傳輸協定伺服器下載與該工單編號對應的一個以上的所述加工路徑檔案，以該加工程式原點為位置的基準，該標記物的影像為比例尺的基準，逐行讀取其中一加工路徑檔案，繪製一加工路徑在該工件影像上。 In order to achieve the above-mentioned creative purpose, the present invention provides a method for displaying the origin and processing path of the processing program on a processed part in an augmented reality method, which is applied to a device that includes a communication module and a camera module, And a screen, the method includes: controlling the camera module to scan a marker on the surface of a workpiece, identifying the marker to obtain a work order number, the camera module continuously photographing the workpiece, and displaying an image of the workpiece on the screen; the The communication module communicates with the augmented reality program interface server to obtain the relative distance and position between the origin of the processing program corresponding to the work order number and the marker, and the image of the marker captured by the camera module As a reference for the position and scale, the origin of the machining program is displayed on the workpiece image on the screen; and the communication module communicates with the augmented reality program interface server to obtain more than one corresponding to the work order number The location and quantity of the processing path file on a file transfer protocol server, from which one or more of the processing path files corresponding to the work order number are downloaded from the file transfer protocol server, taking the origin of the processing program as the position reference, The image of the marker is the reference of the scale, one of the processing path files is read line by line, and a processing path is drawn on the workpiece image.

進一步，本發明該標記物為一維條碼或二維條碼；在該螢幕的該工件影像上顯示該加工程式原點時，還顯示了X軸、Y軸。 Further, in the present invention, the marker is a one-dimensional bar code or a two-dimensional bar code; when the origin of the processing program is displayed on the workpiece image on the screen, the X axis and the Y axis are also displayed.

更進一步，本發明對應該工單編號的一個以上的所述加工路徑檔案具有不同編號，在該螢幕顯示一按鈕，通過按壓該按鈕，切換逐行讀取另一加工路徑檔案，以該加工程式原點為位置為位置的基準，該標記物的影像為比例尺的基準，繪製另一加工路徑在該工件影像上。 Furthermore, in the present invention, more than one processing path file corresponding to the work order number has different numbers, a button is displayed on the screen, and by pressing the button, another processing path file is switched line by line, and the processing program The origin is the position as the reference of the position, the image of the marker is the reference of the scale, and another processing path is drawn on the image of the workpiece.

較佳的，本發明在控制該相機模組掃描一工件表面的標記物前，進行一前置處理，將多個所述的工單編號分別生成所述的標記物，將所述標記物雷射雕刻至對應的所述工件上。 Preferably, before controlling the camera module to scan the marker on the surface of a workpiece, the present invention performs a pre-processing, generates the markers from a plurality of the work order numbers, and mines the markers. Shoot engraving onto the corresponding workpiece.

較佳的，本發明該通訊模組與該擴增實境程式介面伺服器溝通，取得一個以上分別與所述各加工路徑檔案對應的工件模型檔案；對應逐行讀取的該加工路徑檔案，讀取與該加工路徑檔案對應的該工件模型檔案，以該原點座標為位置的基準，該標記物的影像為比例尺的基準，繪製一透明的工件模型影像重疊在該工件影像上。 Preferably, the communication module of the present invention communicates with the augmented reality programming interface server to obtain one or more workpiece model files corresponding to each of the processing path files; corresponding to the processing path files read line by line, Read the workpiece model file corresponding to the processing path file, use the origin coordinates as the position reference, and the image of the marker as the scale reference, and draw a transparent workpiece model image to overlap the workpiece image.

本發明還提供一種以擴增實境方式於加工件上顯示其加工程式原點及加工路徑的裝置，包括：一螢幕；一相機模組，用以掃描、辨識一工件表面的標記物取得一工單編號，該相機模組持續拍攝該工件，在該螢幕上顯示一工件影像；一通訊模組，用以與一擴增實境程式介面伺服器溝通，取得與該工單編號對應的加工程式原點與該標記物的相對距離與位置，以及與該工單編號對應的一個以上的加工路徑檔案在一檔案傳輸協定伺服器的位置與數量，該通訊模組從該檔案傳輸協定伺服器下載與該工單編號對應的一個以上的所述加工路徑檔案；一記憶體，儲存該通訊模組取得及下載的資料以及一擴增實境程式；以及一處理器，分別電連接、控制該螢幕、該相機模組、該通訊模組以及該記憶體，該處理器執行該擴增實境程式，將該相機模組拍攝到的該標記物的影像作為位置及比例尺的基準，配合上述加工程式原點與該標記物的相對距離與位置的資訊，在該螢幕的該工件影像上顯示該加工程式原點，並且以該加工程式原點為位置的基準，該標記物的影像為比例尺的基準，讀取該加工路徑檔案而繪製一加工路徑在該工件影像上。 The present invention also provides a device for displaying the origin of the processing program and the processing path on a processed part in an augmented reality manner, which includes: a screen; a camera module for scanning and identifying markers on the surface of a workpiece to obtain a Work order number. The camera module continuously shoots the workpiece and displays an image of the workpiece on the screen. A communication module is used to communicate with an augmented reality programming interface server to obtain the processing corresponding to the work order number. The relative distance and position of the program origin and the marker, and the position and quantity of one or more processing path files corresponding to the work order number on a file transfer protocol server from which the communication module transfers Download more than one processing path file corresponding to the work order number; A memory for storing data obtained and downloaded by the communication module and an augmented reality program; and a processor for electrically connecting and controlling the screen, the camera module, the communication module, and the memory, respectively, The processor executes the augmented reality program, uses the image of the marker captured by the camera module as a reference for the position and scale, and coordinates the relative distance and position information between the origin of the processing program and the marker. The workpiece image on the screen displays the origin of the machining program, and the origin of the machining program is the position reference, the image of the marker is the scale reference, the processing path file is read and a processing path is drawn on the workpiece image on.

進一步，本發明該標記物為一維條碼或二維條碼，在該螢幕的該工件影像上顯示一X軸、一Y軸。 Further, in the present invention, the marker is a one-dimensional bar code or a two-dimensional bar code, and an X axis and a Y axis are displayed on the image of the workpiece on the screen.

更進一步，本發明對應該工單編號的一個以上的所述加工路徑檔案具有不同編號，在該螢幕顯示一按鈕，通過按壓該按鈕，切換逐行讀取另一加工路徑檔案，以該加工程式原點為位置的基準，該標記物的影像為比例尺的基準，繪製另一加工路徑在該工件影像上。 Furthermore, in the present invention, more than one processing path file corresponding to the work order number has different numbers, a button is displayed on the screen, and by pressing the button, another processing path file is switched line by line, and the processing program The origin is the reference of the position, the image of the marker is the reference of the scale, and another processing path is drawn on the image of the workpiece.

較佳的，本發明該通訊模組用以與該擴增實境程式介面伺服器溝通取得一個以上分別與所述各加工路徑檔案對應的工件模型檔案，所述一個以上的工件模型檔案儲存於該記憶體；該處理器執行該擴增實境程式，對應逐行讀取的該加工路徑檔案，讀取與該加工路徑檔案對應的該工件模型檔案，以該原點座標為位置的基準，該標記物的影像為比例尺的基準，繪製一透明的工件模型影像重疊在該工件影像上。 Preferably, the communication module of the present invention is used to communicate with the augmented reality programming interface server to obtain one or more workpiece model files corresponding to each of the processing path files, and the one or more workpiece model files are stored in The memory; the processor executes the augmented reality program, corresponds to the processing path file read line by line, reads the workpiece model file corresponding to the processing path file, and uses the origin coordinates as the position reference, The image of the marker is the reference of the scale, and a transparent image of the workpiece model is drawn and superimposed on the image of the workpiece.

使用本發明的方法或裝置，能夠將對應工單編號的加工程式原點與加工路徑以擴增實境的方式顯示在擴增實境裝置的螢幕，同時顯示真實環境與虛擬的加工資訊，協助CNC工具機的操作員確認欲加工的工件的該加工程式原點的位置或加工路徑是否正確，達到可避免加工者設定錯誤，減少不良品的產生及機臺撞機的功效。 Using the method or device of the present invention, the origin of the processing program and the processing path corresponding to the work order number can be displayed on the screen of the augmented reality device in the form of augmented reality, and the real environment and virtual processing information can be displayed at the same time. The operator of the CNC machine tool confirms the processing program of the workpiece to be processed Whether the position of the origin or the processing path is correct, so as to prevent the processor from setting errors, reduce the generation of defective products and the effect of machine collision.

10:裝置 10: device

11:處理器 11: processor

12:螢幕 12: screen

13:相機模組 13: Camera module

14:通訊模組 14: Communication module

15:記憶體 15: Memory

20:製造執行系統 20: Manufacturing Execution System

21:擴增實境程式介面伺服器 21: Augmented Reality Programming Interface Server

22:檔案傳輸協定伺服器 22: File Transfer Protocol Server

A:工單編號 A: Work order number

B:加工路徑檔案 B: Processing path file

B1:工件模型檔案 B1: Workpiece model file

C:標記物 C: Marker

D:擴增實境程式 D: Augmented reality program

D1:按鈕 D1: Button

E:工件 E: Workpiece

E1:工件影像 E1: Workpiece image

E2:加工程式原點 E2: Origin of processing program

E3:加工路徑 E3: Processing path

E4:工件模型影像 E4: Workpiece model image

S01-S03:步驟 S01-S03: steps

S031-S035:步驟 S031-S035: Step

圖1是本發明較佳實施例的方法步驟流程圖。 Fig. 1 is a flowchart of the method steps of a preferred embodiment of the present invention.

圖2是本發明較佳實施例的建置系統與裝置的構造方塊圖。 2 is a block diagram of the construction system and device of the preferred embodiment of the present invention.

圖3是本發明較佳實施例的前置處理示意圖。 Fig. 3 is a schematic diagram of pre-processing of a preferred embodiment of the present invention.

圖4是本發明較佳實施例的掃描與在擴增實境顯示加工程式原點的示意圖。 4 is a schematic diagram of scanning and displaying the origin of the processing program in the augmented reality of the preferred embodiment of the present invention.

圖5是本發明較佳實施例的加工程式原點與標記物的相對距離與位置圖。 5 is a diagram of the relative distance and position of the origin of the processing program and the marker in the preferred embodiment of the present invention.

圖6是本發明較佳實施例的在擴增實境顯示加工路徑與切換加工路徑的示意圖。 FIG. 6 is a schematic diagram showing the processing path and switching the processing path in the augmented reality according to the preferred embodiment of the present invention.

圖7是本發明較佳實施例的在擴增實境顯示工件模型影像的示意圖。 FIG. 7 is a schematic diagram of displaying an image of a workpiece model in an augmented reality according to a preferred embodiment of the present invention.

為能詳細瞭解本發明的技術特徵及實用功效，並可依照說明書的內容來實施，進一步以如圖式所示的較佳實施例，詳細說明如下。 In order to understand the technical features and practical effects of the present invention in detail, and implement it in accordance with the content of the specification, the preferred embodiments shown in the drawings are further described in detail as follows.

本發明提供一種以擴增實境方式於加工件上顯示其加工程式原點及加工路徑的方法，請參看圖1至圖6所示的本發明較佳實施例，是應用在一裝置10，該裝置10是擴增實境(AR，Augmented Reality)裝置，可以是光學投影系統、行動裝置、頭戴式顯示器、抬頭顯示器，在本較佳實施例中該裝置10是行動裝置的智慧手機，請參看圖1所示的步驟流程圖，該方法包括前置處理與應用模式的步驟，並在前置處理前進行建置環境的步驟，上述步驟說明如下： The present invention provides a method for displaying the origin of the processing program and the processing path on a processed part in an augmented reality manner. Please refer to the preferred embodiment of the present invention shown in FIGS. 1 to 6, which is applied to a device 10, The device 10 is an augmented reality (AR, Augmented Reality) device, which can be an optical projection system, a mobile device, a head-mounted display, or a head-up display. In this preferred embodiment, the device 10 is a smart phone of a mobile device. Please refer to the step flow chart shown in Figure 1. The method includes the steps of pre-processing and application mode, and the steps of building the environment before the pre-processing. The above steps are described as follows:

(S01)建置環境：請參看圖2所示，設置一製造執行系統(MES，Manufacturing Execution System)20，該製造執行系統20分別與一擴增實境程式介面伺服器(API Server，Application Programming Interface Server)21以及一檔案傳輸協定伺服器(FTP Server，File Transfer Protocol Server)22溝通，藉此達到加工項目確認的實務應用。 (S01) Build environment: Please refer to FIG. 2 and set up a Manufacturing Execution System (MES, Manufacturing Execution System) 20, which is separately associated with an augmented reality program The interface server (API Server, Application Programming Interface Server) 21 communicates with a file transfer protocol server (FTP Server, File Transfer Protocol Server) 22, so as to achieve the practical application of processing project confirmation.

，如圖2至圖3所示，當該製造執行系統20收到產品製作下單(此單會給予一工單編號A)後，即會以自動或者人工的方式透過程序生成一個以上的CNC加工機的加工路徑，將該些加工路徑的加工路徑檔案B，以及分別對應該些加工路徑檔案B的一個以上的工件模型檔案B1(加工路徑B用以加工的工件外形的檔案)上傳至該檔案傳輸協定伺服器22對應該工單編號A的資料夾中。 As shown in Figures 2 to 3, when the manufacturing execution system 20 receives a product production order (this order will be given a work order number A), it will automatically or manually generate more than one CNC through the program The processing path of the processing machine, the processing path files B of these processing paths, and one or more workpiece model files B1 (files of the workpiece shape used for processing by the processing path B) corresponding to the processing path files B are uploaded to the The file transfer protocol server 22 is in the folder corresponding to the work order number A.

當該些加工路徑檔案B上傳成功後，紀錄該些加工路徑檔案B的儲存位置、數量，以及加工路徑的加工程式原點與標記物C的相對距離與位置的資訊至該擴增實境程式介面伺服器21，以備後續的擴增實境程式D使用，前述標記物C在本較佳實施例中是採用二維條碼(QR Code)，該標記物C周圍輪廓為基本幾何形狀，例如方形因此能作為比例尺的基準，該標記物C記載對應工序的工單編號A的資訊。 After the processing path files B are uploaded successfully, record the storage location and quantity of the processing path files B, as well as the relative distance and position information of the processing program origin of the processing path and the marker C to the augmented reality program The interface server 21 is used for the subsequent augmented reality program D. In the preferred embodiment, the aforementioned marker C adopts a QR Code, and the surrounding outline of the marker C is a basic geometric shape, for example The square can therefore be used as a reference for the scale, and the marker C records the information of the work order number A of the corresponding process.

為了同時讓該裝置10能夠在執行該擴增實境程式D時，能夠顯示真實環境和虛擬加工資訊，如圖2所示，本較佳實施例中該裝置10採用蘋果公司生產的iPhone智慧手機，包括一處理器11，以及分別與該處理器11電連接的一螢幕12、一相機模組13、一通訊模組14以及一記憶體15，該裝置10支援作業系統iOS 11.3以上且該處理器11至少為A9處理器，本較佳實施例的擴增實境程式D是由蘋果公司生產的ARKit擴增實境軟體開發平台所編寫，iOS 11.3作業系統支援ARKit的2D圖像辨識功能，能用於追蹤該標記物C，而A9處理器則是執行由ARKit編寫出的該擴增實境程式D的最低硬體要求，該擴增實境程式D儲存於該記憶體15供該處理器11執行。 In order to simultaneously enable the device 10 to display real environment and virtual processing information when executing the augmented reality program D, as shown in FIG. 2, the device 10 in this preferred embodiment uses an iPhone smartphone produced by Apple. , Including a processor 11, and a screen 12, a camera module 13, a communication module 14 and a memory 15 electrically connected to the processor 11, the device 10 supports the operating system iOS 11.3 or higher and the processing The device 11 is at least the A9 processor. The augmented reality program D of this preferred embodiment is written by the ARKit augmented reality software development platform produced by Apple. The iOS 11.3 operating system supports ARKit’s 2D image recognition function. It can be used to track the marker C, and the A9 processor executes the minimum hardware requirements of the augmented reality program D written by ARKit, and the augmented reality program D is stored in the memory 15 for the processing器11Execute.

(S02)前置處理：請配合參看圖2至圖3所示，當收到產品製作下單後，會進入準備材料和模具階段。在這個階段會將該工單編號A生成所述的標記物C，在本較佳實施例中為二維條碼(QR Code)，再使用雷射雕刻機將該標記物C雷射雕刻到欲加工的工件E上；在該工件E表面雕刻該標記物C的位置以及大小，是依照前述加工程式原點與標記物C的相對距離與位置的設計，使該標記物C能成為對應工單編號A的加工路徑的加工程式原點相較於該標記物C的位置及比例尺的基準。 (S02) Pre-processing: Please refer to Figure 2 to Figure 3. After receiving the product production order, it will enter the stage of preparing materials and molds. At this stage, the work order number A will be used to generate the aforementioned marker C, which is a QR Code in this preferred embodiment, and a laser engraving machine will be used to laser engrave the marker C to the desired On the processed workpiece E; the position and size of the marker C engraved on the surface of the workpiece E are designed in accordance with the relative distance and position of the origin of the processing program and the marker C, so that the marker C can become a corresponding work order The origin of the processing program of the processing path of the number A is compared with the position of the marker C and the reference of the scale.

(S03)應用模式：包括初始化擴增實境環境、掃描標記物確認工單編號、取得加工程式原點顯示在工件上、取得加工路徑檔案顯示在工件上，以及點擊按鈕切換顯示的路徑等步驟，其中： (S03) Application mode: including the steps of initializing the augmented reality environment, scanning the marker to confirm the work order number, obtaining the origin of the processing program and displaying it on the workpiece, obtaining the processing path file and displaying it on the workpiece, and clicking the button to switch the displayed path. ,among them:

(S031)初始化擴增實境環境：在產品製作開始加工前，該裝置10會開啟加工項目確認系統，也就是該處理器11執行儲存在該記憶體15內的擴增實境程式D。 (S031) Initialize the augmented reality environment: before the product is produced and processed, the device 10 will turn on the processing item confirmation system, that is, the processor 11 executes the augmented reality program D stored in the memory 15.

(S032)掃描標記物確認工單編號：如圖2至圖4所示，利用該相機模組13掃描該工件E表面的該標記物C(QR Code)，辨識取得對應的該工單編號A。辨識該標記物C的過程中盡量讓裝置10盡量與該工件E平行(由於QR Code在周圍具有三個位置探測圖形，傾斜角度時亦能被辨識讀取，但以正面拍攝讀取效果最佳)，並讓該標記物C可以完整顯示在該相機模組13拍攝的視角範圍內以成功辨識標記物C；當成功辨識該標記物C取得該工單編號A後，該相機模組13持續拍攝該工件E，在該螢幕12上顯示拍攝該工件E的工件影像E1，並且該處理器11執行ARKit 2D圖像辨識的功能，在辨識到該標記物C的影像位置時，持續追蹤、修正偏移的情況，更新該標記物C顯示在該螢幕12上的位置。 (S032) Scan the marker to confirm the work order number: as shown in Figures 2 to 4, the camera module 13 is used to scan the marker C (QR Code) on the surface of the workpiece E to identify and obtain the corresponding work order number A . In the process of recognizing the marker C, try to make the device 10 parallel to the workpiece E as much as possible (because the QR Code has three position detection patterns around it, it can be recognized and read even when it is tilted, but it is best to take a front view to read it. ), and allow the marker C to be fully displayed within the field of view taken by the camera module 13 to successfully identify the marker C; when the marker C is successfully identified and the work order number A is obtained, the camera module 13 continues The workpiece E is photographed, the workpiece image E1 of the workpiece E is displayed on the screen 12, and the processor 11 executes the ARKit 2D image recognition function. When the image position of the marker C is recognized, it continues to track and correct In case of deviation, the position of the marker C displayed on the screen 12 is updated.

(S033)取得加工程式原點顯示在工件上：該裝置10通過該通訊模組14與該擴增實境程式介面伺服器21溝通，取得與該工單編號A對應的加工程式原點E2與該標記物C的相對距離與位置，該加工程式原點E2與標記物C的相對距離與位置包含的資訊如圖5所示，將該相機模組13拍攝到的該標記物C的影像(經由ARKit 2D圖像辨識的功能辨識其位置與尺寸)作為位置及比例尺的基準，在該螢幕12的該工件影像E1上顯示該加工程式原點E2、X軸，以及Y軸(該X軸、Y軸可選擇不顯示)。 (S033) Obtain the origin of the processing program and display it on the workpiece: the device 10 communicates with the augmented reality program interface server 21 through the communication module 14 to obtain the processing program corresponding to the work order number A The relative distance and position of the origin E2 and the marker C, the relative distance and position of the origin E2 of the processing program and the marker C include information as shown in FIG. 5, the marker captured by the camera module 13 The image of object C (recognize its position and size by the ARKit 2D image recognition function) is used as a reference for position and scale. The workpiece image E1 of the screen 12 displays the origin E2 of the machining program, the X axis, and the Y axis (The X axis and Y axis can be selected not to be displayed).

由於該相機模組13拍攝該工件E的角度會產生變動，與擴增實境環境的初始三軸座標不一定一致，所以在該螢幕12的該工件影像E1上顯示該加工程式原點E2、X軸，以及Y軸時，要即時判斷三軸方向的改變，利用辨識出的該標記物C為已知座標的基準，配合加工程式原點與該標記物C的相對距離與位置的資訊進行位移，取得該加工程式原點E2的座標後，在該螢幕12上繪製、顯示該加工程式原點E2、X軸，以及Y軸。 Since the angle at which the camera module 13 photographs the workpiece E will change, and may not be consistent with the initial three-axis coordinates of the augmented reality environment, the workpiece image E1 on the screen 12 displays the origin E2 of the machining program. For X-axis and Y-axis, it is necessary to judge the changes of the three-axis directions in real time. Use the identified marker C as the reference of the known coordinates, and cooperate with the relative distance and position information of the origin of the processing program and the marker C. Displacement, after obtaining the coordinates of the origin E2 of the processing program, draw and display the origin E2, X axis, and Y axis of the processing program on the screen 12.

(S034)取得加工路徑檔案顯示在工件上：接著該通訊模組14與該擴增實境程式介面伺服器21溝通，取得與該工單編號A對應的一個以上的加工路徑檔案B在一檔案傳輸協定伺服器22的位置與數量，依此從該檔案傳輸協定伺服器22下載與該工單編號A對應的一個以上的所述加工路徑檔案B，以該加工程式原點E2為位置的基準，該標記物C受辨識的影像為比例尺的基準，逐行讀取其中一加工路徑檔案B，如圖6所示，根據讀取的相對位置、圓心和正逆時針旋轉來繪製直線和圓弧，繪製、顯示一加工路徑E3在該螢幕12的該工件影像E1上，讓該裝置10的使用者例如加工產品的CNC加工機的操作員確認欲加工的工件E、該加工程式原點E2的位置或加工路徑檔案B顯示的加工路徑是否正確。 (S034) Obtain the processing path file and display it on the workpiece: Then the communication module 14 communicates with the augmented reality programming interface server 21 to obtain one or more processing path files B corresponding to the work order number A in a file The location and quantity of the transmission protocol server 22, and the one or more processing path files B corresponding to the work order number A are downloaded from the file transmission protocol server 22, and the processing program origin E2 is taken as the position reference , The recognized image of the marker C is the reference of the scale, read one of the processing path files B line by line, as shown in Figure 6, draw straight lines and arcs according to the read relative position, center of circle, and forward and counterclockwise rotation. Draw and display a machining path E3 on the workpiece image E1 on the screen 12, allowing the user of the device 10, such as the operator of a CNC processing machine to process the product, to confirm the workpiece E to be processed and the position of the origin E2 of the machining program Or whether the processing path displayed in the processing path file B is correct.

上述在該螢幕上12繪製、顯示該加工路徑E3時，同樣需要隨著該相機模組13拍攝該工件E角度不同來即時判斷三軸方向的改變，利用該標記物C作為已知座標的基準來位移該加工路徑E3的座標的方式，與前述位移該加工程式原點E2座標的方式相同。並且上述由該擴增實境程式介面伺服器21取得的資訊，或由該檔案傳輸協定伺服器22下載的資訊，都儲存於該記憶體15來供該處理器11存取。 When the processing path E3 is drawn and displayed on the screen 12, it is also necessary to determine the change of the three-axis direction in real time as the camera module 13 photographs the workpiece E at different angles, and use the marker C as a reference for the known coordinates. The method of displacing the coordinates of the machining path E3 is the same as the manner of displacing the coordinates of the origin E2 of the machining program. And the above-mentioned information obtained by the augmented reality programming interface server 21 Information or information downloaded by the file transfer protocol server 22 are stored in the memory 15 for the processor 11 to access.

選擇性的，在從該檔案傳輸協定伺服器22下載與該工單編號A對應的一個以上的所述加工路徑檔案B時，可以下載對應各加工路徑檔案B的所述工件模型檔案B1，讀取與該加工路徑檔案B對應的該工件模型檔案B1，以該原點座標E2為位置的基準，該標記物C的影像為比例尺的基準，如圖7所示，繪製一透明的工件模型影像E4重疊在該工件影像E上，讓該裝置10的使用者例如加工產品的CNC加工機的操作員確認欲加工的工件E是否正確。 Optionally, when downloading more than one processing path file B corresponding to the work order number A from the file transfer protocol server 22, the workpiece model file B1 corresponding to each processing path file B may be downloaded, and read Take the workpiece model file B1 corresponding to the processing path file B, take the origin coordinate E2 as the position reference, and the image of the marker C as the scale reference, as shown in Figure 7, draw a transparent workpiece model image E4 is superimposed on the workpiece image E, so that the user of the device 10, such as the operator of a CNC processing machine that processes the product, can confirm whether the workpiece E to be processed is correct.

(S035)點擊按鈕切換顯示的路徑：如圖6所示，當該對應該工單編號A的加工路徑檔案B不只一個時，也就是產品製作的過程包括兩個以上加工路徑的工序時，各加工路徑檔案B具有不同編號，該擴增實境程式D在該螢幕12的下方例如右下角產生、顯示一按鈕D1，按壓該按鈕D1可以按照下載的編號進行切換，切換逐行讀取不同的，例如由原本的加工路徑檔案B切換為讀取另一加工路徑檔案B，同樣以該加工程式原點E2為位置的基準，該標記物C辨識出的影像為比例尺的基準，繪製另一加工路徑E3在該工件影像E1上。 (S035) Click the button to switch the displayed path: As shown in Figure 6, when there is more than one processing path file B corresponding to the work order number A, that is, when the product manufacturing process includes more than two processing path processes, each The processing path file B has different numbers. The augmented reality program D generates and displays a button D1 at the bottom of the screen 12, for example, in the lower right corner. Press the button D1 to switch according to the downloaded number, and switch to read different line by line. For example, switch from the original processing path file B to reading another processing path file B, and also use the origin E2 of the processing program as the position reference, and the image recognized by the marker C as the reference of the scale, draw another processing The path E3 is on the workpiece image E1.

本發明除前述較佳實施例，該標記物C是二維條碼以外，也可以將該標記物C設為一維條碼；該裝置10除了採用蘋果公司生產的iPhone智慧手機以外，也可以是安卓智慧手機，這時編寫該擴增實境程式D的平台改為美商科高國際有限公司(Google LLC)發表的ARCore擴增實境軟體開發平台，此平台開發的擴增實境程式D適用於搭載Android 7.0 Nougat版本以上作業系統的安卓智慧手機。 In addition to the aforementioned preferred embodiment of the present invention, the marker C is a two-dimensional bar code, and the marker C can also be a one-dimensional bar code; the device 10 can also be an Android smartphone in addition to the iPhone smart phone produced by Apple. Smartphones, the platform for writing the augmented reality program D was changed to the ARCore augmented reality software development platform published by Google LLC. The augmented reality program D developed by this platform is suitable for carrying Android smartphones with operating systems above Android 7.0 Nougat.

以上所述僅為本發明的較佳實施例而已，並非用以限定本發明主張的權利範圍，凡其它未脫離本發明所揭示的精神所完成的等效改變或修飾，均應包括在本發明的申請專利範圍內。 The foregoing descriptions are only preferred embodiments of the present invention, and are not intended to limit the scope of rights claimed by the present invention. All other equivalent changes or modifications completed without departing from the spirit disclosed by the present invention should be included in the present invention. Within the scope of patent application.

S01-S03:步驟 S01-S03: steps

S031-S035:步驟 S031-S035: Step

Claims

一種以擴增實境方式於加工件上顯示其程式原點及加工路徑的方法，應用於一裝置，該裝置包括一通訊模組、一相機模組，以及一螢幕，該方法包括：控制該相機模組掃描一工件表面的一標記物，辨識該標記物取得一工單編號，該相機模組持續拍攝該工件，在該螢幕上顯示一工件影像；該通訊模組與一擴增實境程式介面伺服器溝通，取得與該工單編號對應的程式原點與該標記物的相對距離與位置，將該相機模組拍攝到的該標記物的影像作為位置及比例尺的基準，在該螢幕的該工件影像上顯示該程式原點；該通訊模組與該擴增實境程式介面伺服器溝通，取得與該工單編號對應的一個以上的加工路徑檔案在一檔案傳輸協定伺服器的位置與數量，從該檔案傳輸協定伺服器下載與該工單編號對應的一個以上的所述加工路徑檔案，以該程式原點為位置的基準，該標記物的影像為比例尺的基準，逐行讀取其中一加工路徑檔案，繪製一加工路徑在該工件影像上；以及該通訊模組與該擴增實境程式介面伺服器溝通，取得一個以上分別與所述各加工路徑檔案對應的一工件模型檔案；讀取與該加工路徑檔案對應的該工件模型檔案，以一原點座標為位置的基準，該標記物的影像為比例尺的基準，繪製一透明的工件模型影像重疊在該工件影像上。 A method for displaying the program origin and processing path on a processed part in an augmented reality method is applied to a device that includes a communication module, a camera module, and a screen. The method includes: controlling the The camera module scans a marker on the surface of a workpiece, recognizes the marker and obtains a work order number, the camera module continuously shoots the workpiece, and displays an image of the workpiece on the screen; the communication module and an augmented reality The program interface server communicates, obtains the relative distance and position of the program origin corresponding to the work order number and the marker, and uses the image of the marker captured by the camera module as the reference for the position and scale. The program origin is displayed on the workpiece image; the communication module communicates with the augmented reality program interface server to obtain the location of one or more processing path files corresponding to the work order number on a file transfer protocol server And quantity, download one or more of the processing path files corresponding to the work order number from the file transfer protocol server, take the origin of the program as the position reference, the image of the marker as the scale reference, read line by line Take one of the processing path files and draw a processing path on the workpiece image; and the communication module communicates with the augmented reality programming interface server to obtain one or more workpiece models corresponding to each of the processing path files File; read the workpiece model file corresponding to the processing path file, take an origin coordinate as the position reference, the image of the marker as the scale reference, and draw a transparent workpiece model image to overlap the workpiece image.

如請求項1之以擴增實境方式於加工件上顯示其程式原點及加工路徑的方法，其中該標記物為一維條碼或二維條碼；在該螢幕的該工件影像上顯示該程式原點時，還顯示了X軸、Y軸。 For example, the method for displaying the program origin and the processing path on the workpiece in the augmented reality mode of the request item 1, wherein the marker is a one-dimensional bar code or a two-dimensional bar code; the program is displayed on the image of the workpiece on the screen At the origin, the X axis and Y axis are also displayed.

如請求項1或2之以擴增實境方式於加工件上顯示其程式原點及加工路徑的方法，其中對應該工單編號的一個以上的所述加工路徑檔案具有不同編號，在該螢幕顯示一按鈕，通過按壓該按鈕，切換逐行讀取另一加工路徑檔案，以該程式原點為位置為位置的基準，該標記物的影像為比例尺的基準，繪製另一加工路徑在該工件影像上。 For example, the method of displaying the program origin and processing path on the workpiece in the augmented reality mode of request item 1 or 2, wherein more than one processing path file corresponding to the number of the work order has different numbers, on this screen Display a button, by pressing the button, switch to read another processing path line by line For the file, take the program origin as the position as the position reference, the image of the marker as the scale reference, and draw another processing path on the workpiece image.

如請求項3之以擴增實境方式於加工件上顯示其程式原點及加工路徑的方法，其中在控制該相機模組掃描一工件表面的該標記物前，進行一前置處理，將多個所述的工單編號分別生成所述的該標記物，將所述該標記物雷射雕刻至對應的所述工件上。 For example, the method of displaying the program origin and the processing path on the workpiece in the augmented reality mode of claim 3, wherein before controlling the camera module to scan the marker on the surface of a workpiece, a pre-processing is performed, and the A plurality of the work order numbers respectively generate the marker, and laser engrave the marker on the corresponding workpiece.

一種以擴增實境方式於加工件上顯示其程式原點及加工路徑的裝置，包括：一螢幕；一相機模組，用以掃描、辨識一工件表面的一標記物取得一工單編號，該相機模組持續拍攝該工件，在該螢幕上顯示一工件影像；一通訊模組，用以與一擴增實境程式介面伺服器溝通，取得與該工單編號對應的程式原點與該標記物的相對距離與位置，以及與該工單編號對應的一個以上的加工路徑檔案在一檔案傳輸協定伺服器的位置與數量，該通訊模組從該檔案傳輸協定伺服器下載與該工單編號對應的一個以上的所述加工路徑檔案；一記憶體，儲存該通訊模組取得及下載的資料以及一擴增實境程式；以及一處理器，分別電連接、控制該螢幕、該相機模組、該通訊模組以及該記憶體，該處理器執行該擴增實境程式，將該相機模組拍攝到的該標記物的影像作為位置及比例尺的基準，配合上述程式原點與該標記物的相對距離與位置的資訊，在該螢幕的該工件影像上顯示該程式原點，並且以該程式原點為位置的基準，該標記物的影像為比例尺的基準，讀取該加工路徑檔案而繪製一加工路徑在該工件影像上；以及該通訊模組用以與該擴增實境程式介面伺服器溝通取得一個以上分別與所述各加工路徑檔案對應的一工件模型檔案，所述一個以上的工件模型檔案儲存於該記憶體；該處理器執行該擴增實境程式，對應逐行讀取的該加工路徑檔案，讀取與該加工路徑檔案對應的該工件模型檔案，以一原點座標為位置的基準，該標記物的影像為比例尺的基準，繪製一透明的工件模型影像重疊在該工件影像上。 A device for displaying the program origin and processing path on a workpiece in an augmented reality method, comprising: a screen; a camera module for scanning and identifying a marker on the surface of a workpiece to obtain a work order number, The camera module continuously shoots the workpiece and displays an image of the workpiece on the screen; a communication module is used to communicate with an augmented reality programming interface server to obtain the program origin corresponding to the work order number and the The relative distance and position of the marker, and the position and quantity of one or more processing path files corresponding to the work order number on a file transfer protocol server from which the communication module downloads the work order One or more of the processing path files corresponding to the number; a memory that stores the data obtained and downloaded by the communication module and an augmented reality program; and a processor that electrically connects and controls the screen and the camera module respectively Group, the communication module and the memory, the processor executes the augmented reality program, uses the image of the marker captured by the camera module as a reference for position and scale, and matches the origin of the program and the marker The relative distance and position information of the object, the program origin is displayed on the workpiece image on the screen, and the program origin is the position reference, the image of the marker is the scale reference, and the processing path file is read And draw a processing path on the workpiece image; and the communication module is used to communicate with the augmented reality programming interface server to obtain more than one workpiece model file corresponding to each processing path file, the one The above workpiece model file storage In the memory; the processor executes the augmented reality program, corresponds to the processing path file read line by line, reads the workpiece model file corresponding to the processing path file, and uses an origin coordinate as the position reference , The image of the marker is the reference of the scale, and a transparent image of the workpiece model is drawn and superimposed on the image of the workpiece.

如請求項5之以擴增實境方式於加工件上顯示其程式原點及加工路徑的裝置，其中該標記物為一維條碼或二維條碼，在該螢幕的該工件影像上顯示一X軸、一Y軸。 For example, the device for displaying the program origin and processing path on the workpiece in the form of augmented reality in claim 5, where the marker is a one-dimensional bar code or a two-dimensional bar code, and an X is displayed on the image of the workpiece on the screen Axis, a Y axis.

如請求項5或6之以擴增實境方式於加工件上顯示其程式原點及加工路徑的裝置，其中對應該工單編號的一個以上的所述加工路徑檔案具有不同編號，在該螢幕顯示一按鈕，通過按壓該按鈕，切換逐行讀取另一加工路徑檔案，以該程式原點為位置的基準，該標記物的影像為比例尺的基準，繪製另一加工路徑在該工件影像上。 For example, the device for displaying the program origin and processing path on the processed part in the form of augmented reality in request item 5 or 6, wherein more than one processing path file corresponding to the work order number has different numbers, in this screen Display a button, by pressing the button, switch to read another processing path file line by line, with the program origin as the position reference, the marker image as the scale reference, and draw another processing path on the workpiece image .