TWI793457B - Virtual reality based automatic attention-deficit diagnosis method and system - Google Patents
Virtual reality based automatic attention-deficit diagnosis method and system Download PDFInfo
- Publication number
- TWI793457B TWI793457B TW109135186A TW109135186A TWI793457B TW I793457 B TWI793457 B TW I793457B TW 109135186 A TW109135186 A TW 109135186A TW 109135186 A TW109135186 A TW 109135186A TW I793457 B TWI793457 B TW I793457B
- Authority
- TW
- Taiwan
- Prior art keywords
- attention
- virtual reality
- test
- algorithm
- performance
- Prior art date
Links
Images
Landscapes
- Eye Examination Apparatus (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
Description
本發明係有關於一種基於虛擬實境的自動化診斷方法與系統，尤其是指用於評估注意力缺陷的虛擬實境自動化診斷方法與系統。 The present invention relates to a virtual reality-based automatic diagnosis method and system, in particular to a virtual reality automatic diagnosis method and system for evaluating attention deficits.
虛擬實境(virtual reality,VR)技術是一種提供具有沉浸式體驗(immersive experiences)、可互動和富有想像力的電腦模擬技術，可以創建或模擬出一個由電腦虛構的仿真實環境，並允許使用者與之互動，近年來受惠於感測元件技術的進步以及軟體效能的優化，VR相關技術快速興起，諸如VR顯示器以及人機互動技術，且VR技術之應用已不再侷限於遊戲產業，開始在商業、娛樂、教育和醫療等產業蓬勃發展。 Virtual reality (virtual reality, VR) technology is a computer simulation technology that provides immersive experiences (immersive experiences), interactive and imaginative. To interact with it, thanks to the advancement of sensor technology and the optimization of software performance in recent years, VR-related technologies have risen rapidly, such as VR displays and human-computer interaction technology, and the application of VR technology is no longer limited to the game industry. Thrive in industries such as commerce, entertainment, education and healthcare.
相對於具有實體的物理環境，VR技術的特色在於能以較低的成本，創造與模擬出需要的環境，虛擬世界內部的所有物理參數、事件(event)或干擾(distraction)，都可以透過軟體給予精確的控制，同時，虛擬環境具有較高的安全性，也具有良好的生態有效性。 Compared with the physical environment with entities, the characteristic of VR technology is that it can create and simulate the required environment at a lower cost. All physical parameters, events (events) or disturbances (distraction) in the virtual world can be transmitted through the software. Giving precise control, at the same time, the virtual environment has high security and good ecological validity.
加上可穿戴感測技術(wearable sensing)逐漸邁向成熟，許多原本需要仰賴大型感測設備的感測項目，已經能夠微型化製作為可穿戴設備，大幅擴展可穿戴感測技術的應用層面，尤其可以更便捷地量測各式神 經行為，例如：腦波(EEG)、眼球追蹤(eye tracking)、心律變異(HRV)、肌肉電(EMG)、慣性感測器(IMU)等。 Coupled with the gradual maturity of wearable sensing technology, many sensing projects that originally relied on large sensing devices have been miniaturized into wearable devices, greatly expanding the application level of wearable sensing technology. In particular, it is more convenient to measure various gods Behavior, such as: brain wave (EEG), eye tracking (eye tracking), heart rhythm variation (HRV), muscle electricity (EMG), inertial sensor (IMU), etc.
在醫療和健康照護方面，VR和可穿戴感測技術的運用，與傳統診斷方法相比具有相當大的優勢，例如，VR模擬環境可以根據不同患者的不同需求，透過調整輸入參數而快速制定適合的系統和方案，獲得不同面向的病人資訊，並且可以以較低的成本重複使用系統，而且透過可穿戴感測器，原本在傳統診斷和評估方式中不易量測的動作及生理資訊變得容易取得，更多樣的醫療資訊能夠使診斷結果更加準確。 In terms of medical and health care, the application of VR and wearable sensing technology has considerable advantages compared with traditional diagnostic methods. The system and solution can obtain different patient information, and the system can be reused at a lower cost, and through wearable sensors, motion and physiological information that were not easy to measure in traditional diagnosis and evaluation methods become easy Obtaining more diverse medical information can make the diagnosis more accurate.
因此VR技術結合穿戴式感測技術，也逐漸受到神經認知(cognitive)領域的接受，並開始應用作為評估認知缺陷、認知能力水平和行為表現、認知研究、或認知缺陷矯正治療的有力工具，許多的發明家與研究人員紛紛嘗試將可穿戴感測技術與VR技術整合，透過虛擬環境提供各式的刺激與干擾，然後評估受測者的神經認知反應狀態、進行腦認知科學的探索、甚至針對神經性行為疾病進行行為刺激治療。 Therefore, VR technology combined with wearable sensing technology has gradually been accepted by the field of neurocognitive (cognitive), and has begun to be used as a powerful tool for evaluating cognitive defects, cognitive ability levels and behavioral performance, cognitive research, or cognitive defect correction therapy. Many inventors and researchers have tried to integrate wearable sensing technology with VR technology, provide various stimuli and disturbances through the virtual environment, and then evaluate the neurocognitive response status of the subjects, explore brain cognitive science, and even target Behavioral stimulation therapy for neurobehavioral disorders.
但綜觀現有VR技術結合穿戴式感測技術在神經認知領域的應用，大致有以下的不足之處，亟待進一步提出新技術予以克服：(1)VR評估與測試內容無法和臨床測試內容完全匹配；(2)缺少對人體生理與動作指標的處理能力，例如頭部轉動、肢體動作、腦波(EEG)、眼球軌跡追蹤(eye-tracking)等；(3)熱衷採用統計分析方法；以及(4)對於神經認知功能評估，皆採取個別指標進行評估，缺少整合個別指標而進行的總體評估或複合評估，易造成偏估。 However, looking at the application of the existing VR technology combined with wearable sensing technology in the field of neurocognition, there are roughly the following shortcomings, which need to be overcome by further new technologies: (1) VR evaluation and test content cannot completely match the clinical test content; (2) Lack of ability to process human physiological and movement indicators, such as head rotation, body movements, brain waves (EEG), eye-tracking, etc.; (3) Passionate about using statistical analysis methods; and (4) ) For neurocognitive function assessment, individual indicators are used for assessment, and there is a lack of overall assessment or composite assessment that integrates individual indicators, which may easily lead to biased estimation.
職是之故，有鑑於習用技術中存在的缺點，發明人經過悉心 嘗試與研究，並一本鍥而不捨之精神，終構思出本案「注意力缺陷虛擬實境自動化診斷方法與系統」，能夠克服上述缺點，以下為本發明之簡要說明。 As a result, in view of the shortcomings existing in the conventional technology, the inventor has painstakingly Attempts and research, and a spirit of perseverance, finally conceived the "Attention Deficit Virtual Reality Automated Diagnosis Method and System", which can overcome the above-mentioned shortcomings. The following is a brief description of the present invention.
本發明提出注意力缺陷虛擬實境自動化檢測系統與方法，在VR技術的基礎上，結合可穿戴神經行為感測器，向受測者展示台灣本土教室的VR仿真情境，並在VR情境中內置多種注意力檢測，涵蓋選擇性注意力、持續性注意力以及執行功能，並可在檢測過程中發出各式干擾源，從而得到對於受測者注意力的最真實量化分析。 The present invention proposes a virtual reality automatic detection system and method for attention deficits. On the basis of VR technology, combined with a wearable neurobehavioral sensor, the VR simulation situation of a local classroom in Taiwan is shown to the subject, and a built-in A variety of attention tests, covering selective attention, sustained attention, and executive function, and various sources of interference can be emitted during the test, so as to obtain the most realistic quantitative analysis of the subject's attention.
本發明系統在檢測過程將進一步實施多種評估量表檢測與多種任務表現檢測，最終結合神經行為感測器所感測到的神經行為表現，並綜合這三者，透過本發明建立的一個注意力評估電腦輔助診斷模型，來選擇性加權合併評估或診斷受測者的注意力缺陷程度；本發明提出的系統與方法，可以克服習用技術認知缺陷診斷存在的多數問題。 In the detection process, the system of the present invention will further implement a variety of evaluation scale detection and a variety of task performance detection, and finally combine the neurobehavioral performance sensed by the neurobehavioral sensor, and integrate the three, through an attention assessment established by the present invention A computer-aided diagnosis model is used to selectively weight and combine to evaluate or diagnose the degree of attention deficit of the subject; the system and method proposed by the present invention can overcome most of the problems existing in conventional cognitive defect diagnosis.
據此本發明提出一種注意力缺陷虛擬實境自動化診斷方法，包含：透過穿戴式虛擬實境套件在虛擬實境中向受測者展示與實施至少一種注意力測試方法，並感測與收集該受測者因應該至少一種注意力測試方法之複數任務表現與複數量表表現；透過神經行為感測套件量測該受測者因應該至少一種注意力測試方法的複數神經行為表現；以及執行基於機器學習方法建構之注意力評估電腦輔助診斷模型，以根據該等量表表現、該等任務表現以及該等神經行為表現綜合評估該受測者的注意力缺陷程度。 Accordingly, the present invention proposes a virtual reality automatic diagnosis method for attention deficit, including: displaying and implementing at least one attention testing method to the subject in virtual reality through a wearable virtual reality kit, and sensing and collecting the The subject's multiple task performance and multiple scale performance in response to at least one attention test method; measuring the subject's multiple neurobehavioral performance in response to the at least one attention test method through a neurobehavioral sensing kit; The attention assessment computer-aided diagnosis model constructed by the machine learning method is used to comprehensively evaluate the degree of attention deficit of the subject according to the scale performance, the task performance and the neurobehavioral performance.
較佳的，所述之注意力缺陷虛擬實境自動化診斷方法，還包 含以下步驟其中之一：在該虛擬實境中置入注意力測試任務模組、干擾測試模組、以及該神經行為感測套件；經由該注意力測試任務模組指揮該虛擬實境對該受測者實施該至少一種注意力測試方法；經由該干擾測試模組在該至少一種注意力測試方法實施過程中，對該受測者發出至少一干擾事件；透過該神經行為感測套件量測該受測者與該虛擬實境進行互動過程所產生的複數神經行為表現；透過該穿戴式虛擬實境套件量測該受測者與該虛擬實境進行互動過程所產生的複數反應參數；以及在該至少一種注意力測試方法的實施過程同步實施至少一種評估量表診斷以及至少一種任務表現診斷，以基於該等反應參數分別評估該受測者因應該至少一種注意力測試方法的該等量表表現以及該等任務表現。 Preferably, the described attention deficit virtual reality automatic diagnosis method also includes One of the following steps is included: placing an attention test task module, an interference test module, and the neurobehavioral sensing kit in the virtual reality; directing the virtual reality to the The subject implements the at least one attention test method; through the interference test module, during the implementation of the at least one attention test method, at least one interference event is sent to the subject; through the neurobehavioral sensing kit to measure The multiple neurobehavioral performances produced by the subject's interaction with the virtual reality; the multiple response parameters produced by the testee's interaction with the virtual reality through the wearable virtual reality kit; and Simultaneously implement at least one assessment scale diagnosis and at least one task performance diagnosis during the implementation of the at least one attention test method, so as to evaluate the subject's response to the at least one attention test method based on the response parameters. table performance and such task performance.
較佳的，所述之注意力缺陷虛擬實境自動化診斷方法，其中該注意力評估電腦輔助診斷模型係經由實施以下步驟其中之一而建立：對該等量表表現、該等任務表現以及該等神經行為表現之複數評估指標，實施統計顯著性檢定，以計算該等評估指標的顯著檢定值；對該等評估指標，實施統計主成分分析法，以計算該等評估指標的特徵值；以及選出該特徵值與該顯著檢定值皆高於門檻值的該等評估指標作為訓練集資料而輸入該機器學習方法，經由訓練該機器學習方法而建立該等評估指標相對於注意力缺陷的關聯模式、診斷模式或者決策模式，以建立該注意力評估電腦輔助診斷模型。 Preferably, the virtual reality automatic diagnosis method for attention deficit, wherein the attention assessment computer-aided diagnosis model is established by implementing one of the following steps: the performance of these scales, the performance of these tasks and the For multiple evaluation indicators of neurobehavioral performance, perform statistical significance test to calculate the significant test value of these evaluation indicators; for these evaluation indicators, implement statistical principal component analysis method to calculate the eigenvalues of these evaluation indicators; and Select the evaluation indicators whose characteristic value and the significant test value are higher than the threshold value as the training set data and input into the machine learning method, and establish the correlation mode of the evaluation indicators with respect to the attention deficit by training the machine learning method , diagnosis mode or decision-making mode, to establish the computer-aided diagnosis model for the attention assessment.
本發明進一步提出一種注意力缺陷虛擬實境自動化診斷系統，包含：穿戴式虛擬實境套件，經配置向受測者展示虛擬實境以向該受測者實施至少一種注意力測試方法，並感測與收集該受測者因應該至少一 種注意力測試方法之複數任務表現與複數量表表現；神經行為感測套件，經配置量測該受測者因應該至少一種注意力測試方法的複數神經行為表現；以及運算單元，經配置向該穿戴式套件提供該虛擬實境，並執行基於機器學習方法建構之注意力評估電腦輔助診斷模型，以根據該等量表表現、該等任務表現以及該等神經行為表現綜合評估該受測者的注意力缺陷程度。 The present invention further proposes a virtual reality automatic diagnosis system for attention deficit, comprising: a wearable virtual reality kit, which is configured to show a virtual reality to the subject to implement at least one attention testing method to the subject, and feel Measure and collect the respondent should have at least one Multiple task performance and multiple scale performance of an attention test method; a neurobehavioral sensing kit configured to measure the subject's multiple neurobehavioral performance in response to the at least one attention test method; and an arithmetic unit configured to The wearable kit provides the virtual reality and executes the attention assessment computer-aided diagnosis model constructed based on the machine learning method to comprehensively evaluate the subject according to the scale performance, the task performance and the neurobehavioral performance degree of attention deficit.
上述發明內容旨在提供本揭示內容的簡化摘要，以使讀者對本揭示內容具備基本的理解，此發明內容並非揭露本發明的完整描述，且用意並非在指出本發明實施例的重要/關鍵元件或界定本發明的範圍。 The above summary of the invention is intended to provide a simplified summary of the disclosure to enable readers to have a basic understanding of the disclosure. This summary of the invention is not intended to disclose a complete description of the invention, and is not intended to point out important/key elements or components of the embodiments of the invention. define the scope of the invention.
1:注意力缺陷虛擬實境自動化診斷系統 1: Attention deficit virtual reality automatic diagnosis system
10:穿戴式虛擬實境套件 10: Wearable virtual reality kit
11:頭戴式VR顯示裝置 11: Head-mounted VR display device
12:控制器 12: Controller
13:動作感測套組 13:Motion sensing kit
20:神經行為感測套件 20: Neurobehavioral Sensing Kit
21:腦波感測器 21: Brainwave sensor
22:眼球軌跡追蹤器 22:Eye tracker
23:頭部動作感應器 23: Head motion sensor
24:肢體動作感應器 24: Body motion sensor
30:運算單元 30: Operation unit
31:注意力評估電腦輔助診斷模型 31: Attention Assessment Computer Aided Diagnosis Model
ΘOFF:頭部轉動角度 ΘOFF: head rotation angle
110:沉浸式仿真互動VR環境模組 110: Immersive simulation interactive VR environment module
120:注意力測試模組 120: Attention Test Module
130:干擾測試模組 130:Interference test module
140:神經行為感測模組 140:Neural Behavior Sensing Module
200:本發明注意力缺陷虛擬實境自動化診斷方法 200: Automatic Diagnosis Method of Attention Deficit Virtual Reality in the Present Invention
201~208:實施步驟 201~208: Implementation steps
第1圖係揭示本發明注意力缺陷虛擬實境自動化診斷方法之整體架構示意圖； Figure 1 is a schematic diagram of the overall structure of the virtual reality automatic diagnosis method for attention deficit of the present invention;
第2圖係揭示本發明所建構之沉浸式仿真互動VR環境示意圖； Figure 2 is a schematic diagram showing the immersive simulation interactive VR environment constructed by the present invention;
第3圖係揭示本發明透過沉浸式虛擬實境仿真互動環境產生持續性表現測驗之示意圖； Fig. 3 is a schematic diagram showing that the present invention generates a continuous performance test through an immersive virtual reality simulation interactive environment;
第4圖係揭示本發明透過沉浸式虛擬實境仿真互動環境產生威斯康辛卡分類測驗之示意圖； Figure 4 is a schematic diagram showing that the present invention generates the Wisconsin Card Sorting Test through an immersive virtual reality simulation interactive environment;
第5圖係揭示本發明注意力測試任務之測試流程圖； Fig. 5 discloses the test flowchart of the attention test task of the present invention;
第6圖係揭示本發明眼球軌跡追蹤之目標區域之定義示意圖； Figure 6 is a schematic diagram showing the definition of the target area of the eye track tracking of the present invention;
第7圖係揭示本發明頭部轉動總角度之定義示意圖； Figure 7 is a schematic diagram showing the definition of the total head rotation angle of the present invention;
第8圖係揭示本發明注意力評估電腦輔助診斷模型之建構流程示意 圖； Figure 8 is a schematic diagram showing the construction process of the attention assessment computer-aided diagnosis model of the present invention picture;
第9圖係揭示本發明注意力缺陷虛擬實境自動化診斷系統之系統架構視圖；以及 Fig. 9 discloses the system architecture view of the attention deficit virtual reality automatic diagnosis system of the present invention; and
第10圖係揭示本發明注意力缺陷虛擬實境自動化診斷方法之實施步驟流程圖。 Fig. 10 is a flowchart showing the implementation steps of the virtual reality automatic diagnosis method for attention deficit of the present invention.
本發明將可由以下的實施例說明而得到充分瞭解，使得熟習本技藝之人士可以據以完成之，然本發明之實施並非可由下列實施案例而被限制其實施型態；本發明之圖式並不包含對大小、尺寸與比例尺的限定，本發明實際實施時其大小、尺寸與比例尺並非可經由本發明之圖式而被限制。 The present invention can be fully understood by the following examples, so that those skilled in the art can complete it, but the implementation of the present invention can not be limited by the following examples of implementation; the drawings of the present invention are not limited No limitation on size, dimension and scale is included, and the size, dimension and scale of the present invention are not limited by the drawings of the present invention during the actual implementation.
本文中用語“較佳”是非排他性的，應理解成“較佳為但不限於”，任何說明書或請求項中所描述或者記載的任何步驟可按任何順序執行，而不限於請求項中所述的順序，本發明的範圍應僅由所附請求項及其均等方案確定，不應由實施方式示例的實施例確定；本文中用語“包含”及其變化出現在說明書和請求項中時，是一個開放式的用語，不具有限制性含義，並不排除其他特徵或步驟。 The word "preferred" in this article is non-exclusive and should be understood as "preferably but not limited to". order, the scope of the present invention should be determined only by the appended claims and their equivalents, not by the examples illustrated in the implementation; when the term "comprising" and its variations appear in the specification and claims, it is An open-ended term without a restrictive meaning that does not exclude other features or steps.
注意力缺陷多動障礙(attention-deficit/hyperactivity disorder,ADHD)即兒童過動症是認知缺陷(cognitive deficiency)的一種，也是一種常見於兒童期的神經行為障礙，患病率約為3%至12%，患有ADHD的兒童智力基本正常，但在學校、社會、家庭等多種場合將表現出與年齡不符合的注意力集中困難，注意力持續時間短，碎動多或衝動行為等，患有ADHD 的兒童常伴有行為障礙或學習困難，易導致患者的學業、職業、社會生活的表現低下，嚴重影響了患者的生活品質與未來發展。 Attention-deficit/hyperactivity disorder (ADHD) is a kind of cognitive deficit (cognitive deficiency), and it is also a common neurobehavioral disorder in childhood, with a prevalence rate of about 3% to 12%, children with ADHD have basically normal intelligence, but in school, society, family and other occasions, they will show difficulty in concentration, short attention span, frequent movement or impulsive behavior, etc. that do not match their age. have ADHD The children are often accompanied by behavioral disorders or learning difficulties, which can easily lead to poor academic, occupational, and social life performance of patients, seriously affecting the quality of life and future development of patients.
ADHD通常可在學齡前經診斷而確診，如果不及時有效地治療，約60%到70%的ADHD兒童其症狀將持續到青少年期甚至成人期，如果不接受相關治療和訓練，ADHD對患者的影響將持續終身，根據研究資料，在情緒管理方面，由於成年期自我控制能力的提高，患者的過動症狀將大大緩解甚至消失；然而，仍然可能存在著不同程度的注意力缺陷、情緒障礙和繼發性行為障礙，例如難以保持專注，不負責任，情緒上的自卑和行為上的衝動仍然存在。 ADHD can usually be diagnosed before school age. If it is not treated in time and effectively, about 60% to 70% of ADHD children will continue to have symptoms until adolescence or even adulthood. If they do not receive relevant treatment and training, ADHD will affect the patient's health The impact will last for life. According to research data, in terms of emotional management, due to the improvement of self-control ability in adulthood, the patient's hyperactivity symptoms will be greatly relieved or even disappeared; however, there may still be varying degrees of attention deficit, emotional disorders and Secondary behavioral disturbances such as difficulty staying focused, irresponsibility, emotional low self-esteem, and impulsive behavior persist.
ADHD的診斷和評估是進行治療的重要依據，現行的ADHD兒童評估工具主要有量表和神經心理測驗兩類，雖然評估量表在進行ADHD診斷時簡單方便，但是大部分的量表為他評量表，透過父母親或老師的觀察進行填寫，在填寫量表時帶有主觀性，不容易量化兒童的行為，影響評估結果的客觀性，例如使用SNAP-IV或是CONNERS等問卷式量表，再計算由量表得到的分數來判定兒童是否患有ADHD。 The diagnosis and evaluation of ADHD is an important basis for treatment. The current evaluation tools for children with ADHD mainly include scales and neuropsychological tests. The scale is filled in through the observation of parents or teachers. It is subjective when filling out the scale. It is not easy to quantify the behavior of children, which affects the objectivity of the evaluation results. For example, use questionnaires such as SNAP-IV or CONNERS , and then calculate the scores obtained by the scale to determine whether children suffer from ADHD.
然而，傳統診斷方式的執行相當耗時，且在量化行為這方面的證據比較缺乏說服力，主要原因是它缺乏客觀的人體活動測量，使得行為過動的部分不易評估，而且可預測性十分有限，ADHD的診斷要求患兒的注意力缺陷或衝動在兩種及以上的場合出現才能被診斷，但是這在臨床中較難實現，只能從他人角度反映兒童在某個特定場合的特定表現，影響評估結果的正確性。 However, traditional diagnostic methods are time-consuming to perform and the evidence for quantifying behavior is relatively weak, mainly because it lacks objective measures of human activity, making the hyperactive component difficult to assess and with limited predictability , the diagnosis of ADHD requires that the child's attention deficit or impulsiveness can be diagnosed in two or more occasions, but this is difficult to achieve in clinical practice, and it can only reflect the specific performance of the child on a specific occasion from the perspective of others. affect the correctness of the assessment results.
而神經心理測驗通常為透過電腦設備進行測驗，例如CPT測 試，測試分數可以部份反映兒童的行為表現，但是CPT測試過程缺少與真實情景結合，難以觀察兒童在自然情景中出現的行為表現，這類傳統測試仍然存在生態有效性的問題。 Neuropsychological tests are usually conducted through computer equipment, such as the CPT test. Test scores can partially reflect children's behavioral performance, but the CPT test process lacks integration with real situations, and it is difficult to observe children's behavioral performance in natural situations. This type of traditional test still has the problem of ecological validity.
因此本發明以ADHD的認知缺陷為例，說明並提出注意力缺陷虛擬實境自動化檢測系統與方法，在VR技術的基礎上，結合可穿戴神經行為感測技術，包括腦波、眼球軌跡追蹤、頭部轉動以及肢體動作，向患兒展示台灣本土教室的VR情境，並在VR情境中內置多種注意力檢測，涵蓋選擇性注意力、持續性注意力以及執行功能，尤其在患兒接受檢測過程中，系統還可對患兒發出各式干擾源，包括視覺干擾、聽覺干擾、嗅覺干擾、綜合性干擾等，從而得到對於兒童注意力影響的最真實量化分析，且系統可以在檢測過程進一步實施多種評估量表檢測與多種任務表現檢測，以綜合評估患兒的量表表現以及任務表現，本發明提出的系統與方法，可以克服習用技術認知缺陷診斷存在的多數問題，尤其是ADHD診斷中遭遇到的問題。 Therefore, the present invention takes the cognitive defect of ADHD as an example to illustrate and propose a virtual reality automatic detection system and method for attention deficit. Head rotation and body movements show the children the VR situation of Taiwan’s local classroom, and built a variety of attention detection in the VR situation, covering selective attention, sustained attention and executive function, especially during the testing process of children In addition, the system can also send out various sources of interference to children, including visual interference, auditory interference, olfactory interference, comprehensive interference, etc., so as to obtain the most realistic quantitative analysis of the impact on children's attention, and the system can be further implemented in the detection process A variety of evaluation scale detection and a variety of task performance detection to comprehensively evaluate the scale performance and task performance of children. The system and method proposed by the present invention can overcome most of the problems existing in the diagnosis of cognitive deficits in conventional techniques, especially in the diagnosis of ADHD. to the problem.
第1圖係揭示本發明注意力缺陷虛擬實境自動化診斷方法之整體架構示意圖；本發明注意力缺陷虛擬實境自動化診斷方法涉及四個主要模組：沉浸式仿真互動VR環境模組110、注意力測試模組120、干擾測試模組130、以及神經行為感測模組140，在沉浸式仿真互動VR環境模組110的部分，主要透過一組穿戴式虛擬實境套件，向受測者提供一個沉浸式仿真互動VR環境場景，穿戴式虛擬實境套件包含頭戴式VR顯示裝置、控制器、以及動作感測套組等元件。 Figure 1 is a schematic diagram of the overall structure of the virtual reality automatic diagnosis method for attention deficit of the present invention; the virtual reality automatic diagnosis method for attention deficit of the present invention involves four main modules: immersive simulation interactive VR environment module 110, attention The force test module 120, the interference test module 130, and the neurobehavioral sensing module 140, in the part of the immersive simulation interactive VR environment module 110, mainly provide the testees with a set of wearable virtual reality kits. An immersive simulated interactive VR environment scene, the wearable virtual reality kit includes components such as a head-mounted VR display device, a controller, and a motion sensing kit.
第2圖係揭示本發明所建構之沉浸式仿真互動VR環境示意 圖；本發明系統以台灣國民基本教育與義務教育制度中常見的傳統本土教室作為虛擬實境的3D建模對象，從而構建一個虛擬教室的3D場景環境，並透過頭戴式VR顯示裝置向受測者展示，場景中包括教室內部場景、虛擬教師、虛擬學生、投影螢幕、電視和窗外的室外場景等基本元素，所有的基本元素皆以台灣本土真實素材為主，而非採用卡通式人物或其他示意性元素，受測者頭戴式VR顯示裝置所觀看的VR場景如第2圖所揭示。 Figure 2 shows the schematic diagram of the immersive simulation interactive VR environment constructed by the present invention Figure; the system of the present invention takes the common traditional local classroom in Taiwan's national basic education and compulsory education system as the 3D modeling object of virtual reality, thereby constructing a 3D scene environment of a virtual classroom, and presenting to the recipient through a head-mounted VR display device The testers showed that the scenes include basic elements such as classroom interior scenes, virtual teachers, virtual students, projection screens, TVs, and outdoor scenes outside the window. All the basic elements are mainly based on real materials from Taiwan, rather than cartoon characters or Other schematic elements, the VR scene watched by the subject's head-mounted VR display device is shown in Figure 2.
在本實施例，頭戴式VR顯示裝置較佳是選用例如但不限於VR頭盔顯示器、Oculus或者HTC Vive的頭戴式VR眼鏡，沉浸式仿真互動VR環境模組將進一步連結手持式控制器與動作感測套組，動作感測套組較佳是例如但不限於：光學位置追蹤套組、紅外光位置追蹤套組、雷射光位置追蹤套組、超音波位置追蹤套組或者電磁式位置追蹤套組，在本實施例，動作感測套組較佳是採用Valve軟體開發的Lighthouse光學位置追蹤套組，而構成一個沉浸式仿真互動VR環境。 In this embodiment, the head-mounted VR display device is preferably a head-mounted VR glasses such as but not limited to a VR helmet-mounted display, Oculus or HTC Vive, and the immersive simulation interactive VR environment module will further link the handheld controller and Motion sensing kit, the motion sensing kit is preferably such as but not limited to: optical position tracking kit, infrared light position tracking kit, laser light position tracking kit, ultrasonic position tracking kit or electromagnetic position tracking As for the kit, in this embodiment, the motion sensing kit preferably adopts the Lighthouse optical position tracking kit developed by Valve software to form an immersive simulation interactive VR environment.
配置在虛擬教室前方的投影螢幕，將專用於向受測者顯示至少一種注意力測試任務，注意力測試模組會透過投影螢幕顯示注意力測試任務，注意力測試模組較佳將提供至少三種測試方法，包含音頻測驗，持續性表現測驗(continuous performance tests,CPT)和威斯康辛卡分類測驗(Wisconsin card sorting test,WCST)等，以供檢測至少三種注意力，包含持續性注意力、選擇性注意力以及執行功能等。 The projection screen configured in front of the virtual classroom will be dedicated to displaying at least one attention test task to the subjects. The attention test module will display the attention test task through the projection screen. The attention test module will preferably provide at least three Test methods, including audio tests, continuous performance tests (continuous performance tests, CPT) and Wisconsin card sorting tests (Wisconsin card sorting test, WCST), etc., to detect at least three types of attention, including continuous attention, selective attention force and executive function.
音頻測驗是一項聽力任務，在每次測試之前給出確認標準，並且當在播放音頻時聽到特定數字的受測者，將被要求按下控制器上的「確認」按鈕，數字將從0到9當中選出，每輪的標準都會改變，例如，第一輪 採用的標準是，當聽到數字7然後接著聽到數字3的時候，按下確認按鈕，音頻播放的數字序列是27273293，受測者應該在聽到第一個3的時候按下確認按鈕，任務結果的輸出是反應時間以及受測者是否按下每個數字對應的按鈕。如果受測者正確按下按鈕，則輸出為True；否則輸出為False。 The audio test is a listening task where confirmation criteria are given before each test, and when the testee hears a specific number while the audio is played, the testee will be asked to press the "confirm" button on the controller, the number will change from 0 to to be selected out of 9, the criteria change each round, for example, the first round The standard adopted is, when hearing the number 7 and then hearing the number 3, press the confirmation button, the sequence of numbers played in the audio is 27273293, the subject should press the confirmation button when hearing the first 3, the results of the task The outputs were reaction times and whether or not the subject pressed the button for each number. If the subject presses the button correctly, the output is True; otherwise, the output is False.
第3圖係揭示本發明透過沉浸式虛擬實境仿真互動環境產生持續性表現測驗之示意圖；CPT測試是一項視覺任務，在每一輪中，一系列由A到H、J、L和X組成的字母將出現在投影螢幕上，當字母X出現在螢幕上的字母A後面時，受測者應按下控制器上的「確認」按鈕，在其他情況下，受測者不應該按下按鈕，任務結果的輸出是反應時間以及受測者是否正確按下與每個字母對應的按鈕。 Fig. 3 is a schematic diagram showing the present invention to generate a continuous performance test through an immersive virtual reality simulation interactive environment; the CPT test is a visual task, in each round, a series consisting of A to H, J, L and X The letters will appear on the projection screen. When the letter X appears behind the letter A on the screen, the subject should press the "OK" button on the controller. In other cases, the subject should not press the button , the output of the task outcome is the reaction time and whether the subject correctly pressed the button corresponding to each letter.
第4圖係揭示本發明透過沉浸式虛擬實境仿真互動環境產生威斯康辛卡分類測驗之示意圖；WCST測試是由問題庫中的四張刺激卡和128張響應卡組成。每張卡片都有不同數字，顏色和形狀的組合。數字從1到4，顏色為紅色，綠色，黃色和藍色，形狀為三角形，圓形，十字形和星形。受測者需要按數字，顏色或形狀對響應卡進行排序．排序規則由計算機設置為數字，顏色和形狀的循環。受測者不會事先被告知排序規則，並且需要通過反複試驗，分析和推理找到計算機設置的規則。當受測者連續六次或使用128個響應卡時確定正確的排序規則時，測試完成，任務結果的輸出是反應時間以及受測者是否正確排序每張卡。 Fig. 4 is a schematic diagram showing that the present invention generates the Wisconsin Card Sorting Test through an immersive virtual reality simulation interactive environment; the WCST test is composed of four stimulus cards and 128 response cards in the question bank. Each card has a combination of different numbers, colors and shapes. The numbers are from 1 to 4, the colors are red, green, yellow and blue, and the shapes are triangles, circles, crosses and stars. Respondents were asked to sort the response cards by number, color or shape. The sorting rules are set by the computer as cycles of numbers, colors and shapes. Subjects are not told the sorting rules in advance, and need to find the rules set by the computer through trial and error, analysis and reasoning. The test is complete when the subject determines the correct sorting rule six times in a row or when using 128 response cards, and the output of the task outcome is reaction time and whether the subject correctly sorted each card.
為了研究教室內干擾源對於兒童的注意力影響，並探討干擾源對於患有ADHD兒童以及健康兒童的差異，本發明進一步在虛擬教室內置入干擾測試模組，在虛擬教室內對受測者發出一系列干擾，干擾的形式分 為：視覺干擾、聽覺干擾、嗅覺干擾、綜合性干擾，干擾的內容是以事件(event)的方式呈現，各種干擾事件如下述表1所列示。 In order to study the influence of interference sources in the classroom on children's attention, and to explore the differences between interference sources for children with ADHD and healthy children, the present invention further built an interference test module into the virtual classroom, and issued a test to the subjects in the virtual classroom. A series of disturbances, in the form of disturbances They are: visual interference, auditory interference, olfactory interference, and comprehensive interference. The content of the interference is presented in the form of events. Various interference events are listed in Table 1 below.
表1：
為了同步量測注意力測試過程中的神經行為，本研究將虛擬教室與多模態神經行為感測器整合，多模態神經行為感測器至少包含：(1)腦波感測器：在本實施例較佳選用頭戴式陣列腦波感測設備，可以和頭戴式VR顯示裝置進行整合，量測位置為注意力相關的腦部位置，例如AF3和AF4；(2)眼球軌跡追蹤器：在本實施例較佳選用例如但不限於HTC Vive Pro Eye內置的TOBII眼球軌跡追蹤感測設備；(3)頭部動作感應器：在本實施例較佳選用例如但不限於HTC Vive的Lighthouse進行動作捕捉，量測兒童受測過程中的轉動軌跡；以及(4)肢體動作感應器：在本實施例較佳選用例如但不限於Myo手環的多軸慣性感測器，可穿戴在受測者的手腕上，量測手腕的三軸加速度資料。上述所有神經行為感測器皆與虛擬教室的數位內容同步化，以在注意力測試任務實施過程中，即時感測受測者的神經行為表現。 In order to simultaneously measure the neurobehavior during the attention test, this study integrated the virtual classroom with multimodal neurobehavioral sensors. The multimodal neurobehavioral sensors include at least: (1) EEG sensors: In this embodiment, a head-mounted array brainwave sensing device is preferably selected, which can be integrated with a head-mounted VR display device, and the measurement position is the brain position related to attention, such as AF3 and AF4; (2) Eye track tracking Device: In this embodiment, it is better to choose such as but not limited to HTC Vive Pro Eye's built-in TOBII eye track tracking sensing device; (3) head motion sensor: In this embodiment, it is preferable to use Lighthouse such as but not limited to HTC Vive for motion capture, and measure the child's rotation track during the test process; And (4) Body motion sensor: In this embodiment, a multi-axis inertial sensor such as but not limited to Myo wristband is preferably used, which can be worn on the wrist of the subject to measure the three-axis acceleration data of the wrist. All the neurobehavioral sensors mentioned above are synchronized with the digital content of the virtual classroom, so as to sense the subject's neurobehavioral performance in real time during the implementation of the attention test task.
第5圖係揭示本發明注意力測試任務之測試流程圖；在本實施例，將注意力測試任務流程大致分為兩個階段，第一階段是在無干擾事件情況下的測試，第二階段加入各式干擾事件，每個階段持續4-5分鐘，所以每個測試任務將持續8-10分鐘，這個時間長度足以誘發注意力不集中的行為。 Fig. 5 discloses the test flowchart of the attention test task of the present invention; In the present embodiment, the attention test task flow process is roughly divided into two stages, the first stage is the test under the situation of no disturbance event, the second stage A variety of distracting events were added, and each session lasted 4-5 minutes, so each test task would last 8-10 minutes, which is long enough to induce inattentive behavior.
本發明提出的注意力評估電腦輔助(computer-assisted)診斷模型係以選擇性加權合併的方式，涵括三類型的評估結果，包含評估量表、測試任務表現以及神經行為感測資料等，其中在評估量表的部分，至少涵括CONNERS量表、SNAP-IV量表以及Weiss’s量表，測試任務表現則至少涵括反應時間、遺漏錯誤率、替代性錯誤率、完成時間等評估指標，神經行為感測資料則至少涵括腦波、眼球軌跡追蹤、頭部轉動、肢體動作等項目。 The computer-assisted diagnosis model for attention assessment proposed by the present invention includes three types of assessment results in a selective weighted and combined manner, including assessment scales, test task performance, and neurobehavioral sensing data. In the part of the evaluation scale, at least the CONNERS scale, SNAP-IV scale and Weiss's scale are included, and the performance of the test task includes at least the evaluation indicators such as reaction time, omission error rate, substitution error rate, and completion time. Behavioral sensing data include at least brain waves, eye tracking, head rotation, body movements, and other items.
為了量化所量測的神經行為原始資料，依據虛擬教室的環境特性，分別設計不同的神經行為指標，以便神經行為感測資料將進一步量化為對應的神經行為指標，應用於後續基於機器學習方法的模型訓練。 In order to quantify the measured neurobehavioral raw data, different neurobehavioral indicators are designed according to the environmental characteristics of the virtual classroom, so that the neurobehavioral sensing data will be further quantified into corresponding neurobehavioral indicators, which can be applied to subsequent research based on machine learning methods. Model training.
在腦波指標的部分，區分為時域特徵和頻域特徵指標，時域特徵包含：Higuchi碎形維度(HFD)、Katz碎形維度(KFD)或者Petrosian碎形 維度(PFD)，頻域特徵包含：α波頻帶功率(band-power)、β波頻帶功率或者β波頻帶功率/θ波頻帶功率比。 In the part of brainwave indicators, it is divided into time-domain features and frequency-domain feature indicators. Time-domain features include: Higuchi Fractal Dimension (HFD), Katz Fractal Dimension (KFD) or Petrosian Fractal Dimension (PFD), Frequency Domain Features Including: α wave band power (band-power), β wave band power or β wave band power/ θ wave band power ratio.
第6圖係揭示本發明眼球軌跡追蹤之目標區域之定義示意圖；在眼球軌跡追蹤的部分，本發明提出眼球軌跡總長度指標以及目標區域注視比例指標，目標區域是虛擬教室前方的投影螢幕所形成之區域，如第6圖所揭示，眼球軌跡總長度L EYE 係透過以下公式計算： Figure 6 is a schematic diagram showing the definition of the target area of the eye track tracking of the present invention; in the part of the eye track tracking, the present invention proposes an index of the total length of the eye track and a gaze ratio index of the target area, and the target area is formed by the projection screen in front of the virtual classroom area, as revealed in Figure 6, the total length of the eye track LEYE is calculated by the following formula:
其中n是眼球注視位置總數，P i 是眼球注視位置。 Among them, n is the total number of eyeball fixation positions, and P i is the eyeball fixation position.
目標區域注視比例E FOCUS 透過以下方式計算： The target area gaze ratio E FOCUS is calculated in the following way:
EE. FOCUSFOCUS =N=N inin /N/N TotalTotal
其中N in 是目標區域內的眼球注視位置數目，N Total 是眼球注視位置總數。 Among them, N in is the number of eye gaze positions in the target area, and N Total is the total number of eye gaze positions.
第7圖係揭示本發明頭部轉動總角度之定義示意圖；在頭部轉動的部分，本發明提出頭部轉動總角度指標以及頭部專注區比例指標，頭部轉動總角度θ HEAD 係透過以下公式計算： Figure 7 is a schematic diagram showing the definition of the total head rotation angle of the present invention; in the part of head rotation, the present invention proposes the total head rotation angle index and the head focus area ratio index, the total head rotation angle θ HEAD is through the following Formula calculation:
其中n是頭部轉動總次數，θ i 是頭部轉動角度。 where n is the total number of head turns and θi is the head turn angle.
頭部專注區比例H FOCUS 係透過以下公式計算： The proportion of the head focus area H FOCUS is calculated by the following formula:
Hh FOCUSFOCUS =N=N ΘOFF<10ΘOFF<10 /N/N TotalTotal
其中N ΘOFF < 10 是頭部轉動角度小於10度的次數，N Total 是頭部轉動總次數。 Where N ΘOFF < 10 is the number of head rotations less than 10 degrees, and N Total is the total number of head rotations.
在肢體動作的部分，本發明提出肢體動作軌跡長度指標，肢 體動作軌跡長度L Hand 係透過以下公式計算： In the part of body movement, the present invention proposes the index of body movement trajectory length, and the body movement trajectory length L Hand is calculated by the following formula:
其中Ai是加速度，N是動作總次數。 Where Ai is the acceleration and N is the total number of actions.
因此透過穿戴式虛擬實境套件包含的控制器與動作感測套組、以及神經行為感測器，可以感測並收集到受測者因應注意力測試的反應與神經行為指標等資料，這些資料將輸入注意力評估電腦輔助診斷模型進行診斷評估。 Therefore, through the controller and motion sensing kit included in the wearable virtual reality kit, as well as the neurobehavioral sensor, it is possible to sense and collect data such as the subject's response to the attention test and neurobehavioral indicators. The attentional assessment computer-aided diagnostic model will be input into the diagnostic assessment.
第8圖係揭示本發明注意力評估電腦輔助診斷模型之建構流程示意圖；本發明注意力評估電腦輔助診斷模型主要是以選擇性加權合併的方式，涵括下表的三類型的評估結果，大致涵蓋評估量表、測試任務表現以及神經行為感測等三大面向，每一類資料分別有不同的細節指標，如下述表2所列示。 Figure 8 is a schematic diagram showing the construction process of the computer-aided diagnosis model for attention assessment of the present invention; the computer-aided diagnosis model for attention assessment of the present invention is mainly based on a selective weighted combination, including the three types of assessment results in the following table, roughly Covering the three major aspects of assessment scale, test task performance, and neurobehavioral sensing, each type of information has different detailed indicators, as listed in Table 2 below.
表2：
注意力評估電腦輔助診斷模型係經由以下步驟建立，分述如下： The attention assessment computer-aided diagnosis model is established through the following steps, which are described as follows:
(1)評估各項指標對於注意力區分之顯著性的分析，首先對 於表2所列的各項指標，針對ADHD兒童以及健康兒童兩個群體，以t-test進行檢定，探索哪些指標對於ADHD兒童以及健康兒童的區分具有顯著性。 (1) Assess the analysis of the significance of each index for attention distinction, first of all For the indicators listed in Table 2, t-test was used to test the two groups of ADHD children and healthy children, and to explore which indicators were significant for the distinction between ADHD children and healthy children.
(2)以主成分分析法篩選與識別關鍵的評估指標，為了改善並增進後續評估運算的效能，對於具有高維度特性的資料指標，必須進行關鍵評估指標的識別。本發明提出以主成分分析法進行分析，篩選與識別關鍵的評估指標。 (2) Use principal component analysis to screen and identify key evaluation indicators. In order to improve and enhance the performance of subsequent evaluation operations, key evaluation indicators must be identified for data indicators with high-dimensional characteristics. The present invention proposes to use the principal component analysis method to analyze, screen and identify key evaluation indicators.
(3)以機器學習方法訓練以建立注意力評估電腦輔助診斷模型，保留R的特徵值大於1的且具統計顯著性的關鍵評估指標，使用機器學習方法訓練關鍵評估指標，建立ADHD兒童以及健康兒童的分類器，作為注意力評估電腦輔助診斷模型。 (3) Train with machine learning methods to establish a computer-aided diagnosis model for attention assessment, retain statistically significant key assessment indicators with an eigenvalue of R greater than 1, use machine learning methods to train key assessment indicators, and establish ADHD children and their health A classifier for children as a computer-aided diagnosis model for attention assessment.
機器學習方法較佳選自深度學習(deep learning)演算法、類神經網路(ANN)演算法、深度神經網路(DNN)演算法、遞歸神經網路(RNN)演算法、卷積神經網路(CNN)演算法、卷積遞歸神經網路(CRNN)演算法、生成對抗網路(GAN)演算法、深度信念網路(DBN)演算法、全卷積神經網路(FCN)演算法、多列卷積神經網路(MCNN)演算法、遞歸神經網路(RNN)演算法、雙向神經網路(BRNN)演算法、深層循環神經網路(DRNN)演算法、殘差網路(DRN)演算法、限制玻爾茲曼機(RBM)演算法、多層感知(MLP)演算法、自編碼器演算法、注意力網路演算法、梯度提升樹方法、強梯度提升機方法、弱梯度提升機方法、回歸樹方法、隨機森林方法、決策樹方法、弱學習方法、強學習方法、強投票方法、弱投票方法、支援向量機(support vector machines)分類器、關聯法則及其組合其中之一。 The machine learning method is preferably selected from a deep learning algorithm, a neural network (ANN) algorithm, a deep neural network (DNN) algorithm, a recurrent neural network (RNN) algorithm, a convolutional neural network Road (CNN) algorithm, Convolutional Recurrent Neural Network (CRNN) algorithm, Generative Adversarial Network (GAN) algorithm, Deep Belief Network (DBN) algorithm, Fully Convolutional Neural Network (FCN) algorithm , Multi-column Convolutional Neural Network (MCNN) Algorithm, Recurrent Neural Network (RNN) Algorithm, Bidirectional Neural Network (BRNN) Algorithm, Deep Recurrent Neural Network (DRNN) Algorithm, Residual Network ( DRN) algorithm, restricted Boltzmann machine (RBM) algorithm, multi-layer perception (MLP) algorithm, autoencoder algorithm, attention network algorithm, gradient boosting tree method, strong gradient boosting machine method, weak gradient Hoisting machine method, regression tree method, random forest method, decision tree method, weak learning method, strong learning method, strong voting method, weak voting method, support vector machines (support vector machines) classifiers, association rules and combinations thereof one.
將具有統計顯著性的關鍵評估指標切割為訓練集(training set)與測試集(test set)，交由機器學習(machine learning)技術大量讀取、辨識、學習和訓練，以找出隱含在這些關鍵評估指標中之關聯性與分類法則，以反覆訓練機器學習方法找出一套評估診斷模式與決策模式。 Cut statistically significant key evaluation indicators into a training set (training set) and the test set (test set), which are read, identified, learned, and trained by machine learning technology in large quantities to find out the correlation and classification rules hidden in these key evaluation indicators for repeated training The machine learning method finds a set of evaluation diagnosis mode and decision mode.
針對干擾源對於注意力之影響係經由實施以下步驟而評估： The influence of distractors on attention was assessed by implementing the following steps:
(1)整體干擾對於健康兒童的注意力影響，對於表2所列關於測試任務表現以及神經行為感測的評估指標，使用健康兒童的資料，按測試的兩徊階段(無干擾/有干擾)分別運算該階段的評估指標，以t-test進行檢定，可分辨出哪些指標對於無干擾狀態以及有干擾狀態的差異具有顯著性。 (1) The effect of overall interference on the attention of healthy children. For the evaluation indicators of test task performance and neurobehavioral sensing listed in Table 2, using the data of healthy children, according to the two stages of the test (no interference/interference) Calculate the evaluation indicators of this stage separately, and use t-test to verify which indicators are significant for the difference between the non-interference state and the interference state.
(2)整體干擾對於ADHD兒童的注意力影響，對於表2所列關於測試任務表現以及神經行為感測的評估指標，使用ADHD兒童的資料，按測試的兩個階段(無干擾/有干擾)分別運算該階段的評估指標，以t-test進行檢定，可分辨出哪些指標對於無干擾狀態以及有干擾狀態的差異具有顯著性。 (2) The impact of overall interference on the attention of ADHD children. For the evaluation indicators of test task performance and neurobehavioral sensing listed in Table 2, using the data of ADHD children, according to the two stages of the test (without interference / with interference) Calculate the evaluation indicators of this stage separately, and use t-test to verify which indicators are significant for the difference between the non-interference state and the interference state.
(3)個別干擾對於健康兒童的注意力影響，對於健康兒童有干擾階段的資料，每一個干擾出現前2秒、干擾出現期問、出現後2秒等三個時間段落，分別運算表2所列關於測試任務表現以及神經行為感測的評估指標，以單因子變異數分析(one-way ANOVA)，探討該干擾出現前、中、後的差異表現。 (3) The influence of individual disturbances on the attention of healthy children. For the data of the disturbance stage in healthy children, the three time periods of 2 seconds before the appearance of each disturbance, the period of the disturbance, and 2 seconds after the appearance of each disturbance were respectively calculated in Table 2. List the evaluation indicators of test task performance and neurobehavioral sensing, and use one-way ANOVA to explore the difference in performance before, during, and after the disturbance.
(4)個別干擾對於ADHD兒童的注意力影響，對於ADHD兒童有干擾階段的資料，每一個干擾出現前2秒、干擾出現期間、出現後2秒的三個時間段落，分別運算表2所列關於測試任務表現以及神經行為感測的評估指標，以單因子變異數分析(one-way ANOVA)，探討該干擾出現前、中、 後的差異表現。 (4) The influence of individual disturbances on the attention of ADHD children. For the data of ADHD children in the disturbance stage, the three time periods of 2 seconds before the appearance of each disturbance, during the period of disturbance, and 2 seconds after the appearance of each disturbance are listed in Table 2. With regard to the evaluation indicators of test task performance and neurobehavioral sensing, one-way ANOVA was used to explore the interference before, during, and after the appearance of the disturbance. subsequent differential performance.
第9圖係揭示本發明注意力缺陷虛擬實境自動化診斷系統之系統架構視圖(view)；小結而言，本發明提出的注意力缺陷虛擬實境自動化診斷系統1係包含：穿戴式虛擬實境套件10、神經行為感測套件20以及運算單元30等模組，穿戴式虛擬實境套件10包含頭戴式VR顯示裝置11、控制器12、以及動作感測套組13等元件，穿戴式虛擬實境套件10經配置向使用者展示虛擬實境以向使用者實施至少一種注意力測試方法，並感測與收集使用者應對該至少一種注意力測試方法之任務表現與量表表現。
Fig. 9 discloses the system architecture view (view) of the attention deficit virtual reality automatic diagnosis system of the present invention; in summary, the attention deficit virtual reality
神經行為感測套件20則包含腦波感測器21、眼球軌跡追蹤器22、頭部動作感應器23、肢體動作感應器24等元件，神經行為感測套件20經配置量測使用者應對該至少一種注意力測試方法的神經行為表現，而運算單元30內安裝一個注意力評估電腦輔助診斷模型31，經配置向穿戴式套件10提供該虛擬實境，並執行注意力評估電腦輔助診斷模型31以選擇性加權合併量表表現、任務表現以及神經行為表現，以綜合評估使用者的注意力缺陷程度。 The neurobehavioral sensing kit 20 includes components such as brainwave sensors 21, eye track trackers 22, head motion sensors 23, and body motion sensors 24. The neurobehavioral sensing kit 20 is configured to measure the user's response to the The neurobehavioral performance of at least one attention test method, and an attention assessment computer-aided diagnosis model 31 is installed in the computing unit 30, which is configured to provide the virtual reality to the wearable kit 10, and execute the attention assessment computer-aided diagnosis model 31 Scale performance, task performance and neurobehavioral performance were combined with selective weighting to comprehensively assess the degree of attention deficit of users.
第10圖係揭示本發明注意力缺陷虛擬實境自動化診斷方法之實施步驟流程圖；小結而言，本發明注意力缺陷虛擬實境自動化診斷方法200，較佳包含下列步驟：透過一穿戴式虛擬實境套件在一虛擬實境中向一受測者展示與實施至少一種注意力測試方法(步驟201)；在該虛擬實境中置入一注意力測試任務模組、一干擾測試模組、以及該神經行為感測套件(步驟202)；經由該注意力測試任務模組指揮該虛擬實境對該受測者實施該至少一種注意力測試方法(步驟203)；經由該干擾測試模組在該至少一種注意力 測試方法實施過程中，對該受測者發出至少一干擾事件(步驟204)；在該至少一種注意力測試方法的實施過程同步實施至少一種評估量表診斷以及至少一種任務表現診斷(步驟205)；透過穿戴式虛擬實境套件感測與收集該受測者因應該至少一種注意力測試方法之複數任務表現與複數量表表現(步驟206)；透過神經行為感測套件量測該受測者因應該至少一種注意力測試方法的複數神經行為表現(步驟207)；執行注意力評估電腦輔助診斷模型以選擇性加權合併該等量表表現、該等任務表現以及該等神經行為表現以綜合評估該受測者的注意力缺陷程度(步驟208)。 Fig. 10 is a flowchart showing the implementation steps of the virtual reality automatic diagnosis method for attention deficit of the present invention; in summary, the attention deficit virtual reality automatic diagnosis method 200 of the present invention preferably includes the following steps: through a wearable virtual reality The reality kit demonstrates and implements at least one attention test method (step 201) to a subject in a virtual reality environment; in the virtual reality environment, an attention test task module, an interference test module, And the neurobehavioral sensing suite (step 202); command the virtual reality to implement the at least one attention test method (step 203) to the subject via the attention test task module; at least one kind of attention During the implementation of the test method, at least one disturbance event is sent to the subject (step 204); during the implementation of the at least one attention test method, at least one assessment scale diagnosis and at least one task performance diagnosis are simultaneously implemented (step 205) ; Sensing and collecting the subject's multiple task performance and multiple scale performance (step 206) in response to at least one attention test method through a wearable virtual reality kit; measuring the subject through a neurobehavioral sensing kit Multiple neurobehavioral performances corresponding to the at least one attention test method (step 207); performing an attention assessment computer-aided diagnosis model to selectively weight and combine the scale performances, the task performances, and the neurobehavioral performances for a comprehensive assessment The subject's attention deficit degree (step 208).
本發明還包含至少以下優點：(1)VR虛擬教室模擬了台式教室場景，內含豐富的環境元素，具有較好的生態有效性；(2)將多個注意力測試任務嵌入VR虛擬教室，有別於一般單純的電腦測試環境；(3)VR虛擬教室設計了多種形態的環境干擾因素，可以人為控制環境干擾的種類、強度、頻率，有助於探索環境干擾對於注意力的影響；(4)VR虛擬教室測試環境整合多種神經感測設備，包括腦波、眼球軌跡追蹤、頭部轉動、肢體動作，可以同步量測注意力測試過程中的多種神經行為資料，這些資料可以更真實反應受測者的認知行為反應；(5)使用機器學習方法，同時整合評估量表、測試任務表現以及神經行為感測資料等多種不同形態的資料，建立自動化評估/輔助診斷模型；(6)結合統計方法和主成分分析法，從眾多的評估指標篩選出關鍵的評估指標，降低資料分析的維度，提高自動化評估/輔助診斷模型的準確度以及運算效能；以及(7)同步所有環境干擾事件以及多種神經感測設備，可以精準的探索每一個環境干擾事件對於注意力的影響。 The present invention also includes at least the following advantages: (1) the VR virtual classroom simulates the desktop classroom scene, contains rich environmental elements, and has good ecological effectiveness; (2) multiple attention test tasks are embedded in the VR virtual classroom, It is different from the general simple computer test environment; (3) VR virtual classroom is designed with various forms of environmental interference factors, which can artificially control the type, intensity and frequency of environmental interference, and help to explore the influence of environmental interference on attention; ( 4) The VR virtual classroom test environment integrates a variety of neural sensing devices, including brain waves, eye tracking, head rotation, and body movements, and can simultaneously measure a variety of neurobehavioral data during the attention test process, which can be more realistic. Cognitive and behavioral responses of the subjects; (5) use machine learning methods to integrate various forms of data such as assessment scales, test task performance, and neurobehavioral sensing data to establish an automated assessment/aided diagnosis model; (6) combine Statistical methods and principal component analysis methods, which screen out key evaluation indicators from numerous evaluation indicators, reduce the dimension of data analysis, improve the accuracy and computing efficiency of automatic evaluation/aided diagnosis models; and (7) synchronize all environmental disturbance events and A variety of neural sensing devices can accurately explore the impact of each environmental disturbance event on attention.
本發明以上各實施例彼此之間可以任意組合或者替換，從而 衍生更多之實施態樣，但皆不脫本發明所欲保護之範圍，茲進一步提供更多本發明實施例如次： The above embodiments of the present invention can be arbitrarily combined or replaced with each other, so that Deriving more implementation forms, but none of them departing from the scope of protection intended by the present invention, hereby further provide more embodiments of the present invention as follows:
實施例1：一種注意力缺陷虛擬實境自動化診斷方法，包含：透過穿戴式虛擬實境套件在虛擬實境中向受測者展示與實施至少一種注意力測試方法，並感測與收集該受測者因應該至少一種注意力測試方法之複數任務表現與複數量表表現；透過神經行為感測套件量測該受測者因應該至少一種注意力測試方法的複數神經行為表現；以及執行基於機器學習方法建構之注意力評估電腦輔助診斷模型，以根據該等量表表現、該等任務表現以及該等神經行為表現綜合評估該受測者的注意力缺陷程度。 Embodiment 1: a virtual reality automatic diagnosis method for attention deficit, comprising: displaying and implementing at least one attention test method to the subject in the virtual reality through the wearable virtual reality kit, and sensing and collecting the subject The multiple task performance and the multiple scale performance of the examinee in response to at least one attention test method; the multiple neurobehavioral performance of the subject in response to the at least one attention test method is measured through a neurobehavioral sensing kit; and performing machine-based The attention assessment computer-aided diagnosis model constructed by the learning method is used to comprehensively evaluate the degree of attention deficit of the subject according to the scale performance, the task performance and the neurobehavioral performance.
實施例2：如實施例1所述之注意力缺陷虛擬實境自動化診斷方法，還包含以下步驟其中之一：在該虛擬實境中置入注意力測試任務模組、干擾測試模組、以及該神經行為感測套件；經由該注意力測試任務模組指揮該虛擬實境對該受測者實施該至少一種注意力測試方法；經由該干擾測試模組在該至少一種注意力測試方法實施過程中，對該受測者發出至少一干擾事件；透過該神經行為感測套件量測該受測者與該虛擬實境進行互動過程所產生的複數神經行為表現；透過該穿戴式虛擬實境套件量測該受測者與該虛擬實境進行互動過程所產生的複數反應參數；以及在該至少一種注意力測試方法的實施過程同步實施至少一種評估量表診斷以及至少一種任務表現診斷，以基於該等反應參數分別評估該受測者因應該至少一種注意力測試方法的該等量表表現以及該等任務表現。
Embodiment 2: the virtual reality automatic diagnosis method for attention deficit as described in
實施例3：如實施例1所述之注意力缺陷虛擬實境自動化診斷方法，其中該注意力評估電腦輔助診斷模型係經由實施以下步驟其中之一
而建立：對該等量表表現、該等任務表現以及該等神經行為表現之複數評估指標，實施統計顯著性檢定，以計算該等評估指標的顯著檢定值；對該等評估指標，實施統計主成分分析法，以計算該等評估指標的特徵值；以及選出該特徵值與該顯著檢定值皆高於門檻值的該等評估指標作為訓練集資料而輸入機器學習方法，經由訓練該機器學習方法而建立該等評估指標相對於注意力缺陷的關聯模式、診斷模式或者決策模式，以建立該注意力評估電腦輔助診斷模型。
Embodiment 3: the virtual reality automatic diagnosis method for attention deficit as described in
實施例4：如實施例3所述之注意力缺陷虛擬實境自動化診斷方法，其中該機器學習方法係選自深度學習(deep learning)演算法、類神經網路(ANN)演算法、深度神經網路(DNN)演算法、遞歸神經網路(RNN)演算法、卷積神經網路(CNN)演算法、卷積遞歸神經網路(CRNN)演算法、生成對抗網路(GAN)演算法、深度信念網路(DBN)演算法、全卷積神經網路(FCN)演算法、多列卷積神經網路(MCNN)演算法、遞歸神經網路(RNN)演算法、雙向神經網路(BRNN)演算法、深層循環神經網路(DRNN)演算法、殘差網路(DRN)演算法、限制玻爾茲曼機(RBM)演算法、多層感知(MLP)演算法、自編碼器演算法、注意力網路演算法、梯度提升樹方法、強梯度提升機方法、弱梯度提升機方法、回歸樹方法、隨機森林方法、決策樹方法、弱學習方法、強學習方法、強投票方法、弱投票方法、支援向量機(support vector machines)分類器、關聯法則及其組合其中之一。 Embodiment 4: the virtual reality automatic diagnosis method for attention deficit as described in embodiment 3, wherein the machine learning method is selected from deep learning (deep learning) algorithm, neural network (ANN) algorithm, deep neural network Network (DNN) Algorithm, Recurrent Neural Network (RNN) Algorithm, Convolutional Neural Network (CNN) Algorithm, Convolutional Recurrent Neural Network (CRNN) Algorithm, Generative Adversarial Network (GAN) Algorithm , Deep Belief Network (DBN) Algorithm, Fully Convolutional Neural Network (FCN) Algorithm, Multi-column Convolutional Neural Network (MCNN) Algorithm, Recurrent Neural Network (RNN) Algorithm, Bidirectional Neural Network (BRNN) Algorithm, Deep Recurrent Neural Network (DRNN) Algorithm, Residual Network (DRN) Algorithm, Restricted Boltzmann Machine (RBM) Algorithm, Multilayer Perception (MLP) Algorithm, Autoencoder Algorithm, Attention Network Algorithm, Gradient Boosting Tree Method, Strong Gradient Boosting Machine Method, Weak Gradient Boosting Machine Method, Regression Tree Method, Random Forest Method, Decision Tree Method, Weak Learning Method, Strong Learning Method, Strong Voting Method, One of weak voting methods, support vector machines classifiers, association rules, and combinations thereof.
實施例5：如實施例1所述之注意力缺陷虛擬實境自動化診斷方法，其中該穿戴式虛擬實境套件還包含頭戴式虛擬實境顯示裝置、麥克風耳機組、控制器以及動作感測套組，該動作感測套組係選自光學位置追
蹤套組、紅外光位置追蹤套組、雷射光位置追蹤套組、超音波位置追蹤套組以及電磁式位置追蹤套組其中之一。
Embodiment 5: The virtual reality automatic diagnosis method for attention deficit as described in
實施例6：如實施例1所述之注意力缺陷虛擬實境自動化診斷方法，其中該神經行為感測套件還包含腦波感測器、眼球軌跡追蹤器、頭部動作感應器以及肢體動作感應器其中之一，該等神經行為表現係為腦波指標、眼球軌跡追蹤、頭部轉動以及肢體動作其中之一。
Embodiment 6: The virtual reality automatic diagnosis method for attention deficit as described in
實施例7：如實施例2所述之注意力缺陷虛擬實境自動化診斷方法，其中該至少一種注意力測試方法還包含音頻測驗、持續性表現測驗以及威斯康辛卡分類測驗，以分別檢測該受測者之選擇性注意力、持續性注意力以及執行功能。 Embodiment 7: the virtual reality automatic diagnosis method of attention deficit as described in embodiment 2, wherein this at least one attention test method also comprises audio frequency test, continuous performance test and Wisconsin card classification test, to detect this tested respectively selective attention, sustained attention, and executive function.
實施例8：如實施例2所述之注意力缺陷虛擬實境自動化診斷方法，其中該等量表表現係基於CONNERS評估量表、SNAP-IV評估量表以及Weiss’s評估量表其中之一而評估。 Embodiment 8: the virtual reality automatic diagnosis method for attention deficit as described in embodiment 2, wherein these scale performances are evaluated based on one of CONNERS evaluation scale, SNAP-IV evaluation scale and Weiss's evaluation scale .
實施例9：如實施例2所述之注意力缺陷虛擬實境自動化診斷方法，其中該干擾事件係選自教室外出現噪音之事件、教室內播放廣播之事件、打雷事件、教室內老師起立之事件、教室內有紙飛機飛過之事件、教室同學交頭接耳之事件、閃電事件、燃燒煙味事件、垃圾味事件、教室同學打哈欠之事件、教室外有人推門進入然後離關之事件、教室窗外出現嗚笛而過的消防車之事件以及教室內播放電視節目之事件其中之一。 Embodiment 9: The virtual reality automatic diagnosis method for attention deficit as described in embodiment 2, wherein the interference event is selected from the event of noise outside the classroom, the event of broadcasting in the classroom, the thunder event, and the teacher standing up in the classroom Events, events where paper airplanes fly by in the classroom, events where classmates whisper to each other, lightning events, events that smell like burning smoke, events that smell like garbage, events where classmates yawn, events where someone outside the classroom pushes the door to enter and then leaves, the classroom One of the events of a fire truck whistling past the window and the event of a TV program playing in the classroom.
實施例10：一種注意力缺陷虛擬實境自動化診斷系統，包含：穿戴式虛擬實境套件，經配置向受測者展示虛擬實境以向該受測者實施至少一種注意力測試方法，並感測與收集該受測者因應該至少一種注意 力測試方法之複數任務表現與複數量表表現；神經行為感測套件，經配置量測該受測者因應該至少一種注意力測試方法的複數神經行為表現；以及運算單元，經配置向該穿戴式套件提供該虛擬實境，並執行基於機器學習方法建構之注意力評估電腦輔助診斷模型，以根據該等量表表現、該等任務表現以及該等神經行為表現綜合評估該受測者的注意力缺陷程度。 Embodiment 10: a virtual reality automatic diagnosis system for attention deficit, comprising: a wearable virtual reality kit, configured to show a virtual reality to the subject to implement at least one attention testing method to the subject, and feel measure and collect the respondent should respond to at least one attention Multiple task performance and multiple scale performance of a force test method; a neurobehavioral sensing kit configured to measure a plurality of neurobehavioral performance of the subject in response to the at least one attention test method; and a computing unit configured to communicate with the wearable The formula kit provides the virtual reality, and implements the attention assessment computer-aided diagnosis model based on the machine learning method to comprehensively evaluate the subject's attention according to the scale performance, the task performance and the neurobehavioral performance. strength deficit.
本發明各實施例彼此之間可以任意組合或者替換，從而衍生更多之實施態樣，但皆不脫本發明所欲保護之範圍，本發明保護範圍之界定，悉以本發明申請專利範圍所記載者為準。 The various embodiments of the present invention can be combined or replaced arbitrarily with each other, thereby deriving more implementation forms, but none of them depart from the intended protection scope of the present invention, and the definition of the protection scope of the present invention is fully defined by the patent scope of the present invention application The recorder shall prevail.
200:本發明注意力缺陷虛擬實境自動化診斷方法 200: Automatic Diagnosis Method of Attention Deficit Virtual Reality in the Present Invention
201~208:實施步驟 201~208: Implementation steps
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW109135186A TWI793457B (en) | 2020-10-12 | 2020-10-12 | Virtual reality based automatic attention-deficit diagnosis method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW109135186A TWI793457B (en) | 2020-10-12 | 2020-10-12 | Virtual reality based automatic attention-deficit diagnosis method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW202215451A TW202215451A (en) | 2022-04-16 |
| TWI793457B true TWI793457B (en) | 2023-02-21 |
Family
ID=82197221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW109135186A TWI793457B (en) | 2020-10-12 | 2020-10-12 | Virtual reality based automatic attention-deficit diagnosis method and system |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI793457B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI819919B (en) * | 2022-12-08 | 2023-10-21 | 國立陽明交通大學 | Early assistive diagnosis system of adhd |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180177451A1 (en) * | 2013-10-09 | 2018-06-28 | Nedim T. SAHIN | Systems, environment and methods for identification and analysis of recurring transitory physiological states and events using a portable data collection device |
| TW201944429A (en) * | 2018-03-04 | 2019-11-16 | 美商阿奇力互動實驗室公司 | Cognitive screens, monitor and cognitive treatments targeting immune-mediated and neuro-degenerative disorders |
| TWI689334B (en) * | 2019-05-30 | 2020-04-01 | 弘光科技大學 | Virtual reality rehabilitation training system with replacement movement judgement |
-
2020
- 2020-10-12 TW TW109135186A patent/TWI793457B/en active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180177451A1 (en) * | 2013-10-09 | 2018-06-28 | Nedim T. SAHIN | Systems, environment and methods for identification and analysis of recurring transitory physiological states and events using a portable data collection device |
| TW201944429A (en) * | 2018-03-04 | 2019-11-16 | 美商阿奇力互動實驗室公司 | Cognitive screens, monitor and cognitive treatments targeting immune-mediated and neuro-degenerative disorders |
| TWI689334B (en) * | 2019-05-30 | 2020-04-01 | 弘光科技大學 | Virtual reality rehabilitation training system with replacement movement judgement |
Non-Patent Citations (2)
| Title |
|---|
| 專書 黃舒萱 國小ADHD兒童及一般兒童在敘事能力與執行功能之研究 國立屏東教育大學特殊教育學系碩士論文 102年6月;期刊 劉昱志;劉士愷;商志雍;林健禾;杜長齡;高淑芬 注意力缺陷過動症中文版 Swanson, Nolan, and Pelham, Version IV (SNAP -IV)量表之常模及信效度 台灣精神醫學 第20卷第4期 2006年 * |
| 期刊 劉昱志;劉士愷;商志雍;林健禾;杜長齡;高淑芬 注意力缺陷過動症中文版 Swanson, Nolan, and Pelham, Version IV (SNAP -IV)量表之常模及信效度 台灣精神醫學 第20卷第4期 2006年 |
Also Published As
| Publication number | Publication date |
|---|---|
| TW202215451A (en) | 2022-04-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108042147B (en) | Stimulus information compiling method for personality trait value test | |
| CN101797150B (en) | Computerized test apparatus and methods for quantifying psychological aspects of human responses to stimuli | |
| JP2010527642A (en) | Inspection and training of visual cognitive ability and cooperative behavior | |
| CN104921723B (en) | A kind of state of consciousness detecting system based on multi-mode brain-computer interface | |
| Scruggs et al. | Quantifying physical activity in first-through fourth-grade physical education via pedometry | |
| CN108922629A (en) | The screening and its application of brain function corelation behaviour normal form index | |
| CN111325109A (en) | Attention training method | |
| CN105105772A (en) | Stimulation information compilation method for testing cognitive competence value | |
| KR101332593B1 (en) | Norm-based Cognitive Measuring and Evaluation System | |
| Lee et al. | ADHD assessment and testing system design based on virtual reality | |
| Giannakos et al. | Monitoring children's learning through wearable eye-tracking: the case of a making-based coding activity | |
| Tan et al. | Virtual classroom: An ADHD assessment and diagnosis system based on virtual reality | |
| TWI793457B (en) | Virtual reality based automatic attention-deficit diagnosis method and system | |
| KR101118276B1 (en) | Ubiquitous-learning middleware device for generating study emotion index related to study concentration from bio-signal emotion index and context information | |
| Türkmen | The effects of taekwondo courses on multiple intelligence development–a case study on the 9th grade students | |
| KR20120097098A (en) | Ubiquitous-learning study guiding device for improving study efficiency based on study emotion index generated from bio-signal emotion index and context information | |
| Giannakos et al. | Sensing-based analytics in education: The rise of multimodal data enabled learning systems | |
| CN104992080B (en) | A kind of stimulus information preparation method of latent energy value test | |
| Zizzi et al. | Impact of a brief workshop on stages of change profiles in athletes | |
| Stavroulia et al. | A virtual reality environment simulating drug use in schools: effect on emotions and mood states | |
| Chessa et al. | Human-computer interaction approaches for the assessment and the practice of the cognitive capabilities of elderly people. | |
| TWI785329B (en) | Personal Temperament Potential Detection and Tracking System and Method | |
| Ali et al. | Development of Vision Screening Tool in Virtual Reality: A Usability Assessment Study | |
| Popescu et al. | Predictors of the choice of extracurricular sports activities for high school students | |
| Muna et al. | Design And Trial Selective Attention Computer-Based “Letvis” Program For Children With Attention Deficit Disorder |