WO2012121341A1 - 画像データ処理装置および経頭蓋磁気刺激装置 - Google Patents
画像データ処理装置および経頭蓋磁気刺激装置 Download PDFInfo
- Publication number
- WO2012121341A1 WO2012121341A1 PCT/JP2012/055995 JP2012055995W WO2012121341A1 WO 2012121341 A1 WO2012121341 A1 WO 2012121341A1 JP 2012055995 W JP2012055995 W JP 2012055995W WO 2012121341 A1 WO2012121341 A1 WO 2012121341A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- head
- subject
- dimensional
- magnetic stimulation
- Prior art date
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N2/00—Magnetotherapy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
- A61B5/004—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
- A61B5/0042—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the brain
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves
- A61B5/055—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1113—Local tracking of patients, e.g. in a hospital or private home
- A61B5/1114—Tracking parts of the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
- A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
- A61B5/7425—Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N2/00—Magnetotherapy
- A61N2/004—Magnetotherapy specially adapted for a specific therapy
- A61N2/006—Magnetotherapy specially adapted for a specific therapy for magnetic stimulation of nerve tissue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/003—Reconstruction from projections, e.g. tomography
- G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/593—Depth or shape recovery from multiple images from stereo images
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/06—Devices, other than using radiation, for detecting or locating foreign bodies ; determining position of probes within or on the body of the patient
- A61B5/061—Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
- A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N2/00—Magnetotherapy
- A61N2/02—Magnetotherapy using magnetic fields produced by coils, including single turn loops or electromagnets
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30016—Brain
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Definitions
- the present invention relates to an image data processing device and a transcranial magnetic stimulation device using the image data processing device.
- transcranial magnetic stimulation therapy In recent years, interest in transcranial magnetic stimulation therapy has increased as a treatment method for many patients with neurological diseases for which drug treatment is not always effective.
- treatment and / or alleviation of symptoms can be achieved by applying magnetic stimulation to a specific part of the brain (for example, nerve in the brain) by a magnetic field generation source placed on the scalp surface of the patient.
- a magnetic field generation source placed on the scalp surface of the patient.
- this is a relatively new treatment that is non-invasive and requires less burden on the patient. It is expected to spread.
- transcranial magnetic stimulation therapy As a specific method of such transcranial magnetic stimulation therapy, an electric current is passed through a coil located near the surface of the patient's scalp to locally generate a small pulsed magnetic field, and the principle of electromagnetic induction is used for intracranial A method is known in which an eddy current is generated in a brain to stimulate a nerve in the brain immediately below the coil (see, for example, Patent Document 1).
- Patent Document 1 it is confirmed that refractory neuropathic pain is effectively reduced by transcranial magnetic stimulation treatment performed by such a method, and more accurate local stimulation realizes a higher pain reduction effect.
- the optimal stimulation site varies slightly depending on the individual patient.
- transcranial magnetic stimulation therapy practitioners use the positioning function of the optical tracking system while referring to the three-dimensional information in the skull obtained by MRI images, It is possible to guide the treatment coil to accurately apply magnetic stimulation.
- infrared reflective markers are installed at a fixed position (for example, a bed on which a patient lies) associated with a patient's head and a treatment coil.
- the current position of the treatment coil is estimated from the positional relationship between the two obtained by detecting these markers, and the optimal stimulation site on the patient's head is referred to while referring to the three-dimensional information in the skull obtained from the MRI image.
- the treatment coil is guided. Accordingly, accurate alignment between the patient's head and the MRI image is required. For this reason, accurate alignment with the MRI image is performed by designating eyes, ears, nose, and the like using a calibration marker while the patient's head is fixed to the bed.
- the present invention is an image data processing method useful for reducing the burden on the patient and the troublesomeness of the practitioner when performing transcranial magnetic stimulation therapy, and a method using such an image data processing method.
- the basic object is to provide a cranial magnetic stimulation device.
- the image data processing apparatus includes: a) storage means for storing a 3D MRI image of the subject's head that has been captured in advance; and b) a 3D appearance image of the subject's head.
- a three-dimensional appearance image generating means for generating, c) an image generating means for aligning the three-dimensional MRI image and the three-dimensional appearance image, and generating the aligned subject head three-dimensional image, and d) A post-movement image generating means for generating the aligned post-movement subject head three-dimensional image when the subject head moves, and e) a specific part of the subject head on the three-dimensional MRI image
- An operation object image generating means for generating an operation object image indicating the current position of the operation object to be moved to maintain the positional relationship; and f) the subject head three-dimensional image after the movement and the operation object Object image in the same image
- display means for displaying, that is obtained by it said.
- the image data processing apparatus is the position of the first image that is a three-dimensional MRI image of the subject's head and the second image that is a three-dimensional appearance image of the subject's head.
- An image data processing apparatus for performing alignment a) For each of N points a i included in the first image, from among a plurality of points b j included in the second image, selection means for selecting each point m i that satisfy the determined conditions, b) at each point m i selected by the selection means, from the points contained in the first image, the corresponding said second
- the error function E (R, t) consisting of a predetermined calculation procedure using the rotation matrix R and the translation vector t is minimized.
- the rotation matrix R and the translation vector Parameter determining means for determining the torque t, and c) the points a i are converted to the rotation matrix R and the average movement vector t until the value of the error function E (R, t) is equal to or less than a predetermined threshold value.
- the data processing means to perform said rotation matrix R and the translation vector t determine the by selecting said parameter determining means for each point m i by said selection means It is characterized by comprising.
- the selection unit may select each point m i having the smallest Euclidean distance from the plurality of points b j .
- an image data processing device for tracking the position and orientation of a subject's head, and a) generates a three-dimensional appearance image of the subject's head.
- a post-movement image generating means for generating a three-dimensional appearance image of the later subject's head, and d) moving the template image on the three-dimensional image of the subject's head after the movement,
- a feature region determining means for determining a position where the correlation is maximized as a position of the feature region after movement; and e) each point included in the feature region before the movement is included in each feature region after the movement.
- the error function E (R, t) may satisfy the following equation (Equation 2).
- N is the number of feature points that are points included in the feature region and is a number of 2 or more
- xi is a three-dimensional position of each feature point included in the initial head image
- y i is a three-dimensional position of each feature point included in the moving back head image
- w i is a weighting coefficient of each feature point.
- the three-dimensional appearance image of the subject's head may be generated using parallax of images taken from a plurality of viewpoints, or light or ultrasonic waves from one viewpoint. It may be generated using the arrival time.
- the transcranial magnetic stimulation apparatus is a transcranial magnetic stimulation apparatus for applying a magnetic stimulation to a specific part in the subject's head using a magnetic field generating means outside the head.
- the magnetic field generating means configured to be able to change the position and posture according to the operation;
- a storage means for storing a pre-captured three-dimensional MRI image of the subject's head;
- the above 3D appearance image generating means for generating a 3D appearance image of the subject's head; d) aligning the 3D MRI image and the 3D appearance image, and obtaining the aligned subject head 3D image;
- the transcranial magnetic stimulation apparatus is a transcranial magnetic stimulation apparatus for applying magnetic stimulation to a specific part in the subject's head using magnetic field generation means outside the head.
- an image data processing device for aligning a first image that is a three-dimensional MRI image of the subject's head and a second image that is a three-dimensional appearance image of the subject's head.
- the image data processing apparatus a) determines a predetermined condition from a plurality of points b j included in the second image for each of the N points a i included in the first image.
- selection means for selecting each point m i satisfying, b) at each point m i selected by the selection means, from the points contained in the first image, included in the corresponding said second image Rotation line as a parameter for rigid transformation to each point Parameter determining means for determining the rotation matrix R and the translation vector t so that the value of the error function E (R, t) comprising a predetermined calculation procedure using the column R and the translation vector t is minimized.
- the selection unit may select each point m i having the smallest Euclidean distance from the plurality of points b j .
- the transcranial magnetic stimulation device is a transcranial magnetic stimulation device for applying magnetic stimulation to a specific part in the subject's head using magnetic field generation means outside the head.
- an image data processing device for tracking the position and orientation of the subject's head, the image data processing device comprising: a) an image generation means for generating a three-dimensional appearance image of the subject's head; B) Extraction and storage means for extracting at least one feature region from the three-dimensional appearance image and storing it as a three-dimensional template image; and c) the subject head after movement when the subject head moves.
- a post-movement image generating means for generating a three-dimensional appearance image of d)
- the template image is moved on the three-dimensional image of the subject head after the movement, and the correlation between the two image data is maximized.
- the position after moving A feature region determining means for determining the position of the region; and e) a rotation matrix R as a parameter for rigidly transforming each point included in the feature region before movement to each point included in the feature region after movement.
- parameter determination means for determining the rotation matrix R and the translation vector t so that the value of the error function E (R, t) consisting of a predetermined calculation procedure using the translation vector t is minimized. , Is characterized by that.
- the error function E (R, t) may satisfy the following equation (Equation 4).
- N is the number of feature points that are points included in the feature region and is a number of 2 or more
- xi is a three-dimensional position of each feature point included in the initial head image
- y i is a three-dimensional position of each feature point included in the moving back head image
- w i is a weighting coefficient of each feature point.
- the three-dimensional appearance image of the subject's head may be generated using parallax of images taken from a plurality of viewpoints, or light from one viewpoint. Or you may produce
- the three-dimensional MRI image of the subject's head is aligned with the appearance image of the subject's head, and the aligned subject's head three-dimensional image is generated.
- the troublesomeness at the time of performing calibration in the initial state in which the subject head is fixed to the bed and accurately aligned with the MRI image is reduced.
- the 3D MRI image of the subject's head is aligned with the appearance image of the subject's head after the movement, a 3D MRI image of the subject's head after the movement from the initial state is automatically obtained.
- the operation object is positioned at a predetermined position with respect to a specific part on the three-dimensional image of the subject's head. The moving operation when leading to the relationship can be performed more easily.
- the 1st image which is a 3D MRI image of a test subject's head and the 2nd image which is a 3D appearance image of the said test subject's head For each of N points a i included in the first image, each satisfying a predetermined condition from among a plurality of points b j included in the second image.
- the rotation matrix R and the translation vector t are set so that the value of the error function E (R, t) consisting of a predetermined calculation procedure using the rotation matrix R and the translation vector t as parameters for Parameter decision to decide.
- the image data processing apparatus when tracking the position and orientation of the subject's head, when the subject's head moves, the three-dimensional appearance of the subject's head after the movement An image is generated, the template image is moved on the three-dimensional image of the subject's head after the movement, and the position where the correlation between the two image data is maximized is defined as the position of the feature region after the movement.
- the rotation matrix R and the translation vector t are used as parameters for rigidly transforming each point included in the feature region before movement to each point included in the feature region after movement.
- the three-dimensional MRI image of the subject's head is aligned with the appearance image of the subject's head, and the aligned subject's head 3
- the troublesomeness in performing the calibration in the initial state in which the subject head is fixed to the bed and accurately aligned with the MRI image is reduced.
- the 3D MRI image of the subject's head is aligned with the appearance image of the subject's head after the movement, a 3D MRI image of the subject's head after the movement from the initial state is automatically obtained.
- the operation object is positioned at a predetermined position with respect to a specific part on the three-dimensional image of the subject's head. The moving operation when leading to the relationship can be performed more easily.
- the first image that is a three-dimensional MRI image of the subject's head and the second that is a three-dimensional appearance image of the subject's head.
- An image data processing device for performing alignment with an image is included, and the image data processing device includes N points a i included in the first image in the second image.
- the transcranial magnetic stimulation device has an image data processing device for tracking the position and orientation of the head of the subject, When the subject head moves, the template image is moved on the after-movement image generating means for generating a three-dimensional appearance image of the subject head after movement, and the three-dimensional image of the subject head after movement.
- a feature region determining means for determining the position where the mutual correlation between the two image data is maximum as the position of the feature region after the movement, and each point included in the feature region before the movement, As a parameter for rigid body transformation to each point included in the feature region, the value of the error function E (R, t) consisting of a predetermined calculation procedure using the rotation matrix R and the translation vector t is minimized.
- FIG. 1 is a schematic configuration diagram of a transcranial magnetic stimulation device according to an embodiment of the present invention. It is a part of flowchart for demonstrating the flow of the magnetic stimulation treatment performed using the apparatus of FIG. It is a part of flowchart for demonstrating the flow of the said magnetic stimulation treatment. It is a figure for demonstrating the process which performs positioning of a head 3D MRI image and a head 3D appearance image using the apparatus of FIG. It is a figure which shows the operation
- FIG. 1 performs on a face, and detecting the position of a face more correctly. It is a figure for demonstrating the block matching parallelization process which the apparatus of FIG. 1 performs. It is a figure which shows an example of the image of the magnetic flux which the brain surface which the image display means of the apparatus of FIG. 1 displays, and a treatment coil produce
- FIG. 1 is an explanatory diagram schematically showing the outline of the configuration of the transcranial magnetic stimulation apparatus according to the present embodiment.
- the transcranial magnetic stimulation apparatus 1 is for performing treatment by applying magnetic stimulation to a specific site (optimal stimulation site) in the head 2h of a subject 2 (for example, a patient or a test examinee).
- a transcranial magnetic stimulation device 1 (hereinafter simply referred to as “device” as appropriate) includes an image monitor unit 10, a device body unit 20, a magnetic stimulation coil unit 30, as main components.
- a stereo camera 40 and a projector 50 are provided.
- the stereo camera 40 included in the device 1 is an example for obtaining the spatial coordinate information of the object in the three-dimensional space. According to another aspect as will be described later, It is also possible to obtain the spatial coordinate information of the face and the magnetic stimulation coil unit 30.
- the image monitor unit 10 includes a monitor screen such as a CRT screen or a liquid crystal screen, and has a function of displaying image information.
- a monitor screen such as a CRT screen or a liquid crystal screen
- an image display unit of a personal computer may be used.
- the practitioner (not shown) of the magnetic stimulation treatment looks at the three-dimensional MRI image of the subject 2 displayed on the image monitor unit 10 and the position and posture of the magnetic stimulation coil unit 30 in the space, and the magnetic flux for magnetic stimulation.
- the magnetic stimulation coil unit 30 is changed in position and posture so as to correctly reach the optimal stimulation site, and appropriate magnetic stimulation treatment is performed.
- the image monitor unit 10 may display on the screen a graphic corresponding to the magnetic flux irradiated from the magnetic stimulation coil unit 30 (for example, refer to an elongated rectangular shape in FIG. 14 described later).
- the apparatus main body unit 20 is configured to hold the following components integrally or partly separately, and each held configuration includes the following. Each of these components is divided into a plurality of components for convenience of explanation. Needless to say, these components may be realized as execution software installed in a personal computer.
- the image display control unit 21 included in the apparatus main body unit 20 holds a pre-photographed 3D MRI image of the head 2h of the subject 2 in a readable manner and displays various images to be displayed on the image monitor unit 10. Control is performed.
- the three-dimensional MRI image may be readable and held in a memory device attached to the image display control unit 21 or attached to the outside of the apparatus body unit 20.
- the magnetic stimulation coil control unit 22 controls on / off of the magnetic flux generation current applied to the magnetic stimulation coil unit 30 and the current. Further, the three-dimensional information generation unit 23 uses the parallax of a plurality of images (for example, two in the present embodiment) input from the stereo camera 40, and in the space between the subject head 2h and the magnetic stimulation coil unit 30. In addition to generating position and orientation information, control of the random dot pattern projection operation performed by the projector 50 is performed.
- the image display control unit 21, the magnetic stimulation coil control unit 22, and the three-dimensional information generation unit 23 described above are each configured with necessary control circuits, arithmetic circuits, and the like. Specific operations of the image display control unit 21, the magnetic stimulation coil control unit 22, and the three-dimensional information generation unit 23 will be described later.
- control by the apparatus may be realized as execution software installed in a personal computer.
- the apparatus is programmed by a programmed computer or recorded on a recording medium. Is executed by a computer that reads and executes.
- a program for executing necessary control and calculation to be described later using a computer and further, at least a part of data necessary for such control and calculation are linked to be communicable with the apparatus, for example.
- the computer is used to perform necessary control and computation by downloading necessary programs and data each time in response to a request from the device side. You can also.
- the magnetic stimulation coil unit 30 has an operation unit (not shown) after the practitioner holds the grip portion 31 and freely changes the position and orientation (posture) in a predetermined range of space and appropriately approaches the optimal stimulation site.
- the magnetic stimulation treatment is performed by applying a magnetic flux of a predetermined intensity to generate an induced current in the brain of the subject's head 2h and applying magnetic stimulation to the optimal stimulation site. It is for performing.
- the magnetic stimulation coil unit 30 includes a magnetic stimulation coil 33 (hereinafter referred to as “treatment coil” or simply “coil” as appropriate) and a stereo camera 40 that generates a parallax image.
- the marker unit 32 for generating information on the position and posture of the coil unit 30 (that is, the position and posture of the treatment coil 33).
- the marker unit 32 has a specific graphic pattern.
- the posture of the therapeutic coil means the direction and angle of the therapeutic coil 33
- the direction of the therapeutic coil means the coil 33 on the scalp surface of the subject's head 2 h.
- the “angle of the therapeutic coil” means the angle formed by the normal of the scalp surface of the subject's head 2 h and the magnetic field direction of the coil 33.
- the stereo camera 40 detects the position and orientation of the subject head 2h and the magnetic stimulation coil unit 30 in the space using the parallax between the two images output by the imaging cameras 41 and 42 arranged in a pair on the left and right. In addition, these subjects are photographed from the left and right imaging cameras 41 and 42, and respective images are output.
- the projector 50 is for projecting a random dot pattern on the surface of the subject's head 2h to serve as an extraction point for image processing.
- the inventor of the present application analyzed the requirements that the transcranial magnetic stimulation apparatus 1 should have, and as a result, obtained the following knowledge.
- the position and orientation (posture) of the subject's head 2 h that changes from moment to moment are analyzed, and MRI data and head images are always analyzed. Processing to match is required.
- a three-dimensional face image intended for the face of the subject 2 including many easy-to-designate feature points is used as the three-dimensional appearance image of the head 2h of the subject 2.
- the magnetic stimulation coil unit 30 it is necessary to analyze the position and posture of the magnetic stimulation coil unit 30 for performing magnetic stimulation and always know which region of the brain of the subject's head 2h is to be stimulated. Furthermore, the practitioner (such as a doctor) needs to perform stimulation while referring to intracranial information (an image of the brain epidermis) based on a three-dimensional MRI image, so the information on the brain surface, the posture of the head (face) and An interface that displays the orientation of the magnetic stimulation coil unit 30 in an easy-to-understand manner is also necessary.
- intracranial information an image of the brain epidermis
- a three-dimensional MRI image so the information on the brain surface, the posture of the head (face) and An interface that displays the orientation of the magnetic stimulation coil unit 30 in an easy-to-understand manner is also necessary.
- the requirements to be satisfied in constructing the transcranial magnetic stimulation device 1 are: (1) It is provided with a posture collation function between the three-dimensional MRI measurement data and the current posture of the head (face) of the subject 2. (2) Provide a function for tracking the posture of the head (face) of the subject 2 in real time. (3) Provide a function of tracking the position and posture of the magnetic stimulation coil unit 30 in real time. (4) Provide an interface function that makes it easy to grasp the stimulation status, such as magnetic stimulation points on the brain surface. It is important to realize these four requirements with sufficient accuracy, operability and economy.
- the transcranial magnetic stimulation apparatus 1 projects a random dot on the spatial coordinate information acquisition unit of the object in the three-dimensional space exemplified by the stereo camera 40 and the subject head 2h, Using an optical device such as an image projector (projector 50) as a measurement marker, the subject 2 automatically measures the head posture and facial shape change just by taking a resting posture at the time of medical examination.
- a device for visualizing the state of magnetic stimulation by the coil 33 is used.
- the positions of the subject head 2h and the treatment coil 33 can be grasped.
- the limit is expanded to a range where the position in the three-dimensional space can be grasped, such as the imaging limit area of the stereo camera 40, and the limit where the subject 2 lies down and the therapeutic coil 33 can be moved is expanded.
- the convenience of the treatment is improved, and the burden on the subject 2 is also reduced.
- etc. The test subject 2 is not restrained or wearing a fixing tool, and a burden is eased.
- FIG. 2A and FIG. 2B are flowcharts for explaining the flow of magnetic stimulation treatment using the apparatus 1 including the operation of guiding the treatment coil 33 to the optimum position.
- an image frame (initial image frame) including the subject head 2h and the magnetic stimulation coil unit 30 is acquired using, for example, the stereo camera 40, and then the step In # 2, the three-dimensional face image of the subject 2 in the initial state obtained based on the initial image frame and the subject head 2h held in the image display control unit 21 of the apparatus main body unit 20 so as to be readable.
- the initial alignment with the three-dimensional MRI image is performed. This process corresponds to the aforementioned “(I) initial alignment” process.
- the MRI data of the patient's head to be treated Matching with patient face shape data on the same coordinate system. Details of the initial alignment process will be described later.
- a three-dimensional face image for the face of the subject 2 including many feature points that can be easily specified is used as the three-dimensional appearance image of the head 2h of the subject 2.
- the acquisition of the image frame is performed every moment using the stereo camera 40 (Step # 3), and based on the acquired current image frame, A three-dimensional face image of the current subject 2 is acquired (step # 4). That is, the posture of the subject head 2h is tracked in real time. Then, the three-dimensional position and posture of the three-dimensional MRI image of the subject head 2h are collated with the current three-dimensional face image of the subject 2 (step # 5). Thereby, the current three-dimensional MRI image of the subject's head 2h is obtained.
- Step # 2 the result of the initial alignment in step # 2 is reflected in the real-time tracking result of the posture of the subject head 2h, so that the current 3D face image and 3D MRI image of the subject 2 are correctly positioned. And can be superimposed in posture.
- the processes of Step # 4 and Step # 5 correspond to the above-described “(II) Tracking the posture of the subject's head”.
- the marker information of the current therapeutic coil 33 (that is, the tracking marker attached to the magnetic stimulation coil unit 30).
- Image information of the unit 32) is acquired (step # 6).
- the position and orientation of the coil 33 are tracked.
- the current three-dimensional position and orientation of the marker unit 32 are calculated (step # 7), and the current three-dimensional position and orientation of the coil 33 (preferably, the three-dimensional position of the magnetic field). And direction) are calculated (step # 8).
- Steps # 6 to # 8 correspond to the above-described “(III) treatment coil tracking” step.
- step # 9 Based on the result of step # 5 and the result of step # 8, at least the current 3D MRI image of the subject's head 2h and the current 3D position and posture of the coil 33 are more preferably In addition to these, the current face image and the current three-dimensional position and direction of the magnetic field are displayed in the three-dimensional image representing the same space (step # 9).
- the process of step # 9 corresponds to the “(IV) tracking result display” process described above.
- the three-dimensional position and direction of the magnetic field are displayed on the screen using a figure corresponding to the magnetic flux irradiated by the therapeutic coil 33 (for example, refer to an elongated rectangular shape in FIG. 14 described later). can do.
- the current position and posture of the therapeutic coil 33 that is, the therapeutic magnetic flux is determined by the subject. Which part of the brain surface of 2 is directed can be displayed.
- the series of steps from Step # 3 to Step # 9 are continuously and repeatedly executed until the magnetic stimulation treatment is finished and the apparatus 1 is stopped.
- the coil 33 is moved to guide to the optimal stimulation position and posture (step # 10), and the current three-dimensional position of the coil 33 and It is determined whether or not the posture (preferably, the three-dimensional position and direction of the magnetic field) has reached the optimal position (position corresponding to the optimal stimulation site of the subject's head 2h) and posture (step # 11).
- the optimum position and posture are reached (step # 11: YES)
- magnetic stimulation treatment using the coil 33 is performed (step # 12). That is, the practitioner operates the magnetic stimulation coil control unit 22 to apply a magnetic flux having a predetermined intensity from the therapeutic coil 33 to generate an induced current in the brain of the subject's head 2h, and magnetize the optimal stimulation site. Add stimulus.
- Step # 13: NO the magnetic stimulation treatment is continuously performed and the magnetic stimulation treatment is completed.
- Step # 13: YES the operation of the device 1 is stopped. In this way, a series of steps from Step # 3 to Step # 13 is continuously executed repeatedly until the treatment is finished after the treatment is finished.
- Head MRI image data obtained by an MRI apparatus installed in a medical institution prior to magnetic stimulation treatment, and stereo measurement (stereoscopic measurement using parallax) by a stereo camera 40 which is one exemplary embodiment of the apparatus 1 ) Is measured with a different measuring machine and with different patient postures, and when three-dimensionally displayed on the same coordinate system, a deviation occurs between the two data (FIG. 3 (a)). )reference). For this reason, it is necessary to match these two data.
- the two data after alignment are shown in FIG. This process is called alignment, and corresponds to obtaining a 3-by-3 rotation matrix R and a three-dimensional translation vector t, which are rigid body transformation parameters that determine the posture of each data.
- an ICP (Iterative Closest Point) algorithm is used as this alignment method.
- This algorithm is a technique for obtaining a rigid transformation parameter that minimizes the distance between corresponding points by iterative calculation. By using this method, it is possible to perform alignment with high accuracy without requiring correspondence between measurement data and prior calibration between the measurement apparatus and the target object.
- Initial alignment is performed by executing the processes in the following order.
- Reading of MRI data (2) Capturing a face using two cameras (left camera 41, right camera 42) (3) Adaboost (described later) from images obtained from left and right cameras 41, 42 The used face detection is performed, and the face area in the image is extracted. (4) Perform stereo measurement on the face area and measure the face shape. (5) Align the MRI data and the face shape data obtained by the stereo measurement using the ICP algorithm.
- Stereo measurement which is one aspect of the three-dimensional position detection method used by the apparatus 1 will be described.
- Stereo measurement is a kind of optical three-dimensional shape measurement technique, in which a measurement object is photographed with two cameras arranged on the left and right sides, and a three-dimensional position is estimated from the parallax information by triangulation. is there.
- Stereo measurement requires two processes: (a) search for corresponding points and (b) calculation of three-dimensional positions.
- FIG. 4 shows an example of two left and right images used in stereo measurement. Since the stereo camera 40 used in the present embodiment is parallel stereo (the optical axis of the right camera 42 and the optical axis of the left camera 41 are parallel), the corresponding points shift only in the horizontal direction. Therefore, the corresponding point search need only consider the horizontal direction, and all the corresponding points of the right eye image viewed from the left eye image are on the left side of the left eye image.
- the input image is considered as a two-dimensional array having respective pixel values, and a small region centered on the target pixel of the left image is overlapped while moving in the right image, and the difference in pixel value is taken.
- This is a method in which a region having the smallest sum of squares of differences (SSD) is used as a corresponding point.
- FIG. 5 illustrates the relationship of the three-dimensional position between the parallel stereo and the measurement target.
- the three-dimensional position of the gazing point can be calculated by the following equation (Equation 6).
- B is the distance between the cameras
- f is the focal length of each camera. Since the values of B and f are known at the time of measurement, the three-dimensional position of the gazing point can be calculated by using the parallax d obtained by the corresponding point search.
- ⁇ Face detection by Adaboost method Only the three-dimensional data on the face surface is used for alignment between the MRI data and the data obtained by stereo measurement. For MRI data, only necessary areas are extracted in advance. On the other hand, for stereo measurement data, a face area is detected from an image obtained by the camera 40, and three-dimensional data of the area is used.
- object detection using the Haar-like feature amount, which is an image feature amount, and the Adaboost algorithm, which is a learning algorithm is used as face extraction processing. This object detection process was improved by Rainer Lienhart et al. Based on research on object detection such as Paul Viola (Paul Viola and Michael Jones: “Object Detection using a Boosted Cascade of Simple”, IEEE CVPR, 2001). Rainer Lienhart and Jochen Maydt: “An Extended Set of Haar-lide Feature for Rapid Object Detection”, IEEE ICIP 2002, vol. 1, pp. 900-903 (2002)), can detect objects at high speed.
- ICP Intelligent Closest Point
- An ICP algorithm is used as a method for aligning MRI data and data obtained by stereo measurement.
- the ICP algorithm is a method proposed by Besl et al. In 1992 (PJ Best and ND McKay: “A Method for Registration of 3-D Shapes”, IEEE Trans. Pattern Anal. Machine Intell, vol. 14, No. 2, pp. 239-256 (1992-2)), which is a technique for obtaining a rigid transformation parameter that minimizes the distance between corresponding points by iterative calculation.
- Equation 7 The Euclidean distance d between the two points r 1 and r 2 in the three-dimensional space can be expressed as the following equation (Equation 7).
- the distance between the point a i included in the point group A and the point group B is defined as the distance from the closest point among the points included in the point group B (see the following equation (Equation 9) and FIG. 7). ), A distance d (a i , B) between each point a i of the point group A and the point group B is obtained.
- the alignment rigid body parameter can be obtained by the following procedure.
- (I) The closest point m i with the point group B at each point a i of the point group A is obtained.
- (Ii) A rigid transformation parameter that minimizes the error E is obtained.
- (Iii) The point group A is converted using the obtained parameters (R, t).
- (Iv) If the error E is less than or equal to the threshold value, the iterative calculation is terminated. In other cases, the process returns to (i) and the same steps are repeated.
- the method for determining the rigid body transformation parameter described above is merely an example, and the point at which the distance minimum is used as the starting point of the approximate calculation and the error calculation method shown in Equation 5 are converted to other methods. It is possible. Any other method may be used as long as the degree of coincidence of the position and orientation (6 degrees of freedom) of the rigid body in the three-dimensional space, that is, the magnitude of the error can be evaluated numerically. The same applies to the following description.
- the parallax of the stereo camera 40 that is, the parallax of images taken from a plurality of viewpoints, is used as a method for obtaining positional information in the three-dimensional space of the face of the subject 2 and the magnetic stimulation coil unit 30.
- the present invention is not limited to such a method, and the acquisition of the position information can be realized in another aspect.
- a light projecting unit such as a projector or a laser irradiating unit and an image capturing unit having only one viewpoint such as a video camera (not a system using parallax of images taken from a plurality of viewpoints)
- the light emitted from the light projecting means is reflected by the object, and the reflected light is captured by the image capturing means. From the angle information of each optical axis to the object by the same triangulation principle as described above. As a result, the spatial coordinates of the light reflection point of the object can be obtained.
- a laser radar measures the time when the laser projection light is reflected by the object and returns to the light receiving sensor to know the distance of the object
- an ultrasonic rangefinder also the projected ultrasonic wave.
- the distance information and the projection angle of the projection light or the ultrasonic wave are combined with a distance meter that knows the distance of the point of the object (such as the time when the sound wave returns) and a scanning means that scans the measurement point. From this information, spatial coordinate information of each point of the object can be obtained.
- a subject to be measured is photographed by an imaging unit using a solid-state imaging device such as a CCD, and light projected from the light projecting unit to the subject is reflected to each of the solid-state imaging devices.
- a solid-state imaging device such as a CCD
- the time from the projection until reaching the pixel is detected by the phase difference between the projection light and the pixel arrival light.
- Devices for calculating the distance between subject points imaged in one pixel are commercially available. For example, MESA Imaging Corp. in Zurich, Switzerland has introduced a device named “SR4000” to the market, and a related technology is disclosed in, for example, JP 2009-515147 A Yes.
- the method according to the “other aspects (second to fourth aspects)” as described above is not limited to the method using the parallax of the stereo camera described above, and can be used to acquire position information in the three-dimensional space. The same applies to the following description.
- the MRI data and the data obtained by stereo measurement are in a state in which the initial states are identical in the same coordinate system by using the ICP algorithm.
- a three-dimensional face image for the face of the subject 2 including many feature points that can be easily specified is used as the three-dimensional appearance image of the head 2h of the subject 2. Accordingly, in this case, the “posture of the subject head 2 h” can also be expressed as the “face posture” of the subject 2.
- the ICP algorithm was used to obtain the rigid body transformation parameters for alignment. According to this method, points with unknown correspondence can be matched with high accuracy. However, since it requires a large number of iterative calculations and processing takes time, real-time processing is performed after initial alignment. It is not suitable for the tracking process of the face posture performed in.
- the calculation amount is greatly reduced compared with the case of using the ICP algorithm, and the calculation is performed.
- the time and cost required can be reduced. Therefore, in the present embodiment, 7 points of the eye corners and eyes, the mouth (both ends), and the nasal head of both eyes are designated as the facial features, and the rigid transformation parameters are calculated by tracking the facial features using template matching. I tried to do it.
- the face feature area was characterized by having a pattern suitable for tracking and characteristic in the face image.
- a face image in an initial posture is acquired using a stereo camera.
- (2) Designate each feature area (both eye corners and eyes, mouth (both ends), nasal head), and store an image (template) and three-dimensional coordinates of each area.
- (3) A face image in the current posture is acquired using a stereo camera.
- (4) Using template matching, the feature point positions in the left and right images are examined and their three-dimensional coordinates are obtained.
- the change from the initial posture is obtained by the steepest descent method (that is, the rigid body transformation for fitting the measured value in the initial posture to the current posture).
- the processes (1) and (2) are initialization processes and need only be performed once at the start of tracking.
- the face tracking performed in real time is performed by repeating the processes (3) to (5).
- ⁇ Template matching> The process of associating in which part in another image a certain image (template) exists is called template matching.
- template matching As shown in FIG. 8, this is a method of preparing an image called a template in advance and superimposing it on the target image while moving it, and examining the correlation between the template and the target image.
- a correlation coefficient C is used as a scale for measuring the difference between two images.
- This correlation function C is expressed by the following equation (Equation 11), where I (m, n) is the target image and T (m, n) (image size: M ⁇ N) is the template image.
- the correlation between the images increases as the value of the correlation coefficient C increases, and the region having the largest correlation coefficient value in the image is set as a corresponding region.
- the initial posture is set as the initial posture, and the three-dimensional coordinates of the feature region are acquired by stereo vision.
- the face feature area is searched by template matching for the current frame, and the three-dimensional coordinates of each area are obtained as a result of stereo viewing.
- the problem of obtaining the position and orientation of the head from the measurement result of the three-dimensional position of each feature region is the rotation matrix R that is a rigid transformation parameter that minimizes the error function E shown in the following equation (Equation 10): This results in the problem of obtaining the translation vector t.
- the equation (Equation 12) is the same as the equations (Equation 2) and (Equation 4) described above.
- N is the number of feature points
- xi is the three-dimensional position of each feature point in the initial posture
- yi is the three-dimensional position of each feature region in the current face posture.
- wi is a weighting coefficient for each feature point, and the product of the respective correlation coefficients obtained when the feature area is detected from the left and right images using template matching is used as this coefficient.
- this rigid body transformation is obtained by using a so-called steepest descent method.
- FIGS. 10A to 10F are a series of explanatory diagrams showing specific processing of marker recognition. Specific processing of marker recognition when the marker shown in FIG. 10A is used is as follows.
- (I) The image from the camera is binarized and the dark part of the image is searched. : An image input from the camera (FIG. 10B) is displayed using a threshold value, a region brighter than the threshold value is displayed in black, and a dark region is displayed in white (FIG. 10C).
- (Ii) Search and label closed areas in dark areas. : Search the closed region for the white region in the binarized image. Further, a number (label) is assigned to each closed area so as to be distinguished (labeling process). In FIG. 10 (d), the state of the distinction of the closed region is shown by the color difference.
- (Iii) The number of vertices is examined in each closed region, and a region having four vertices is determined to be a tetragon. : Check the number of vertices in each closed area, determine that the area with the number of vertices of 4 is a tetragon, and use it as a marker candidate area (see FIG. 10E). At this time, a region where the area of the closed region is very small or very large is excluded. (Iv) Simplify the image within the rectangle. : Using the affine transformation on the quadrangular area, the area is corrected to be square (see FIG. 10F). (V) Comparison between simplified image and registered pattern: A pixel comparison between the simplified image and the registered marker is performed to calculate an error. The area with the smallest error among all the square areas is determined as the marker area.
- the current head posture is matched with the three-dimensional brain model, and the transformed three-dimensional brain model
- the display of the stimulation point by the therapeutic coil 33 displays a prism that penetrates the center of the coil so that the current stimulation site can be determined from the relationship between the prism and the brain surface (see FIG. 10 described later).
- FIG. 10 it can be seen that the position and direction of the magnetic flux for treatment are displayed so that the relative relationship can be grasped with respect to the three-dimensional MRI image of the brain surface of the subject 2 or the head 2h. .
- transcranial magnetic stimulation apparatus 1 Next, specific examples of the transcranial magnetic stimulation apparatus 1 according to the embodiment of the present invention described above will be described.
- the present embodiment was realized using the equipment and development language shown in Table 1 below.
- initial alignment was performed according to the following processing procedure.
- a three-dimensional model of a face acquired from MRI data is read.
- Random dots are projected from the projector to the patient, and captured from the left and right cameras with an image size of 640 ⁇ 480 pixels.
- Face recognition is performed on the left and right images, and a face area in the image is detected. This process is realized by using the function of Open CV.
- Edge detection is performed on the left and right images using a 3 ⁇ 3 pixel Sobel filter, and block matching is performed on 7 pixels around the edge. For the blow-out matching, a block of 11 ⁇ 11 pixels was used.
- processing is parallelized using a GPU, enabling high-speed search for corresponding points.
- a three-dimensional position is obtained by the triangulation method from the parallax information obtained by block matching.
- the face shape measured and the face shape obtained from the MRI data are aligned using the ICP algorithm.
- the ICP algorithm is implemented using a function of VTK (Visualization Tool Kit), and a rotation matrix R and a translation vector t are obtained by executing the ICP algorithm.
- the result of the head scan is obtained as a set of cross-sectional images as shown in FIG.
- the MRI image used in the present example has a size of, for example, 256 ⁇ 256 pixels and is composed of 130 cross-sectional images.
- the slice interval of the cross section is, for example, 1.40 mm, and the size of one pixel is, for example, 0.98 ⁇ 0.98 mm.
- the image is scanned from the maximum value in the x direction.
- a pixel having a luminance value of 30 or more was acquired as the face surface.
- the acquired pixel is a slice image number N (0 ⁇ N ⁇ 130), and the acquired pixel value I (i, j) is a slice interval (for example, 1.40 mm) and a pixel size (for example, 0.98 ⁇ 0). .98 mm) can be converted into three-dimensional coordinates (X, Y, Z) T as in the following equation (Equation 13).
- FIG. 11C shows a three-dimensional model of the face surface reconstructed from the MRI image.
- the point cloud used for ICP was not the entire face surface, but only the center area of the face including the characteristic areas of the nose, eyes, and mouth. The size of the area to be cut out is determined empirically.
- ⁇ Noise reduction by random dot pattern projection> The most difficult problem in passive stereo measurement is search for corresponding points. As described above, in the corresponding point search by block matching, the regions having the smallest difference in pixel values in the block are associated with each other. Therefore, when searching for corresponding points in a region having a small surface feature, a difference in pixel values is unlikely to occur, and erroneous correspondence is likely to occur. Therefore, in this embodiment, a random dot pattern is projected from the projector, and a surface feature is added to the measurement target in a pseudo manner. In addition, by detecting edges (regions with large color changes on the image) using a 3 ⁇ 3 pixel Sobel filter and searching for corresponding points only in the edges and surrounding pixels, noise due to incorrect correspondence is generated. Reduced.
- FIG. 12 shows a comparison of measurement results by edge extraction and pattern projection.
- the distance image represents the distance from the camera by color change.
- the color change of the distance image does not coincide with the target shape, and it can be seen that an incorrect correspondence has occurred.
- distance measurement is performed for a characteristic area in the image, but an area with little color change such as a cheek is not extracted as an edge. As the distance is not measured.
- pattern projection and edge extraction are used together, a distance image very similar to the face shape is obtained, and the entire face can be measured while suppressing noise generation compared to the former. Recognize.
- CUDA is a parallel computing architecture for GPU developed by NVIDIA. Since the GPU has a large number of simple arithmetic units, it is possible to demonstrate high arithmetic performance compared with the CPU in arithmetic processing with high parallelism.
- programming for the GPU can be performed using the C language.
- the corresponding point search process which has the highest calculation cost and takes a long time, is performed in parallel, thereby speeding up the process and reducing the cost (see FIG. 13).
- FIG. 13 illustrates a state where block matching of 5 ⁇ 5 pixels is performed using 10 threads (THREAD). Calculate and save one row of SSD in each thread. Then, the SSDs of the entire block can be obtained by adding the SSDs of the columns stored in the threads of the left and right columns of the target pixel. The parallax of the entire image can be obtained by fixing the left image, obtaining the SSD by moving the right image pixel by pixel, and obtaining the amount of movement for each thread when the SSD is the smallest. In this embodiment, 64 points are used, and corresponding point search is performed with a block size of 11 ⁇ 11 pixels.
- Images of the initial posture are acquired from the left and right cameras, for example, with a size of 240 ⁇ 320 pixels.
- a feature region in the left image is detected by template matching, and the three-dimensional position of each feature region is stored by stereo vision.
- a feature region is searched using the template image acquired in the process (2) for the left and right camera images obtained for each frame. The three-dimensional position of the feature region is obtained from the search result by stereo vision.
- ARTToolkit is used to recognize this marker.
- the ARTToolkit is a C language library for realizing augmented reality (AR) (Hirokazu Kato, Mark Billinghurst, Koichi Asano, Keihachiro Tachibana: "Augmented reality system based on marker tracking and its Calibration ”; Transactions of the Virtual Reality Society of Japan, vol. 4, No. 4 (1999)).
- AR augmented reality
- the marker recognition function of this library is used.
- the four corners of the marker are detected from the left and right images. Then, the marker position in the three-dimensional space is obtained by stereo vision.
- the marker was installed vertically in the central portion of the therapeutic coil 33. From the three-dimensional coordinates of the four corners of the marker, a straight line passing through the center of the marker and the center of the coil 33 is obtained, and as shown in FIG. 14, a figure corresponding to the magnetic flux irradiated by the therapeutic coil 33 along the straight line. For example, a long and narrow rectangular shape is displayed, and this rectangular shape is displayed together with the brain surface in a three-dimensional space so that the magnetic stimulation points can be grasped.
- the practitioner determines the magnetic stimulation site with reference to the pattern of the brain surface of the subject 2. Therefore, it is necessary to create a three-dimensional brain model from cross-sectional data obtained by MRI. Therefore, the following procedure was performed.
- a cross-sectional image of the subject's head 2h is acquired using software MRIcro (see FIG. 15A).
- the brain region image is manually cut out from the acquired cross-sectional image of the subject's head 2h (see FIG. 15B).
- a brain image is three-dimensionally reconstructed from the clipped brain region image using the above-described three-dimensional reconstruction method. Thereby, a three-dimensional point cloud of the brain as shown in FIG. 15C can be acquired.
- the pattern of the brain surface may be difficult to see depending on the display angle.
- a huge number of points must be displayed, and the calculation cost becomes very high, which is not suitable for real-time display. Therefore, we created a mesh model along the brain surface, displayed only the surface using polygons, and mapped the brain surface pattern as a texture. Next, a method for creating a brain mesh model and a texture image will be described.
- ⁇ Create texture image> For the creation of the texture image, the color information of the three-dimensional point cloud of the brain is used. Polar coordinates with the center of the brain as the origin are set, and the polar coordinates of each point in the point cloud data are obtained as shown in FIG. For a point represented by (x, y, z) in three-dimensional coordinates, the angle ( ⁇ , ⁇ ) and the distance r of the polar coordinate element are obtained by the following equation (Formula 14).
- a 180 ⁇ 180 array is prepared, and the array number corresponds to the angle ⁇ and the row number corresponds to the angle ⁇ (see FIG. 16B).
- the angle ⁇ and the angle ⁇ are each grouped with a width of 1 degree from 0 degrees to 180 degrees, and the point having the largest distance r among the points included in each group is defined as a brain surface point.
- the color information of the brain surface points is stored in an array and used as texture information.
- FIG. 17 shows a texture image created from the point group in FIG.
- ⁇ Mesh model creation> The mesh model is created based on the cross-sectional image.
- the brain boundary coordinates of the cross-sectional image are acquired, and the polar coordinates and three-dimensional coordinates thereof are acquired.
- An array similar to that used when creating a texture image is prepared, and the acquired points are stored in the array based on polar coordinates. Thereby, as shown to Fig.18 (a), the acquired point can be mapped two-dimensionally. From this point, Delaunay triangulation method (Hiroyuki Yamamoto, Junji Uchiyama, Hideyuki Tamura: "Droney network generation method for 3D shape modeling", IEICE Transactions D-11, Vol. J83-D-11, No. 5, pp.
- FIG. 18C shows a three-dimensional display of the brain surface and texture mapping using a texture image as shown in FIG.
- FIG. 19A shows MRI data and stereo measurement data as shown in FIG. 19A, and the accuracy evaluation of the IPC algorithm was performed.
- FIG. 19B shows a result of matching two data using the rigid body transformation parameters acquired by the IPC algorithm. By referring to the position of the nasal muscles and eyes, it can be seen that the two data substantially match.
- FIG. 20 is a diagram showing an MRI image displayed in a superimposed manner with stereo measurement data in the initial alignment state.
- a vertical cross section and a horizontal cross section obtained by cutting along the vertical (vertical) plane and the horizontal (horizontal) plane shown in FIG. 20A are shown in FIG. 20B and FIG. 20C, respectively.
- Stereo measurement data is displayed as a solid curve on the face surface in FIGS. 20 (b) and 20 (c).
- the line segment displayed in the lower right of these figures represents 1 cm, and it can be seen that the two data are matched with almost no error.
- FIG. 21A shows the rotation amount around each axis
- FIG. 21B shows the parallel movement amount in each axis direction
- FIG. 21C shows the error average of each feature point.
- the ideal result is that the z-axis rotation (indicated by a two-dot chain line) in the graph of FIG. 21A changes as an ideal line (solid line). It is desirable that all changes be 0 (zero). As long as you look at the graph of rotation change and translation, you can get a tracking result close to the ideal change with some errors.
- the average rotation error around each axis was 0.7 degree around the x axis, 0.5 degree around the y axis, and 1.1 degree around the z axis.
- the average movement error in each axial direction was 4 mm in the x-axis direction, 3 mm in the y-axis direction, and 0 mm in the z-axis direction, and the average error between each feature point and the actually measured value was 6 mm.
- a moving operation for bringing the magnetic stimulation coil closer to the magnetic stimulation site and therefore the position of the magnetic stimulation coil and the subject's head with higher accuracy.
- An embodiment corresponding to the purpose of alignment will be described.
- Various position detection methods have detection errors. For example, if the method using the stereo camera 40 of FIG. 1 is described, the distance from the camera to the subject is measured by the well-known triangulation principle, so the theoretical error is as the distance of the subject from the camera increases. Expanding.
- a transcranial magnetic stimulation device When a transcranial magnetic stimulation device is used at a patient's home (at home) instead of being used at a medical institution such as a hospital, a medical person cannot be present at the place of treatment.
- the irradiation point of the magnetic stimulation coil must be brought close to the sweet spot until it enters a distance range that is sufficient to exert a therapeutic effect. If the accuracy required for treatment cannot be obtained during the moving operation, a more efficient magnetic stimulation coil can be used so that the necessary stimulation magnetic field can be obtained at the required irradiation site even if there is some error. In order to achieve this, it is necessary to increase the size and number of turns of the coil or increase the value of the coil applied current.
- the various position detection methods always have errors that cannot be avoided theoretically.
- the stereo camera 40 has the head 2h of the subject and the magnetic stimulation coil 30.
- these theoretical inevitable errors are accumulated and compared with a single position measurement. As a result, the error may increase additively.
- the stereo camera 70 that is a position detection unit is configured integrally with the magnetic stimulation coil 60 via a connecting portion 62.
- the relative distance and posture difference between the magnetic stimulation coil 60 and the stereo camera 70 remain unchanged.
- another configuration different from the above configuration may be adopted as long as it is a mechanism that relatively fixes the magnetic stimulation coil 60 and the stereo camera 70.
- the position and orientation of the magnetic stimulation coil 60 viewed from the measurement coordinate origin of the stereo camera 70 are determined by design, or once measurement is performed at the start of use, subsequent measurement is unnecessary. Therefore, there is only one subject to be measured during treatment, and the accumulation of theoretical errors can be avoided reliably.
- a stereo camera 70 which is an electronic device, is disposed in the vicinity of the magnetic stimulation coil 60 that generates a large magnetic field, the electrical and physical damage caused by the large induced current generated by the coil 60, or the physical and mechanism associated with the magnetic induction.
- the stereo camera 70 may be magnetically shielded with a metal plate or the like.
- a countermeasure such as arranging the stereo camera 70 at a position on the rotational axis 64 of the two spiral coils 63 where the induced magnetic field is theoretically zero from the structure of the magnetic stimulation coil 60 may be considered.
- stereo camera 70 instead of the stereo camera 70, other position detection means may be used.
- the stereo camera 70 since the imaging viewpoint is in the vicinity of the magnetic stimulation coil 60 and is away from the viewpoint of the subject 2 who performs the operation, the viewpoint is determined based on the detected position information. Coordinate conversion is performed so that the position of the stereo camera 40 shown in FIG.
- the subject's head 2h is photographed using the above-described second to fourth modes for obtaining positional information in the three-dimensional space, such as a monocular camera instead of a stereo camera, and the contour of the head or the pupil, It is also possible to perform superposition with the head MRI image or superposition with the head image after movement from the feature point on the image such as the nasal bridge.
- a transcranial magnetic stimulation device 1b for applying magnetic stimulation using a coil 60 is provided.
- the transcranial magnetic stimulation apparatus 1b includes a magnetic stimulation coil 60 configured to be able to change its position and posture according to an operation, and a stereo camera or the like whose relative position and posture are fixed with respect to the magnetic stimulation coil 60.
- a control unit 80 is provided that displays a screen for teaching to move the magnetic stimulation coil 60 to the sweet spot.
- the control unit 80 records and holds the head MRI three-dimensional image of the subject in which the position of the specific part to be magnetically stimulated is marked, and the head MRI three-dimensional image and the head appearance image captured by the imaging unit 70.
- the head of the current subject can change the relative distance and posture with respect to the magnetic stimulation coil 60 configured so that the patient moves with the grip 61 by overlapping the corresponding parts.
- the distance and posture difference between the current head appearance image captured by the imaging means 70 and the head appearance image used for superposition are calculated, and the magnetic stimulation coil 60 is calculated using the result of the calculation.
- the relative distance and posture difference from the current sweet spot is comprised so that the screen display for the teaching which moves the magnetic stimulation coil 60 to a sweet spot may be performed using the result of the measurement.
- all of the above explanation is a process of relieving neuropathic pain by applying magnetic stimulation to nerves in the brain with a coil for magnetic stimulation arranged on the scalp surface of a subject (for example, a patient or a test examinee).
- a subject for example, a patient or a test examinee.
- this invention is not limited to such a case, It can apply effectively also in another magnetic stimulation use.
- the present invention is capable of accurately irradiating a magnetic flux to a target region in a narrow range, particularly when performing magnetic stimulation treatment on the head, and the treatment practitioner can determine the three-dimensional position and magnetic flux of the magnetic stimulation means.
- an image processing method and a transcranial magnetic stimulation device that can grasp the orientation of the patient in a wide range and reduce the burden on the patient.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Radiology & Medical Imaging (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Surgery (AREA)
- Medical Informatics (AREA)
- Heart & Thoracic Surgery (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Neurology (AREA)
- Physiology (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- High Energy & Nuclear Physics (AREA)
- Multimedia (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
- Magnetic Treatment Devices (AREA)
Abstract
Description
この特許文献1においては、かかる方法で施した経頭蓋磁気刺激治療により難治性の神経障害性疼痛が有効に軽減され、更に、より正確な局所刺激がより高い疼痛軽減効果を実現することが確認されている。但し、最適刺激部位は個々の患者によって微妙に異なることも明らかにされている。
かかる治療用コイルの位置決めについては、例えば赤外線を用いた光学式トラッキングシステムを利用して患者頭部に対する治療用コイルの位置決めを行う構成のものが公知であり(例えば、特許文献2,3参照)、既に一部には市販され臨床応用されている。
図1は、本実施形態に係る経頭蓋磁気刺激装置の構成の概要を模式的に示す説明図である。この経頭蓋磁気刺激装置1は、被験者2(例えば、患者や検査受検者など)の頭部2h内の特定部位(最適刺激部位)に磁気刺激を加えて治療を行なうためのものである。
図1に示すように、経頭蓋磁気刺激装置1(以下、適宜、単に「装置」と略称する)は、その主な構成として、画像モニタ部10,装置本体ユニット20,磁気刺激コイルユニット30,ステレオカメラ40及びプロジェクタ50を備えている。尚、前記装置1に含まれるステレオカメラ40は、3次元空間内における対象物の空間座標情報を得るための一例を示したものであり、後で説明するような他の態様により、被験者2の顔や磁気刺激コイルユニット30の空間座標情報を得るように構成することも可能である。
装置本体ユニット20に含まれる画像表示制御部21は、予め撮影された被験者2の頭部2hの3次元MRI画像を読み出し可能に保持すると共に、画像モニタ部10に表示させるべき各種の画像の表示制御を行うものである。尚、前記3次元MRI画像は、画像表示制御部21に付設された若しくは装置本体ユニット20の外部に付設されたメモリ装置に、読み出し可能に保持されていてもよい。磁気刺激コイル制御部22は、磁気刺激コイルユニット30に印加する磁束生成電流のオン/オフおよび電流を制御するものである。また、3次元情報生成部23は、ステレオカメラ40から入力される複数(本実施形態では、例えば2つ)の画像の視差を利用して、被験者頭部2hおよび磁気刺激コイルユニット30の空間内における位置および姿勢の情報を生成するとともに、プロジェクタ50が行なうランダムドットパターン投影動作の制御を行うものである。以上の画像表示制御部21,磁気刺激コイル制御部22及び3次元情報生成部23は、それぞれ所要の制御回路および演算回路等を備えて構成されている。これら画像表示制御部21,磁気刺激コイル制御部22及び3次元情報生成部23の具体的な動作については後述する。
尚、本明細書において、「治療用コイルの姿勢」とは、治療用コイル33の方向および角度を意味し、「治療用コイルの方向」とは、被験者頭部2hの頭皮表面におけるコイル33の向きのことであり、「治療用コイルの角度」とは、被験者頭部2hの頭皮表面の法線とコイル33の磁場方向とがなす角度を意味するものとする。
また、プロジェクタ50は、被験者頭部2hの表面にランダムドットパターンを投影し、画像処理のための抽出点とするためのものである。
従来の経頭蓋磁気刺激装置が有する技術的な課題を克服するため、本願発明者は、経頭蓋磁気刺激装置1が備えるべき要件を分析した結果、以下の知見を得た。
まず、磁気刺激治療を行うためには、被験者頭部2hの3次元撮影画像と3次元MRIデータとの正確な位置合わせを行わなければならない。被験者2を拘束せずにこのような正確な位置合わせを行うためには、時々刻々と変化する被験者頭部2hの位置と向き(姿勢)を解析し、常に、MRIデータと頭部画像とを一致させる処理が必要である。本実施形態では、被験者2の頭部2hの3次元外観画像として、指定し易い特徴点を数多く含む被験者2の顔を対象とした3次元顔画像を用いることとした。
さらに、施術者(医師等)は3次元MRI画像による頭蓋内の情報(脳の表皮の画像)を参照しながら刺激をおこなう必要があるため、脳表の情報,頭部(顔)の姿勢および磁気刺激コイルユニット30の姿勢をわかりやすく表示するインタフェースも必要である。
(1)3次元MRI計測データと被験者2の現在の頭部(顔)の姿勢との姿勢照合機能を備えること。
(2)リアルタイムに被験者2の頭部(顔)の姿勢を追跡する機能を備えること。
(3)リアルタイムに磁気刺激コイルユニット30の位置および姿勢を追跡する機能を備えること。
(4)脳表における磁気刺激ポイントなど、刺激状況の把握が容易なインタフェース機能を備えること。
であり、これら4つの要件を、十分な精度と操作性および経済性の下で実現することが重要である。
図1に示す構成を備えた前記経頭蓋磁気刺激装置1の基本的な動作について、具体的に説明する。尚、以下の説明では、画像処理の具体的な算法など、データを処理する手順や方法を主眼として説明を行うので、図1を用いて先に説明をした本装置1の各構成の機能や動作として直接言及されない場合がある。しかし、その場合でも、これら説明がなされる機能や動作は、図1に図示した経頭蓋磁気刺激装置1の機能や動作として実現されているため、装置1のどの構成に対応するかは容易に特定が行なえるものである。
(I)初期位置合わせ
(II)被験者頭部の姿勢の追跡
(III)治療用コイルの追跡
(IV)追跡結果の表示
装置1の作動がスタートすると、まず、ステップ#1で、例えばステレオカメラ40を用いて、被験者頭部2h及び磁気刺激コイルユニット30を含む画像フレーム(初期画像フレーム)が取得され、次に、ステップ#2で、この初期画像フレームに基づいて得られた初期状態での被験者2の3次元顔画像と、前記装置本体ユニット20の画像表示制御部21に読み出し可能に保持されていた被験者頭部2hの3次元MRI画像との初期位置合わせが行われる。この工程が、前述の「(I)初期位置合わせ」工程に相当している。
このとき、被験者頭部2hの姿勢のリアルタイム追跡結果に、ステップ#2での初期位置合わせの結果を反映することで、今現在の被験者2の3次元顔画像と3次元MRI画像とを正しい位置および姿勢で重ね合わせることができる。このスッテプ#4及びステップ#5の工程が、前述の「(II)被験者頭部の姿勢の追跡」工程に相当している。
前記ステップ#3からステップ#9の一連のステップは、磁気刺激治療を終えて装置1が停止されるまで、常時、継続して繰り返し実行される。
磁気刺激治療に先立って医療機関に設置されたMRI装置により得られた頭部MRI画像データと,本装置1の一つの例示態様であるステレオカメラ40によるステレオ計測(視差を利用した3次元位置計測)により得られた計測データとは、異なる計測機で、しかも異なる患者の姿勢で計測されており、同―座標系上で三次元表示すると2つのデータ間にズレが発生する(図3(a)参照)。このため、この2つのデータを一致させる必要がある。位置あわせ後の2つのデータを図3(b)に示す。
この処理を位置合わせといい、各データの姿勢を決める剛体変換パラメータである3行3列の回転行列Rと、3次元の平行移動ベクトルtを求めることに相当する。本装置1では、この位置合わせの手法として、ICP(Iterative Closest Point)アルゴリズムを用いる。このアルゴリズムは、反復計算により対応点間の距離を最小化するような剛体変換パラメータを求める手法である。この手法を用いることで、計測データ間の対応関係や、計測装置と対象物体との事前のキャリブレーションを必要とせずに、高精度に位置合わせを行うことができる。
(1)MRIデータの読み込み
(2)2台のカメラ(左カメラ41,右カメラ42)を用いた顔のキャプチャ
(3)左右のカメラ41,42から得られた画像からAdaboost(後述する)を用いた顔検出を行い、画像中の顔領域を抜き出す。
(4)顔領域に対してステレオ計測を行い、顔形状を計測する
(5)MRIデータとステレオ計測により得られた顔形状データとを、ICPアルゴリズムを用いて位置合わせを行なう。
ここで、本装置1が用いる3次元位置検出方法の一つの態様であるステレオ計測について説明する。
ステレオ計測とは、光学的な3次元形状計測手法の1種であり、左右に配置された2台のカメラで計測対象を撮影し、視差情報から三角測量法により3次元位置を推測する手法である。ステレオ計測は、(a)対応点の探索と(b)3次元位置の算出の2つの処理を行う必要がある。
三角測量法を用いて3次元位置を求める場合、左カメラ41で撮影された画像が右カメラ42で撮影された画像のどの部分に対応するかを調べ、対応点のズレ(視差)を求める必要がある。
図4に、ステレオ計測で用いる左右2つの画像の例を示す。本実施形態で用いるステレオカメラ40は平行ステレオ(右のカメラ42の光軸と左のカメラ41の光軸とが平行)であるために、横方向にのみ対応点のずれが発生する。したがって、対応点探索は横方向のみを考慮すればよく、左目画像からみた右目画像の対応点は、すべて左目画像より左側にあることになる。
以上の対応点探索によって求められた視差dを用いて、既知の三角測量法により3次元位置を算出する。
図5に、平行ステレオと計測対象との3次元位置の関係を例示する。平行ステレオでは、対応点が水平線上に存在するため、奥行きは視差の反比例として算出することができる。注視点の3次元位置は、次式(数6)によって計算できる。
ここで、Bはカメラ間の距離,fは各カメラの焦点距離である。B及びfの値は計測時に既知であるので、対応点探索によって求まる視差dを用いることで、注視点の3次元位置を算出することができる。
MRIデータとステレオ計測により得られたデータとの位置合わせには、顔表面の3次元データのみを用いる。MRIデータについては、必要な領域のみを事前に抽出しておく。一方、ステレオ計測データについては、カメラ40により得られた画像から顔領域を検出し、その領域の3次元データを用いる。本装置1では、顔抽出処理として、画像特微量であるHaar-like特微量と学習アルゴリズムであるAdaboostアルゴリズムを用いたオブジェクト検出を用いる。このオブジェクト検出処理は、Paul Viola等のオブジェクト検出の研究(Paul Viola and Michael Jones: "Object Detection using a Boosted Cascade of Simple", IEEE CVPR, 2001)を元に、Rainer Lienhart等が改良したもので(Rainer Lienhart and Jochen Maydt: "An Extended Set of Haar-lide Feature for Rapid Object Detection", IEEE ICIP 2002, vol. 1, pp.900-903 (2002))、高速にオブジェクトを検出することができる。
MRIデータとステレオ計測で得られたデータとの位置合わせの手法として、ICPアルゴリズムを用いる。ICPアルゴリズムとは、1992年にBesl等により提案された手法で(P. J. Best and N. D. McKay: “A Method for Registration of 3-D Shapes", IEEE Trans. Pattern Anal. Machine Intell, vol. 14, No. 2, pp. 239-256 (1992-2))、反復計算により対応点間の距離を最小化するような剛体変換パラメータを求める手法である。
剛体変換パラメータである回転行列R,平行移動ベクトルtは、次式(数10)に示す誤差関数E(R,t)を最小化することで求めることができる。尚、(数10)の式は、前述の(数1)及び(数3)の式と同一のものである。
(i)点群Aの各点aiにおける点群Bとの最近点miを求める。
(ii)誤差Eを最小にする剛体変換パラメータを求める。
(iii)点群Aを求められたパラメータ(R,t)を用いて変換する。
(iv)誤差Eが閾値以下であれば反復計算を終了する。それ以外の場合には、(i)に戻って、同様のステップを繰り返して実行する。
以上の説明では、対象被験者2の顔や磁気刺激コイルユニット30の3次元空間内の位置情報を得るための方法として、ステレオカメラ40の視差、つまり、複数の視点から撮影した画像の視差、を利用していたが(第1の態様)、かかる方法に限定されることなく、他の態様にて前記位置情報の取得を実現することができる。
例えば第2の態様として、プロジェクタやレーザ照射手段のような投光手段と、ビデオカメラのような一つの視点のみを有する(複数の視点から撮影した画像の視差を利用する方式ではない)撮像手段とを用い、投光手段から発した光が対象物で反射し、その反射光を撮像手段が捉えた状態で、各光軸の角度の情報から上記と同様の三角測量原理によって、対象物までの距離および角度を知り、この結果、対象物の光反射点の空間座標を得ることができる。
前述のように、MRIデータとステレオ計測により得られたデータは、ICPアルゴリズムを用いることで、同一座標系において初期状態が一致した状態にある。ここで、被験者2を拘束しない状態で磁気刺激を行うためには、リアルタイムに被験者頭部2hの初期姿勢からの変化を追跡し、今現在の姿勢への剛体変換パラメータを求める必要がある。
尚、前述のように、本実施形態では、被験者2の頭部2hの3次元外観画像として、指定し易い特徴点を数多く含む被験者2の顔を対象とした3次元顔画像を用いている。従って、この場合、「被験者頭部2hの姿勢」は被験者2の「顔姿勢」と表現することもできる。
そこで、本実施形態では、顔特徴として、両眼の目尻および目頭,口元(両端),鼻頭の7点を指定し、この顔特徴についてテンプレートマッチングを用いて追跡することで、剛体変換パラメータを算出するようにした。顔特徴領域は、顔画像中で特徴的でありトラッキングに適したパターンを持っていることを選択の基準とした。
(1)ステレオカメラを用いて初期姿勢での顔画像を取得する。
(2)各特徴領域(両眼の目尻および目頭,口元(両端),鼻頭)を指定し、各領域の画像(テンプレート)と3次元座標とを保存する。
(3)現在の姿勢での顔画像を、ステレオカメラを用いて取得する。
(4)テンプレートマッチングを用いて、左右の画像中の特徴点位置を調べ、その3次元座標を求める。
(5)最急降下法により、初期姿勢からの変化を求める(つまり、初期姿勢での測定値を現在の姿勢にフィッテイングさせる剛体変換を求める)。
ここに、前記(1)及び(2)の処理は、初期化処理であり、追跡開始時に一度だけ行えばよい。リアルタイムで行なう顔の追跡は、前記(3)~(5)の処理を繰り返すことで行う。
ここで、前記テンプレートマッチング法について説明する。
ある画像(テンプレート)が、他の画像中のどの部分に存在するかを対応づける処理をテンプレートマッチングという。これは、図8に示すように、テンプレートと呼ばれる画像を予め用意し、これを移動させながら対象画像と重ね合わせ、テンプレートと対象画像との相関を調べる方法である。
二つの画像の違いを測る尺度としては、相関係数Cを用いる。この相関関数Cは、対象画像をI(m,n),テンプレート画像をT(m,n)(画像サイズ:M×N)とすると、次式(数11)で表される。このとき、相関係数Cの値が大きいほど画像間の相関は大きく、画像中で最も相関係数の値が大きくなった領域を対応する領域とする。
顔姿勢の追跡を行うためには、初期姿勢からの姿勢変化を求める必要がある。本装置1では、前述のように、両眼の目尻および目頭,口元(両端),鼻頭の7点の特徴領域の3次元的な姿勢変化(回転R(α,β,γ)と平行移動t(x,y,z)を求める姿勢追跡を行う(図9参照)。
次に、各特徴領域の3次元位置の計測結果から、頭部の位置および姿勢を求める問題は、次式(数10)に示す誤差関数Eを最小とする剛体変換パラメータである回転行列Rと平行移動ベクトルtを求める問題に帰着する。尚、(数12)の式は、前述の(数2)及び(数4)の式と同一のものである。
磁気刺激治療を行うためには、治療用コイル33の3次元位置および姿勢を把握し、対象に対して正確に刺激できているかを常に観測する必要がある。磁気刺激治療にとって有効とされる精度は、一般に、頭蓋内部で直径1cm程度であり、そのスポットをターゲットとして治療用コイル33から磁束ビームを指向させる必要がある。治療用コイル33の追跡では、既知のマーカ(画像データ上から特徴点を抽出するために、被写体表面に配置した図形パターン)を用い、このマーカを追跡することでコイル33の3次元位置や姿勢を求める。
(1)ステレオカメラ44を用いて画像を取得する。
(2)左右の画像に対してマーカ認識を行い、マーカ32の四隅の画素を探索する。
(3)ステレオ視によりマーカ32の四隅の3次元位置を求める。
(4)マーカ平面の法線ベクトルを求め、磁束の方向(刺激方向)を求める。
治療用コイル33の追跡における最も重要な技術は、画像中のマーカ領域を正確に把握することである。マーカ認識では、使用するマーカ32を事前に登録しておく必要があり、探索画像中のマーカ候補領域を探索し、登録マーカとの相関を調べることでマーカ領域を確定する。
図10(a)~(f)は、マーカ認識の具体的な処理を示す一連の説明図である。図10(a)に示すマーカを用いた場合における、マーカ認識の具体的な処理は、以下に示す通りである。
:カメラから入力した画像(図10(b))を、閾値を用い、閾値より明るい領域を黒で、暗い領域を白で表示する(図10(c))。
(ii)暗い領域において、閉領域を探索しラベリングする。
:2値化した画像中の白の領域に対して、閉領域を探索する。さらに、各閉領域に番号(ラベル)を割り振ることで区別できるようにする(ラベリング処理)。図10(d)では、閉領域の区別の様子を色の違いで示している。
(iii)各閉領域で頂点数を調べ、4つの頂点を持つ領域を4角形と判断する。
:各閉領域の頂点数を調べ、頂点数が4の領域を4角形と判断して、マーカの候補領域とする(図10(e)参照)。このとき、閉領域の面積が非常に小さいか、又は非常に大きい領域は、除外する。
(iv)4角形内の画像を単純化する。
:4角形の領域に対してアフィン変換を用いることで、領域が正方形になるように修正する(図10(f)参照)。
(v)単純化された画像と登録パターンとの比較
:単純化された画像と登録マーカとの画素比較を行い、誤差を計算する。全ての4角形領域の中で最も誤差の小さな領域を、マーカ領域と判断する。
実際の医療現場で経頭蓋磁気刺激装置を運用するためには、頭部追跡およびコイル追跡の結果として、治療のための磁束が、現在、脳のどの部分を刺激しようとしているのかをユーザに伝えるためのユーザ・インターフェースが必要となる。
本実施形態では、施術者が治療用コイル33を用いて磁気刺激を行う際には、脳表の模様を参考に刺激部位を決定していることから、脳の3次元モデルを表示し、表示角度や大きさを自由に変更できるインタフェースを採用した。
また、治療用コイル33による刺激点の表示には、コイルの中心を貫く角柱を表示し、角柱と脳表との関係から現在の刺激部位を判断できるようにした(後述する図10参照)。この図10では、被験者2の脳表或いは頭部2hの3次元MRI画像に対して、治療用の磁束の位置と向きを、相対的な関係が把握できるように表示がなされていることがわかる。
本実施例は、次表1の機材および開発言語などを用いて実現した。
前述のように、初期位置合わせでは、ICPアルゴリズムを用いて、MRIデータとステレオ計測データとの初期位置を一致させる。そのためには、MRIで得られた断面画像からICPに用いる顔表面の3次元データを事前に取得しておく必要がある。また、ステレオ計測時にノイズが発生するとICPの結果に大きな影響を与えるため、このノイズを低減させる必要もある。更に、対応点探索は計算コストが非常に高く、また、処理に時間も掛かる。
そこで、本実施例では、CUDAというGPU(Graphics Processing Unit)向けの開発環境を用いて処理を並列化することで、対応点探索の高速化を実現した。
(1)MRIデータから取得した顔の3次元モデルを読み込む。
(2)プロジェクタから患者に対してランダムドットを投影し、640×480画素の画像サイズで左右のカメラから取り込む。
(3)左右の画像に対して顔認識を行い、画像中の顔領域を検出する。この処理は、Open CVの機能を用いて実現している。
(4)左右の画像に対して、3×3画素のSobelフィルタを用いてエッジ検出を行い、エッジの周囲7画素に対して、ブロックマッチングを行う。ブロ尽くマッチングは11×11画素のブロックを用いた。また、GPUを用いて処理を並列化しており、高速な対応点探索を可能にしている。エッジを検出し、その周囲のみ対応点探索することで、誤対応によるノイズの発生を低減させることができる。
(5)ブロックマッチングにより得られた視差情報から、三角測量法により3次元位置を求める。
(6)計測された顔形状とMRIデータから得た顔形状どうしを、ICPアルゴリズムを用いて位置合わせする。ICPアルゴリズムは、VTK(Visualization Tool Kit)の機能を用いて実装しており、実行することで回転行列Rと平行移動ベクトルtとが得られる。
被験者頭部2hのMRIの計測では、図11(a)のような断面画像の集合として頭部スキャンの結果が得られる。ICPアルゴリズムを用いて位置合わせを行うためには、この断面画像からマッチングに必要な領域(顔の表面)の3次元点群を取得することが必要である。
本実施例に使用したMRI画像は、例えば、256×256ピクセルの大きさで、130枚の断面画像で構成されている。断面のスライス間隔は例えば1.40mmであり、1ピクセルの大きさは例えば0.98×0.98mmである。画像中の顔の輪郭領域は、白く表示されていることに注目し、図11(b)の矢印線(11-1)のように、画像のx方向の最大値から走査し、例えば、最初に輝度値が30以上になる画素を顔の表面として取得した。取得した画素は、断面画像の番号N(0≦N<130)、取得した画素値目I(i,j)とすると、スライス間隔(例えば1.40mm),画素サイズ(例えば0.98×0.98mm)を用いて、次式(数13)のように3次元座標(X,Y,Z)Tに変換できる。
パッシブステレオ計測において最も困難な問題は対応点探索である。前述のように、ブロックマッチングによる対応点探索では、ブロック内の画素値の差が最も小さい領域どうしを対応させる。従って、表面の特徴が少ない領域の対応点探索では、画素値に差が生じ難く、誤対応が発生しやすくなる。
そこで、本実施例ではプロジェクタからランダムドットパターンを投影し、計測対象に擬似的に表面特徴を付加した。また、3×3画素のSobelフィルタを用いてエッジ(画像上での色変化の大きな領域)を検出し、エッジとその周囲の画素のみを対応点探索することで、誤対応によるノイズの発生を低減させた。
CUDAとは、NVIDIA社が開発したGPU向けの並列コンピューティングアーキテクチャである。GPUは、シンプルな演算ユニットを多数搭載しているため、並列性の高い演算処理では、CPUと比較して高い演算能力を発揮できる。CUDAを用いることで、C言語を用いてGPU向けのプログラミングを行うことができる。本実施例では、最も計算コストが高く時間も掛かる対応点探索処理を、並列計算させることで、処理の高速化およびコスト低減を図るようにした(図13参照)。
左画像を固定し、右画像を1画素ずつ移動させてSSDを求め、最もSSDが小さくなるときの移動量を各スレッドについて求めることで、画像全体の視差を求めることができる。本実施例では、64個のスレッドを用い、11×11画素のブロックサイズで対応点探索を行った。
初期位置合わせ処理が終了した後、顔姿勢の追跡が開始される。前述のように、顔姿勢追跡では顔の特徴領域(両眼の目尻および目頭,口元(両端),鼻頭の7点)を追跡し、初期状態からの3次元的な変化量を求める。本実施例に係るシステムでは、この特徴領域を手動で与える必要がある。また、前提条件として、追跡開始時の初期姿勢と初期位置合わせ時の姿勢が一致している必要がある。具体的には、以下の処理手順で顔姿勢の追跡を行った。
(2)右画像中の両眼の目尻および目頭,口元(両端),鼻頭の7点をマウスでクリックすることで、その領域を、例えば17×17画素のテンプレート画像として取得する。また、取得したテンプレート画像を用いて、左画像中の特徴領域をテンプレートマッチングによって検出し、ステレオ視により各特徴領域の3次元位置を保存する。
(3)毎フレーム得られる左右カメラの画像に対して、処理(2)で取得したテンプレート画像を用いて特徴領域を探索する。探索結果からステレオ視により特徴領域の3次元位置を求める。
(4)前記処理(2)で取得した初期姿勢の3次元位置と前記処理(3)で取得した現在の姿勢の3次元位置との剛体変換パラメータを、前述の[数2]で示される式の誤差関数Eを最小にすることで求める。この最適化計算には、最急降下法を用いている。
治療用コイルの位置および姿勢は、既知のマーカを用いて追跡を行った。本実施例では、このマーカを認識するのに、ARToolkitを用いた。このARToolkitとは、拡張現実感(AR:Augmented Reality)を実現するためのC言語ライブラリである(加藤博一,Mark Billinghurst,浅野浩一,橘啓八郎:「マーカ追跡に基づく拡張現実感システムとそのキャリブレーション」;日本バーチャルリアリティ学会論文誌,vol. 4, No. 4 (1999))。本実施例では、このライブラリのマーカ認識機能を利用している。
<脳の三次元表示>
前述のように、施術者は被験者2の脳表の模様を参考に磁気刺激部位を決定する。従って、MRIにより得られた断面データから脳の3次元モデルを作成する必要がある。そこで、次の手順にて実施を行なった。
(1)ソフトウェアMRIcroを用いて、被験者頭部2hの横断面画像を取得する(図15(a)参照)。
(2)取得した被験者頭部2hの横断面画像から、手動で脳領域の画像を切り取る(図15(b)参照)。
(3)切り取った脳領域の画像から、前述の3次元再構成手法を用いて、脳画像を3次元再構成する。これにより、図15(c)に示すような脳の3次元点群を取得することができる。
そこで、脳表に沿ったメッシュモデルを作成し、ポリゴンによって表面のみを表示し、そこに、脳表模様をテクスチャとしてマッピングするようにした。次に、脳のメッシュモデルとテクスチャ画像の作成方法について説明する。
テクスチャ画像の作成には、脳の3次元点群の色情報を用いる。脳の中心を原点とする極座標を設定し、図16(a)に示すように、点群データの各点の極座標を求める。3次元座標で(x,y,z)で表される点について、極座標要素の角度(θ,φ)と距離rは、次式(数14)で求まる。
メッシュモデルは、断面画像を基にして作成する。断面画像の脳の境界線座標を取得し、その極座標と3次元座標を取得する。テクスチャ画像の作成時と同様の配列を用意し、取得した点を極座標を基に配列に格納する。これにより、図18(a)に示すように、取得した点を2次元状にマッピングすることができる。この点から、ドロネー三角形分割法(山本裕之,内山晋二,田村秀行:「三次元形状モデリングのためのドロネー網生成法」,電気情報通信学会論文誌D-11,Vol. J83-D-11, No. 5, pp. 745-753 (1995-5))を用いて、面情報を取得する(図18(b)参照)。
こうして取得した面情報と3次元座標を用いて、脳表の3次元モデルとすることができる。脳表を3次元表示し、図17に示すようなテクスチャ画像を用いてテクスチャマッピングしたものを図18(c)に示す。
以上、説明した実施例について、前述の「初期位置合わせ」,「被験者頭部の姿勢(顔姿勢)の追跡」,「治療用コイルの追跡」の各工程での精度評価を行った。
精度評価に用いた機材等は前述の表1に示したものと同様である。また、ステレオカメラから計測対象までの距離は、70cmから100cmとした。
図19(a)に示すようなMRIデータとステレオ計測データとを用意し、前述のIPCアルゴリズムの精度評価を行った。IPCアルゴリズムにより取得した剛体変換パラメータを用いて2つのデータをマッチングさせたものを図19(b)に示す。鼻筋や目の位置を参照すれば、2つのデータが略一致していることが分かる。
顔姿勢追跡の精度評価には、人形の顔模型を用い、この顔模型を回転板(回転軸:z軸)に載せて、z軸廻りに±12.5度ずつ、2.5度刻みで回転させたときに得られる、回転行列,平行移動ベクトル,各特徴点の誤差平均をそれぞれ取得した。その結果を図21に示す。図21(a)は各軸廻りの回転量を、図21(b)は各軸方向の平行移動量を、図21(c)は各特徴点の誤差平均を、それぞれ示している。
各軸廻りの平均回転誤差は、x軸廻りが0.7度,y軸廻りが0.5度,z軸廻りが1.1度であった。また、各軸方向への平均移動誤差は、x軸方向に4mm,y軸方向に3mm,z軸方向に0mmであり、各特徴点と実測値との平均誤差は6mmであった。
コイル位置の追跡の精度評価として、マーカ中心の観測値と3次元空間中の実際の位置との誤差平均を算出した。平均誤差として、x軸方向に4mm,y軸方向に4mm,z軸方向に0mmであり、精度良くマーカの3次元位置を検出できていることが確認できた。
様々な位置検出方法は、それぞれ検出誤差を有している。例えば、図1のステレオカメラ40を用いた方式で説明すれば、よく知られた三角測量の原理でカメラから被写体までの距離を測定するので、カメラから被写体の距離が遠ざかるにつれて理論上の誤差は拡大する。
ステレオカメラ70の測定座標原点から見た磁気刺激コイル60の位置及び姿勢は設計上で決定されるか、あるいは使用開始時に一度測定をすれば、その後の測定は不要である。従って、治療に際して測定すべき対象は、被験者2の頭部2hただ一つとなり、理論誤差の積み重ねが確実に回避できる。
この経頭蓋磁気刺激装置1bは、操作に応じて位置および姿勢を変更可能に構成された磁気刺激コイル60と、磁気刺激コイル60に対して相対的な位置及び姿勢が固定されたステレオカメラ等の撮像手段70とを備えると共に、磁気刺激コイル60をスイートスポットまで移動操作する教示のための画面表示を行う制御部80を備えている。
この制御部80は、磁気刺激を行うべき特定部位の位置がマークされた被験者の頭部MRI3次元画像を記録保持しており、頭部MRI3次元画像と、撮像手段70が撮影した頭部外観画像とを、対応する部位が重なるよう重ね合わせを行い、グリップ61を持って患者が移動操作するよう構成された磁気刺激コイル60に対する相対的な距離及び姿勢が変わり得る、現在の当該被験者の頭部を撮像手段70が撮影した現在の頭部外観画像と、重ね合わせに用いられた頭部外観画像との間の距離及び姿勢の差を算出し、この算出の結果を用いて、磁気刺激コイル60と、現在のスイートスポットとの相対的な距離及び姿勢の差を測定する。そして、その測定の結果を用いて、磁気刺激コイル60をスイートスポットまで移動操作する教示のための画面表示を行うように構成されている。
2 被験者
2h 被験者の頭部
10 画像モニタ部
20 装置本体ユニット
21 画像表示制御部
22 磁気刺激コイル制御部
23 3次元情報生成部
30,60 磁気刺激コイルユニット
32 マーカ部
33,63 治療用コイル
40,70 ステレオカメラ
41 左カメラ
42 右カメラ
80 制御部
Claims (18)
- 予め撮影された被験者頭部の3次元MRI画像を保存する保存手段と、
前記被験者頭部の3次元外観画像を生成する3次元外観画像生成手段と、
前記3次元MRI画像と前記3次元外観画像とを位置合わせし、その位置合わせされた被験者頭部3次元画像を生成する画像生成手段と、
被験者頭部が移動した際に、前記位置合わせされた移動後の被験者頭部3次元画像を生成する移動後画像生成手段と、
前記被験者頭部の3次元MRI画像上での特定部位との位置関係を維持すべく移動操作される操作対象物の現在位置を示す操作対象物画像を生成する操作対象物画像生成手段と、
移動後の前記被験者頭部3次元画像と前記操作対象物画像とを同一画像内に表示する表示手段と、
を備えることを特徴とする画像データ処理装置。 - 被験者頭部の3次元MRI画像である第一の画像と、当該被験者頭部の3次元外観画像である第二の画像との位置合わせを行なうための画像データ処理装置であって、
前記第一の画像に含まれるN個の点aiそれぞれについて、前記第二の画像に含まれる複数の点bjの中から、予め決められた条件を満足する各点miを選択する選択手段と、
前記選択手段により選択された各点miにおいて、前記第一の画像に含まれる各点から、対応する前記第二の画像に含まれる各点へ剛体変換するためのパラメータとして、回転行列Rおよび平行移動ベクトルtを用いた所定の算出手順から成る誤差関数E(R,t)の値が最小となるように、前記回転行列Rおよび前記平行移動ベクトルtを決定するパラメータ決定手段と、
前記誤差関数E(R,t)の値が予め決められた閾値以下となるまで、前記各点aiを前記回転行列Rおよび前記平均移動ベクトルtで剛体変換し、変換後の各点aiについて、前記選択手段による前記各点miの選択と前記パラメータ決定手段による前記回転行列Rおよび前記平行移動ベクトルtの決定とを行わせるデータ処理手段と、
を備えることを特徴とする画像データ処理装置。 - 前記選択手段は、前記N個の点aiそれぞれについて、前記複数の点bjの中からユークリッド距離が最小である各点miを選択する、ことを特徴とする請求項2に記載の画像データ処理装置。
- 被験者の頭部の位置と向きを追跡するための画像データ処理装置であって、
前記被験者頭部の3元外観画像を生成する画像生成手段と、
前記3次元外観画像から、少なくとも一つの特徴領域を抽出して、3次元のテンプレート画像として保存する抽出保存手段と、
被験者頭部が移動した際に、移動後の被験者頭部の3次元外観画像を生成する移動後画像生成手段と、
前記移動後の被験者頭部の3次元画像上にて、前記テンプレート画像を移動させ、両画像データの相互の相関が最大となる位置を、移動後の前記特徴領域の位置として決定する特徴領域決定手段と、
移動前の前記特徴領域に含まれる各点を、前記移動後の特徴領域に含まれる各点へ剛体変換するためのパラメータとして、回転行列Rおよび平行移動ベクトルtを用いた所定の算出手順から成る誤差関数E(R,t)の値が最小となるように、前記回転行列Rおよび前記平行移動ベクトルtを決定するパラメータ決定手段と、
を備えることを特徴とする画像データ処理装置。 - 前記被験者頭部の3次元外観画像は、複数の視点から撮影した画像の視差を利用して生成される、ことを特徴とする請求項1から6の何れか一に記載の画像データ処理装置。
- 前記被験者頭部の3次元外観画像は、1つの視点から光または超音波の到達時間を利用して生成される、ことを特徴とする請求項1から6の何れか一に記載の画像データ処理装置。
- 前記操作対象物に対する、前記3次元外観画像生成手段の被験者頭部を撮影する撮像手段の相対的な位置及び姿勢が固定されている、ことを特徴とする請求項1から8の何れか一に記載の画像データ処理装置。
- 被験者頭部内の特定部位に対し、頭部外にある磁場発生手段を用いて磁気刺激を加えるための経頭蓋磁気刺激装置であって、
操作に応じて位置および姿勢を変更可能に構成された前記磁場発生手段と、
予め撮影された被験者頭部の3次元MRI画像を保存する保存手段と、
前記被験者頭部の3次元外観画像を生成する3次元外観画像生成手段と、
前記3次元MRI画像と前記3次元外観画像とを位置合わせし、その位置合わせされた被験者頭部3次元画像を生成する画像生成手段と、
被験者頭部が移動した際に、前記位置合わせされた移動後の被験者頭部3次元画像を生成する移動後画像生成手段と、
前記被験者頭部の3次元MRI画像上での特定部位との位置関係を維持すべく操作される前記磁場発生手段の現在位置を示す磁場発生手段画像を生成する磁場発生手段画像生成手段と、
移動後の前記被験者頭部3次元画像と前記磁場発生手段画像とを同一画像内に表示する表示手段と、
を備えることを特徴とする経頭蓋磁気刺激装置。 - 被験者頭部内の特定部位に対し、頭部外にある磁場発生手段を用いて磁気刺激を加えるための経頭蓋磁気刺激装置であって、
被験者頭部の3次元MRI画像である第一の画像と、当該被験者頭部の3次元外観画像である第二の画像との位置合わせを行なうための画像データ処理装置を有し、該画像データ処理装置は、
前記第一の画像に含まれるN個の点aiそれぞれについて、前記第二の画像に含まれる複数の点bjの中から、予め決められた条件を満足する各点miを選択する選択手段と、
前記選択手段により選択された各点miにおいて、前記第一の画像に含まれる各点から、対応する前記第二の画像に含まれる各点へ剛体変換するためのパラメータとして、回転行列Rおよび平行移動ベクトルtを用いた所定の算出手順から成る誤差関数E(R,t)の値が最小となるように、前記回転行列Rおよび前記平行移動ベクトルtを決定するパラメータ決定手段と、
前記誤差関数E(R,t)の値が予め決められた閾値以下となるまで、前記各点aiを前記回転行列Rおよび前記平均移動ベクトルtで剛体変換し、変換後の各点aiについて、前記選択手段による前記各点miの選択と前記パラメータ決定手段による前記回転行列Rおよび前記平行移動ベクトルtの決定とを行わせるデータ処理手段と、
を備えることを特徴とする経頭蓋磁気刺激装置。 - 前記選択手段は、前記N個の点aiそれぞれについて、前記複数の点bjの中からユークリッド距離が最小である各点miを選択する、ことを特徴とする請求項11に記載の経頭蓋磁気刺激装置。
- 被験者頭部内の特定部位に対し、頭部外にある磁場発生手段を用いて磁気刺激を加えるための経頭蓋磁気刺激装置であって、
被験者の頭部の位置と向きを追跡するための画像データ処理装置を有し、該画像データ処理装置は、
前記被験者頭部の3元外観画像を生成する画像生成手段と、
前記3次元外観画像から、少なくとも一つの特徴領域を抽出して、3次元のテンプレート画像として保存する抽出保存手段と、
被験者頭部が移動した際に、移動後の被験者頭部の3次元外観画像を生成する移動後画像生成手段と、
前記移動後の被験者頭部の3次元画像上にて、前記テンプレート画像を移動させ、両画像データの相互の相関が最大となる位置を、移動後の前記特徴領域の位置として決定する特徴領域決定手段と、
移動前の前記特徴領域に含まれる各点を、前記移動後の特徴領域に含まれる各点へ剛体変換するためのパラメータとして、回転行列Rおよび平行移動ベクトルtを用いた所定の算出手順から成る誤差関数E(R,t)の値が最小となるように、前記回転行列Rおよび前記平行移動ベクトルtを決定するパラメータ決定手段と、
を備えている、ことを特徴とする経頭蓋磁気刺激装置。 - 前記被験者頭部の3次元外観画像は、複数の視点から撮影した画像の視差を利用して3次元画像を生成するよう構成されている、ことを特徴とする請求項10から15の何れか一に記載の経頭蓋磁気刺激装置。
- 前記被験者頭部の3次元外観画像は、1つの視点から光または超音波の到達時間を利用して生成される、ことを特徴とする請求項10から15の何れか一に記載の経頭蓋磁気刺激装置。
- 前記磁場発生手段に対する、前記3次元外観画像生成手段の被験者頭部を撮影する撮像手段の相対的な位置及び姿勢が固定されている、ことを特徴とする請求項10~17の何れか一に記載の経頭蓋磁気刺激装置。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP15159364.7A EP2919194B1 (en) | 2011-03-09 | 2012-03-08 | Image data processing device and transcranial magnetic stimulation apparatus |
JP2013503607A JP6161004B2 (ja) | 2011-03-09 | 2012-03-08 | 画像データ処理装置および経頭蓋磁気刺激装置 |
EP12754418.7A EP2684518A4 (en) | 2011-03-09 | 2012-03-08 | IMAGE DATA PROCESSING DEVICE AND DEVICE FOR TRANSCRANIAL MAGNETIC STIMULATION |
US14/004,060 US9993655B2 (en) | 2011-03-09 | 2012-03-08 | Image data processing device and transcranial magnetic stimulation apparatus |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2011-051871 | 2011-03-09 | ||
JP2011051871 | 2011-03-09 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012121341A1 true WO2012121341A1 (ja) | 2012-09-13 |
Family
ID=46798296
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2012/055995 WO2012121341A1 (ja) | 2011-03-09 | 2012-03-08 | 画像データ処理装置および経頭蓋磁気刺激装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US9993655B2 (ja) |
EP (2) | EP2684518A4 (ja) |
JP (1) | JP6161004B2 (ja) |
ES (1) | ES2818078T3 (ja) |
WO (1) | WO2012121341A1 (ja) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015098155A1 (ja) * | 2013-12-24 | 2015-07-02 | 国立大学法人大阪大学 | 操作教示装置および経頭蓋磁気刺激装置 |
WO2015122369A1 (ja) * | 2014-02-14 | 2015-08-20 | 国立大学法人東京大学 | 脳内電流シミュレーション方法とその装置,及び脳内電流シミュレーション装置を含む経頭蓋磁気刺激システム |
JP2017029629A (ja) * | 2015-08-06 | 2017-02-09 | 東芝メディカルシステムズ株式会社 | 医用画像処理装置 |
JP2017164075A (ja) * | 2016-03-15 | 2017-09-21 | 富士フイルム株式会社 | 画像位置合せ装置、方法およびプログラム |
JP2018047035A (ja) * | 2016-09-21 | 2018-03-29 | 学校法人自治医科大学 | 医療支援方法および医療支援装置 |
JP2018061837A (ja) * | 2016-10-11 | 2018-04-19 | バイオセンス・ウエブスター・(イスラエル)・リミテッドBiosense Webster (Israel), Ltd. | 撮像装置を備える磁気追跡システムの重ね合わせ法 |
JP2019533218A (ja) * | 2016-08-10 | 2019-11-14 | コー・ヤング・テクノロジー・インコーポレーテッド | 3次元データ整合装置及び方法 |
CN113243923A (zh) * | 2021-05-19 | 2021-08-13 | 中科芯未来微电子科技成都有限公司 | 一种提高心电图监测的准确性的方法和装置 |
JP2021180866A (ja) * | 2017-08-25 | 2021-11-25 | ニューロフェット インコーポレイテッドNeurophet Inc. | パッチガイド方法及びプログラム |
CN114190922A (zh) * | 2020-09-18 | 2022-03-18 | 四川大学 | Tms头动检测方法 |
JP7486603B2 (ja) | 2020-04-22 | 2024-05-17 | ブレイン ナビ バイオテクノロジー カンパニー リミテッド | 拡張現実画像上に医用スキャン画像情報を補う方法及びシステム |
Families Citing this family (24)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101470411B1 (ko) * | 2012-10-12 | 2014-12-08 | 주식회사 인피니트헬스케어 | 가상 환자 모델을 이용한 의료 영상 디스플레이 방법 및 그 장치 |
WO2015045554A1 (ja) * | 2013-09-26 | 2015-04-02 | シャープ株式会社 | 生体情報取得装置および生体情報取得方法 |
US9477878B2 (en) * | 2014-01-28 | 2016-10-25 | Disney Enterprises, Inc. | Rigid stabilization of facial expressions |
ES2834553T3 (es) * | 2015-06-12 | 2021-06-17 | Accenture Global Services Ltd | Un método y sistema de realidad aumentada de medición y/o fabricación |
US10248883B2 (en) * | 2015-08-20 | 2019-04-02 | Align Technology, Inc. | Photograph-based assessment of dental treatments and procedures |
US11311193B2 (en) * | 2017-03-30 | 2022-04-26 | The Trustees Of Columbia University In The City Of New York | System, method and computer-accessible medium for predicting response to electroconvulsive therapy based on brain functional connectivity patterns |
CN110520056B (zh) * | 2017-04-07 | 2022-08-05 | 国立研究开发法人产业技术综合研究所 | 测量仪安装辅助装置和测量仪安装辅助方法 |
US11986319B2 (en) | 2017-08-25 | 2024-05-21 | NEUROPHET Inc. | Patch guide method and program |
KR102060483B1 (ko) * | 2017-09-11 | 2019-12-30 | 뉴로핏 주식회사 | Tms 자극 내비게이션 방법 및 프로그램 |
US11058887B2 (en) | 2017-12-21 | 2021-07-13 | NeuraLace Medical, Inc. | Devices, systems, and methods for non-invasive chronic pain therapy |
US11291852B2 (en) | 2018-02-06 | 2022-04-05 | The Board Of Trustees Of The Leland Stanford Junior University | Accurate patient-specific targeting of cranial therapy using a brain atlas |
CN109260593B (zh) * | 2018-09-27 | 2020-09-08 | 武汉资联虹康科技股份有限公司 | 一种经颅磁刺激治疗方法及设备 |
CN110896609B (zh) * | 2018-09-27 | 2023-09-26 | 武汉资联虹康科技股份有限公司 | 一种用于经颅磁刺激治疗的tms定位导航方法 |
WO2020172779A1 (zh) * | 2019-02-26 | 2020-09-03 | 武汉资联虹康科技股份有限公司 | 一种用于经颅磁刺激诊疗的摄像*** |
CN110382046B (zh) * | 2019-02-26 | 2023-11-24 | 武汉资联虹康科技股份有限公司 | 一种基于摄像头的经颅磁刺激诊疗检测*** |
CN110896611B (zh) * | 2019-02-26 | 2023-11-21 | 武汉资联虹康科技股份有限公司 | 一种基于摄像头的经颅磁刺激诊疗导航*** |
CN111436905B (zh) * | 2020-04-03 | 2024-03-19 | 图湃(北京)医疗科技有限公司 | 一种光学相干层析血管造影方法及装置 |
US11779218B2 (en) * | 2020-05-18 | 2023-10-10 | The Trustees Of Columbia University In The City Of New York | System, method and computer-accessible medium for predicting response to electroconvulsice therapy based on brain functional connectivity patterns |
CN111657947B (zh) * | 2020-05-21 | 2022-07-05 | 四川大学华西医院 | 一种神经调控靶区的定位方法 |
CN111553979B (zh) * | 2020-05-26 | 2023-12-26 | 广州雪利昂生物科技有限公司 | 基于医学影像三维重建的手术辅助***及方法 |
CN112704486B (zh) * | 2021-01-14 | 2021-10-26 | 中国科学院自动化研究所 | 基于电磁仿真计算的tms线圈位姿图谱生成方法 |
CN114224486A (zh) * | 2021-12-15 | 2022-03-25 | 中国医学科学院生物医学工程研究所 | 一种用于声场和磁场正交定位的神经导航定位*** |
CN114176776A (zh) * | 2021-12-15 | 2022-03-15 | 中国医学科学院生物医学工程研究所 | 一种用于同步双线圈磁刺激的神经导航定位*** |
DE102023104761B3 (de) | 2023-02-27 | 2024-06-06 | Forbencap Gmbh | Vorrichtung zur nicht-invasiven Neurostimulation und chirurgische Vorrichtung |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09511430A (ja) * | 1994-09-01 | 1997-11-18 | マサチューセッツ インスティチュート オブ テクノロジー | 三次元データ組の登録システムおよび登録方法 |
JP2003180649A (ja) | 2001-10-17 | 2003-07-02 | Nexstim Oy | 磁気刺激量の計算方法及び装置 |
JP2004000636A (ja) | 2002-05-31 | 2004-01-08 | Nexstim Oy | 脳の磁気刺激のターゲティング方法及び装置 |
JP2007209531A (ja) * | 2006-02-09 | 2007-08-23 | Hamamatsu Univ School Of Medicine | 手術支援装置、方法及びプログラム |
WO2007123147A1 (ja) | 2006-04-18 | 2007-11-01 | Osaka University | 経頭蓋磁気刺激用頭部固定具及び経頭蓋磁気刺激装置 |
JP2008526422A (ja) * | 2005-01-13 | 2008-07-24 | メイザー サージカル テクノロジーズ リミテッド | 鍵穴脳神経外科用画像ガイドロボットシステム |
JP2008262555A (ja) * | 2007-03-20 | 2008-10-30 | National Univ Corp Shizuoka Univ | 形状情報処理方法、形状情報処理装置及び形状情報処理プログラム |
JP2009515147A (ja) | 2005-10-19 | 2009-04-09 | メサ・イメージング・アー・ゲー | 変調電磁波場を復調する装置およびその方法 |
JP2009529951A (ja) * | 2006-03-13 | 2009-08-27 | ブラッコ イメージング エス.ピー.エー. | 手術ナビゲーションプロセスを記録し精査するための方法および装置 |
WO2010143400A1 (ja) * | 2009-06-10 | 2010-12-16 | 三菱電機株式会社 | 画像照合装置及びこれを用いた患者位置決め装置 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3512992B2 (ja) | 1997-01-07 | 2004-03-31 | 株式会社東芝 | 画像処理装置および画像処理方法 |
US7087008B2 (en) * | 2001-05-04 | 2006-08-08 | Board Of Regents, The University Of Texas System | Apparatus and methods for delivery of transcranial magnetic stimulation |
US7711431B2 (en) * | 2003-08-04 | 2010-05-04 | Brainlab Ag | Method and device for stimulating the brain |
JP4868382B2 (ja) | 2005-05-17 | 2012-02-01 | 公立大学法人広島市立大学 | 磁気刺激における刺激部位の特定あるいはターゲッティングを行うための装置 |
US7925066B2 (en) * | 2006-09-13 | 2011-04-12 | Nexstim Oy | Method and apparatus for correcting an error in the co-registration of coordinate systems used to represent objects displayed during navigated brain stimulation |
US8811692B2 (en) * | 2007-04-17 | 2014-08-19 | Francine J. Prokoski | System and method for using three dimensional infrared imaging for libraries of standardized medical imagery |
US8010177B2 (en) | 2007-04-24 | 2011-08-30 | Medtronic, Inc. | Intraoperative image registration |
WO2012164172A1 (en) * | 2011-06-03 | 2012-12-06 | Nexstim Oy | Method and system for combining anatomical connectivity patterns and navigated brain stimulation |
CA2887370C (en) * | 2011-09-27 | 2021-03-23 | The Maclean Hospital Corporation | Magnetic field stimulation |
-
2012
- 2012-03-08 EP EP12754418.7A patent/EP2684518A4/en not_active Withdrawn
- 2012-03-08 ES ES15159364T patent/ES2818078T3/es active Active
- 2012-03-08 WO PCT/JP2012/055995 patent/WO2012121341A1/ja active Application Filing
- 2012-03-08 JP JP2013503607A patent/JP6161004B2/ja not_active Expired - Fee Related
- 2012-03-08 US US14/004,060 patent/US9993655B2/en not_active Expired - Fee Related
- 2012-03-08 EP EP15159364.7A patent/EP2919194B1/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09511430A (ja) * | 1994-09-01 | 1997-11-18 | マサチューセッツ インスティチュート オブ テクノロジー | 三次元データ組の登録システムおよび登録方法 |
JP2003180649A (ja) | 2001-10-17 | 2003-07-02 | Nexstim Oy | 磁気刺激量の計算方法及び装置 |
JP2004000636A (ja) | 2002-05-31 | 2004-01-08 | Nexstim Oy | 脳の磁気刺激のターゲティング方法及び装置 |
JP2008526422A (ja) * | 2005-01-13 | 2008-07-24 | メイザー サージカル テクノロジーズ リミテッド | 鍵穴脳神経外科用画像ガイドロボットシステム |
JP2009515147A (ja) | 2005-10-19 | 2009-04-09 | メサ・イメージング・アー・ゲー | 変調電磁波場を復調する装置およびその方法 |
JP2007209531A (ja) * | 2006-02-09 | 2007-08-23 | Hamamatsu Univ School Of Medicine | 手術支援装置、方法及びプログラム |
JP2009529951A (ja) * | 2006-03-13 | 2009-08-27 | ブラッコ イメージング エス.ピー.エー. | 手術ナビゲーションプロセスを記録し精査するための方法および装置 |
WO2007123147A1 (ja) | 2006-04-18 | 2007-11-01 | Osaka University | 経頭蓋磁気刺激用頭部固定具及び経頭蓋磁気刺激装置 |
JP2008262555A (ja) * | 2007-03-20 | 2008-10-30 | National Univ Corp Shizuoka Univ | 形状情報処理方法、形状情報処理装置及び形状情報処理プログラム |
WO2010143400A1 (ja) * | 2009-06-10 | 2010-12-16 | 三菱電機株式会社 | 画像照合装置及びこれを用いた患者位置決め装置 |
Non-Patent Citations (8)
Title |
---|
HIROKAZU KATO; MARK BILLINGHURST; KOICHI ASANO; KEIHACHIRO TACHIBANA: "An Augmented Reality System and its Calibration based on Marker Tracking", TRANSACTIONS OF THE VIRTUAL REALITY SOCIETY OF JAPAN, vol. 4, no. 4, 1999 |
HIROYUKI YAMAMOTO; SHINJI UCHIYAMA; HIDEYUKI TAMURA: "Method for Generating The Delaunay Mesh for Three-Dimensional Modeling", IEICE TRANSACTIONS D-11, vol. J83-D-11, no. 5, May 1995 (1995-05-01), pages 745 - 753 |
MASAYUKI HIRATA ET AL.: "Preoperative and intraoperative evaluation of visual function using magnetoencephalography, electrocorticogram, fiber tracking and transcranial magnetic stimulation", RINSHO NOHA, vol. 51, no. 12, 1 December 2009 (2009-12-01), pages 721 - 728, XP008171556 * |
P. J. BESL; N. D. MCKAY: "A Method for Registration of 3-D Shapes", IEEE TRANS. PATTERN ANAL. MACHINE INTELL, vol. 14, no. 2, February 1992 (1992-02-01), pages 239 - 256 |
PAUL J.BESL ET AL.: "A Method for Registration of 3-D Shapes", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 14, no. 2, February 1992 (1992-02-01), pages 239 - 256, XP000248481 * |
PAUL VIOLA; MICHAEL JONES: "Object Detection using a Boosted Cascade of Simple", IEEE CVPR, 2001 |
RAINER LIENHART; JOCHEN MAYDT: "An Extended Set of Haar-like Feature for Rapid Object Detection", IEEE ICIP 2002, vol. 1, 2002, pages 900 - 903 |
See also references of EP2684518A4 |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2015098155A1 (ja) * | 2013-12-24 | 2017-03-23 | 国立大学法人大阪大学 | 操作教示装置および経頭蓋磁気刺激装置 |
US10675479B2 (en) | 2013-12-24 | 2020-06-09 | Osaka University | Operation teaching device and transcranial magnetic stimulation device |
WO2015098155A1 (ja) * | 2013-12-24 | 2015-07-02 | 国立大学法人大阪大学 | 操作教示装置および経頭蓋磁気刺激装置 |
US10292645B2 (en) | 2014-02-14 | 2019-05-21 | The University Of Tokyo | Intracerebral current simulation method and device thereof, and transcranial magnetic stimulation system including intracerebral current simulation device |
WO2015122369A1 (ja) * | 2014-02-14 | 2015-08-20 | 国立大学法人東京大学 | 脳内電流シミュレーション方法とその装置,及び脳内電流シミュレーション装置を含む経頭蓋磁気刺激システム |
JP2017029629A (ja) * | 2015-08-06 | 2017-02-09 | 東芝メディカルシステムズ株式会社 | 医用画像処理装置 |
JP2017164075A (ja) * | 2016-03-15 | 2017-09-21 | 富士フイルム株式会社 | 画像位置合せ装置、方法およびプログラム |
JP2019533218A (ja) * | 2016-08-10 | 2019-11-14 | コー・ヤング・テクノロジー・インコーポレーテッド | 3次元データ整合装置及び方法 |
US11481867B2 (en) | 2016-08-10 | 2022-10-25 | Koh Young Technology Inc. | Device and method for registering three-dimensional data |
JP2018047035A (ja) * | 2016-09-21 | 2018-03-29 | 学校法人自治医科大学 | 医療支援方法および医療支援装置 |
JP2018061837A (ja) * | 2016-10-11 | 2018-04-19 | バイオセンス・ウエブスター・(イスラエル)・リミテッドBiosense Webster (Israel), Ltd. | 撮像装置を備える磁気追跡システムの重ね合わせ法 |
JP7046553B2 (ja) | 2016-10-11 | 2022-04-04 | バイオセンス・ウエブスター・(イスラエル)・リミテッド | 撮像装置を備える磁気追跡システムの重ね合わせ法 |
JP2021180866A (ja) * | 2017-08-25 | 2021-11-25 | ニューロフェット インコーポレイテッドNeurophet Inc. | パッチガイド方法及びプログラム |
JP7271000B2 (ja) | 2017-08-25 | 2023-05-11 | ニューロフェット インコーポレイテッド | パッチガイド方法及びプログラム |
JP7486603B2 (ja) | 2020-04-22 | 2024-05-17 | ブレイン ナビ バイオテクノロジー カンパニー リミテッド | 拡張現実画像上に医用スキャン画像情報を補う方法及びシステム |
CN114190922A (zh) * | 2020-09-18 | 2022-03-18 | 四川大学 | Tms头动检测方法 |
CN114190922B (zh) * | 2020-09-18 | 2023-04-21 | 四川大学 | Tms头动检测方法 |
CN113243923A (zh) * | 2021-05-19 | 2021-08-13 | 中科芯未来微电子科技成都有限公司 | 一种提高心电图监测的准确性的方法和装置 |
Also Published As
Publication number | Publication date |
---|---|
JP6161004B2 (ja) | 2017-07-12 |
US9993655B2 (en) | 2018-06-12 |
JPWO2012121341A1 (ja) | 2014-07-17 |
EP2684518A1 (en) | 2014-01-15 |
EP2919194A1 (en) | 2015-09-16 |
US20130345491A1 (en) | 2013-12-26 |
EP2684518A4 (en) | 2014-09-17 |
EP2919194B1 (en) | 2020-08-19 |
ES2818078T3 (es) | 2021-04-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6161004B2 (ja) | 画像データ処理装置および経頭蓋磁気刺激装置 | |
US11744465B2 (en) | Method and program for generating three-dimensional brain map | |
EP3309749B1 (en) | Registration of a magnetic tracking system with an imaging device | |
US11745012B2 (en) | Method and program for navigating TMS stimulation | |
US11576645B2 (en) | Systems and methods for scanning a patient in an imaging system | |
CN103371870B (zh) | 一种基于多模影像的外科手术导航*** | |
ES2550455T3 (es) | Método y aparato para corregir un error en el registro conjunto de sistemas de coordenadas usados para representar objetos visualizados durante la estimulación guiada del cerebro | |
EP2965285B1 (en) | Scan region determining apparatus | |
Boctor et al. | A novel closed form solution for ultrasound calibration | |
US11576578B2 (en) | Systems and methods for scanning a patient in an imaging system | |
Ettinger et al. | Experimentation with a transcranial magnetic stimulation system for functional brain mapping | |
RU2594811C2 (ru) | Визуализация для навигационного указания | |
JP6644795B2 (ja) | 解剖学的オブジェクトをセグメント化する超音波画像装置及び方法 | |
JP6550660B2 (ja) | 操作教示装置および経頭蓋磁気刺激装置 | |
EP3292835B1 (en) | Ent image registration | |
US20230074362A1 (en) | Method and system for non-contact patient registration in image-guided surgery | |
US20220309690A1 (en) | Method and system for non-contact patient registration in image-guided surgery | |
EP4091567A1 (en) | Technique of providing user guidance for obtaining a registration between patient image data and a surgical tracking system | |
CN116580820B (zh) | 基于多模态医学图像的经******穿刺智能麻醉*** | |
JP6795744B2 (ja) | 医療支援方法および医療支援装置 | |
Ettinger | Hierarchical three-dimensional medical image registration | |
Abdalbari | A framework of Image Processing for Image-guided Procedures | |
Chemaly | Multimodality image registration for visualization in robotic assisted breast biopsy | |
CN116982079A (zh) | 用于图像引导手术中的非接触式患者配准的方法和*** | |
WO2024069498A1 (en) | Three-dimensional model generation for tumor treating fields transducer layout |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12754418 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2013503607 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14004060 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2012754418 Country of ref document: EP |