WO2020172779A1 - Système de caméra pour thérapie par stimulation magnétique transcrânienne - Google Patents

Système de caméra pour thérapie par stimulation magnétique transcrânienne Download PDF

Info

Publication number
WO2020172779A1
WO2020172779A1 PCT/CN2019/076100 CN2019076100W WO2020172779A1 WO 2020172779 A1 WO2020172779 A1 WO 2020172779A1 CN 2019076100 W CN2019076100 W CN 2019076100W WO 2020172779 A1 WO2020172779 A1 WO 2020172779A1
Authority
WO
WIPO (PCT)
Prior art keywords
head
patient
camera
magnetic stimulation
treatment
Prior art date
Application number
PCT/CN2019/076100
Other languages
English (en)
Chinese (zh)
Inventor
孙聪
王波
蔡胜安
Original Assignee
武汉资联虹康科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 武汉资联虹康科技股份有限公司 filed Critical 武汉资联虹康科技股份有限公司
Priority to CN201980001093.0A priority Critical patent/CN110300993B/zh
Priority to PCT/CN2019/076100 priority patent/WO2020172779A1/fr
Publication of WO2020172779A1 publication Critical patent/WO2020172779A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/30—Assessment of water resources

Definitions

  • the invention relates to the technical field of medical equipment, in particular to a camera system used for transcranial magnetic stimulation diagnosis and treatment.
  • the total number of patients with mental illness in my country has exceeded 100 million, but the public awareness rate of mental illness is less than 50%, and the rate of consultation is even lower. At present, about 20% of these mental patients receive timely treatment, and 80% of the mental patients are not treated in time, or even the most basic treatment.
  • the number of patients with mental illness is as high as 16 million.
  • global mental illness medications have exceeded 36 billion US dollars, accounting for 5% of total drug sales.
  • domestically the current mental illness drug market is still relatively small, accounting for about 1.5% of the total hospital sales.
  • Transcranial Magnetic Stimulation is a technique that uses a pulsed magnetic field to generate an electric current in the local cerebral cortex to temporarily activate or inhibit the cortex.
  • Transcranial magnetic stimulation technology has been widely used in epilepsy, neuropsychology, rehabilitation, pediatrics and other fields.
  • medical staff need to aim the treatment equipment at the patient’s head to be treated. Discomfort caused by the patient’s long-term maintenance of the same posture will cause the patient to change the head posture.
  • Medical staff need to always pay attention to the treatment instrument and Whether the patient’s head is aligned or not, this method is costly, time-consuming, and large in error. Therefore, medical staff urgently need a fast, high-precision head posture tracking system that can truly feedback the patient’s head posture to the mechanical control equipment .
  • the traditional TMS treatment method usually includes the following steps: the operator finds the treatment magnetic stimulation point area on the patient's head (location based on experience); places the TMS coil on this area, ensuring that the center of the coil is directly opposite the magnetic stimulation point, and the coil plane is with the scalp Tangent; Use mechanical devices to clamp and fix the TMS coil; start TMS and start treatment. Because TMS treatment usually takes a long time, according to the course of treatment, the magnetic stimulation points that are located purely by experience and memory cannot guarantee the accuracy of each treatment.
  • the problem of TMS accurate repeat positioning has been studied worldwide, but most of the other methods currently are based on the infrared positioning method of the light guide ball.
  • a special light guide ball must be worn for the patient and the TMS coil every time it is positioned, and the positioning is performed according to the relative position of the light guide ball in the infrared camera. Because the position of wearing is not fixed each time, it is necessary to additionally use a special positioning pen to determine the relative position of the light guide ball and the human head through complex positioning. Therefore, the operation is quite inconvenient and inefficient.
  • the time for wearing and positioning the light guide ball is about 30 minutes per person. Since a TMS treatment only takes 10-20 minutes, the current positioning method is not suitable for TMS Repeat positioning efficiently.
  • a more accurate positioning method can be a transcranial magnetic stimulation navigation system.
  • a machine vision positioning navigation system such as the Chinese patent "Transcranial Magnetic Stimulation Navigation System and Transcranial Magnetic Stimulation Coil Positioning Method” 201210281507.X , 201210281472.X; the other is an optical positioning navigation system, such as the Chinese patent "a navigational transcranial magnetic stimulation treatment system” 201010235826.8, "a calibration device and method for repetitive transcranial magnetic stimulation optical positioning navigation system” 201010235828.7.
  • the machine vision positioning and navigation system requires a positioning cap.
  • the thickness of the positioning cap increases the distance between the coil and the cerebral cortex, increases the stimulation intensity, and causes a waste of energy, especially when transcranial magnetic stimulation and EEG recording are required at the same time.
  • Positioning caps and EEG electrode caps will increase the distance between the coil and the cerebral cortex.
  • RGBD camera is one of the cameras that can obtain image information and distance information at the same time. Therefore, the present invention combines the advantages that the RGBD camera can obtain image information and distance information at the same time.
  • the RGBD camera is used to photograph the patient's posture, and then the photographed image is processed to locate the therapeutic magnet. Stimulation point, without the need to wear a light guide ball to ensure the precise positioning of the magnetic stimulation point during each treatment.
  • the purpose of the present invention is to solve the problems existing in the prior art and provide a camera system for transcranial magnetic stimulation diagnosis and treatment.
  • the patient's head posture can be obtained quickly and accurately, and adjusted according to the head posture
  • Treatment of magnetic stimulation points improves the accuracy of magnetic stimulation point positioning and repeated positioning in TMS treatment.
  • a camera system for transcranial magnetic stimulation diagnosis and treatment comprising a first photographing subsystem for head modeling, a second photographing subsystem for diagnosis and treatment, and an intelligent terminal.
  • the first photographing subsystem includes 3D scanning
  • the second photographing subsystem includes a 3D camera, a lying bed, and a headrest; the first photographing subsystem sends the photographed head data to the intelligent terminal for head modeling, and obtains A 3D head model that identifies facial feature points; the second photographing subsystem sends the face data captured in real time to a smart terminal for face detection, and the smart terminal compares the detected face image with the 3D head model The matching is performed to obtain the location information of the magnetic stimulation point for transcranial magnetic stimulation diagnosis and treatment.
  • the 3D scanner includes a 3D camera and a rotating bracket, the 3D camera is mounted on the rotating bracket, the rotating bracket is driven to rotate by a motor, and the motor is electrically connected to the smart terminal;
  • the intelligent terminal controls the motor to drive the rotating bracket to rotate at a constant speed, so that the 3D camera moves in a circular motion around the patient's head at a constant speed, and collects 3D image data of the patient's head from all directions.
  • the 3D scanner includes a plurality of 3D cameras and a fixed bracket, and the plurality of 3D cameras are all installed on the fixed bracket; when collecting 3D image data of the patient's head, the plurality of 3D cameras are controlled by an intelligent terminal The camera simultaneously collects 3D image data of the patient's head from different directions.
  • the image data captured by the 3D camera includes color images, depth images, and 3D point cloud images.
  • the 3D camera in the second photographing sub-system is arranged above the patient's face and can fully incorporate his face into the photographing range.
  • the positioning cap is a white hood made of elastic material, which is used to cover the patient's hair; the positioning cap is provided with a number of Mark points to facilitate the 3D camera to collect image data.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment modeling system adopts the camera system, and the modeling method of the modeling system includes the following steps:
  • S1 Start the smart terminal, collect 3D image data of the patient's head wearing the positioning cap from various directions through the 3D scanner, and send the collected 3D image data to the smart terminal;
  • step S2 the method for integrating the 3D image data collected from various directions is: calculating the matching relationship between the images by identifying the feature points in the images collected in each direction, and then using 3D
  • the point cloud ICP algorithm obtains the spatial position relationship between the point cloud images collected in various directions, and finally, according to the matching relationship and the spatial position relationship, all the point cloud image data are rotated and translated to obtain a complete head of the patient 3D point cloud image.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment detection system adopts the camera system, and the detection method of the detection system includes the following steps:
  • the 3D camera is used to take image data of the patient's head, and the intelligent terminal is used to model the head to establish a 3D head model of the patient's head;
  • the 3D camera is used to take a real-time facial image of the patient
  • the smart terminal is used to perform pose matching
  • the real-time facial image is matched with the established 3D head model for position matching, which further includes:
  • the facial feature points used for matching are marked in the 3D head model;
  • the facial feature points of the patient's real-time facial image are automatically recognized by the 3D camera;
  • the conversion matrix is obtained by affine transformation through feature point matching, and the patient's real-time facial image is calculated
  • the conversion relationship with the established 3D head mold calculate the position of the 3D head mold in space; calculate the position coordinates of the magnetic stimulation points on the 3D head mold in space.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment navigation system using the camera system, further comprising a manipulator and a TMS coil, the manipulator and the TMS coil are respectively electrically connected to an intelligent terminal, and the navigation method of the navigation system includes the following steps :
  • S3 Match the position of the patient's head model with the actual position of the patient's head through the 3D camera and the smart terminal, and determine the spatial position of the magnetic stimulation point on the patient's head model to be magnetically stimulated;
  • step S5 Put the device model built in step S4 and the patient head model built in step S2 in the same spatial coordinate system; and then calculate through the smart terminal that the TMS coil model arrives on the head model for magnetic stimulation
  • the smart terminal automatically navigates the movement of the manipulator according to the optimal path, and finally moves the TMS coil to the magnetic stimulation point on the patient's head for treatment.
  • step S3 the method of matching the position of the patient's head mold with the actual position of the patient's head includes the following steps:
  • step S33 Perform a matching calculation between the facial feature points marked in step S31 and the facial feature points identified in step S32 to obtain the rotation and translation relationship between the patient's head and the patient's head model;
  • S34 Perform rotation and translation operations on the patient's head mold according to the rotation and translation relationship, so that the position of the patient's head mold matches the actual position of the patient's head.
  • step S4 after modeling the manipulator, TMS coil, and 3D camera, the spatial positions of the manipulator model, TMS coil model, and 3D camera model need to be compared with the actual spatial positions of the manipulator, TMS coil, and 3D camera.
  • Matching the specific matching method is:
  • step S43 Perform matching calculation on the feature points marked in step S41 and the feature points identified in step S42 to obtain the rotation and translation relationship between the manipulator model and the manipulator;
  • step S45 Perform rotation and translation operations on the manipulator model, TMS coil model, and 3D camera model according to the rotation and translation relationship in step S43 and step S44, so that the manipulator model, TMS coil model, and 3D camera model are positioned in space They are matched with the actual spatial positions of the manipulator, TMS coil and 3D camera.
  • the detection method further includes: during the magnetic stimulation treatment of the patient's head, the smart terminal also uses a 3D camera to follow the positioning of the patient's head; during the treatment, it will record each time the positioning is completed.
  • the position information of the magnetic stimulation point on the patient's head If the distance between the current and the previous magnetic stimulation point is more than 5mm due to the patient's head movement in the next moment, follow-up positioning will be started; if it is not more than 5mm, follow-up positioning will not be started .
  • the navigation method further includes a following positioning step, and the following positioning step includes: adjusting the spatial pose of the patient's head model through the smart terminal so that the spatial pose of the patient's head model is consistent with that of the patient's head.
  • the current actual spatial pose is matched, and then the latest magnetic stimulation point is repositioned on the head model, and finally the moving path of the manipulator is re-planned, and the TMS coil is moved to the latest magnetic stimulation point for treatment.
  • the present invention uses a camera to capture video image data of the patient’s head, models the patient’s head, and based on the modeling data and real-time facial video captured The image detects the patient’s face posture, obtains the patient’s face posture data, and then performs manipulator navigation based on the face posture data to locate the TMS treatment magnetic stimulation point.
  • This method has high positioning accuracy and good real-time performance, which solves the existing problem.
  • the problem of precise positioning and repeated positioning in TMS diagnosis and treatment 2)
  • the present invention uses the camera to directly obtain the face image, performs 3D modeling, and combines the face detection and recognition algorithm to estimate the accurate face pose.
  • the pose positioning is accurate and real-time It has strong performance and does not need to wear any auxiliary tools on the patient's head;
  • the present invention uses a manipulator to carry out TMS coil navigation and positioning, with rapid positioning and high repeat accuracy.
  • Fig. 1 is a schematic diagram of a camera system according to an embodiment of the present invention
  • Fig. 2 is a schematic diagram of a first photographing subsystem according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of a first photographing subsystem according to another embodiment of the present invention.
  • FIG. 4 is a schematic diagram of the distribution of cameras in the first photographing subsystem according to another embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of a modeling system according to another embodiment of the present invention.
  • Fig. 6 is a schematic structural diagram of a detection system according to another embodiment of the present invention.
  • FIG. 7 is a schematic flowchart of a detection system according to another embodiment of the present invention.
  • FIG. 8 is a schematic flowchart of a navigation system according to another embodiment of the present invention.
  • connection should be understood in a broad sense.
  • it can be a mechanical connection or an electrical connection, or the internal communication between two components, which can be directly connected, or through an intermediate connection.
  • the media is indirectly connected, and those of ordinary skill in the art can understand the specific meanings of the above-mentioned terms according to specific circumstances.
  • a camera system for transcranial magnetic stimulation diagnosis and treatment includes a first photographing subsystem for head modeling and a second photographing sub-system for diagnosis and treatment.
  • the first photographing subsystem includes a 3D scanner and a positioning cap
  • the second photographing subsystem includes a 3D camera, a lying bed and a headrest; the first photographing subsystem will photograph the head
  • the part data is sent to the smart terminal for head modeling to obtain a 3D head model with identified facial feature points
  • the second photographing subsystem sends the real-time photographed face data to the smart terminal for face detection
  • the smart terminal matches the detected face image with the 3D head model to obtain magnetic stimulation point location information for transcranial magnetic stimulation diagnosis and treatment.
  • the 3D scanner includes a 3D camera and a rotating bracket, the 3D camera is mounted on the rotating bracket, the rotating bracket is driven to rotate by a motor, and the motor is electrically connected to the smart terminal. Connection; when collecting 3D image data of the patient's head, the intelligent terminal controls the motor to drive the rotating bracket to rotate at a uniform speed, so that the 3D camera moves in a circular motion around the patient's head at a uniform speed, and collects 3D image data of the patient's head from all directions. The collected 3D image data is sent to an intelligent terminal, and the intelligent terminal performs head modeling to obtain a 3D model of the patient's head.
  • the 3D camera in the second photographing subsystem is set above the patient’s face and can fully incorporate the patient’s face into the position of the photographing range, and is used to photograph the patient’s face and facial data in real time during the TMS diagnosis and treatment process, and to photograph
  • the face and face data of is sent to the smart terminal, and the smart terminal performs face detection, and performs manipulator navigation according to the face detection result, and controls the TMS coil to move to the therapeutic magnetic stimulation point.
  • the image data captured by the 3D camera includes color images, depth images, and 3D point cloud images.
  • the positioning cap is a white hood made of elastic material, which is used to cover the patient's hair; because the 3D scanner cannot scan black hair without heat, the white hood needs to be used to cover the hair and expose the patient
  • the facial features and forehead are marked with characteristic points (brow center, nose tip, etc.);
  • the positioning cap is flexible, suitable for a wide range of people, and convenient to wear;
  • the positioning cap is provided with a number of Mark points to facilitate the 3D camera to collect image data.
  • the 3D scanner in the first photographing subsystem includes three 3D cameras and a fixed bracket.
  • the fixed bracket is provided with three camera installation positions, the angle between two adjacent camera installation positions is 120 degrees, and the three 3D cameras are respectively installed on the three camera installation positions ;
  • the three 3D cameras are controlled by an intelligent terminal to simultaneously collect 3D image data of the patient's head from three directions.
  • the 3D image data of the patient's head is simultaneously collected through three 3D cameras, and the collected data is sent to the intelligent terminal for head modeling, which has good real-time performance.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment head model modeling system which includes a 3D scanner, a positioning cap, a seat, and a smart terminal.
  • the 3D scanner and the smart terminal are electrically connected ;
  • the smart terminal may be a computer.
  • the modeling method of the head model modeling system includes the following steps:
  • S1 The patient sits on the seat and wears the positioning cap, starts the smart terminal, collects 3D image data of the patient's head from various directions through the 3D scanner, and then integrates the collected 3D image data Sent to the smart terminal;
  • the positioning cap is a white hood made of elastic material, which is used to cover the patient’s hair; because the 3D scanner cannot scan black hair without heat, it needs to be covered by the white hood Hair, exposing the facial features and forehead of the patient, and marking the characteristic points (brow center, nose tip, etc.); the positioning cap is flexible, suitable for a wide range of people, and convenient to wear; the positioning cap is provided with several Mark points, which is convenient for 3D camera to collect images data.
  • step S2 the method for integrating the 3D image data collected from various directions is: calculating the matching relationship between the images by identifying the feature points in the images collected in each direction, and then using 3D
  • the point cloud ICP algorithm obtains the spatial position relationship between the point cloud images collected in various directions, and finally, according to the matching relationship and the spatial position relationship, all the point cloud image data are rotated and translated to obtain a complete head of the patient 3D point cloud image.
  • 3D scan data of the patient's head it is necessary to collect 3D scan data of the patient's head through a 3D camera.
  • a 3D camera takes a photo
  • a color map, a depth map, and a 3D point cloud map are generated.
  • These three images are generated at the same time, so each The points on the image have a fixed correspondence. This correspondence is known and is obtained through the calibration of the camera; 3D scanning is to take a series of images around the patient’s head, and then stitch these images into a complete image.
  • the image stitching needs to find the same parts in the two images and match them; in the 3D camera, the 3D point cloud cannot be obtained for the hair, and the 3D data of the skull is needed in the medical treatment of the head model (no hair), so the patient is in the head model scan You need to wear a specific positioning cap. In order to make the matching more accurate, some mark points are usually set on the cap; 3D scanning finally needs to stitch the 3D point cloud, and the rotation and translation relationship between the point clouds of each image is required for stitching.
  • the splicing of point clouds mainly relies on the ICP algorithm, which sometimes fails, so rough matching is required first.
  • the mapping method is to compare the four points NZ, CZ, AL, and AR on the patient's head with the four points on the skull model to obtain the skull model conversion matrix, and then transfer the data in the MNI space
  • the point of is multiplied by the transformation matrix to obtain the coordinate points of the patient’s head model; where NZ represents the root of the nose, AL represents the left ear, AR represents the right ear, and CZ represents the line connecting the root of the nose and the occipital protuberance to the left and right ears. The intersection of the lines.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment system includes a lying bed, a headrest 2, a 3D camera, a manipulator, a TMS coil, and a smart terminal; the 3D camera, manipulator, and TMS coil are connected to the smart terminal respectively; the smart terminal can choose a computer or a notebook , Tablet, etc.
  • the lying bed is a horizontal translation platform, which can move back and forth, and is used to adjust the relative position of the patient's head and the camera.
  • the headrest 2 mainly functions as a bracket, and the supporting site is the skull and also includes the neck.
  • the function is to limit the movement of the patient without causing discomfort to the patient, and cannot hinder the magnetic stimulation of the head.
  • the 3D camera is used to obtain the patient's head posture data and real-time facial posture data. Before the treatment, the 3D camera is used to obtain the patient's head posture data, and the intelligent terminal is used for head 3D modeling; after the treatment is started , Use a 3D camera to obtain the patient's real-time facial data, combine the intelligent terminal to process the real-time facial data, and match the modeled 3D head model with the real-time facial image.
  • the 3D camera is also used to obtain the spatial pose of the manipulator and the TMS coil, so that the manipulator is used for navigation and clamps the TMS coil to the position of the magnetic stimulation point.
  • the manipulator is also used to clamp the TMS coil to stimulate the magnetic stimulation points on the patient's head for magnetic stimulation treatment.
  • the detection method of the detection system includes the following steps:
  • the patient lies flat on the reclining bed, and adjusting the front and back positions of the reclining bed so that the reclining bed reaches the treatment position;
  • the 3D camera is used to capture the image data of the patient's head, and the intelligent terminal is used for modeling to establish a 3D head model of the patient's head;
  • the 3D camera is used to take a real-time facial image of the patient
  • the smart terminal is used to perform pose matching
  • the real-time facial image is matched with the established 3D head model for position matching, which further includes:
  • the facial feature points for matching are marked in the 3D head model, and the facial feature points are automatically recognized by the camera during the modeling process;
  • the facial feature points of the patient’s real-time facial image are automatically recognized by the 3D camera;
  • the feature points are passed Matching and performing affine transformation to obtain a conversion matrix, calculate the conversion relationship between the patient’s real-time facial image and the established 3D head model; calculate the position of the 3D head model in the camera coordinate system; calculate the magnetic field on the 3D head model The position coordinates of the stimulus point in space.
  • modeling the patient's head includes the following steps:
  • S22 The smart terminal integrates the 3D image data to obtain a complete 3D point cloud image of the patient's head, and then obtains the complete 3D head model data of the patient's head through sampling, smoothing, and plane fitting processing;
  • the method of integrating the 3D image data is: calculating the matching relationship between the images by identifying the feature points in the images collected in each direction, and then obtaining the point clouds collected in each direction through the ICP algorithm of the 3D point cloud The spatial position relationship between the images, and finally all the point cloud image data are rotated and translated according to the matching relationship and the spatial position relationship to obtain a complete 3D point cloud image of the patient's head;
  • the 3D image captured by the 3D camera in real time only has facial information of the patient, and no head information. Therefore, the head model built in S2 must be registered with the facial data captured in real time. Due to the ICP algorithm The amount of calculation is too large to meet the requirements of real-time detection.
  • the position registration method is to first mark the facial feature points (corners of the eyes, nose tip, etc.) for registration in the head model, and then automatically identify the facial feature points in the real-time image. Through feature point matching, the conversion relationship between the real-time image and the head mold is calculated, and the position of the head mold in space is calculated, and then the position coordinates of the magnetic stimulation points on the head mold in space are calculated.
  • the conversion relationship includes the rotation and translation relationship between the patient's real-time facial image and the 3D head model in the camera coordinate system.
  • the 3D head model is rotated and translated according to the rotation and translation relationship, and the 3D The head model is matched to the patient's real-time facial image.
  • a camera-based transcranial magnetic stimulation diagnosis and treatment navigation system includes a lying bed, a headrest 2, a 3D camera, a manipulator, a TMS coil, and a smart terminal; the 3D camera, the manipulator, and the TMS coil are respectively electrically connected to the smart terminal; the smart terminal may be a computer;
  • the lying bed is a horizontal translation platform, which can move back and forth, and is used to adjust the relative position of the patient's head and the camera;
  • the headrest 2 mainly functions as a bracket, and the supporting site is the skull and also includes the neck.
  • the function is to limit the movement of the patient without causing discomfort to the patient, and cannot hinder the magnetic stimulation of the back of the head;
  • the 3D camera is used to obtain the spatial pose of the patient's head and the manipulator, so as to navigate the manipulator;
  • the manipulator is used to clamp the TMS coil to stimulate the magnetic stimulation points on the patient's head for magnetic stimulation treatment;
  • the navigation method of the navigation system includes the following steps:
  • S3 Match the position of the patient's head model with the actual position of the patient's head through the 3D camera and the smart terminal, and determine the spatial position of the magnetic stimulation point on the patient's head model to be magnetically stimulated;
  • step S5 Put the device model built in step S4 and the patient head model built in step S2 in the same spatial coordinate system; and then calculate through the smart terminal that the TMS coil model arrives on the head model for magnetic stimulation
  • the optimal path of the point the shortest moving distance and no collision with other devices during the movement
  • the smart terminal then automatically navigates the movement of the manipulator according to the optimal path, and finally moves the TMS coil to the patient
  • the head is to be treated by magnetic stimulation at the magnetic stimulation point.
  • modeling the patient's head includes the following steps:
  • S22 Integrate the 3D image data through the smart terminal to obtain a complete 3D point cloud image of the patient's head, and then obtain complete 3D head model data of the patient's head after sampling, smoothing, and plane fitting;
  • the method of integrating the 3D image data is: calculating the matching relationship between the images by identifying the feature points in the images collected in each direction, and then obtaining the point clouds collected in each direction through the ICP algorithm of the 3D point cloud The spatial position relationship between the images, and finally all the point cloud image data are rotated and translated according to the matching relationship and the spatial position relationship to obtain a complete 3D point cloud image of the patient's head;
  • step S3 the 3D image captured by the 3D camera in real time has only the patient’s facial information and no head information. Therefore, the head model built in S2 must be registered with the real-time facial data. Due to the ICP algorithm The amount of calculation is too large to meet the requirements of real-time detection.
  • the position registration method is to first mark the facial feature points (brow center, earlobe, corner of the eye, nose tip, corner of the mouth and chin) for registration in the head model, and then in the real-time image Automatically recognize facial feature points, calculate the conversion relationship between the real-time image and the head model through feature point matching, and calculate the position of the head model in space, and then calculate the position coordinates of the magnetic stimulation points on the head model in space; the specific steps are as follows :
  • step S33 Perform a matching calculation between the facial feature points marked in step S31 and the facial feature points identified in step S32 to obtain the rotation and translation relationship between the patient's head and the patient's head model;
  • S34 Perform rotation and translation operations on the patient's head mold according to the rotation and translation relationship, so that the position of the patient's head mold matches the actual position of the patient's head.
  • step S4 SolidWorks software can be used to model the manipulator, TMS coil and 3D camera.
  • the spatial positions of the manipulator model, TMS coil model, and 3D camera model need to be compared with the manipulator, The actual spatial position of the TMS coil and the 3D camera are matched; the specific matching method is:
  • step S43 Perform matching calculation between the feature points marked in step S41 and the feature points identified in step S42 to obtain the rotation and translation relationship between the manipulator model and the manipulator;
  • step S45 Perform rotation and translation operations on the manipulator model, TMS coil model, and 3D camera model according to the rotation and translation relationship in step S43 and step S44, so that the manipulator model, TMS coil model, and 3D camera model are positioned in space They are matched with the actual spatial positions of the manipulator, TMS coil and 3D camera.
  • step S5 the general movement path planning algorithm of the manipulator is relatively complicated. Since the model, obstacles, and path in this embodiment are all known, the method of manually planning the path is adopted, and the distance from the head mold is far (greater than Use a straight path at the position of 30mm), and use a circular path near the head mold (less than/equal to 30mm) to move the TMS coil around the head to the next magnetic stimulation point; since the 3D data of the head mold is known, Therefore, the head mold data can be enlarged to leave a safe distance for operation, and the shortest arc path between two points on the head mold can be calculated.
  • the present invention avoids the influence of human factors on the treatment effect, and at the same time improves the patient's sense of experience.
  • the present invention also has a following positioning function. In the process of navigating the manipulator, or in the process of face detection, even if the posture of the patient's head changes, the posture of the patient's head can be tracked in real time through the 3D camera to ensure the accuracy of treatment and improve Treatment effect and patient experience.
  • the smart terminal also uses a 3D camera to follow the positioning of the patient's head; during the treatment, the position information of the patient's head is recorded every time the positioning is completed, If the distance between the current and the previous magnetic stimulation point is more than 5mm due to the movement of the patient’s head at the next moment, follow-up positioning will be activated; if it does not exceed 5mm, follow-up positioning will not be activated; if the patient’s head turns more often, Then the 3D camera and the manipulator will be suspended, and the magnetic stimulation of the TMS coil will be suspended at the same time; if the patient is not within the adjustable range of the 3D camera or left, the magnetic stimulation action of the manipulator and the coil will be stopped.
  • the step of following positioning is: adjusting the spatial pose of the patient's head model through the smart terminal, so that the spatial pose of the patient's head model is matched with the current actual spatial pose of the patient's head, Then reposition the latest magnetic stimulation site on the head model, and finally re-plan the moving path of the manipulator, and move the TMS coil to the latest magnetic stimulation site for treatment.
  • the invention uses a camera to capture video image data of the patient's head, models the patient's head, and detects and estimates the patient's facial posture based on the modeling data and the captured facial video image to obtain the patient's facial posture According to the data, the manipulator navigation is carried out according to the face posture data, and the TMS treatment magnetic stimulation point is adjusted. It ensures the accurate positioning of the magnetic stimulation point during each treatment without the need to wear a light guide ball, which solves the problem of TMS positioning and repeated positioning .

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Epidemiology (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)
  • Magnetic Treatment Devices (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un système de caméra pour une thérapie par stimulation magnétique transcrânienne. Le système de caméra comprend un premier sous-système de photographie pour la modélisation de la tête, un second sous-système de photographie pour la thérapie, et un terminal intelligent (9), le premier sous-système de photographie comprenant un scanner 3D (8) et un embout de positionnement, et le second sous-système de photographie comprenant une caméra 3D (3), un lit de type inclinable (1) et un appui-tête (2). Le premier sous-système de photographie envoie des données de la tête photographiée au terminal intelligent (9) pour la modélisation de la tête pour obtenir un modèle de tête 3D avec des points de caractéristique faciale marqués ; le second sous-système de photographie envoie des données faciales photographiées en temps réel au terminal intelligent (9) pour la détection faciale ; et le terminal intelligent (9) fait coïncider une image faciale détectée avec le modèle de tête 3D pour obtenir des informations de positionnement du point de stimulation magnétique pour une thérapie de stimulation magnétique transcrânienne. La posture de tête d'un patient est acquise de manière rapide et précise au moyen de la caméra 3D (3), et des points de stimulation magnétique thérapeutique sont ajustés sur la base de la posture de tête, ce qui permet d'améliorer la précision de positionnement et de repositionnement du point de stimulation magnétique pendant une thérapie TMS.
PCT/CN2019/076100 2019-02-26 2019-02-26 Système de caméra pour thérapie par stimulation magnétique transcrânienne WO2020172779A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980001093.0A CN110300993B (zh) 2019-02-26 2019-02-26 一种用于经颅磁刺激诊疗的摄像***
PCT/CN2019/076100 WO2020172779A1 (fr) 2019-02-26 2019-02-26 Système de caméra pour thérapie par stimulation magnétique transcrânienne

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/076100 WO2020172779A1 (fr) 2019-02-26 2019-02-26 Système de caméra pour thérapie par stimulation magnétique transcrânienne

Publications (1)

Publication Number Publication Date
WO2020172779A1 true WO2020172779A1 (fr) 2020-09-03

Family

ID=68033040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/076100 WO2020172779A1 (fr) 2019-02-26 2019-02-26 Système de caméra pour thérapie par stimulation magnétique transcrânienne

Country Status (2)

Country Link
CN (1) CN110300993B (fr)
WO (1) WO2020172779A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634336A (zh) * 2020-12-31 2021-04-09 华科精准（北京）医疗科技有限公司 一种配准方法及***

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111216109A (zh) * 2019-10-22 2020-06-02 东北大学 一种用于临床治疗与检测的视觉跟随装置及其方法
CN111249622B (zh) * 2020-01-17 2024-04-09 上海市精神卫生中心（上海市心理咨询培训中心） 基于增强现实的精准经颅磁刺激在线导航方法
CN111729200B (zh) * 2020-07-27 2022-06-17 浙江大学 基于深度相机和磁共振的经颅磁刺激自动导航***和方法
CN113041500A (zh) * 2021-03-31 2021-06-29 南京伟思医疗科技股份有限公司 一种可记忆经颅磁刺激导航定位装置与定位方法
CN115008753A (zh) * 2022-06-28 2022-09-06 德清县第三人民医院 一种3d打印精准定位导航经颅磁刺激头模及制作方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060161039A1 (en) * 2005-01-20 2006-07-20 Neuronetics, Inc. Articulating arm
CN102814002A (zh) * 2012-08-08 2012-12-12 深圳先进技术研究院 经颅磁刺激导航***及经颅磁刺激线圈定位方法
US20130345491A1 (en) * 2011-03-09 2013-12-26 A School Corporation Kansai University Image data processing device and transcranial magnetic stimulation apparatus
CN107497049A (zh) * 2017-09-30 2017-12-22 武汉资联虹康科技股份有限公司 一种用于经颅磁刺激器的电磁定位导航方法及装置
US20170372006A1 (en) * 2014-02-10 2017-12-28 Neuronetics, Inc. Head modeling for a therapeutic or diagnostic procedure
CN109173063A (zh) * 2018-09-21 2019-01-11 武汉资联虹康科技股份有限公司 一种3d扫描的经颅磁刺激***及经颅磁刺激方法
CN109260593A (zh) * 2018-09-27 2019-01-25 武汉资联虹康科技股份有限公司 一种经颅磁刺激治疗方法及设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7651459B2 (en) * 2004-01-06 2010-01-26 Neuronetics, Inc. Method and apparatus for coil positioning for TMS studies
FR2970638B1 (fr) * 2011-01-26 2014-03-07 Inst Nat Rech Inf Automat Procede et systeme d'aide au positionnement d'un outil medical sur la tete d'un sujet
EP2911587B1 (fr) * 2012-10-26 2018-10-17 Varian Medical Systems, Inc. Ciblage guidé d'images proche infrarouge
US10980273B2 (en) * 2013-11-12 2021-04-20 VMR Products, LLC Vaporizer, charger and methods of use
CN106955122A (zh) * 2017-03-20 2017-07-18 苏州网颢信息科技有限公司 放射治疗中基于单摄像模组的头部三维检测方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060161039A1 (en) * 2005-01-20 2006-07-20 Neuronetics, Inc. Articulating arm
US20130345491A1 (en) * 2011-03-09 2013-12-26 A School Corporation Kansai University Image data processing device and transcranial magnetic stimulation apparatus
CN102814002A (zh) * 2012-08-08 2012-12-12 深圳先进技术研究院 经颅磁刺激导航***及经颅磁刺激线圈定位方法
US20170372006A1 (en) * 2014-02-10 2017-12-28 Neuronetics, Inc. Head modeling for a therapeutic or diagnostic procedure
CN107497049A (zh) * 2017-09-30 2017-12-22 武汉资联虹康科技股份有限公司 一种用于经颅磁刺激器的电磁定位导航方法及装置
CN109173063A (zh) * 2018-09-21 2019-01-11 武汉资联虹康科技股份有限公司 一种3d扫描的经颅磁刺激***及经颅磁刺激方法
CN109260593A (zh) * 2018-09-27 2019-01-25 武汉资联虹康科技股份有限公司 一种经颅磁刺激治疗方法及设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112634336A (zh) * 2020-12-31 2021-04-09 华科精准（北京）医疗科技有限公司 一种配准方法及***

Also Published As

Publication number Publication date
CN110300993B (zh) 2023-10-27
CN110300993A (zh) 2019-10-01

Similar Documents

Publication Publication Date Title
EP3858433B1 (fr) Dispositif de navigation et positionnement pour un traitement de stimulation magnétique transcrânienne
WO2020172779A1 (fr) Système de caméra pour thérapie par stimulation magnétique transcrânienne
WO2020172781A1 (fr) Système de navigation de traitement et de diagnostic de stimulation magnétique transcrânienne à base de caméra
CN110896609B (zh) 一种用于经颅磁刺激治疗的tms定位导航方法
TWI678181B (zh) 手術導引系統
WO2020172782A1 (fr) Diagnostic de stimulation magnétique transcrânienne et système de détection de traitement basé sur une caméra
CN109173062B (zh) 一种高效的tms重复定位方法
CN111729200B (zh) 基于深度相机和磁共振的经颅磁刺激自动导航***和方法
WO2015098155A1 (fr) Appareil d'enseignement d'opération et appareil de stimulation magnétique transcrânienne
CN106139423A (zh) 一种基于摄像头的图像引导粒子植入***
US11948247B2 (en) Camera-based transcranial magnetic stimulation diagnosis and treatment head modeling system
CN116630382B (zh) 神经调控图像监测配准***和控制方法
CN115006737A (zh) 基于深度相机的放射治疗***监测***
CN110896610A (zh) 一种经颅磁刺激诊疗设备
CN113081013A (zh) 定位片扫描方法、装置及***
US20240123248A1 (en) Transcranial magnetic stimulation system and method
US20240122537A1 (en) Transcranial magnetic stimulation system and method
EP2521007A1 (fr) Procédé pour l'identification de la position 3D basée sur l'analyse de plusieurs images s'appliquant au suivi du regard
FI20225840A1 (en) Dental X-ray imaging system and method for dental X-ray imaging of a patient
US20240041415A1 (en) Chest x-ray system and method
Shin et al. Development of stereo camera module using webcam for navigation Transcranial Magnetic Stimulation system
Yasumuro et al. Coil positioning system for repetitive transcranial magnetic stimulation treatment by ToF camera ego-motion
CN115008753A (zh) 一种3d打印精准定位导航经颅磁刺激头模及制作方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19917179

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19917179

Country of ref document: EP

Kind code of ref document: A1