WO2022165812A1 - System for synthesizing real-time image by using optical body surface motion signals - Google Patents

System for synthesizing real-time image by using optical body surface motion signals Download PDF

Info

Publication number
WO2022165812A1
WO2022165812A1 PCT/CN2021/075866 CN2021075866W WO2022165812A1 WO 2022165812 A1 WO2022165812 A1 WO 2022165812A1 CN 2021075866 W CN2021075866 W CN 2021075866W WO 2022165812 A1 WO2022165812 A1 WO 2022165812A1
Authority
WO
WIPO (PCT)
Prior art keywords
body surface
image
time
real
deformation
Prior art date
Application number
PCT/CN2021/075866
Other languages
French (fr)
Chinese (zh)
Inventor
张艺宝
黄宇亮
李晨光
吴昊
刘宏嘉
Original Assignee
北京肿瘤医院(北京大学肿瘤医院)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京肿瘤医院(北京大学肿瘤医院) filed Critical 北京肿瘤医院(北京大学肿瘤医院)
Priority to CN202180000460.2A priority Critical patent/CN113574565A/en
Priority to PCT/CN2021/075866 priority patent/WO2022165812A1/en
Publication of WO2022165812A1 publication Critical patent/WO2022165812A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20068—Projection on vertical or horizontal image axis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Definitions

  • the invention relates to the technical field of medical equipment, in particular to a system for synthesizing real-time images using optical body surface motion signals.
  • Respiratory motion increases the risk of off-target radiation during radiotherapy, causing tumor recurrence and metastasis and normal organ damage.
  • the existing technology has respiratory gating, 1D body surface motion substitute signals have limited accuracy, and internal structures cannot be visualized in real time, and anatomical changes between 4D planned CT and actual treatment cannot be reflected by 1D respiratory signals.
  • X-ray fluoroscopy superimposed 2D anatomical information, poor image quality, and imaging at an orthogonal angle to the treatment rays, there is a blind spot for monitoring the movement in the same direction as the imaging rays, and the fluoroscopy accumulates a large amount of radiation dose, increasing the risk of radiation damage and secondary cancer.
  • 3DCBCT or 4DCBCT imaging before treatment, cannot be visualized and navigated in real time during treatment.
  • Optical body surface, internal structure cannot be visualized in real time.
  • a real-time system that can display the patient's internal anatomical structure information during treatment is an urgent technical problem to be solved.
  • the purpose of the present invention is to solve the technical problem in the prior art that the patient's internal anatomical structure information cannot be displayed in real time during the treatment process.
  • the technical scheme adopted in the present invention is: a system for synthesizing real-time images by using optical body surface motion signals, comprising: an acquisition unit, which acquires real-time body surface signals; The mapping relationship of the internal anatomical structure information displayed by the image is obtained by acquiring the body surface signal to obtain the internal anatomical structure information displayed by the 4D medical image corresponding to the time.
  • the 4D medical image includes a 4D image obtained by direct shooting, or a reconstructed and synthesized 4D image.
  • it also includes a 4D image reconstruction and synthesis unit, which acquires CBCT before treatment and simultaneously collects synchronous body surface data, divides the projections of each angle of CBCT into different time phases, and associates synchronous optical body surface data with breathing time. Reconstruction based on sparse projection data is performed on each phase projection of CBCT, and the body surface data is correlated with the in vivo anatomical structure images of the same respiratory phase.
  • the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: register each phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and obtain The deformation field is decomposed into the weighted sum of the principal components; (2) Sorting of body surface data: The body surface contour is segmented from each time phase of the 4D CT, and it is correlated with the respiratory time phase, so as to obtain the inference based on the instantaneous body surface The method of respiratory phase; (3) Reconstruction based on CBCT sparse projection data: according to the optical body surface data collected synchronously during the CBCT scanning process, sort according to the relative CBCT two-dimensional projection during respiration; The digital reconstructed image is nearly consistent with the actual collected CBCT two-dimensional projection, and the weight of each principal component in the prior respiratory motion model is calculated; the deformation field is determined according to the weight, and the reconstructed and synthesized 4D image
  • real-time optical body surface signals are collected during treatment, and correlated with the acquired body surface data before treatment, and then through the mapping relationship between the body surface data and the in vivo anatomical structure images in the same respiratory phase, the in vivo anatomical structure is further obtained. 4D images.
  • a 4D image reconstruction and synthesis unit is also included, the boundary of the anatomical structure in the body corresponds to the change of the body surface, and the deformation of the anatomical structure in the body is inferred from the deformation of the body surface.
  • the relationship between body surface deformation and internal body deformation is mined from historical data.
  • the specific process is as follows: collect four-dimensional CT images of patients, segment the body surface contour, and use the body surface deformation or four-dimensional CT image or body surface contour as the input of the model. , taking the CT of a certain phase as the reference image, and registering the reference image to the CT of other phases, the output is a time series model of the deformation field of the anatomical structure in the body; during treatment, the real-time optical body surface data is input into the model to predict The deformation field of the anatomical structure in the body can be applied to the reference image to obtain the image at that moment.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the corresponding respiratory phase is determined according to the optical body surface collected synchronously, and the four-dimensional image is predicted, thereby generating the same phase,
  • the digital reconstruction projection image of the same angle; the reconstruction projection image is compared with the two-dimensional projection map of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the respiration through the synchronously collected optical body surface. time phase, and then reconstruct the 4D image through the 4D image reconstruction and synthesis unit.
  • the deformation field of the anatomical structure in the body should satisfy the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image and the real acquisition two-dimensional projection. The graphs tend to be consistent. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the respiration through the synchronously collected optical body surface.
  • the deformation field of the anatomical structure in the body should meet the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection map, and the deformation satisfying this condition is calculated. field weights and then retrain the model.
  • the present invention can display the patient's internal anatomical structure information in real time during the treatment process, reduce the risk of off-target radiation during the radiotherapy process, and has the advantages of good compatibility, low cost, high efficiency, non-invasiveness, safety, reliability and so on.
  • FIG. 1 is a schematic diagram of a system of the present invention for synthesizing real-time images using optical body surface motion signals
  • FIG. 2 is a schematic diagram of a system of the present invention for synthesizing a real-time image using an optical body surface motion signal
  • FIG. 3 is a schematic diagram of body surface deformation of the present invention.
  • FIG. 5 is a schematic diagram of the loss function in the model training process of the present invention.
  • system is a method used to distinguish different components, elements, parts, parts or assemblies at different levels. However, other words may be replaced by other expressions if they serve the same purpose.
  • system and unit may be implemented by software or hardware, and may be a physical or virtual name having the functional part.
  • Embodiment 1 As shown in FIG. 1 , the technical solution adopted in the present invention is: a system for synthesizing real-time images using optical body surface motion signals, comprising: an acquisition unit for acquiring real-time body surface signals; an image conversion unit for The mapping relationship between the signal and the internal anatomical structure information displayed by the 4D medical image, by acquiring the body surface signal, the internal anatomical structure information displayed by the 4D medical image corresponding to the time is obtained.
  • 4D (four-dimensional) medical images include 4DCT (Four-dimensional Computed Tomography), 4DMR (four-dimensional nuclear magnetic resonance), 4DPET (Four-Dimensional positron emission tomography), 4DCBCT (Four- Dimensional Cone beam Computed Tomography four-dimensional cone beam X-ray computed tomography) and so on.
  • Body surface signals include optical body surface 4D signals, or body surface data extracted from 4DCBCT based on other methods, such as obtaining motion signals through body surface reflective blocks, in vivo markers such as diaphragm movement, and implanted markers.
  • the system that uses optical body surface motion signals to synthesize real-time images can display the patient's internal anatomical structure information in real time during the treatment process, reducing the risk of off-target radiation during radiotherapy.
  • Embodiment 2 As shown in FIG. 2 , on the basis of Embodiment 1, a system for synthesizing real-time images using optical body surface motion signals, 4D medical images include 4D images obtained by direct shooting, or reconstructed and synthesized 4D images.
  • the 4D images obtained by direct shooting specifically include directly shooting 4D images before radiotherapy, such as 4DCT, 4DMR, 4DPET, 4DCBCT, and the like.
  • mapping relationship between the simultaneous optical body surface data and the 4D image obtained by direct imaging, or from the 4DCBCT extracted from direct imaging based on other methods such as body surface reflection blocks, in vivo markers such as diaphragm movement, implant markers, etc.
  • the mapping relationship between the body surface data and the directly captured 4DCBCT images is used to correlate with the optical body surface signal data acquired during the radiotherapy process, so as to realize the mapping and display from the body surface to the body structure.
  • the reconstructed and synthesized 4D image further includes a 4D image reconstruction and synthesis unit to acquire CBCT images before treatment including but not limited to positioning.
  • the CBCT images used for setup refer to the CBCT images taken each day before treatment.
  • Simultaneously collect synchronous body surface data divide the projections of each angle of CBCT into different time phases, use the prior breathing motion model obtained in 4DCT to obtain the deformation field of the anatomical structure, and correlate the synchronous optical body surface data with the breathing time.
  • Reconstruction based on sparse projection data is performed on each phase projection of CBCT, and the body surface data is correlated with the in vivo anatomical structure images of the same respiratory phase.
  • each time phase of the four-dimensional CT is registered with the four-dimensional CT at a certain moment as a reference, and the obtained
  • the deformation vector field is decomposed into the weighted sum of the principal components
  • Instantaneous kV 2D projection raw data from 3D-CBCT were correlated with motion signals from 4D-CT.
  • the patient's 4D-CT prior anatomical structure is used to make up for the insufficient number of instantaneous kV two-dimensional projections, while retaining the anatomical information on the treatment day or the closest to the treatment day reflected by the kV two-dimensional projections.
  • the specific method is:
  • the new image can be expressed as:
  • (I, j, k) is the voxel position
  • D is the deformation of the new image I relative to I 0
  • Dx, Dy, and Dz are the components of the deformation field D in the x, y, and z directions, respectively.
  • W1, W2 and W3 are the weights of each principal component.
  • DRR represents the digital reconstruction projection of I.
  • the weight of each deformation principal component can be obtained by solving the above formula through gradient descent, and the estimation of the 4D image can be obtained.
  • the B-spline base can be further used to fine-tune the deformation field so that I and the two-dimensional projection of the same phase kV have better consistency.
  • Sparse projection refers to a small number of projections. Due to the small number of projections belonging to one phase, the reconstruction effect is poor. Optimization algorithms, such as gradient descent, can be used to calculate the optimal weights of each principal component in the prior respiratory motion model, so that the 2D digital reconstructed image calculated after CBCT deformation is nearly consistent with the actual collected CBCT 2D projection; Near uniformity means that the error is within a set tolerable threshold, and the threshold can be adjusted.
  • the CBCT image obtained according to the deformation field and the reference image represents the anatomical structure of the patient on that day, so as to ensure that the anatomical structure of the patient is not "distorted” during treatment, and at the same time improve the quality of the reconstructed image.
  • the period required by this method is short, and only one period is needed to reconstruct the synthetic 4D image.
  • real-time optical body surface signals are collected during treatment, and are associated with the acquired body surface data before treatment.
  • the mapping relationship between the body surface data and the in vivo anatomical structure images in the same respiratory phase the in vivo anatomy is further obtained. 4D image of the structure. That is, the acquisition of real-time optical body surface signals during treatment is associated and mapped with the above-mentioned reconstructed body surface data of the 4D image of the patient during treatment, and the corresponding internal dynamic anatomical structure information is displayed to realize "virtual perspective".
  • the two-dimensional projection map of the patient can be obtained by means of X-ray imaging, and the corresponding respiratory phase can be determined according to the optical body surface collected synchronously, and the four-dimensional image can be predicted by the above-mentioned method of 4D image reconstruction and synthesis. image, thereby generating a digital reconstructed projection image of the same time phase and the same angle; the reconstructed projection image is compared with the two-dimensional projection map of the patient obtained by X-ray imaging to verify the accuracy of the above method.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface.
  • the specific process of reconstructing the 4D image through the 4D image reconstruction and synthesis unit to obtain the 4D image of the patient’s anatomical structure of the day is to update the synthetic image of the day.
  • the obtained 2D digital reconstruction projection is nearly consistent with the real collected 2D projection map. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
  • Embodiment 3 As shown in FIG. 2 , on the basis of Embodiment 1, a system for synthesizing real-time images by using optical body surface motion signals further includes a 4D image reconstruction and synthesis unit, which corresponds to the deformation of the anatomical structure in vivo (DVF).
  • the relationship between body surface deformation and internal body deformation is mined from historical data, avoiding complex physical modeling.
  • the specific training process is as follows: collect historical 4D CT images of patients, segment the body surface contour, use the body surface deformation or 4D CT image or body surface contour as the input of the model, and use the CT of a certain time phase as the reference image to register the reference image
  • the output is a time series model of the in vivo anatomical structure deformation field, that is, the in vivo anatomical structure deformation field is the prediction target of the model.
  • the model can use the classic convolutional neural network and functions with similar functions; during treatment, the real-time optical body surface data is input into the model to predict the deformation field of the anatomical structure in the body, and the four-dimensional image at that moment can be obtained by acting on the reference image.
  • the deformation field has many parameters, and dimensionality reduction is conducive to the rapid training of the model.
  • PCA principal component analysis
  • ICA independent component analysis
  • IsoMap methods are used to decompose the deformation field into the weighted sum of the principal components, including Not limited to using a priori breathing motion model, the prediction target of the model is transformed into the weight of different principal components, and the deformation can be obtained by calculating the weight.
  • Model application stage collect the optical body surface data of the patient during the treatment process, convert it into a form associated with the body surface data segmented from the CT image in the training stage, and input it into the model to predict the deformation field of the anatomical structure in the body.
  • the image at this moment can be obtained from the reference image.
  • the model has been verified by experimental data, the body surface deformation is shown in Figure 3, an example of the predicted principal component weight is shown in Figure 4, and the loss function in the model training process is shown in Figure 5.
  • the two-dimensional projection map of the patient can be obtained by means of X-ray imaging, and the corresponding respiratory phase can be determined according to the optical body surface collected synchronously, and the four-dimensional image can be predicted by the above model, thereby generating the same time.
  • the digital reconstructed projection image of the same phase and the same angle; the reconstructed projection image is compared with the two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. Therefore, the deformation field of the in vivo anatomical structure obtained by the above model should satisfy the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection map, and the calculation of the Deform the field weights and retrain the model. Approaching consistency means that the error is within a set tolerable threshold, and the threshold can be adjusted.
  • Embodiment 4 A method for synthesizing a real-time image by using an optical body surface motion signal, which corresponds to the above-mentioned Embodiments 1 to 3, and the explanation part is described above for details.
  • the 4D medical image includes a 4D image obtained by direct shooting, or a reconstructed and synthesized 4D image.
  • the CBCT before treatment was acquired and synchronized body surface data were collected, the projections of each angle of CBCT were divided into different time phases, the synchronized optical body surface data was correlated with the respiratory time phase, and the projections of each time phase of CBCT were performed based on sparseness. Reconstruction of projection data, correlating body surface data with images of in vivo anatomical structures in the same respiratory phase.
  • the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: each time phase of the four-dimensional CT is registered with the four-dimensional CT at a certain moment as a reference, and the obtained The deformation field is decomposed into the weighted sum of the principal components; (2) the ordering of the body surface data: the body surface contour is segmented from each time phase of the 4D CT, and it is correlated with the respiratory time, so as to obtain the inferred respiration according to the instantaneous body surface Time-phase method; (3) Reconstruction based on CBCT sparse projection data: according to the optical body surface data collected synchronously during the CBCT scanning process, sort according to the relative CBCT two-dimensional projection during respiration; The reconstructed image is close to the actual collected CBCT 2D projection, and the weight of each principal component in the prior respiratory motion model is calculated; the deformation field is determined according to the weight, and the reconstructed and synthesized 4D image is
  • real-time optical body surface signals are collected during treatment and correlated with the acquired body surface data before treatment, and then through the mapping relationship between the body surface data and the in vivo anatomical structure images of the same respiratory phase, a 4D 4D in vivo anatomical structure is further obtained. image.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. Then the 4D image is reconstructed by the 4D image reconstruction and synthesis unit. At this time, the deformation field of the anatomical structure in the body should satisfy the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image and the real collected two-dimensional projection map. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
  • the boundaries of the in vivo anatomical structure correspond to the changes of the body surface, and the deformation of the in vivo anatomical structure is inferred from the deformation of the body surface.
  • the relationship between body surface deformation and internal body deformation was mined from historical data, and the specific process was as follows: collecting four-dimensional CT images of patients, segmenting the body surface contour, and using the body surface deformation or the four-dimensional CT image or body surface contour as the input of the model, Taking the CT of a certain phase as the reference image, the reference image is registered to the CT of other phases, and the output is a time series model of the deformation field of the anatomical structure in the body; during the treatment, the real-time optical body surface data is input into the model to predict the in vivo model. The deformation field of the anatomical structure acts on the reference image to obtain the image at that moment.
  • the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface.
  • the deformation field of the anatomical structure in the body should meet the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection image, and the deformation field that satisfies this condition is calculated. weights, and then retrain the model.
  • a two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the corresponding respiratory phase is determined according to the optical body surface collected synchronously, and a four-dimensional image is predicted, thereby generating the same phase and the same phase.
  • the digital reconstructed projection image of the angle; the reconstructed projection image is compared with the two-dimensional projection map of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Quality & Reliability (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Disclosed in the present invention is a system for synthesizing a real-time image by using optical body surface motion signals, comprising: an acquisition unit, configured to acquire a real-time body surface signal; and an image conversion unit, configured to, according to a mapping relationship between the body surface signal and internal anatomical structure information displayed on a 4D medical image, obtain the internal anatomical structure information displayed on the 4D medical image at a corresponding time by acquiring the body surface signal. The present invention has the beneficial effects: respiratory movement increases an off-target risk of rays during radiotherapy, causing tumor recurrence and metastasis and normal organ damage; however, the system for synthesizing the real-time image by using the optical body surface motion signals can display the internal anatomical structure information of a patient in real time in a treatment process, can reduce the off-target risk of rays during radiotherapy, and has the advantages of good compatibility, low cost, high efficiency, noninvasiveness, safety, reliability and the like.

Description

一种利用光学体表运动信号合成实时图像的***A system for synthesizing real-time images using optical body surface motion signals 技术领域technical field
本发明涉及医疗设备技术领域，特别是涉及一种利用光学体表运动信号合成实时图像的***。The invention relates to the technical field of medical equipment, in particular to a system for synthesizing real-time images using optical body surface motion signals.
背景技术Background technique
呼吸运动会增加放疗过程中射线的脱靶风险，引起肿瘤复发转移和正常器官损伤。现有技术呼吸门控，1D体表运动替代信号精度有限，且内部结构不能实时可视化，4D计划CT与实际治疗之间的解剖变化，无法由1D呼吸信号体现。X线透视，2D解剖信息叠加，图像质量差，与治疗射线正交角度成像，存在与成像射线同向运动监测盲区，透视累积大量辐射剂量，增加辐射损伤和二次致癌风险。3DCBCT或4DCBCT，治疗前成像，不能治疗过程中实时可视化导航。光学体表，内部结构不能实时可视化。Respiratory motion increases the risk of off-target radiation during radiotherapy, causing tumor recurrence and metastasis and normal organ damage. The existing technology has respiratory gating, 1D body surface motion substitute signals have limited accuracy, and internal structures cannot be visualized in real time, and anatomical changes between 4D planned CT and actual treatment cannot be reflected by 1D respiratory signals. X-ray fluoroscopy, superimposed 2D anatomical information, poor image quality, and imaging at an orthogonal angle to the treatment rays, there is a blind spot for monitoring the movement in the same direction as the imaging rays, and the fluoroscopy accumulates a large amount of radiation dose, increasing the risk of radiation damage and secondary cancer. 3DCBCT or 4DCBCT, imaging before treatment, cannot be visualized and navigated in real time during treatment. Optical body surface, internal structure cannot be visualized in real time.
一种实时的能够在治疗过程中显示患者内部解剖结构信息的***是亟待解决的技术问题。A real-time system that can display the patient's internal anatomical structure information during treatment is an urgent technical problem to be solved.
发明内容SUMMARY OF THE INVENTION
本发明的目的是为了解决现有技术中不能实时的在治疗过程中显示患者内部解剖结构信息的技术问题。The purpose of the present invention is to solve the technical problem in the prior art that the patient's internal anatomical structure information cannot be displayed in real time during the treatment process.
为解决上述技术问题，本发明采用的技术方案是：一种利用光学体表运动信号合成实时图像的***，包含：获取单元，获取实时体表信号；图像转化单元，根据体表信号与4D医学图像显示的内部解剖结构信息的映射关系，通过获取体表信号，得到对应时间的4D医学图像显示的内部解剖结构信息。In order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is: a system for synthesizing real-time images by using optical body surface motion signals, comprising: an acquisition unit, which acquires real-time body surface signals; The mapping relationship of the internal anatomical structure information displayed by the image is obtained by acquiring the body surface signal to obtain the internal anatomical structure information displayed by the 4D medical image corresponding to the time.
更优的，4D医学图像包括直接拍摄获取的4D图像，或者重建合成的4D图像。More preferably, the 4D medical image includes a 4D image obtained by direct shooting, or a reconstructed and synthesized 4D image.
更优的，还包括4D图像重建合成单元，获取治疗前的CBCT同时采集同步体表数据，将CBCT的各角度投影划分为不同的时相，将同步光学体表数据与呼吸时相相关联，对CBCT各时相投影进行基于稀疏投影数据的重建,将体表数据与同一呼吸时相的体内解剖结构图像相关联。More preferably, it also includes a 4D image reconstruction and synthesis unit, which acquires CBCT before treatment and simultaneously collects synchronous body surface data, divides the projections of each angle of CBCT into different time phases, and associates synchronous optical body surface data with breathing time. Reconstruction based on sparse projection data is performed on each phase projection of CBCT, and the body surface data is correlated with the in vivo anatomical structure images of the same respiratory phase.
更优的，4D图像重建合成单元重建4D图像的具体过程如下：(1)先验呼吸运动模型的生成：对四维CT的各个时相与以某一时刻四维CT为参考进行配准，将得到的形变场分解为主成分的加权和；(2)体表数据的排序：从四维CT的各个时相中分割出体表轮廓，将其与 呼吸时相相关联，从而得到根据瞬时体表推断呼吸时相的方法；(3)基于CBCT稀疏投影数据的重建：根据CBCT扫描过程中同步采集的光学体表数据，按呼吸时相对CBCT二维投影进行排序；根据CBCT形变后计算出的二维数字重建图像与实际采集的CBCT二维投影趋近一致，计算先验呼吸运动模型中各主成分的权重；根据权重确定形变场，根据该形变场与参考图像获得重建合成的4D图像。More preferably, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: register each phase of the four-dimensional CT with the four-dimensional CT at a certain moment as a reference, and obtain The deformation field is decomposed into the weighted sum of the principal components; (2) Sorting of body surface data: The body surface contour is segmented from each time phase of the 4D CT, and it is correlated with the respiratory time phase, so as to obtain the inference based on the instantaneous body surface The method of respiratory phase; (3) Reconstruction based on CBCT sparse projection data: according to the optical body surface data collected synchronously during the CBCT scanning process, sort according to the relative CBCT two-dimensional projection during respiration; The digital reconstructed image is nearly consistent with the actual collected CBCT two-dimensional projection, and the weight of each principal component in the prior respiratory motion model is calculated; the deformation field is determined according to the weight, and the reconstructed and synthesized 4D image is obtained according to the deformation field and the reference image.
更优的，治疗时采集实时光学体表信号，与获取的治疗前的体表数据相关联，再通过体表数据与同一呼吸时相的体内解剖结构图像的映射关系，进一步得到体内解剖结构的4D图像。More preferably, real-time optical body surface signals are collected during treatment, and correlated with the acquired body surface data before treatment, and then through the mapping relationship between the body surface data and the in vivo anatomical structure images in the same respiratory phase, the in vivo anatomical structure is further obtained. 4D images.
更优的，还包括4D图像重建合成单元，体内解剖结构的边界对应体表的变化，从体表的形变中推断体内解剖结构的形变。More preferably, a 4D image reconstruction and synthesis unit is also included, the boundary of the anatomical structure in the body corresponds to the change of the body surface, and the deformation of the anatomical structure in the body is inferred from the deformation of the body surface.
更优的，从历史数据中挖掘体表形变与体内形变的关系，具体过程如下：搜集患者四维CT图像，分割出体表轮廓，以体表形变或四维CT图像或体表轮廓作为模型的输入，以某一时相的CT作为参考图像，将参考图像配准到其他时相的CT上，得到输出为体内解剖结构形变场的时间序列的模型；治疗时将实时光学体表数据输入模型中预测体内解剖结构的形变场，作用在参考图像上即可得到该时刻的图像。More preferably, the relationship between body surface deformation and internal body deformation is mined from historical data. The specific process is as follows: collect four-dimensional CT images of patients, segment the body surface contour, and use the body surface deformation or four-dimensional CT image or body surface contour as the input of the model. , taking the CT of a certain phase as the reference image, and registering the reference image to the CT of other phases, the output is a time series model of the deformation field of the anatomical structure in the body; during treatment, the real-time optical body surface data is input into the model to predict The deformation field of the anatomical structure in the body can be applied to the reference image to obtain the image at that moment.
更优的，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，并根据同步采集的光学体表确定对应的呼吸时相，预测出四维图像，从而生成同一时相、同一角度的数字重建投影图像；重建投影图像与通过X线成像的方式，得到患者的二维投影图进行比较，验证上述方法的准确性。More preferably, in the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the corresponding respiratory phase is determined according to the optical body surface collected synchronously, and the four-dimensional image is predicted, thereby generating the same phase, The digital reconstruction projection image of the same angle; the reconstruction projection image is compared with the two-dimensional projection map of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.
更优的，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，再通过4D图像重建合成单元重建4D图像，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，如偏差较大，则计算满足该条件的形变场，然后重新训练模型。More preferably, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the respiration through the synchronously collected optical body surface. time phase, and then reconstruct the 4D image through the 4D image reconstruction and synthesis unit. At this time, the deformation field of the anatomical structure in the body should satisfy the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image and the real acquisition two-dimensional projection. The graphs tend to be consistent. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
更优的，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，计算满足该条件的形变场权重，然后重新训练模型。More preferably, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the respiration through the synchronously collected optical body surface. Time phase, at this time, the deformation field of the anatomical structure in the body should meet the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection map, and the deformation satisfying this condition is calculated. field weights and then retrain the model.
有益效果：Beneficial effects:
本发明根据利用光学体表运动信号合成实时图像的***可以在治疗过程中实时显示患者内部解剖结构信息，降低放疗过程中射线的脱靶风险，具有兼容性好、成本低、高效、无创、安全、可靠等优点。According to the system of synthesizing real-time images using optical body surface motion signals, the present invention can display the patient's internal anatomical structure information in real time during the treatment process, reduce the risk of off-target radiation during the radiotherapy process, and has the advantages of good compatibility, low cost, high efficiency, non-invasiveness, safety, reliability and so on.
附图说明Description of drawings
图1是本发明一种利用光学体表运动信号合成实时图像的***示意图；1 is a schematic diagram of a system of the present invention for synthesizing real-time images using optical body surface motion signals;
图2是本发明一种利用光学体表运动信号合成实时图像的***示意图；2 is a schematic diagram of a system of the present invention for synthesizing a real-time image using an optical body surface motion signal;
图3是本发明体表形变示意图；3 is a schematic diagram of body surface deformation of the present invention;
图4是本发明预测的主成分权重示意图；4 is a schematic diagram of the principal component weights predicted by the present invention;
图5是本发明模型训练过程中的损失函数示意图。FIG. 5 is a schematic diagram of the loss function in the model training process of the present invention.
具体实施方式Detailed ways
下面结合附图对本发明的较佳实施例进行详细阐述，以使本发明的优点和特征能更易于被本领域技术人员理解，从而对本发明的保护范围做出更为清楚明确的界定。The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and the protection scope of the present invention can be more clearly defined.
为了更清楚地说明本发明实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本发明的一些示例或实施例，各个实施例的技术特征之间可以相互组合，构成实现发明目的的实际方案，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本发明应用于其它类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present invention, and the technical features of each embodiment can be combined with each other to constitute a practical solution to achieve the purpose of the invention. For those of ordinary skill in the art, On the premise of no creative effort, the present invention can also be applied to other similar situations according to these drawings. Unless obvious from the locale or otherwise specified, the same reference numbers in the figures represent the same structure or operation.
应当理解，本文使用的“***”、“单元”是用于区分不同级别的不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。并且，“***”、“单元”可以由软件或者硬件实施，可以是实体或虚拟的具有该功能部分的称呼。It should be understood that "system", "unit" as used herein is a method used to distinguish different components, elements, parts, parts or assemblies at different levels. However, other words may be replaced by other expressions if they serve the same purpose. In addition, "system" and "unit" may be implemented by software or hardware, and may be a physical or virtual name having the functional part.
本发明中使用了流程图用来说明根据本发明的实施例的***所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。各个实施例中的技术方案可以相互组合实现本发明的目的。Flow diagrams are used in the present invention to illustrate the operations performed by a system according to an embodiment of the present invention. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Instead, the various steps can be processed in reverse order or simultaneously. At the same time, other actions can be added to these procedures, or a step or steps can be removed from these procedures. The technical solutions in each embodiment can be combined with each other to achieve the purpose of the present invention.
实施例一：如图1所示，本发明采用的技术方案是：一种利用光学体表运动信号合成实时图像的***，包含：获取单元，获取实时体表信号；图像转化单元，根据体表信号与4D 医学图像显示的内部解剖结构信息的映射关系，通过获取体表信号，得到对应时间的4D医学图像显示的内部解剖结构信息。Embodiment 1: As shown in FIG. 1 , the technical solution adopted in the present invention is: a system for synthesizing real-time images using optical body surface motion signals, comprising: an acquisition unit for acquiring real-time body surface signals; an image conversion unit for The mapping relationship between the signal and the internal anatomical structure information displayed by the 4D medical image, by acquiring the body surface signal, the internal anatomical structure information displayed by the 4D medical image corresponding to the time is obtained.
4D(四维)医学图像包括4DCT(Four-dimensional Computed Tomography四维X射线计算机断层成像)、4DMR(四维核磁共振)、4DPET(Four-Dimensional positron emission tomography四维正电子发射计算机断层扫描)、4DCBCT(Four-Dimensional Cone beam Computed Tomography四维锥形束X射线计算机断层成像)等。4D (four-dimensional) medical images include 4DCT (Four-dimensional Computed Tomography), 4DMR (four-dimensional nuclear magnetic resonance), 4DPET (Four-Dimensional positron emission tomography), 4DCBCT (Four- Dimensional Cone beam Computed Tomography four-dimensional cone beam X-ray computed tomography) and so on.
体表信号包括光学体表4D信号，或基于其它方式从4DCBCT中提取的体表数据，如通过体表反光块，体内标记如膈肌运动、植入标记等方式获得运动信号。Body surface signals include optical body surface 4D signals, or body surface data extracted from 4DCBCT based on other methods, such as obtaining motion signals through body surface reflective blocks, in vivo markers such as diaphragm movement, and implanted markers.
利用光学体表运动信号合成实时图像的***可以在治疗过程中实时显示患者内部解剖结构信息，降低放疗过程中射线的脱靶风险。The system that uses optical body surface motion signals to synthesize real-time images can display the patient's internal anatomical structure information in real time during the treatment process, reducing the risk of off-target radiation during radiotherapy.
实施例二：如图2所示，在实施例一的基础上，利用光学体表运动信号合成实时图像的***，4D医学图像包括直接拍摄获取的4D图像，或者重建合成的4D图像。Embodiment 2: As shown in FIG. 2 , on the basis of Embodiment 1, a system for synthesizing real-time images using optical body surface motion signals, 4D medical images include 4D images obtained by direct shooting, or reconstructed and synthesized 4D images.
直接拍摄获取的4D图像具体包括，在放疗前直接拍摄4D图像，比如4DCT、4DMR、4DPET、4DCBCT等。The 4D images obtained by direct shooting specifically include directly shooting 4D images before radiotherapy, such as 4DCT, 4DMR, 4DPET, 4DCBCT, and the like.
在放疗前利用同时相光学体表数据与直接拍摄获取的4D图像的映射关系，或从基于其它方式(如体表反光块，体内标记如膈肌运动、植入标记等)提取的直接拍摄的4DCBCT中的体表数据与直接拍摄的4DCBCT图像的映射关系，用于和放疗过程中实施获取的光学体表信号数据进行关联，实现从体表到体内结构的映射和展示。Before radiotherapy, use the mapping relationship between the simultaneous optical body surface data and the 4D image obtained by direct imaging, or from the 4DCBCT extracted from direct imaging based on other methods (such as body surface reflection blocks, in vivo markers such as diaphragm movement, implant markers, etc.) The mapping relationship between the body surface data and the directly captured 4DCBCT images is used to correlate with the optical body surface signal data acquired during the radiotherapy process, so as to realize the mapping and display from the body surface to the body structure.
重建合成的4D图像，进一步地，还包括4D图像重建合成单元，获取治疗前的包括不限于用于摆位的CBCT图像。用于摆位的CBCT图像指得是每一天治疗前拍摄的CBCT图像。The reconstructed and synthesized 4D image further includes a 4D image reconstruction and synthesis unit to acquire CBCT images before treatment including but not limited to positioning. The CBCT images used for setup refer to the CBCT images taken each day before treatment.
同时采集同步体表数据，将CBCT的各角度投影划分为不同的时相，利用4DCT中得到的先验呼吸运动模型得到解剖结构的形变场，将同步光学体表数据与呼吸时相相关联，对CBCT各时相投影进行基于稀疏投影数据的重建,将体表数据与同一呼吸时相的体内解剖结构图像相关联。Simultaneously collect synchronous body surface data, divide the projections of each angle of CBCT into different time phases, use the prior breathing motion model obtained in 4DCT to obtain the deformation field of the anatomical structure, and correlate the synchronous optical body surface data with the breathing time. Reconstruction based on sparse projection data is performed on each phase projection of CBCT, and the body surface data is correlated with the in vivo anatomical structure images of the same respiratory phase.
进一步地，4D图像重建合成单元重建4D图像的具体过程如下：(1)先验呼吸运动模型的生成：对四维CT的各个时相与以某一时刻四维CT为参考进行配准，将得到的形变场(deformation vector field)分解为主成分的加权和；Further, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: each time phase of the four-dimensional CT is registered with the four-dimensional CT at a certain moment as a reference, and the obtained The deformation vector field is decomposed into the weighted sum of the principal components;
采用3D-CBCT的瞬时kV二维投影原始数据与4D-CT的运动信号进行关联。利用患者 4D-CT先验解剖结构弥补瞬时kV二维投影的数量不足的问题，同时保留kV二维投影所反映的治疗当天或者最接近治疗当天的解剖信息。具体做法是：Instantaneous kV 2D projection raw data from 3D-CBCT were correlated with motion signals from 4D-CT. The patient's 4D-CT prior anatomical structure is used to make up for the insufficient number of instantaneous kV two-dimensional projections, while retaining the anatomical information on the treatment day or the closest to the treatment day reflected by the kV two-dimensional projections. The specific method is:
首先以4D-CT某一时相为基准(I0)，则新的图像可表示为：First, taking a certain time phase of 4D-CT as the reference (I0), the new image can be expressed as:
I(I,j,k)＝F(I0,D)＝I0(i+Dx(I,j,k),j+Dy(I,j,k),k+Dz(I,j,k))I(I,j,k)=F(I0,D)=I0(i+Dx(I,j,k),j+Dy(I,j,k),k+Dz(I,j,k) )
其中，(I,j,k)为体素位置，D为新图像Ⅰ相对于I 0的形变，Dx、Dy、Dz分别为形变场D在x,y,z方向上的分量。计算4D-CT各时相与I 0间的形变，计算其平均形变
Figure PCTCN2021075866-appb-000001
并进行主成分分析得到这些形变的前三个主成分D 1、D 2和D 3，则Ⅰ相对于I 0的形变场D可简化为： Among them, (I, j, k) is the voxel position, D is the deformation of the new image I relative to I 0 , and Dx, Dy, and Dz are the components of the deformation field D in the x, y, and z directions, respectively. Calculate the deformation between each phase of 4D-CT and I 0 , and calculate the average deformation
Figure PCTCN2021075866-appb-000001
And carry out principal component analysis to obtain the first three principal components D 1 , D 2 and D 3 of these deformations, then the deformation field D of I relative to I 0 can be simplified as:
Figure PCTCN2021075866-appb-000002
Figure PCTCN2021075866-appb-000002
其中，W1、W2和W3为各主成分的权重。搜寻某一时相的kV二维投影(P)，则该时相的4D图像相对于I 0的形变应满足DRR(F(I 0,D))＝P Among them, W1, W2 and W3 are the weights of each principal component. Search for the kV two-dimensional projection (P) of a certain time phase, then the deformation of the 4D image of this time phase relative to I 0 should satisfy DRR(F(I 0 , D))=P
其中，DRR表示Ⅰ的数字重建投影。可通过梯度下降求解上式得到各形变主成分的权重，得到4D影像的估计。后续可进一步采用B-样条基底对形变场进行微调使Ⅰ与同时相kV二维投影有更好的一致性。where DRR represents the digital reconstruction projection of I. The weight of each deformation principal component can be obtained by solving the above formula through gradient descent, and the estimation of the 4D image can be obtained. In the follow-up, the B-spline base can be further used to fine-tune the deformation field so that I and the two-dimensional projection of the same phase kV have better consistency.
(2)体表数据的排序：从四维CT的各个时相中分割出体表轮廓，以某一时刻(包括不限于呼气末或吸气末)的体表为参考体表，计算其他时相体表与参考体表之差及其梯度，再对同一时相体表差及梯度统计平均值，将其与呼吸时相相关联，包括不限于采用简单的回归模型实现，回归模型包括多项式回归、逻辑回归、岭回归、Lasso回归等，根据瞬时体表推断呼吸时相的方法。(2) Sorting of body surface data: The contour of the body surface is segmented from each phase of the 4D CT, and the body surface at a certain moment (including but not limited to end-expiration or end-inspiration) is used as the reference body surface, and other time periods are calculated. The difference between the body surface and the reference body surface and its gradient, and then the statistical average value of the body surface difference and gradient in the same time phase is correlated with the breathing time, including but not limited to the implementation of a simple regression model, the regression model includes polynomial Regression, logistic regression, ridge regression, Lasso regression, etc., methods of inferring the respiratory phase based on the instantaneous body surface.
(3)基于CBCT稀疏投影数据的重建：根据CBCT扫描过程中同步采集的光学体表数据，按呼吸时相对CBCT二维投影进行排序；根据CBCT形变后计算出的二维数字重建图像与实际采集的CBCT二维投影趋近一致，计算先验呼吸运动模型中各主成分的权重；根据权重确定形变场D，根据该形变场D与参考图像获得重建合成的4D图像。(3) Reconstruction based on CBCT sparse projection data: According to the optical body surface data collected synchronously during CBCT scanning, sort by relative CBCT two-dimensional projection during respiration; The two-dimensional projections of the CBCT are nearly consistent, and the weight of each principal component in the prior respiratory motion model is calculated; the deformation field D is determined according to the weight, and the reconstructed and synthesized 4D image is obtained according to the deformation field D and the reference image.
稀疏投影指的是少量的投影。由于属于一个时相的投影数量较少，重建效果较差。可采用优化算法，如梯度下降等，计算先验呼吸运动模型中各主成分的最优权重，使得CBCT形变后计算出的二维数字重建图像与实际采集的CBCT二维投影趋近一致；趋近一致指的是误差在设定的可容忍的阈值范围内，阈值可以调整。Sparse projection refers to a small number of projections. Due to the small number of projections belonging to one phase, the reconstruction effect is poor. Optimization algorithms, such as gradient descent, can be used to calculate the optimal weights of each principal component in the prior respiratory motion model, so that the 2D digital reconstructed image calculated after CBCT deformation is nearly consistent with the actual collected CBCT 2D projection; Near uniformity means that the error is within a set tolerable threshold, and the threshold can be adjusted.
根据该形变场与参考图得到的CBCT图像即代表了患者当天解剖结构，从而保证患者治 疗时解剖结构不“失真”的同时，提高重建图像的质量。本方法需要的周期短，只需要一个周期即可重建合成4D图像。The CBCT image obtained according to the deformation field and the reference image represents the anatomical structure of the patient on that day, so as to ensure that the anatomical structure of the patient is not "distorted" during treatment, and at the same time improve the quality of the reconstructed image. The period required by this method is short, and only one period is needed to reconstruct the synthetic 4D image.
进一步地，在应用阶段，治疗时采集实时光学体表信号，与获取的治疗前的体表数据相关联，通过体表数据与同一呼吸时相的体内解剖结构图像的映射关系，进一步得到体内解剖结构的4D图像。即治疗时采集实时光学体表信号与上述重建的患者治疗时4D图像的体表数据进行关联和映射，展示其对应的内部动态解剖结构信息，实现“虚拟透视”。Further, in the application stage, real-time optical body surface signals are collected during treatment, and are associated with the acquired body surface data before treatment. Through the mapping relationship between the body surface data and the in vivo anatomical structure images in the same respiratory phase, the in vivo anatomy is further obtained. 4D image of the structure. That is, the acquisition of real-time optical body surface signals during treatment is associated and mapped with the above-mentioned reconstructed body surface data of the 4D image of the patient during treatment, and the corresponding internal dynamic anatomical structure information is displayed to realize "virtual perspective".
进一步地，在治疗过程中，可以通过X线成像的方式，得到患者的二维投影图，并根据同步采集的光学体表确定对应的呼吸时相，采用上述4D图像重建合成的方法预测出四维图像，从而生成同一时相、同一角度的数字重建投影图像；重建投影图像与通过X线成像的方式得到的患者的二维投影图进行比较，验证上述方法的准确性。Further, in the course of treatment, the two-dimensional projection map of the patient can be obtained by means of X-ray imaging, and the corresponding respiratory phase can be determined according to the optical body surface collected synchronously, and the four-dimensional image can be predicted by the above-mentioned method of 4D image reconstruction and synthesis. image, thereby generating a digital reconstructed projection image of the same time phase and the same angle; the reconstructed projection image is compared with the two-dimensional projection map of the patient obtained by X-ray imaging to verify the accuracy of the above method.
进一步地，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，再通过4D图像重建合成单元重建4D图像得到患者当天解剖结构4D图像的具体过程更新当天的合成图像，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，如偏差较大，则计算满足该条件的形变场，然后重新训练模型。Further, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. Then, the specific process of reconstructing the 4D image through the 4D image reconstruction and synthesis unit to obtain the 4D image of the patient’s anatomical structure of the day is to update the synthetic image of the day. The obtained 2D digital reconstruction projection is nearly consistent with the real collected 2D projection map. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
实施例三：如图2所示，在实施例一的基础上，利用光学体表运动信号合成实时图像的***，进一步地，还包括4D图像重建合成单元，体内解剖结构的形变(DVF)对应体内解剖结构的边界变化，体内解剖结构的边界对应体表的变化，从体表的形变中推断体内相应的形变场。Embodiment 3: As shown in FIG. 2 , on the basis of Embodiment 1, a system for synthesizing real-time images by using optical body surface motion signals further includes a 4D image reconstruction and synthesis unit, which corresponds to the deformation of the anatomical structure in vivo (DVF). The boundary changes of the anatomical structure in vivo, the boundary of the anatomical structure in the body corresponds to the change of the body surface, and the corresponding deformation field in the body is inferred from the deformation of the body surface.
进一步地，从历史数据中挖掘体表形变与体内形变的关系，避免了较为复杂的物理建模。具体训练过程如下：搜集患者历史四维CT图像，分割出体表轮廓，以体表形变或四维CT图像或体表轮廓作为模型的输入，以某一时相的CT作为参考图像，将参考图像配准到其他时相的CT上，得到输出为体内解剖结构形变场的时间序列的模型，即体内解剖结构形变场为模型的预测目标。模型可采用经典的卷积神经网络及具有相似功能的函数；治疗时将实时光学体表数据输入模型中预测体内解剖结构的形变场，作用在参考图像上即可得到该时刻的四维图像。Further, the relationship between body surface deformation and internal body deformation is mined from historical data, avoiding complex physical modeling. The specific training process is as follows: collect historical 4D CT images of patients, segment the body surface contour, use the body surface deformation or 4D CT image or body surface contour as the input of the model, and use the CT of a certain time phase as the reference image to register the reference image On the CT of other phases, the output is a time series model of the in vivo anatomical structure deformation field, that is, the in vivo anatomical structure deformation field is the prediction target of the model. The model can use the classic convolutional neural network and functions with similar functions; during treatment, the real-time optical body surface data is input into the model to predict the deformation field of the anatomical structure in the body, and the four-dimensional image at that moment can be obtained by acting on the reference image.
形变场的参数较多，对其进行降维有利于模型的快速训练，如采用PCA(主成分分析)、ICA(independent component analysis)、IsoMap方法，将形变场分解为主成分的加权和，包括 不限于使用先验呼吸运动模型，模型的预测目标转为不同主成分的权重，计算权重即可获得形变。The deformation field has many parameters, and dimensionality reduction is conducive to the rapid training of the model. For example, PCA (principal component analysis), ICA (independent component analysis), and IsoMap methods are used to decompose the deformation field into the weighted sum of the principal components, including Not limited to using a priori breathing motion model, the prediction target of the model is transformed into the weight of different principal components, and the deformation can be obtained by calculating the weight.
模型应用阶段：采集治疗过程中患者的光学体表数据，将其转化与训练阶段CT图像中分割出的体表数据相关联的形式，输入到模型中，预测体内解剖结构的形变场，作用在参考图像上即可得到该时刻的图像。该模型已经得到实验数据验证，体表形变见图3，预测的主成分权重示例见图4，模型训练过程中的损失函数见图5。Model application stage: collect the optical body surface data of the patient during the treatment process, convert it into a form associated with the body surface data segmented from the CT image in the training stage, and input it into the model to predict the deformation field of the anatomical structure in the body. The image at this moment can be obtained from the reference image. The model has been verified by experimental data, the body surface deformation is shown in Figure 3, an example of the predicted principal component weight is shown in Figure 4, and the loss function in the model training process is shown in Figure 5.
模型的验证结果，在十个样本上测试模型的效果，以肺部的DICE(戴斯相似系数)相似系数、靶区的DICE相似系数、肿瘤中心点的距离为评价指标，均得到与采用模型预测前更好且具有统计性差异的结果。The validation results of the model, the effect of the model was tested on ten samples, and the DICE similarity coefficient of the lung, the DICE similarity coefficient of the target area, and the distance of the tumor center point were used as evaluation indicators. Better and statistically different results before prediction.
Figure PCTCN2021075866-appb-000003
Figure PCTCN2021075866-appb-000003
进一步地，在治疗过程中，可以通过X线成像的方式，得到患者的二维投影图，并根据同步采集的光学体表确定对应的呼吸时相，通过上述模型预测四维图像，从而生成同一时相、同一角度的数字重建投影图像；重建投影图像与通过X线成像的方式，得到患者的二维投影图进行比较，验证上述方法的准确性。Further, during the treatment process, the two-dimensional projection map of the patient can be obtained by means of X-ray imaging, and the corresponding respiratory phase can be determined according to the optical body surface collected synchronously, and the four-dimensional image can be predicted by the above model, thereby generating the same time. The digital reconstructed projection image of the same phase and the same angle; the reconstructed projection image is compared with the two-dimensional projection image of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.
进一步地，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，通过上述模型获得体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，计算满足该条件的形变场权重，然后重新训练模型。趋近一致指的是误差在设定的可容忍的阈值范围内，阈值可以调整。Further, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. Therefore, the deformation field of the in vivo anatomical structure obtained by the above model should satisfy the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection map, and the calculation of the Deform the field weights and retrain the model. Approaching consistency means that the error is within a set tolerable threshold, and the threshold can be adjusted.
实施例四：一种利用光学体表运动信号合成实时图像的方法，与上述实施例一至三相对应，解释说明部分具体见上文。获取实时体表信号；根据体表信号与4D医学图像显示的内部解剖结构信息的映射关系，通过获取体表信号，得到对应的时间4D医学图像显示的内部解剖结构信息。Embodiment 4: A method for synthesizing a real-time image by using an optical body surface motion signal, which corresponds to the above-mentioned Embodiments 1 to 3, and the explanation part is described above for details. Obtain real-time body surface signals; according to the mapping relationship between the body surface signals and the internal anatomical structure information displayed by the 4D medical image, by acquiring the body surface signal, the internal anatomical structure information displayed by the corresponding time 4D medical image is obtained.
进一步地，4D医学图像包括直接拍摄获取的4D图像，或者重建合成的4D图像。Further, the 4D medical image includes a 4D image obtained by direct shooting, or a reconstructed and synthesized 4D image.
进一步地，获取治疗前的CBCT同时采集同步体表数据，将CBCT的各角度投影划分为不同的时相，将同步光学体表数据与呼吸时相相关联，对CBCT各时相投影进行基于稀疏投 影数据的重建,将体表数据与同一呼吸时相的体内解剖结构图像相关联。Further, the CBCT before treatment was acquired and synchronized body surface data were collected, the projections of each angle of CBCT were divided into different time phases, the synchronized optical body surface data was correlated with the respiratory time phase, and the projections of each time phase of CBCT were performed based on sparseness. Reconstruction of projection data, correlating body surface data with images of in vivo anatomical structures in the same respiratory phase.
进一步地，4D图像重建合成单元重建4D图像的具体过程如下：(1)先验呼吸运动模型的生成：对四维CT的各个时相与以某一时刻四维CT为参考进行配准，将得到的形变场分解为主成分的加权和；(2)体表数据的排序：从四维CT的各个时相中分割出体表轮廓，将其与呼吸时相相关联，从而得到根据瞬时体表推断呼吸时相的方法；(3)基于CBCT稀疏投影数据的重建：根据CBCT扫描过程中同步采集的光学体表数据，按呼吸时相对CBCT二维投影进行排序；根据CBCT形变后计算出的二维数字重建图像与实际采集的CBCT二维投影趋近一致，计算先验呼吸运动模型中各主成分的权重；根据权重确定形变场，根据该形变场与参考图像获得重建合成的4D图像。Further, the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: each time phase of the four-dimensional CT is registered with the four-dimensional CT at a certain moment as a reference, and the obtained The deformation field is decomposed into the weighted sum of the principal components; (2) the ordering of the body surface data: the body surface contour is segmented from each time phase of the 4D CT, and it is correlated with the respiratory time, so as to obtain the inferred respiration according to the instantaneous body surface Time-phase method; (3) Reconstruction based on CBCT sparse projection data: according to the optical body surface data collected synchronously during the CBCT scanning process, sort according to the relative CBCT two-dimensional projection during respiration; The reconstructed image is close to the actual collected CBCT 2D projection, and the weight of each principal component in the prior respiratory motion model is calculated; the deformation field is determined according to the weight, and the reconstructed and synthesized 4D image is obtained according to the deformation field and the reference image.
进一步地，治疗时采集实时光学体表信号，与获取的治疗前的体表数据相关联，再通过体表数据与同一呼吸时相的体内解剖结构图像的映射关系，进一步得到体内解剖结构的4D图像。Further, real-time optical body surface signals are collected during treatment and correlated with the acquired body surface data before treatment, and then through the mapping relationship between the body surface data and the in vivo anatomical structure images of the same respiratory phase, a 4D 4D in vivo anatomical structure is further obtained. image.
进一步地，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，再通过4D图像重建合成单元重建4D图像，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，如偏差较大，则计算满足该条件的形变场，然后重新训练模型。Further, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. Then the 4D image is reconstructed by the 4D image reconstruction and synthesis unit. At this time, the deformation field of the anatomical structure in the body should satisfy the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image and the real collected two-dimensional projection map. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
进一步地，体内解剖结构的边界对应体表的变化，从体表的形变中推断体内解剖结构的形变。Further, the boundaries of the in vivo anatomical structure correspond to the changes of the body surface, and the deformation of the in vivo anatomical structure is inferred from the deformation of the body surface.
进一步地，从历史数据中挖掘体表形变与体内形变的关系，具体过程如下：搜集患者四维CT图像，分割出体表轮廓，以体表形变或四维CT图像或体表轮廓作为模型的输入，以某一时相的CT作为参考图像，将参考图像配准到其他时相的CT上，得到输出为体内解剖结构形变场的时间序列的模型；治疗时将实时光学体表数据输入模型中预测体内解剖结构的形变场，作用在参考图像上即可得到该时刻的图像。Further, the relationship between body surface deformation and internal body deformation was mined from historical data, and the specific process was as follows: collecting four-dimensional CT images of patients, segmenting the body surface contour, and using the body surface deformation or the four-dimensional CT image or body surface contour as the input of the model, Taking the CT of a certain phase as the reference image, the reference image is registered to the CT of other phases, and the output is a time series model of the deformation field of the anatomical structure in the body; during the treatment, the real-time optical body surface data is input into the model to predict the in vivo model. The deformation field of the anatomical structure acts on the reference image to obtain the image at that moment.
进一步地，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，计算满足该条件的形变场权重，然后重新训练模型。Further, during the treatment process, the two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the acquired projection map is used as correction data to improve and optimize the subsequent results, and determine the breathing time through the synchronously collected optical body surface. At this time, the deformation field of the anatomical structure in the body should meet the requirement that the two-dimensional digital reconstruction projection calculated after the deformation field is applied to the reference image is nearly consistent with the real collected two-dimensional projection image, and the deformation field that satisfies this condition is calculated. weights, and then retrain the model.
进一步地，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，并根据同步 采集的光学体表确定对应的呼吸时相，预测出四维图像，从而生成同一时相、同一角度的数字重建投影图像；重建投影图像与通过X线成像的方式，得到患者的二维投影图进行比较，验证上述方法的准确性。Further, in the course of treatment, a two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the corresponding respiratory phase is determined according to the optical body surface collected synchronously, and a four-dimensional image is predicted, thereby generating the same phase and the same phase. The digital reconstructed projection image of the angle; the reconstructed projection image is compared with the two-dimensional projection map of the patient obtained by means of X-ray imaging to verify the accuracy of the above method.
需要说明的是，不同实施例可能产生的有益效果不同，在不同的实施例里，可能产生的有益效果可以是以上任意一种或几种的组合，也可以是其他任何可能获得的有益效果。It should be noted that different embodiments may have different beneficial effects, and in different embodiments, the possible beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.
上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述详细披露仅仅作为示例，而并不构成对本发明的限定。The basic concept has been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example, and does not constitute a limitation of the present invention.
此外，除非权利要求中明确说明，本发明所述处理元素和序列的顺序、数字字母的使用、或其他名称的使用，并非用于限定本发明流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本发明实施例实质和范围的修正和等价组合。Furthermore, unless explicitly stated in the claims, the order of the processing elements and sequences described in the present invention, the use of numbers and letters, or the use of other names are not intended to limit the order of the processes and methods of the present invention. While the foregoing disclosure discusses by way of various examples some embodiments of the invention that are presently believed to be useful, it is to be understood that such details are for purposes of illustration only and that the appended claims are not limited to the disclosed embodiments, but rather The claims are intended to cover all modifications and equivalent combinations that fall within the spirit and scope of the embodiments of the invention.
同理，应当注意的是，为了简化本发明披露的表述，从而帮助对一个或多个发明实施例的理解，前文对本发明实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本发明对象所需要的特征比权利要求中提及的特征多。Similarly, it should be noted that, in order to simplify the expressions disclosed in the present invention and thereby help the understanding of one or more embodiments of the present invention, in the foregoing description of the embodiments of the present invention, various features may sometimes be combined into one embodiment, in the drawings or descriptions thereof. However, this method of disclosure does not imply that the object of the invention requires more features than are mentioned in the claims.
最后，应当理解的是，本发明中所述实施例仅用以说明本发明实施例的原则。其他的变形也可能属于本发明的范围。因此，作为示例而非限制，本发明实施例的替代配置可视为与本发明的教导一致。相应地，本发明的实施例不仅限于本发明明确介绍和描述的实施例。Finally, it should be understood that the embodiments described in the present invention are only used to illustrate the principles of the embodiments of the present invention. Other variations are also possible within the scope of the present invention. Accordingly, by way of example and not limitation, alternative configurations of embodiments of the present invention may be considered consistent with the teachings of the present invention. Accordingly, the embodiments of the present invention are not limited to those expressly introduced and described in the present invention.

Claims (10)

  1. 一种利用光学体表运动信号合成实时图像的***，包含：A system for synthesizing real-time images using optical body surface motion signals, comprising:
    获取单元，获取实时体表信号；图像转化单元，根据体表信号与4D医学图像显示的内部解剖结构信息的映射关系，通过获取体表信号，得到对应时间的4D医学图像显示的内部解剖结构信息。The acquisition unit acquires the real-time body surface signal; the image conversion unit obtains the internal anatomical structure information displayed by the 4D medical image at the corresponding time by acquiring the body surface signal according to the mapping relationship between the body surface signal and the internal anatomical structure information displayed by the 4D medical image. .
  2. 根据权利要求1所述的利用光学体表运动信号合成实时图像的***，其特征在于，4D医学图像包括直接拍摄获取的4D图像，或者重建合成的4D图像。The system for synthesizing real-time images using optical body surface motion signals according to claim 1, wherein the 4D medical image comprises a 4D image obtained by direct shooting, or a reconstructed and synthesized 4D image.
  3. 根据权利要求1所述的利用光学体表运动信号合成实时图像的***，其特征在于，还包括4D图像重建合成单元，获取治疗前的CBCT同时采集同步体表数据，将CBCT的各角度投影划分为不同的时相，将同步光学体表数据与呼吸时相相关联，对CBCT各时相投影进行基于稀疏投影数据的重建,将体表数据与同一呼吸时相的体内解剖结构图像相关联。The system for synthesizing real-time images using optical body surface motion signals according to claim 1, further comprising a 4D image reconstruction and synthesis unit, which acquires the CBCT before treatment and simultaneously collects synchronous body surface data, and divides the projections of each angle of the CBCT into divisions. For different time phases, the synchronized optical body surface data is correlated with the respiratory time phase, and the CBCT projections of each time phase are reconstructed based on the sparse projection data, and the body surface data is correlated with the in vivo anatomical structure images of the same respiratory time phase.
  4. 根据权利要求3所述的利用光学体表运动信号合成实时图像的***，其特征在于，4D图像重建合成单元重建4D图像的具体过程如下：(1)先验呼吸运动模型的生成：对四维CT的各个时相与以某一时刻四维CT为参考进行配准，将得到的形变场分解为主成分的加权和；(2)体表数据的排序：从四维CT的各个时相中分割出体表轮廓，将其与呼吸时相相关联，从而得到根据瞬时体表推断呼吸时相的方法；(3)基于CBCT稀疏投影数据的重建：根据CBCT扫描过程中同步采集的光学体表数据，按呼吸时相对CBCT二维投影进行排序；根据CBCT形变后计算出的二维数字重建图像与实际采集的CBCT二维投影趋近一致，计算先验呼吸运动模型中各主成分的权重；根据权重确定形变场，根据该形变场与参考图像获得重建合成的4D图像。The system for synthesizing real-time images using optical body surface motion signals according to claim 3, wherein the specific process of reconstructing the 4D image by the 4D image reconstruction and synthesis unit is as follows: (1) Generation of a priori respiratory motion model: for four-dimensional CT Each time phase of the 4D CT is registered with the 4D CT at a certain moment as a reference, and the obtained deformation field is decomposed into the weighted sum of the principal components; (2) Sorting of the body surface data: The body is segmented from each time phase of the 4D CT. (3) Reconstruction based on CBCT sparse projection data: According to the optical body surface data collected synchronously during CBCT scanning, press Sort the relative CBCT 2D projection during breathing; according to the 2D digital reconstruction image calculated after CBCT deformation and the actual collected CBCT 2D projection, the weight of each principal component in the prior breathing motion model is calculated; determined according to the weight Deformation field, based on the deformation field and the reference image to obtain a reconstructed and synthesized 4D image.
  5. 根据权利要求3或4所述的利用光学体表运动信号合成实时图像的***，其特征在于，治疗时采集实时光学体表信号，与获取的治疗前的体表数据相关联，通过体表数据与同一呼吸时相的体内解剖结构图像的映射关系，进一步得到体内解剖结构的4D图像。The system for synthesizing real-time images using optical body surface motion signals according to claim 3 or 4, wherein the real-time optical body surface signals are collected during treatment, and are associated with the acquired body surface data before treatment. The mapping relationship with the in vivo anatomical structure images in the same respiratory phase further obtains a 4D image of the in vivo anatomical structures.
  6. 根据权利要求1所述的利用光学体表运动信号合成实时图像的***，其特征在于，还包括4D图像重建合成单元，体内解剖结构的边界对应体表的变化，从体表的形变中推断体内解剖结构的形变。The system for synthesizing real-time images using optical body surface motion signals according to claim 1, further comprising a 4D image reconstruction and synthesis unit, wherein the boundary of the anatomical structure in the body corresponds to the change of the body surface, and infers the body surface from the deformation of the body surface. Deformation of anatomical structures.
  7. 根据权利要求6所述的利用光学体表运动信号合成实时图像的***，其特征在于，从历史数据中挖掘体表形变与体内形变的关系，具体过程如下：搜集患者四维CT图像，分割出体表轮廓，以体表形变或四维CT图像或体表轮廓作为模型的输入，以某一时相的CT作为参考图像，将参考图像配准到其他时相的CT上，得到输出为体内解剖结构形变场的时间序列的模型；治疗时将实时光学体表数据输入模型中预测体内解剖结构的形变场，作用在参考 图像上即可得到该时刻的图像。The system for synthesizing real-time images using optical body surface motion signals according to claim 6, wherein the relationship between body surface deformation and body deformation is excavated from historical data, and the specific process is as follows: collecting four-dimensional CT images of patients, segmenting the body Surface contour, taking body surface deformation or four-dimensional CT image or body surface contour as the input of the model, taking the CT of a certain phase as the reference image, registering the reference image to the CT of other phases, and obtaining the output as the deformation of the anatomical structure in vivo The model of the time series of the field; during the treatment, the real-time optical body surface data is input into the model to predict the deformation field of the anatomical structure in the body, and the image at this moment can be obtained by acting on the reference image.
  8. 根据权利要求1至7任一项所述的利用光学体表运动信号合成实时图像的***，其特征在于，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，并根据同步采集的光学体表确定对应的呼吸时相，预测出四维图像，从而生成同一时相、同一角度的数字重建投影图像；重建投影图像与通过X线成像的方式，得到患者的二维投影图进行比较，验证上述方法的准确性。The system for synthesizing real-time images using optical body surface motion signals according to any one of claims 1 to 7, characterized in that, during the treatment process, a two-dimensional projection map of the patient is obtained by means of X-ray imaging, and according to The optical body surface collected synchronously determines the corresponding respiratory phase and predicts a four-dimensional image, thereby generating a digital reconstruction projection image of the same phase and angle; the reconstructed projection image and X-ray imaging are used to obtain a two-dimensional projection image of the patient Compare and verify the accuracy of the above methods.
  9. 根据权利要求3至5任一项所述的利用光学体表运动信号合成实时图像的***，其特征在于，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，再通过4D图像重建合成单元重建4D图像，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，如偏差较大，则计算满足该条件的形变场，然后重新训练模型。The system for synthesizing real-time images using optical body surface motion signals according to any one of claims 3 to 5, characterized in that, during the treatment process, a two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the collected The projection map is used as the correction data to improve and optimize the subsequent results. The respiratory phase is determined by the optical body surface collected synchronously, and then the 4D image is reconstructed by the 4D image reconstruction and synthesis unit. At this time, the deformation field of the anatomical structure in the body should satisfy the The 2D digital reconstructed projection calculated after the deformation field acts on the reference image is nearly consistent with the real collected 2D projection image. If the deviation is large, the deformation field that satisfies this condition is calculated, and then the model is retrained.
  10. 根据权利要求6或7所述的利用光学体表运动信号合成实时图像的***，其特征在于，在治疗过程中，通过X线成像的方式，得到患者的二维投影图，采集到的投影图作为修正数据，对后续结果进行改进和优化，通过同步采集的光学体表确定呼吸时相，此时体内解剖结构的形变场，应当满足将该形变场作用在参考图像上后计算出的二维数字重建投影与真实采集的二维投影图趋近一致，计算满足该条件的形变场权重，然后重新训练模型。The system for synthesizing real-time images using optical body surface motion signals according to claim 6 or 7, characterized in that, during the treatment process, a two-dimensional projection map of the patient is obtained by means of X-ray imaging, and the collected projection map As the correction data, the subsequent results are improved and optimized, and the respiratory phase is determined by the optical body surface collected synchronously. At this time, the deformation field of the anatomical structure in the body should satisfy the two-dimensional calculated after the deformation field is applied to the reference image. The digitally reconstructed projections are nearly identical to the real collected 2D projections, the weights of the deformation fields that satisfy this condition are calculated, and then the model is retrained.
PCT/CN2021/075866 2021-02-07 2021-02-07 System for synthesizing real-time image by using optical body surface motion signals WO2022165812A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202180000460.2A CN113574565A (en) 2021-02-07 2021-02-07 System for synthesizing real-time image by using optical body surface motion signal
PCT/CN2021/075866 WO2022165812A1 (en) 2021-02-07 2021-02-07 System for synthesizing real-time image by using optical body surface motion signals

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/075866 WO2022165812A1 (en) 2021-02-07 2021-02-07 System for synthesizing real-time image by using optical body surface motion signals

Publications (1)

Publication Number Publication Date
WO2022165812A1 true WO2022165812A1 (en) 2022-08-11

Family

ID=78174902

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/075866 WO2022165812A1 (en) 2021-02-07 2021-02-07 System for synthesizing real-time image by using optical body surface motion signals

Country Status (2)

Country Link
CN (1) CN113574565A (en)
WO (1) WO2022165812A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117679671A (en) * 2024-02-02 2024-03-12 四川大学华西医院 Method, system and storage medium for reconstructing radiation therapy 4D dose

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114225236A (en) * 2021-12-20 2022-03-25 联影(常州)医疗科技有限公司 Radiotherapy guiding device, radiotherapy guiding method, electronic equipment and storage medium
WO2024108409A1 (en) * 2022-11-23 2024-05-30 北京肿瘤医院(北京大学肿瘤医院) Non-contact four-dimensional imaging method and system based on four-dimensional surface respiratory signal

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090175406A1 (en) * 2008-01-07 2009-07-09 Hui Zhang Target tracking using surface scanner and four-dimensional diagnostic imaging data
CN108853753A (en) * 2016-09-30 2018-11-23 上海联影医疗科技有限公司 Tumour real time monitoring apparatus, radiotherapy system
CN109727672A (en) * 2018-12-28 2019-05-07 江苏瑞尔医疗科技有限公司 Patient's thorax and abdomen malignant respiratory movement predicting tracing method
CN112154483A (en) * 2020-07-15 2020-12-29 北京肿瘤医院(北京大学肿瘤医院) Method and system for synthesizing real-time image by using optical body surface motion signal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090175406A1 (en) * 2008-01-07 2009-07-09 Hui Zhang Target tracking using surface scanner and four-dimensional diagnostic imaging data
CN108853753A (en) * 2016-09-30 2018-11-23 上海联影医疗科技有限公司 Tumour real time monitoring apparatus, radiotherapy system
CN109727672A (en) * 2018-12-28 2019-05-07 江苏瑞尔医疗科技有限公司 Patient's thorax and abdomen malignant respiratory movement predicting tracing method
CN112154483A (en) * 2020-07-15 2020-12-29 北京肿瘤医院(北京大学肿瘤医院) Method and system for synthesizing real-time image by using optical body surface motion signal

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117679671A (en) * 2024-02-02 2024-03-12 四川大学华西医院 Method, system and storage medium for reconstructing radiation therapy 4D dose
CN117679671B (en) * 2024-02-02 2024-05-14 四川大学华西医院 Method and system for reconstructing radiation therapy 4D dose

Also Published As

Publication number Publication date
CN113574565A (en) 2021-10-29

Similar Documents

Publication Publication Date Title
WO2022165812A1 (en) System for synthesizing real-time image by using optical body surface motion signals
US8559689B2 (en) Medical image processing apparatus, method, and program
CN102715906B (en) Method and system for 3D cardiac motion estimation from single scan of c-arm angiography
Ehlke et al. Fast generation of virtual X-ray images for reconstruction of 3D anatomy
WO2022011617A1 (en) Method and system for using optical body surface motion signal to synthesize real-time image
US20210282730A1 (en) Reduced interaction ct scanning
CN112785632B (en) Cross-modal automatic registration method for DR and DRR images in image-guided radiotherapy based on EPID
JP5510619B1 (en) Image processing device
US20110243387A1 (en) Analysis of Radiographic Images
US9355454B2 (en) Automatic estimation of anatomical extents
Alam et al. Medical image registration: Classification, applications and issues
KR101767069B1 (en) Method and apparatus for tumor tracking during radiation treatment using image registration and tumor matching between planning 4d mdct and treatment 4d cbct
Humbert et al. 3D reconstruction of both shape and bone mineral density distribution of the femur from DXA images
Whitmarsh et al. 3D bone mineral density distribution and shape reconstruction of the proximal femur from a single simulated DXA image: an in vitro study
US20080285822A1 (en) Automated Stool Removal Method For Medical Imaging
WO2024108409A1 (en) Non-contact four-dimensional imaging method and system based on four-dimensional surface respiratory signal
CN111954494B (en) Medical image processing device, medical image processing method, and recording medium
CN109325925A (en) A kind of medicine dynamic image respiration motion compensation method based on sparse subspace clustering
Alam et al. Generalizable cone beam ct esophagus segmentation using in silico data augmentation
Liu et al. A shape-navigated image deformation model for 4D lung respiratory motion estimation
KR102440954B1 (en) Method and apparatus to predict 3 dimensional face following orthodontic treatment based on deep learning
Price et al. 4D cone beam CT phase sorting using high frequency optical surface measurement during image guided radiotherapy
Preiswerk et al. Towards more precise, minimally-invasive tumour treatment under free breathing
US20230281842A1 (en) Generation of 3d models of anatomical structures from 2d radiographs
CN118212482A (en) Multi-mode hybrid training method for artificial intelligent medical image analysis

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21923823

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21923823

Country of ref document: EP

Kind code of ref document: A1