WO2019128660A1 - 训练神经网络的方法和设备、图像处理方法和设备以及存储介质 - Google Patents

训练神经网络的方法和设备、图像处理方法和设备以及存储介质 Download PDF

Info

Publication number
WO2019128660A1
WO2019128660A1 PCT/CN2018/119372 CN2018119372W WO2019128660A1 WO 2019128660 A1 WO2019128660 A1 WO 2019128660A1 CN 2018119372 W CN2018119372 W CN 2018119372W WO 2019128660 A1 WO2019128660 A1 WO 2019128660A1
Authority
WO
WIPO (PCT)
Prior art keywords
projection
image
estimated
data
network
Prior art date
Application number
PCT/CN2018/119372
Other languages
English (en)
French (fr)
Inventor
邢宇翔
梁凯超
沈乐
张丽
杨洪恺
康克军
陈志强
李荐民
刘以农
Original Assignee
清华大学
同方威视技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 清华大学, 同方威视技术股份有限公司 filed Critical 清华大学
Publication of WO2019128660A1 publication Critical patent/WO2019128660A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/006—Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N23/00—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00
    • G01N23/02—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material
    • G01N23/04—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/005—Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G06T11/008—Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20088—Trinocular vision calculations; trifocal tensor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/416—Exact reconstruction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/421—Filtered back projection [FBP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2211/00—Image generation
    • G06T2211/40—Computed tomography
    • G06T2211/436—Limited angle

Definitions

  • Embodiments of the present disclosure relate to radiation imaging, and in particular, to a method and apparatus for training a neural network, an image processing method, an image processing apparatus, and a storage medium.
  • X-ray CT Computerized-Tomography
  • the ray source and the detector collect a series of fading signal data according to a certain orbit.
  • the CT image reconstruction process consists in restoring the linear attenuation coefficient distribution from the data acquired by the detector.
  • the filtered back projection (Filtered Back-Projection), the FDK (Feldkmap-Davis-Kress) class analytical reconstruction algorithm and the ART (Algebra Reconstruction Technique), MAP (Maximum A Posterior) and other iterative reconstruction methods are mainly used.
  • a method and apparatus for training a neural network, an image processing method, an image processing apparatus, and a storage medium are proposed, and the neural network obtained through training can improve the quality of the reconstructed image.
  • a method for training a neural network comprising: a projection domain network for processing input projection data to obtain estimated projection data; and an analytical reconstruction network layer for The projection data is estimated and reconstructed to obtain a reconstructed image; the image domain network is used to process the reconstructed image to obtain an estimated image; and the projection layer is used to perform projection operation on the estimated image by using a system projection matrix of the CT scanning system to obtain an estimated image.
  • the statistical model layer for determining the input of the projection data, the estimated projection data, and the projection result of the estimated image based on the consistency of the statistical model;
  • the method comprises: adjusting a convolution kernel parameter of the image domain network and the projection domain network by using a consistency cost function of a data model of the input projection data, the estimated projection data, and the projection result of the estimated image.
  • the method further includes: constructing a cost function consistent with the projection using the projection layer, constructing a likelihood relation cost function using the statistical model layer, and at least one of a cost function and a likelihood relation cost function consistent with the projection A consistent cost function that forms the data model.
  • the convolutional neural network further includes at least one a priori model layer, the prior model layer adjusts an image domain network by using an a priori model cost function based on the estimated image, and reconstructs the network layer to the gradient after analysis Reverse pass to adjust the convolution kernel parameters of the projection domain network.
  • the forward transfer process of the projection domain network, the analytical reconstruction network layer, and the image domain network includes:
  • the estimated projection data output by the projection domain network is expressed as M' ⁇ M, after weighting Obtained after the ramp filter layer
  • the output of the network layer is parsed and reconstructed. make An image representing the function of the image domain network, and an estimated image of the image domain network output
  • the superscript T represents the transpose of the matrix
  • h is the discretized ramp filter operator
  • H R is the M′ ⁇ N-dimensional reconstruction system matrix
  • N is the total number of pixels of the reconstructed image
  • W 1 , W 2 , . , W M ' denotes a weighting coefficient
  • the consistency cost function of the data model is expressed as The error transfer relationship from data model consistency is:
  • the prior model cost function according to an embodiment of the present disclosure Including local variation of total variation, Markov field prior, texture prior, feature space sparsity, or a combination thereof, wherein For the estimated image.
  • the method further includes defining a priori model cost function using a priori error ⁇ Pr
  • Cost function according to each prior model according to an embodiment of the present disclosure The importance of ⁇ in the error feedback process to adjust the image domain network.
  • the a priori error back-transfer of the analytical reconstruction network layer is implemented according to the following transfer relationship:
  • the estimated projection data output by the projection domain network is expressed as M' ⁇ M, after weighting Obtained after the ramp filter layer After back projection, the output of the network layer is parsed and reconstructed.
  • the superscript T indicates the transposition of the matrix
  • h is the discretized slope filter operator
  • H R is the M′ ⁇ N-dimensional reconstruction system matrix
  • N is the total number of pixels of the reconstructed image
  • W 1 , W 2 , ..., W M ' denotes a weighting coefficient.
  • the method further includes: with They are passed together to the projection domain network to update the parameters of each layer.
  • the method further includes: acquiring, by the CT scanning system, the attenuation signal data of the object, and pre-processing the attenuation signal data to obtain the input projection data.
  • the method further includes acquiring, by the CT scanning system, projection data of the object according to one of the following scanning modes: a detector undersampling scan, a sparse angle scan, an inner reconstruction scan, a finite angle scan, and a linear trajectory scan .
  • the projection domain network includes a plurality of parallel convolutional neural network branches.
  • the image domain network includes a U-shaped convolutional neural network.
  • the method further includes training the convolutional neural network with the simulation data set as input projection data.
  • an image processing method including:
  • the convolutional neural network includes: a projection domain network for processing input projection data to obtain estimated projection data; an analytical reconstruction network layer for performing analytical reconstruction on the estimated projection data to obtain a reconstructed image; and an image domain network; For processing the reconstructed image to obtain an estimated image; a projection layer for performing a projection operation on the estimated image by using a system projection matrix of the CT scanning system to obtain a projection result of the estimated image; and a statistical model layer for determining the input projection
  • the data, the estimated projection data, and the projection result of the estimated image are based on the consistency of the statistical model;
  • the image processing method comprises training a convolutional neural network, comprising: adjusting an image domain network and using a consistency cost function of a data model based on the input projection data, the estimated projection data, and the projection result of the estimated image; The convolution kernel parameter of the projection domain network.
  • an apparatus for training a neural network comprising:
  • Memory for storing instructions and data
  • a processor configured to execute the instructions to:
  • Constructing the neural network to include: a projection domain network for processing input projection data to obtain estimated projection data; an analytical reconstruction network layer for performing analytical reconstruction on the estimated projection data to obtain a reconstructed image; and an image domain network; For processing the reconstructed image to obtain an estimated image; a projection layer for performing a projection operation on the estimated image by using a system projection matrix of the CT scanning system to obtain a projection result of the estimated image; and a statistical model layer for determining the input projection
  • the data, the estimated projection data, and the projection result of the estimated image are based on the consistency of the statistical model;
  • the processor is further configured to train the neural network, including adjusting an image domain network and using a consistency cost function of a data model based on the input projection data, the estimated projection data, and the projection result of the estimated image
  • the convolution kernel parameter of the projection domain network is further configured to train the neural network, including adjusting an image domain network and using a consistency cost function of a data model based on the input projection data, the estimated projection data, and the projection result of the estimated image.
  • an image processing apparatus comprising:
  • Memory for storing instructions and data
  • a processor configured to execute the instructions to:
  • the processor is further configured to construct the convolutional neural network to include: a projection domain network for processing input projection data to obtain estimated projection data; and an analytical reconstruction network layer for estimating projection data Analytical reconstruction, obtaining reconstructed image; image domain network for processing the reconstructed image to obtain an estimated image; and a projection layer for performing projection operation on the estimated image by using a system projection matrix of the CT scanning system to obtain a projection result of the estimated image; And a statistical model layer for determining the input of the projection data, the estimated projection data, and the projection result of the estimated image based on the consistency of the statistical model;
  • the processor is further configured to train the convolutional neural network, including adjusting an image domain by using a consistency cost function of a data model of the input projection data, the estimated projection data, and the projection result of the estimated image.
  • a computer readable storage medium having stored therein computer instructions that, when executed by a processor, implement a method in accordance with the present disclosure.
  • FIG. 1 is a schematic structural diagram of a CT apparatus according to an embodiment of the present disclosure
  • FIG. 2 is a schematic structural diagram of a control and data processing apparatus in the CT apparatus shown in FIG. 1;
  • FIG. 3 illustrates an example of a sinogram of projection data in accordance with an embodiment of the present disclosure
  • Figure 4 shows a schematic diagram of data contained in a sinogram in different scanning modes
  • FIG. 5 is a schematic diagram of a scanning device implementing a sparse angle sampling scanning mode according to an embodiment of the present disclosure
  • FIG. 6A shows a schematic structural diagram of a neural network according to an embodiment of the present disclosure
  • FIG. 6B shows another structural schematic diagram of a neural network in accordance with an embodiment of the present disclosure
  • Figure 7 is a schematic diagram depicting images processed by various modules in the neural network architecture as shown in Figures 6A and 6B;
  • FIG. 8 is a block diagram showing a structure of a projection domain network used in a device according to an embodiment of the present disclosure
  • FIG. 9 is a block diagram showing the structure of a parsing reconstruction network layer used in a device according to an embodiment of the present disclosure.
  • FIG. 10 is a diagram showing an example of a structure of an image domain network in a device according to still another embodiment of the present disclosure.
  • FIG. 11 is a block diagram showing the structure of a smooth conditional convolution kernel used in a convolutional neural network according to an embodiment of the present disclosure
  • 12A, 12B, and 12C illustrate schematic views of dimensions of a filter core used in a device in accordance with an embodiment of the present disclosure
  • FIG. 13A is a schematic flowchart depicting an image processing method according to an embodiment of the present disclosure.
  • FIG. 13B is a schematic flowchart depicting a method of training a neural network according to an embodiment of the present disclosure
  • FIG. 14 is a schematic diagram of a scanning device that implements a limited angle CT scan, in accordance with another embodiment of the present disclosure.
  • FIG. 15 is a schematic diagram of a scanning apparatus implementing an internal reconstruction scanning mode according to still another embodiment of the present disclosure.
  • 16 is a schematic diagram of a scanning device implementing a detector undersampling scanning mode according to still another embodiment of the present disclosure
  • FIG. 17 shows a schematic diagram of a scanning device that implements a linear trajectory CT scan in accordance with yet another embodiment of the present disclosure.
  • references to "one embodiment”, “an embodiment”, “an” or “an” or “an” In at least one embodiment.
  • the appearances of the phrase “in one embodiment”, “in the embodiment”, “the” Furthermore, the particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and/or sub-combination.
  • the term “and/or” as used herein includes any and all combinations of one or more of the associated listed items.
  • Embodiments of the present disclosure propose a method and apparatus for training a neural network, and an image processing method and apparatus therefor.
  • the neural network is utilized to process the input projection data to obtain an estimated image of the object.
  • the neural network may include: a projection domain network, an analytical reconstruction network layer, an image domain network, a projection layer, and a statistical model layer.
  • the projection domain network processes the input projection data to obtain estimated projection data.
  • the analytical reconstruction network layer performs analytical reconstruction on the estimated projection data to obtain a reconstructed image.
  • the image domain network processes the reconstructed image to obtain an estimated image.
  • the projection layer performs a projection operation on the estimated image by using a system projection matrix of the CT scanning system to obtain a projection result of the estimated image.
  • the statistical model layer determines the input of the projection data, the estimated projection data, and the projection result of the estimated image based on the consistency of the statistical model.
  • the convolution kernel parameters of the image domain network and the projection domain network are adjusted using a consistency cost function of the data model based on the input projection data, the estimated projection data, and the projection result of the estimated image.
  • FIG. 1 is a block diagram showing the structure of a CT apparatus according to an embodiment of the present disclosure.
  • the CT apparatus includes an X-ray source 10, a mechanical motion device 50, a detector and data acquisition system 20, and a control and data processing device 60 for performing CT scans and data on the object 40 to be inspected. Processing, such as training of neural networks and reconstruction of images using trained networks.
  • the X-ray source 10 can be, for example, an X-ray machine, and a suitable X-ray machine focus size can be selected depending on the resolution of the imaging. In other embodiments, it is also possible to generate an X-ray beam using a linear accelerator or the like without using an X-ray machine.
  • the mechanical motion device 50 includes a stage and a frame, a control system, and the like.
  • the stage can be translated to adjust the position of the center of rotation
  • the frame can be translated to align the X-ray source (X-ray machine) 10, the detector, and the center of rotation.
  • the description is made according to the circumferential scanning trajectory or the spiral trajectory of the rotating stage and the fixed frame. Since the movement of the stage and the frame is relative motion, the method of the embodiment can also be implemented by means of the stage stationary and the rotation of the frame.
  • the detector and data acquisition system 20 includes an X-ray detector, a data acquisition circuit, and the like.
  • the X-ray detector may use a solid state detector, or a gas detector or other detector, and embodiments of the present disclosure are not limited thereto.
  • the data acquisition circuit includes a readout circuit, an acquisition trigger circuit, and a data transmission circuit.
  • the control and data processing device 60 includes, for example, a computer device equipped with a control program and a data processing program, responsible for completing control of the CT system operation process, including mechanical rotation, electrical control, safety interlock control, etc., training the neural network, and utilizing the training The neural network reconstructs a CT image or the like from the projection data.
  • FIG. 2 shows a block diagram of the control and data processing device 200 shown in FIG. 1.
  • the data collected by the detector and data acquisition system 20 is stored in the storage device 210 via the interface unit 270 and the bus 280.
  • Configuration information and a program of the computer data processor are stored in the read only memory (ROM) 220.
  • a random access memory (RAM) 230 is used to temporarily store various data during the operation of the processor 250.
  • a computer program for performing data processing such as a program for training a neural network and a program for reconstructing a CT image, and the like, are also stored in the storage device 210.
  • the internal bus 280 is connected to the above-described storage device 210, read only memory 220, random access memory 230, input device 240, processor 250, display device 260, and interface unit 270.
  • the instruction code of the computer program commands processor 250 to execute an algorithm that trains the neural network and/or an algorithm that reconstructs the CT image.
  • the reconstruction result is obtained, it is displayed on a display device 260 such as an LCD display, or the processing result is directly outputted in the form of a hard copy such as printing.
  • the object to be inspected is subjected to CT scanning using the above-described apparatus to obtain an original attenuation signal.
  • Such attenuated signal data may also be displayed in the form of a two-dimensional image, and FIG. 3 illustrates an example of attenuated signal data obtained in accordance with an embodiment of the present disclosure.
  • the horizontal axis direction of the original attenuation signal as shown in FIG. 3 represents the detector pixel sequence (for example, from 1 to 256), and the vertical axis represents the angle (for example, from 1 degree to 360 degrees).
  • the original attenuated signal is preprocessed to become projection data.
  • the projection data may be preprocessed by a CT scanning system by performing negative logarithmic transformation or the like on the projection data.
  • the processor 250 in the control device performs a reconstruction process, processes the projection data by using the trained neural network, obtains estimated projection data, and performs a reconstruction operation on the estimated projection data through the analytical reconstruction network layer to obtain a reconstructed image.
  • the reconstructed image is further processed to obtain a final image.
  • a reconstructed image is processed using a trained (eg, U-shaped) convolutional neural network to obtain feature maps of different scales, and feature maps of different scales are combined to obtain a resulting image.
  • the projection data is processed in the projection domain using a trained convolutional neural network, and then a reconstruction layer reconstruction operation is performed to reconstruct the image, and finally the image domain network processes the reconstructed image to obtain an estimated image.
  • the convolutional neural network can include a convolutional layer, a pooled, and a fully connected layer.
  • the convolutional layer identifies the characterization of the input data set, with each nonlinear layer having a nonlinear activation function operation. Pooled layer refinement represents the representation of features, typical operations include averaging pooling and maximizing pooling.
  • One or more layers of fully connected layers enable high-order signal nonlinear synthesis, and the fully connected layer also has a nonlinear activation function. Commonly used nonlinear activation functions are Sigmoid, Tanh, ReLU, and so on.
  • Figure 4 shows a schematic diagram of the data contained in a sinogram in different scanning modes.
  • the projection data obtained by the angle sparse sampling CT scan, the limited angle CT scan, the detector undersampled CT scan, and the internal reconstruction CT scan are all incomplete.
  • the projection data is incomplete, with the above scheme, it is possible to reconstruct a higher quality image from these incomplete projection data.
  • FIG. 5 is a schematic diagram of a scanning device implementing a sparse angle sampling scanning mode according to still another embodiment of the present disclosure.
  • the X-rays emitted from the radiation source 10 are transmitted through the object 40 in the field of view 45, are received by the detector 30, converted into electrical signals, and further converted into digital signals representing attenuation values, which are preprocessed as Project the data for reconstruction by a computer.
  • the neural network trained by the method of the present disclosure can reconstruct an image of higher quality. In this way, even if the object to be inspected is subjected to the sparse angle CT scan, it is possible to reconstruct an image of higher quality from the incomplete projection data.
  • FIG. 6A shows a structural schematic diagram of a convolutional neural network in accordance with an embodiment of the present disclosure.
  • the input of the unsupervised X-ray CT image reconstruction neural network is the projection data obtained by preprocessing the attenuation signal after the CT scan.
  • the neural network may mainly include a projection domain network 610, a parsing reconstruction network layer 620, and an image domain network 630. Further, the neural network according to an embodiment of the present disclosure may further include a statistical model layer 640 and a projection layer 650.
  • FIG. 6B illustrates another structural schematic of a convolutional neural network in accordance with an embodiment of the present disclosure. Unlike the neural network structure shown in FIG. 6A, the neural network in FIG. 6B may further include at least one a priori model layer, and FIG. 6B shows three prior model layers 660, 670, and 680 as examples.
  • Figure 7 is a schematic diagram depicting images processed by various modules in the neural network architecture as shown in Figures 6A and 6B.
  • the input projection data can be expressed as g
  • the data processed by the projection domain network is represented as This can be referred to as "estimated projection data.”
  • the data processed by the analytical reconstruction network layer is expressed as
  • the data processed by the image domain network is expressed as This can be referred to as an "estimated image.”
  • the projection domain network 610 is used for restoration of the projection data and missing data estimates to obtain complete projection data.
  • the parsing reconstruction network layer 620 includes multiple layers. Although these layers are constructed based on analytical reconstruction algorithms known to those skilled in the art, they have undergone special matrixing, and the specific construction of these layers will be described in detail below.
  • the image domain network 630 is used to reduce artifacts and errors in the reconstructed image, further improving the quality of the reconstructed image.
  • Projection layer 650 estimates the image using the projection matrix of the CT scanning system A projection operation is performed to obtain a projection of the estimated image, so that a cost function consistent with the projection can be constructed, similar to the cost function of ART.
  • the statistical model layer 640 constitutes a cost branch of the network, describes the statistical fluctuations during projection acquisition, and is used to determine the input of the projection data, the estimated projection data, and the projection result of the estimated image based on the consistency of the statistical model, all of which Or a portion may be, for example, a likelihood relation cost function that the acquired X-ray CT data and the real data satisfy under a statistical model.
  • the neural network may further include a priori model layers 660, 670, 680 constituting an end cost function of the network.
  • a priori model layers 660, 670, 680 constituting an end cost function of the network.
  • the prior model layers 660, 670, 680 may be at least one of a local conditional total variation, a Markov field prior, a texture prior, a feature space sparsity, and other models, respectively.
  • the parameter ⁇ adjusts the importance (or intensity) of the prior model or constraint, and ⁇ 1 , ⁇ 2 , ⁇ 3 are shown in Figure 6, which can be used as the weights for the inverse transfer of the prior model layers 660, 670, 680, respectively.
  • the forward operation flow of the overall network is shown by the solid arrows in Figures 6A and 6B.
  • the estimated projection data output by the projection domain network 610 is Usually M' ⁇ M.
  • W 1 , W 2 , ..., W M ' denote weighting coefficients.
  • Filtered sinogram after ramping the filter layer h is a discretized ramp filter operator, which may be, for example, a Ram-Lak filter or a Shepp-Logan filter. In one example, h can be a discretized ramp convolution kernel.
  • H R is a matrix of M′ ⁇ N-dimensional reconstruction systems, similar to the forward projection matrix H (system projection matrix), determined by the architecture and scanning mode of the CT scanning system.
  • H system projection matrix
  • the back projection process is completed, and the weighted back projection process is completed under fan beam or cone beam scanning.
  • After rebuilding the network Continue to feed forward through the image domain network to get an estimated image of the scanned object
  • the output of each prior model layer is a cost function It may be one of a plurality of total variation such as local conditions, a Markov field prior, a texture prior, a feature space sparsity, or the like.
  • the error back-transfer of the overall network is shown by the dashed arrow shown in Figure 6.
  • the reverse pass is divided into two main lines: the consistency cost of the data model and the coincidence cost of the prior model.
  • Consistency cost function based on prior model Define a priori error
  • the parameter ⁇ defines each The importance of the error feedback process, that is, the importance of the image space prior model.
  • the error of the coincidence degree of the prior model is transmitted to the image domain network, and the derivative of each layer input and each layer parameter to the cost function is calculated by the chain derivation rule. Then, the network layer is reconstructed through parsing, and the a priori error back-transfer of the network layer of the analytical reconstruction network layer is completed as follows:
  • the consistency cost of the data model includes at least one of a likelihood relation cost function and a cost function consistent with the projection.
  • the likelihood relation cost function is defined according to the statistical model of the signal, which may be a Gaussian noise distribution, a Poisson probability distribution, a Gaussian and Poisson mixed probability distribution, etc. (the negative likelihood is obtained under the framework of error minimization) Defined here The smaller the input, the input projection data g and the estimated projection data The more consistent.
  • the cost function consistent with the projection reflects the projection result and estimated projection data of the previous estimation result (estimated image) after projection through the projection layer. difference between.
  • the consistency cost function of the data model can be expressed as:
  • H is the system projection matrix.
  • the training set data may include a simulation tool to generate a simulation model covering the application scenario, and generate a projection data set according to actual CT system parameters and scanning manners. Scan the object on the actual system to obtain CT scan data, some of which are also input into the network as training set data for further training, and use another part to collect data to test the network training effect.
  • FIG. 8 is a block diagram showing the structure of a projection domain convolutional neural network applied to a sparse angle device by a device in accordance with an embodiment of the present disclosure.
  • preprocessing such as negative logarithmic transformation of the original attenuation signal collected by the CT scanning system (preprocessing may also include adding air value correction, consistency correction)
  • projection data g is obtained, and the projection domain network 610 uses the projection data as the projection data.
  • the projection domain network complements the missing data in a convolutional neural network.
  • the input to the parallel network as shown in Figure 8 is the data acquired at the sparse angle.
  • the estimated missing angle data is divided into multiple groups, each of which has the same scale as the acquisition angle, and is a data with a constant angle from the acquisition angle.
  • the existing projection data is used as the input data, and the multi-level feature extraction is performed by concatenation of the convolution layer (Conv) including the activation function, through the fully connected layer (which can be implemented by a 1x1 convolution layer) Get missing projection data.
  • Conv convolution layer
  • the fully connected layer which can be implemented by a 1x1 convolution layer
  • the 2-dimensional convolution kernel of all scales has two dimensions, where the first dimension is defined as the detector direction and the second dimension is the scanning angle direction.
  • the lengths of the convolution kernels of the two dimensions need not be the same.
  • the scale of the convolution kernel in the direction of the detector is larger than the dimension of the scanning angle direction, for example, a convolution kernel of 3*1, 5*3, 7*3, 9*3.
  • the convolution kernel size can also be taken according to the two dimensional proportional relationship of the projection map. Multiple convolution kernels can be set for each scale.
  • the convolutional layer carries an activation function.
  • the projection domain network may include, for example, a 5-branch parallel network as shown in FIG.
  • Each path includes, for example, a seven-layer convolutional neural network.
  • the middle part uses a rectangular convolution kernel (Figs. 12A, 12B, and 12C) due to the proportional relationship between the detector and the projection number.
  • the convolution kernel size at the last layer is 1*1.
  • the feature map direction is fully connected, and each layer of the hoard layer output uses the Relu function as a nonlinear activation function.
  • the angular direction uses its periodic boundary data padding to keep the feature map size the same as the input.
  • the four sets of missing projection estimates are combined with a set of acquired and network-reduced projections to form an estimated projection image of 360 dense angles by interpolation processing. Analyze the reconstruction network layer.
  • FIG. 8 shows a convolutional network including a plurality of parallel branches, those skilled in the art will appreciate that the technical solutions of the present disclosure can also be implemented by other forms of networks.
  • the projection domain network is used to recover the missing data, that is, in the case of CT scanning for obtaining incomplete data, those skilled in the art can understand that the above projection domain network can complete the projection data. Process to improve the quality of the projection data.
  • the analytical reconstruction network layer 620 can include a weighting layer (optional), a ramp filtering layer (optional), and a back projection layer.
  • the weighting layer implements a cosine weighting for each data.
  • the ramp filter layer implements the ramp filter operation in the conventional analytical reconstruction method.
  • the back projection layer implements a back projection from the projection domain to the image domain (for the fan beam CT and the cone beam CT back projection layer is a distance weighted back projection).
  • the analytical reconstruction network layer is designed and implemented according to the architecture of the CT imaging system, and no parameter modification is performed during the network training process.
  • the analytical reconstruction network layer 620 explicitly adds the analytical reconstruction algorithm to the network structure, thereby simplifying the physical laws that the network needs to learn.
  • the analytical reconstruction network layer 620 includes three layers.
  • the first layer is a weighting layer.
  • the cosine vector of the detector direction data is cosine normalized by the cosine vector of 216*1.
  • the angle of the cosine normalized vector is expanded to obtain 216*360.
  • Weighting matrix W where each column value of W is equal. After passing through the W layer, the projected image is expressed as
  • the second layer is the ramp filter layer.
  • the ramp filter performs a discrete filtering operation on the direction of the projection detector.
  • the 216 detector response vector filtering is used for each angle, and the matrix multiplication method can be used to generate the filter matrix F of 216*216 and the multiplication of the weighted projection data matrix to complete the filtering process.
  • the third layer is the back projection layer.
  • the back projection layer reconstructs the filtered projection into an image according to the geometric relationship, and uses the pixel driving method to generate the distance weighted back projection matrix according to the geometric parameters of the application.
  • the filtered image is back-projected to obtain an image domain reconstructed image.
  • FIG. 10 shows an example schematic diagram of an image domain network used in a device in accordance with yet another embodiment of the present disclosure.
  • the image domain network can perform artifact suppression and noise suppression of the image domain.
  • the image domain network 630 shown in FIG. 6 may be the U-shaped network shown in FIG.
  • the image domain network uses a U-shaped network design ( Figure 10). Among them, for example, the reconstructed image of 200*200 is subjected to four pooling, and the feature map size is gradually reduced to increase the global feature of the accepted domain learning image.
  • the convolution kernel sizes at different levels are, for example, 3*3.
  • the number of feature maps gradually increases.
  • the number of feature maps gradually decreases.
  • FIG. 10 illustrates an image domain network as a specific structural example of a U-shaped network
  • those skilled in the art will appreciate that the technical solution of the present disclosure can also be implemented with a U-shaped network of other configurations.
  • those skilled in the art can also think of using other networks as image domain networks, such as an Auto-Encoder, a Fully Convolution Neural Network, etc., and can also implement the technical solutions of the present disclosure. .
  • all convolution kernels of the projection domain network 610 and the image domain network 630 are pending network parameters, which may be randomly initialized, or may be updated in other network pre-training results during the network training process.
  • the network processes the input data in the projection domain and the image domain, respectively, so that the objective function to be optimized (often referred to as the loss function in the depth learning domain) achieves an optimal result. Since the adjacent pixels in the projection domain and the image domain have different geometric relationships, the projection domain convolution layer and the image domain convolution layer can complement each other.
  • the acquired projection data g is a set of samples that conform to an independent Gaussian distribution whose mean is the integral of the linear attenuation coefficient of the scanned object on the corresponding ray path.
  • the first term in the constraint is the Gaussian model likelihood cost function, which completes the maximum likelihood estimation constraint from the sampled g to the distribution true value, and is only applicable to the partially reduced noise projection network of the sparse angle projection.
  • the second term represents the consistency constraint between the projection and the image, and is applicable to each part of the projection domain network.
  • g is the collected sparse angle projection data
  • H is the system projection matrix
  • ⁇ is the Lagrangian multiplier parameter
  • the asymmetry model of the a priori model cost function can be expressed as
  • the network output image is derived, and the gradient is inversely transmitted by the analytical reconstruction layer and then acts on the projection domain network convolution kernel.
  • the two data sets, the simulation data set and the actual data set may be used in the process of training the neural network.
  • the simulated data set is a high quality human CT tomogram from a source such as the network.
  • the angle can be started from 3 degrees, and the simulation projection data is generated in increments of 5 degrees to 358 degrees, and the number of photons is 10 5 .
  • a set of simulated projection data of 216*72 scale is obtained. 4/5 of them are used for unsupervised training of the network, and 1/5 is used as a verification set to control the normalization effect of the network.
  • the actual data set can be projected on the fan beam spiral CT platform at a 5 degree angular interval and the 216 detector array scans the phantom body, and the control tube current time reaches the normal dose.
  • a different set of phantoms was scanned in the same manner to obtain a projection as a test set, which was used to test the network effect after the training was completed.
  • a direct training approach is employed.
  • the projection domain network and the image domain network convolution kernel weight are randomly initialized, and the actual collected data set is trained. After the training is completed, another set of actual collected data is used as a test set to verify the network training effect.
  • pre-training assisted direct training may be employed.
  • the data emulation stage may generate high quality projection data, and the process of training the neural network using the simulated data set may be referred to as "pre-training".
  • the simulation data is used to generate 360 angles of complete projection data, and the supervised individual training parallel projection estimation network.
  • the projection is used to estimate the initial value of the network as the overall network projection domain network, and then the network is trained as a whole in addition to the initial training.
  • the collected data is input into the above training process to obtain the trained network (at this time, the network parameters are fixed), and the reconstructed image is obtained.
  • FIG. 13A is a schematic flow chart describing an image processing method according to an embodiment of the present disclosure.
  • step S131 projection data of the subject is acquired by the CT scanning system.
  • step S132 the projection data is processed using a convolutional neural network to acquire an estimated image of the object.
  • a neural network may include a projection domain network, a resolution reconstruction network layer, and an image domain network.
  • the projection domain network is used to process the input projection data to obtain estimated projection data.
  • the analytical reconstruction network layer performs analytical reconstruction on the estimated projection data to obtain a reconstructed image.
  • the image domain network processes the reconstructed image to obtain an estimated image.
  • a neural network according to an embodiment of the present disclosure may include a projection layer for performing a projection operation on an estimated image using a system projection matrix of a CT scanning system to obtain a projection result of the estimated image; and a statistical model layer for determining the input projection data, Estimating the projection data and the projection result of the estimated image are based on the consistency of the statistical model.
  • FIG. 13B is a schematic flowchart depicting a method of training a neural network, in accordance with an embodiment of the present disclosure.
  • step S1321 in the first training process, the simulation data set is used as the input projection data to train the neural network.
  • the first training process is to pre-train the neural network to speed up the training process.
  • step S1322 in the second training process, the collected real CT data is used as input projection data to further train the neural network.
  • the image domain network may be adjusted by using the prior model cost function of the estimated image, and the gradient is reversely transmitted through the analytical reconstruction network layer to adjust the convolution kernel parameter of the projection domain network.
  • the convolution kernel parameters of the image domain network and the projection domain network are adjusted using a consistency cost function of the data model based on the input projection data, the estimated projection data, and the projection result of the estimated image.
  • angle thin sampling scanning and the like are given above, those skilled in the art will appreciate that the training method of the present disclosure can also be used in limited angle CT scanning, internal reconstruction scanning, detector under sampling scanning, and linear trajectory CT scanning system. .
  • FIG. 14 is a schematic diagram of a scanning device that implements a limited angle CT scan, in accordance with another embodiment of the present disclosure.
  • the X-rays emitted from the radiation source 10 are transmitted through the object 40 in the field of view 45, and are received by the detector 30, converted into electrical signals to obtain attenuation data, and subjected to pre-processing operations to obtain projection data. Rebuild.
  • the trained neural network can reconstruct a higher quality image.
  • FIG. 15 is a schematic diagram of a scanning apparatus implementing an internal reconstruction scanning mode according to still another embodiment of the present disclosure.
  • the X-rays emitted from the radiation source 10 are transmitted through a portion of the object 40 to be inspected in the field of view 45, and then received by the detector 30, converted into an electrical signal, and further converted into a digital signal representing the attenuation value as a projection. Data, reconstructed by a computer.
  • the trained neural network can reconstruct a higher quality image.
  • FIG. 16 is a schematic diagram of a scanning device that implements an undersampling scan mode of a detector, in accordance with yet another embodiment of the present disclosure.
  • the X-rays emitted from the radiation source 10 are transmitted through the object 40 in the field of view 45 and then received by the detector 30, converted into electrical signals and converted into digital signals representing attenuation values, as projection data.
  • the computer is rebuilt.
  • detector 30 is set to undersample, for example, by spacing each detector unit a predetermined distance to achieve undersampling.
  • FIG. 17 shows a schematic diagram of a scanning device that implements a linear trajectory CT scan in accordance with yet another embodiment of the present disclosure.
  • the X-ray transmitted from the radiation source 10 is transmitted through the object 40 in the field of view and then received by the detector 30, converted into an electrical signal and converted into a digital signal representing the attenuation value, as projection data, by the computer.
  • the object under inspection 40 moves along a linear path on a conveyor belt that is parallel to the detector.
  • the detector is as large as possible in the horizontal direction with the angle of the source, covering the object in the vertical direction.
  • the detector array is placed on the opposite side of the source, and the ray horizontal opening angle ⁇ is required to be above 90 degrees to obtain a linear trajectory CT scan projection data.
  • the trained neural network can reconstruct a higher quality image.
  • the training method of the embodiment of the present disclosure may first use the simulation data to perform pre-training, and then use the real data for unsupervised training. In addition, you can directly use a large amount of unsupervised training of real data. In this way, the data information is deeply explored, and the convolutional neural network and system-specific parameters are formed, and an efficient CT image reconstruction method is obtained.
  • the method of the present disclosure can be flexibly applied to different CT scan modes and system architectures, and can be used in medical diagnostics, industrial non-destructive testing, and security inspection.
  • aspects of the embodiments disclosed herein may be implemented in an integrated circuit as a whole or in part, as one or more of one or more computers running on one or more computers.
  • a computer program eg, implemented as one or more programs running on one or more computer systems
  • implemented as one or more programs running on one or more processors eg, implemented as one or One or more programs running on a plurality of microprocessors, implemented as firmware, or substantially in any combination of the above, and those skilled in the art, in accordance with the present disclosure, will be provided with design circuitry and/or write software and / or firmware code capabilities.
  • signal bearing media include, but are not limited to, recordable media such as floppy disks, hard drives, compact disks (CDs), digital versatile disks (DVDs), digital tapes, computer memories, and the like; and transmission-type media such as digital and / or analog communication media (eg, fiber optic cable, waveguide, wired communication link, wireless communication link, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Pathology (AREA)
  • Algebra (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biochemistry (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Surgery (AREA)
  • Immunology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Radiology & Medical Imaging (AREA)
  • Optics & Photonics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

公开了一种训练神经网络的方法和设备、图像处理方法和设备以及存储介质。神经网络包括：投影域网络，处理输入的投影数据，得到估计投影数据；解析重建网络层，由估计投影数据得到重建图像；图像域网络，处理重建图像，得到估计图像；投影层，得到估计图像的投影结果；和统计模型层，确定输入的投影数据、估计投影数据和估计图像的投影结果基于统计模型的一致性。神经网络还可包括先验模型层。所述方法包括：利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果的数据模型的一致性代价函数调整图像域网络和投影域网络的卷积核参数。利用上述方案，训练得到的神经网络能够在投影数据存在缺陷时重建质量更高的图像。

Description

训练神经网络的方法和设备、图像处理方法和设备以及存储介质
本申请要求于2017年12月29日提交的、申请号为201711498783.0的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开的实施例涉及辐射成像，具体涉及一种训练神经网络的方法和设备、图像处理方法、图像处理设备以及存储介质。
背景技术
X射线CT(Computerized-Tomography)成像***在医疗、安检、工业无损检测等领域中都有着广泛的应用。射线源和探测器按照一定的轨道采集一系列的衰减信号数据，经过预处理、图像重建算法的复原可以得到被检查对象的线性衰减系数的三维空间分布。CT图像重建过程在于从探测器采集到的数据中恢复线性衰减系数分布。目前，在实际应用中主要使用滤波反投影(Filtered Back-Projection)、FDK(Feldkmap-Davis-Kress)类的解析重建算法和ART(Algebra Reconstruction Technique)、MAP(Maximum A Posterior)等迭代重建方法。
随着X光CT成像的需求越来越多样化，对降低辐射剂量的要求也越来越高。已经提出了利用卷积神经网络来重建CT图像的技术。但是在利用卷积神经网络来重建方法中，在训练神经网络的过程中需要进行有监督的训练。这样的方法需要获取大量真实图像作为标签，即卷积神经网络既需要稀疏采样的投影数据，又需要完备采样的投影数据(真值)。
发明内容
根据本公开实施例，提出了一种训练神经网络的方法和设备、图像处理方法、图像处理设备以及存储介质，通过训练得到的神经网络能够提高重建图像的质量。
在本公开的一个方面，提出了一种用于训练神经网络的方法，所述神经网络包括：投影域网络，用于处理输入的投影数据，得到估计投影数据；解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；图像域网络，用于对重建图像进行处理，得到估计图像；投影层，用于利用CT扫描***的***投 影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
其中，所述方法包括：利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
根据本公开实施例，所述方法还包括：利用投影层构建与投影一致的代价函数，利用统计模型层构建似然关系代价函数，以及与投影一致的代价函数和似然关系代价函数中的至少一个形成所述数据模型的一致性代价函数。
根据本公开实施例，所述卷积神经网络还包括至少一个先验模型层，所述先验模型层利用基于估计图像的先验模型代价函数调整图像域网络，并经过解析重建网络层对梯度进行反向传递，以调整投影域网络的卷积核参数。
根据本公开实施例，投影域网络、解析重建网络层和图像域网络的正向传递过程包括：
投影域网络的输入投影数据表示为g＝{g 1，g 2，...，g M}，投影域网络输出的估计投影数据表示为
Figure PCTCN2018119372-appb-000001
M′≥M，经过加权后得到
Figure PCTCN2018119372-appb-000002
经过斜坡滤波层后得到
Figure PCTCN2018119372-appb-000003
经过反投影得到解析重建网络层的输出
Figure PCTCN2018119372-appb-000004
令
Figure PCTCN2018119372-appb-000005
表示图像域网络的作用函数，则图像域网络输出的估计图像
Figure PCTCN2018119372-appb-000006
其中，上标T表示矩阵的转置，h为离散化的斜坡滤波算子，H R是M′×N维重建用***矩阵，N是重建图像的像素总数，W 1，W 2，……，W M′表示加权系数。
根据本公开实施例，将所述数据模型的一致性代价函数表示为
Figure PCTCN2018119372-appb-000007
来自数据模型一致性的误差传递关系为：
Figure PCTCN2018119372-appb-000008
其中，
Figure PCTCN2018119372-appb-000009
为似然关系代价函数，
Figure PCTCN2018119372-appb-000010
越小，则投影数据g与估计投影数据
Figure PCTCN2018119372-appb-000011
越吻合，
Figure PCTCN2018119372-appb-000012
为与投影一致的代价函数，β为拉格朗日乘子参数，H为***投影矩阵。
根据本公开实施例，所述先验模型代价函数
Figure PCTCN2018119372-appb-000013
包括局部条件的全变分、马尔科夫场先验、纹理先验、特征空间稀疏度之一或其组合，其中
Figure PCTCN2018119372-appb-000014
为所述估计图像。
根据本公开实施例，所述方法还包括利用先验误差ε Pr定义先验模型代价函数
Figure PCTCN2018119372-appb-000015
根据本公开实施例，根据各个先验模型的代价函数
Figure PCTCN2018119372-appb-000016
在误差反馈过程中的重要性λ来调整图像域网络。
根据本公开实施例，在解析重建网络层，按照如下的传递关系实现解析重建网络层的先验误差反向传递：
Figure PCTCN2018119372-appb-000017
其中，投影域网络的输入投影数据表示为g＝{g 1，g 2，...，g M}，投影域网络输出的估计投影数据表示为
Figure PCTCN2018119372-appb-000018
M′≥M，经过加权后得到
Figure PCTCN2018119372-appb-000019
经过斜坡滤波层后得到
Figure PCTCN2018119372-appb-000020
经过反投影得到解析重建网络层的输出
Figure PCTCN2018119372-appb-000021
其中上标T表示矩阵的转置，h为离散化的斜坡滤波算子，H R是M′×N维重建用***矩阵，N是重建图像的像素总数，W 1，W 2，……，W M′表示加权系数。
根据本公开实施例，其中，假定
Figure PCTCN2018119372-appb-000022
表示图像域网络的作用函数，即
Figure PCTCN2018119372-appb-000023
则按照如下的传递关系实现先验误差反向传递：
Figure PCTCN2018119372-appb-000024
根据本公开实施例，所述方法还包括：将
Figure PCTCN2018119372-appb-000025
和
Figure PCTCN2018119372-appb-000026
共同传递至投影域网络，以便对各层参数进行更新。
根据本公开实施例，所述方法还包括：由CT扫描***获取对象的衰减信号数据，并对衰减信号数据进行预处理得到输入的投影数据。
根据本公开实施例，所述方法还包括由CT扫描***按照如下扫描方式之一来获取对象的投影数据：探测器欠采样扫描、稀疏角度扫描、内重建扫描、有限角扫描、和直线轨迹扫描。
根据本公开实施例，投影域网络包括多个并行的卷积神经网络支路。
根据本公开实施例，图像域网络包括U型卷积神经网络。
根据本公开实施例，所述方法还包括：将仿真数据集合作为输入的投影数据训练所述卷积神经网络。
根据本公开的另一方面，提供了一种图像处理方法，包括：
由CT扫描***获取对象的投影数据；以及
利用卷积神经网络对所述投影数据进行处理，以获取所述对象的估计图像；
其中，所述卷积神经网络包括：投影域网络，用于处理输入的投影数据，得到估计投影数据；解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；图像域网络，用于对重建图像进行处理，得到估计图像；投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
其中，所述图像处理方法包括训练卷积神经网络，包括：利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
根据本公开的另一方面，提供了一种用于训练神经网络的设备，包括：
存储器，用于存储指令和数据，
处理器，配置为执行所述指令，以便：
构建所述神经网络，使其包括：投影域网络，用于处理输入的投影数据，得到估计投影数据；解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；图像域网络，用于对重建图像进行处理，得到估计图像；投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
其中，所述处理器还配置为训练所述神经网络，包括利用基于输入的投影 数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
根据本公开的另一方面，提供了一种图像处理设备，包括：
存储器，用于存储指令和数据，
处理器，配置为执行所述指令，以便：
接收CT扫描***获取的对象的投影数据；以及
利用卷积神经网络对所述投影数据进行处理，以获取所述对象的估计图像；
其中，所述处理器还配置为构建所述卷积神经网络，使其包括：投影域网络，用于处理输入的投影数据，得到估计投影数据；解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；图像域网络，用于对重建图像进行处理，得到估计图像；投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
其中，所述处理器还配置为训练所述卷积神经网络，包括利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
根据本公开的另一方面，提供了一种计算机可读存储介质，其中存储有计算机指令，当所述指令被处理器执行时实现根据本公开的方法。
利用本公开上述实施例的方案，能够在无监督情况下训练神经网络，从而使得重建得到质量更高的图像。
附图说明
为了更好地理解本公开实施例，将根据以下附图对本公开实施例进行详细描述：
图1示出了本公开一个实施例的CT设备的结构示意图；
图2是如图1所示的CT设备中控制与数据处理装置的结构示意图；
图3示出了根据本公开实施例的中投影数据的正弦图的例子；
图4示出了在不同扫描方式下的正弦图中包含的数据的示意图；
图5是根据本公开一实施例的实现稀疏角度采样扫描方式的扫描装置的示 意图；
图6A示出了根据本公开一个实施例的神经网络的一种结构示意图；
图6B示出了根据本公开一个实施例的神经网络的另一种结构示意图；
图7是描述如图6A和图6B所示的神经网络架构中各个模块处理的图像的示意图；
图8示出了根据本公开实施例的设备中使用的投影域网络的结构示意图；
图9示出了根据本公开实施例的设备中使用的解析重建网络层的结构示意图；
图10示出了根据本公开又一实施例的设备中图像域网络的结构示例示意图；
图11示出了本公开实施例的卷积神经网络中使用的平滑条件卷积核的结构示意图；
图12A、图12B和图12C示出了根据本公开实施例的设备中使用的滤波器核的尺寸示意图；
图13A是描述根据本公开的实施例的图像处理方法的示意性流程图；
图13B是描述根据本公开的实施例的训练神经网络方法的示意性流程图；
图14是根据本公开另一实施例的实现有限角度CT扫描的扫描装置的示意图；
图15是根据本公开再一实施例的实现内重建扫描方式的扫描装置的示意图；
图16是根据本公开再一实施例的实现探测器欠采样扫描方式的扫描装置的示意图；以及
图17示出了根据本公开再一实施例的实现直线轨迹CT扫描的扫描装置的示意图。
具体实施方式
下面将详细描述本公开实的具体实施例，应当注意，这里描述的实施例只用于举例说明，并不用于限制本公开实施例。在以下描述中，为了提供对本公开实施例的透彻理解，阐述了大量特定细节。然而，对于本领域普通技术人员显而易见的是：不必采用这些特定细节来实行本公开实施例。在其他实例中，为了避免混淆本公开实施例，未具体描述公知的结构、材料或方法。
在整个说明书中，对“一个实施例”、“实施例”、“一个示例”或“示例” 的提及意味着：结合该实施例或示例描述的特定特征、结构或特性被包含在本公开至少一个实施例中。因此，在整个说明书的各个地方出现的短语“在一个实施例中”、“在实施例中”、“一个示例”或“示例”不一定都指同一实施例或示例。此外，可以以任何适当的组合和/或子组合将特定的特征、结构或特性组合在一个或多个实施例或示例中。此外，本领域普通技术人员应当理解，这里使用的术语“和/或”包括一个或多个相关列出的项目的任何和所有组合。
本公开的实施例提出了一种用于训练神经网络的方法及其设备和一种图像处理方法及其设备。其中利用神经网络来处理输入的投影数据以获得对象的估计图像。该神经网络可以包括：投影域网络、解析重建网络层、图像域网络、投影层和统计模型层。投影域网络处理输入的投影数据，得到估计投影数据。解析重建网络层对估计投影数据进行解析重建，得到重建图像。图像域网络对重建图像进行处理，得到估计图像。投影层利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果。统计模型层确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性。利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。利用本公开上述实施例的方案，训练得到的神经网络能够在投影数据存在缺陷时重建得到质量更高的图像。
图1示出了本公开一个实施例的CT设备的结构示意图。如图1所示，根据本实施例的CT设备包括X射线源10、机械运动装置50、探测器和数据采集***20，以及控制和数据处理装置60，对被检查对象40进行CT扫描和数据处理，例如神经网络的训练和利用训练后的网络重建图像。
X射线源10例如可以为X光机，可以根据成像的分辨率选择合适的X光机焦点尺寸。在其他实施例中也可以不使用X光机，而是使用直线加速器等产生X射线束。
机械运动装置50包括载物台和机架以及控制***等。载物台可平移以调整旋转中心的位置，机架可平移使X射线源(X光机)10、探测器和旋转中心三者对准。本实施例中按照旋转载物台、固定机架的圆周扫描轨迹或者螺旋轨迹进行描述。由于载物台与机架的运动属于相对运动，也可采用载物台静止、机架旋转的方式实现本实施例的方法。
探测器及数据采集***20包括X射线探测器和数据采集电路等。X射线探测器可以使用固体探测器，也可以使用气体探测器或者其他探测器，本公开的实施例不限于此。数据采集电路包括读出电路、采集触发电路及数据传输电路等。
控制和数据处理装置60例如包括安装有控制程序和数据处理程序的计算机设备，负责完成CT***运行过程的控制，包括机械转动、电气控制、安全联锁控制等，训练神经网络，并且利用训练的神经网络从投影数据重建CT图像等。
图2示出了如图1所示的控制和数据处理设备200的结构示意图。如图2所示，探测器及数据采集***20采集得到的数据通过接口单元270和总线280存储在存储设备210中。只读存储器(ROM)220中存储有计算机数据处理器的配置信息以及程序。随机存取存储器(RAM)230用于在处理器250工作过程中暂存各种数据。另外，存储设备210中还存储有用于进行数据处理的计算机程序，例如训练神经网络的程序和重建CT图像的程序等等。内部总线280连接上述的存储设备210、只读存储器220、随机存取存储器230、输入装置240、处理器250、显示设备260和接口单元270。
在用户通过诸如键盘和鼠标之类的输入装置240输入的操作命令后，计算机程序的指令代码命令处理器250执行训练神经网络的算法和/或重建CT图像的算法。在得到重建结果之后，将其显示在诸如LCD显示器之类的显示设备260上，或者直接以诸如打印之类硬拷贝的形式输出处理结果。
根据本公开的实施例，利用上述设备对被检查对象进行CT扫描，得到原始衰减信号。这样的衰减信号数据也可以显示为二维图像的形式，图3示出了根据本公开的实施例得到的衰减信号数据的例子。如图3所示的原始衰减信号的横轴方向表示探测器像素序列(例如从1到256)，而纵轴表示角度(例如从1度到360度)。原始衰减信号进行预处理后成为投影数据。例如，可以由CT扫描***对投影数据进行负对数变换等预处理得到投影数据。然后，控制设备中的处理器250执行重建程序，利用训练的神经网络对投影数据进行处理，得到估计投影数据，进而通过解析重建网络层对估计投影数据进行重建操作，得到重建图像。进一步对重建图像进行处理，得到最终图像。例如，利用训练的(例如U型)卷积神经网络处理重建的图像，得到不同尺度的特征图，并且对不同尺度的特征图进行合并，得到结果图像。
在本公开的实施例中，在投影域利用训练的卷积神经网络对投影数据进行处 理，然后进行重建层重建操作来重建图像，最后图像域网络对重建图像进行处理，得到估计图像。卷积神经网络可以包括卷积层、池化、和全连接层。卷积层识别输入数据集合的特性表征，每个卷积层带一个非线性激活函数运算。池化层精炼对特征的表示，典型的操作包括平均池化和最大化池化。一层或多层的全连接层实现高阶的信号非线性综合运算，全连接层也带非线性激活函数。常用的非线性激活函数有Sigmoid、Tanh、ReLU等等。
虽然上面的描述主要是针对360度圆周扫描得到完备投影数据的情况来描述的，但是本领域的技术人员能够理解，上述方案可以用于对非完备投影数据的情况，例如应用于探测器欠采样、稀疏角度采样、有限角、内重建、或者直线轨迹扫描等方式。
图4示出了在不同扫描方式下的正弦图中包含的数据的示意图。如图4所示，角度稀疏采样CT扫描、有限角度CT扫描、探测器欠采样CT扫描和内重建CT扫描得到的投影数据都是不完备的。尽管投影数据不完备，但是利用上述的方案，也能够从这些不完备的投影数据中重建得到质量较高的图像。
图5是根据本公开再一实施例的实现稀疏角度采样扫描方式的扫描装置的示意图。如图5所示，从射线源10发出的X射线透射视野45中的被检查对象40后，被探测器30接收，转换成电信号并进而转换成表示衰减值的数字信号，预处理后作为投影数据，以便由计算机进行重建。利用上述的方案，即使对被检查对象40进行若干旋转位置下的CT扫描(例如6个位置)，利用本公开的方法训练的神经网络也能够重建得到质量较高的图像。这样，即使对被检查对象进行稀疏角度CT扫描，也能够从非完备的投影数据中重建得到质量较高的图像。
图6A示出了根据本公开的实施例的卷积神经网络的一种结构示意图。如图6A所示，无监督X射线CT图像重建神经网络的输入为CT扫描后的衰减信号经预处理得到的投影数据。神经网络主要可以包括投影域网络610、解析重建网络层620、图像域网络630。进一步，根据本公开实施例的神经网络还可以包括统计模型层640和投影层650。图6B示出了根据本公开的实施例的卷积神经网络的另一种结构示意图。与图6A所示的神经网络结构不同的是，图6B中的神经网络还可以包括至少一个先验模型层，图6B示出了三个先验模型层660、670和680作为示例。
图7是描述如图6A和6B所示的神经网络架构中各个模块处理的图像的示 意图。如图7所示，输入的投影数据可以表示为g，经投影域网络处理后的数据表示为
Figure PCTCN2018119372-appb-000027
可以将其称作“估计投影数据”。经解析重建网络层处理后的数据表示为
Figure PCTCN2018119372-appb-000028
经图像域网络处理后的数据表示为
Figure PCTCN2018119372-appb-000029
可以将其称作“估计图像”。
投影域网络610用于投影数据的恢复和缺失数据估计，以得到完备投影数据。解析重建网络层620包括多个层。这些层虽然是基于本领域技术人员了解的解析重建算法构建的，但是却经过了特殊的矩阵化，下文将详细介绍这些层的特殊构造。图像域网络630用于减少重建图像的伪影和误差，进一步提高重建图像的质量。投影层650利用CT扫描***的投影矩阵对估计图像
Figure PCTCN2018119372-appb-000030
进行投影运算，得到估计图像的投影，从而可构建与投影一致的代价函数，类似于ART的代价函数。统计模型层640构成网络的一个代价支路，描述投影采集时的统计涨落，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性，其全部或一部分可以是例如采集的X光CT数据与真实数据在统计模型下满足的似然关系代价函数。
如图6B所示，根据本公开实施例的神经网络还可以包括先验模型层660、670、680，构成网络的末端代价函数。本领域技术人员可以理解，图6B中的三个先验模型层仅为示例，神经网络也可以包括1个或者其他数目的并联的先验模型层(也可称为约束条件)。先验模型层660、670、680可以分别为局部条件的全变分、马尔科夫场先验、纹理先验、特征空间稀疏度及其他模型中的至少之一。参数λ调节先验模型或约束条件的重要性(或强度)，图6中示出了λ 1、λ 2、λ 3，可以分别作为先验模型层660、670、680反向传递的权重。
整体网络的正向运算流程如图6A和6B的实线箭头所示。用g＝{g 1，g 2，...，g M}表示投影域网络输入的投影数据，投影域网络610输出的估计投影数据为
Figure PCTCN2018119372-appb-000031
通常M′≥M。经过图9所示的加权层后得到
Figure PCTCN2018119372-appb-000032
W 1，W 2，……，W M′表示加权系数。经过斜坡滤波层后得到滤波后正弦图
Figure PCTCN2018119372-appb-000033
h为离散化的斜坡滤波算子，可以是例如Ram-Lak滤波器或者Shepp-Logan滤波器。在一个示例中，h可以为离散化的斜坡卷积核。经过反投影层得到解析重建网络层的输出
Figure PCTCN2018119372-appb-000034
其中上标T表示矩阵的转置。H R是M′×N维重建用***矩阵，与正向投影矩阵H(***投影矩阵)类似，由CT扫描***的架构和扫描方式确定，
Figure PCTCN2018119372-appb-000035
完成反投影过程，在扇束或锥束扫描下完成加权反投影过程。经过重建网络后
Figure PCTCN2018119372-appb-000036
继续前馈 通过图像域网络，得到对扫描物体的估计图像
Figure PCTCN2018119372-appb-000037
各个先验模型层的输出为代价函数
Figure PCTCN2018119372-appb-000038
可以是多种如局部条件的全变分、马尔科夫场先验、纹理先验、特征空间稀疏度等之一或其组合。
整体网络的误差反向传递如图6所示的虚线箭头所示。反向传递分两条主线：数据模型的一致性代价和先验模型的吻合度代价。根据先验模型的吻合度代价函数
Figure PCTCN2018119372-appb-000039
定义先验误差
Figure PCTCN2018119372-appb-000040
参数λ定义了各个
Figure PCTCN2018119372-appb-000041
的在误差反馈过程中的重要度，也就是图像空间先验模型的重要性。先验模型的吻合度代价的误差传递至图像域网络，逐层向前按链式求导法则计算各层输入与各层参数对代价函数的导数。然后通过解析重建网络层，按照如下传递方式完成解析重建网络层的先验误差反向传递：
Figure PCTCN2018119372-appb-000042
假设
Figure PCTCN2018119372-appb-000043
代表图像域网络的作用函数，即
Figure PCTCN2018119372-appb-000044
则反向传递可进一步表示为
Figure PCTCN2018119372-appb-000045
数据模型的一致性代价包括似然关系代价函数和与投影一致的代价函数中的至少一个。似然关系代价函数是根据信号的统计模型定义的，可以是高斯噪声分布、泊松概率分布、高斯和泊松混合概率分布等计算似然函数(在误差最小化的框架下取其负)
Figure PCTCN2018119372-appb-000046
这里定义的
Figure PCTCN2018119372-appb-000047
越小，则输入投影数据g与估计投影数据
Figure PCTCN2018119372-appb-000048
越吻合。与投影一致的代价函数反映了前次估计结果(估计图像)通过投影层投影后的投影结果与估计投影数据
Figure PCTCN2018119372-appb-000049
之间的差异。数据模型的一致性代价函数可以表示为：
Figure PCTCN2018119372-appb-000050
由此，来自数据模型一致性的误差传递关系为：
Figure PCTCN2018119372-appb-000051
Figure PCTCN2018119372-appb-000052
其中β为拉格朗日乘子参数，
Figure PCTCN2018119372-appb-000053
为与投影一致的代价函数，H为***投影矩阵。
Figure PCTCN2018119372-appb-000054
和
Figure PCTCN2018119372-appb-000055
共同传递至投影域网络，对各层参数进行更新。
Figure PCTCN2018119372-appb-000056
传 递到图像域网络，并经过解析重建网络层传递到投影域网络。
根据本公开的实施例，训练集数据可包含仿真工具产生覆盖应用情景的仿真模型，并按照实际CT***参数和扫描方式生成投影数据集。在实际***上扫描物体，获得CT扫描数据，其中一部分也作为训练集数据输入到此网络进行进一步训练，并利用另一部分采集数据对网络训练效果进行测试。
图8示出了根据本公开实施例的设备对稀疏角度应用的投影域卷积神经网络的结构示意图。使用例如CT扫描***对采集的原始衰减信号进行负对数变换等预处理(预处理还可以包括加空气值校正、一致性校正)后，得到投影数据g，投影域网络610使用该投影数据作为输入。投影域网络以卷积神经网络方式补全缺失数据。如图8所示的并联网络的输入是在稀疏角度情况下采集的数据。估计的缺失角度数据分为多组，每一组数据与采集角度规模相同，为与采集角度相差一个恒定角度的数据。对每个组，使用已有的投影数据作为输入数据，通过包含激活函数的卷积层(Conv)的级联完成多级特征的提取，通过全连接层(可以用1x1的卷积层实现)获得缺失的投影数据。
对于投影域的网络610，所有尺度的2维卷积核有两个维度，此处定义第一维度为探测器方向，第二维度为扫描角度方向。两个维度的卷积核长度不必相同，一般设置卷积核在探测器方向的尺度大于扫描角度方向的尺度，例如取3*1，5*3，7*3，9*3的卷积核。也可以根据投影图的两个维度比例关系取卷积核大小。每个尺度可以设置多个卷积核。卷积层带一个激活函数。
例如，投影域网络的主要作用为提升投影数据在角度方向的分辨率。投影域网络可以包括例如图8所示的5支路并行网络。每一支路包括例如七层卷积神经网络，中间部分由于探测器和投影数的比例关系采用长方形卷积核(图12A、12B、12C)，在最后一层卷积核尺寸为1*1实现特征图方向全连接，每一层巻积层输出均使用Relu函数作为非线性激活函数。卷积过程中，角度方向使用其周期性进行边界数据填补使特征图大小始终维持与输入相同。
如图8所示，在七层网络并行分别处理后，经插值处理，将4组缺失投影估计与1组已采集并经过网络降噪的投影合并形成360密集角度的估计投影图，并传向解析重建网络层。虽然图8所示为包括多个并行支路的卷积网络，但是本领域的技术人员可想到用其他形式的网络也能实现本公开的技术方案。虽然在上述 实施例中是用投影域网络来恢复缺失的数据，也就是应用于得到非完备数据的CT扫描情况下，但是本领域的技术人员可以理解，上述投影域网络可以对完备的投影数据进行处理，提高投影数据的质量。
图9示出了根据本公开实施例的设备中使用的解析重建网络层的结构示意图。解析重建网络层620可以包括加权层(可选)、斜坡滤波层(可选)和反投影层。加权层实现对每个数据的余弦加权。斜坡滤波层实现传统解析重建方法中的斜坡滤波运算。反投影层实现从投影域到图像域的反向投影(对于扇束CT和锥形束CT反投影层为距离加权反投影)。通常解析重建网络层根据CT成像***的架构设计和实施，网络训练过程中不作参数修改。
例如，解析重建网络层620将解析重建算法显性加入网络结构，从而简化网络需要学习的物理规律。解析重建网络层620包括三层。
第一层为加权层。根据本应用中CT扫描几何参数用216*1的余弦向量对探测器方向数据进行余弦归一，为实现与估计投影之间点乘，对余弦归一向量在角度方向进行拓展得到216*360的加权矩阵W，这里W的每一列值均相等。经过W层后，投影图表示为
Figure PCTCN2018119372-appb-000057
第二层是斜坡滤波层。斜坡滤波对投影图探测器方向进行离散滤波操作。在本应用中即分别对每一个角度下216探测器响应向量滤波，可用矩阵乘法描述，生成216*216的滤波矩阵F与加权后投影数据矩阵乘法完成滤波过程
Figure PCTCN2018119372-appb-000058
第三层为反投影层。反投影层将滤波后投影按几何关系重建为图像，按本应用几何参数采用像素驱动方法生成距离加权反投影矩阵
Figure PCTCN2018119372-appb-000059
对滤波后图像进行反投影，得到图像域重建图像。
图10示出了根据本公开又一实施例的设备中使用的图像域网络的示例示意图。图像域网络可以完成图像域的伪影抑制和噪声抑制。例如图6所示的图像域网络630可以是图10所示的U型网络。
例如，利用如图10所示的U型卷积神经网络处理重建的图像，可以得到不同尺度的特征图，并且对不同尺度的特征图进行合并，可以得到结果图像。更具体地，利用上采样操作逐级融合多个尺度下的特征图，并最终得到被检查物体的结果图像。例如，在估计投影解析重建的基础之上，图像域网络进一步应用先验知识进行去伪影。在本实例中，图像域网络采用U型网络设计(如图10)。其中，例如200*200的重建图像经过4次池化，逐步缩小特征图尺寸从而增大接受域学 习图像全局特征。随后逐步扩展，并与同尺寸没有降采样的特征图合并，用于防止因降采样导致信息损失，最终再次恢复200*200尺寸经过网络处理后最终重建图像。在图10所示的图像域网络中，在不同层级卷积核大小均例如是3*3。图像在降采样过程中，随着特征图尺寸减小，特征图数量逐渐增多，在升采样过程中，特征图数量再逐渐减少。
虽然图10将图像域网络示例为一种U型网络的具体结构示例，但是本领域的技术人员可想到用其他结构的U型网络也能实现本公开的技术方案。此外，本领域的技术人员也可以想到将其他网络用作图像域网络，例如自编码网络(Auto-Encoder)、全卷积神经网络(Fully convolution neural network)等，也能够实现本公开的技术方案。
根据本公开的实施例，投影域网络610和图像域网络630的所有卷积核为待定的网络参数，可随机初始化，也可使用其它途径的预训练结果，在本网络训练过程中更新。此外，该网络分别对输入数据在投影域和图像域进行处理，使待优化的目标函数(在深度学习领域常称为损失函数)达到最优结果。由于在投影域和图像域相邻像素所具有的几何关系不同，因此投影域卷积层和图像域卷积层可以起到互补作用。
根据本公开的实施例的一个具体示例，在图像域基于图像连续性的先验知识，采用邻域相似性约束，通过一固定卷积核w 0实现此运算。图11表示了一种固定的3*3卷积核。可以使用图像平滑的代价函数作为待优化的目标函数，可表达为
Figure PCTCN2018119372-appb-000060
对数据模型一致性代价函数Ψ的设计，在本示例中，采集的投影数据g是一组符合独立高斯分布的采样，其均值为被扫描对象在对应射线路径上线性衰减系数的积分。约束中的第一项是高斯模型似然代价函数，完成由采样g到分布真值的极大似然估计约束，仅适用于被采集稀疏角度投影部分降噪网络。而第二项代表投影与图像的一致性约束，适用于投影域网络的各部分。
Figure PCTCN2018119372-appb-000061
其中，g为采集到的稀疏角度投影数据，
Figure PCTCN2018119372-appb-000062
为对应于已采集稀数角度投影 数据的射线路径上的估计值，
Figure PCTCN2018119372-appb-000063
为网络输出的估计图像，∑为对角线元素为投影数据方差的对角阵，H为***投影矩阵，β为拉格朗日乘子参数。
先验模型的吻合度代价函数反向传递可以表示为
Figure PCTCN2018119372-appb-000064
对网络输出图像求导，并且梯度经解析重建层反向传递后作用于投影域网络卷积核。
数据保真约束Ψ中同时有
Figure PCTCN2018119372-appb-000065
和
Figure PCTCN2018119372-appb-000066
梯度反向传递同时从
Figure PCTCN2018119372-appb-000067
更新投影域网络以及
Figure PCTCN2018119372-appb-000068
传给图像域网络。
Figure PCTCN2018119372-appb-000069
Figure PCTCN2018119372-appb-000070
在对神经网络训练的过程中可能用到仿真数据集和实际数据集这两个数据集。
仿真数据集是来自网络等来源的高质量人体CT断层图片。根据机械几何关系，举一个示例，角度可以从3度开始，按照以5度为步长递增，到358度结束，来生成仿真投影数据，光子数为10 5。得到一组216*72规模的仿真投影数据。将其中的4/5用于网络的无监督训练，剩下1/5作为验证集，用于控制网络的范化效果。
实际数据集可以是在扇束螺旋CT平台上按5度角度间隔和216探测器阵列对模体扫描产生投影，控制管电流时间达到正常剂量。用另一组不同的模体按同样的方式进行扫描得到投影作为测试集，在训练完成后用于对网络效果进行测试。
根据本发明的一个实施例，采用直接训练方式。在直接训练过程中，随机初始化投影域网络及图像域网络卷积核权值，由实际采集数据集进行训练，训练完成后，用另一组实际采集数据作为测试集以验证网络训练效果。
根据本发明的另一个实施例，可以采用预训练辅助直接训练的方式，数据仿 真阶段是可以产生高质量的投影数据的，可以将使用仿真数据集训练神经网络的过程称为“预训练”。先利用仿真数据生成360个角度完整投影数据，有监督的单独训练并行投影估计网络。待预训练完成后(仿真数据集训练到收敛后)，利用此投影估计网络作为整体网络投影域网络的初始值，再按照直接训练的除了赋予初始值之外的方式整体训练网络。
对于实际CT扫描过程，把采集的数据输入上述训练过程获得已训练网络(此时网络参数固定)，获得重建图像。
图13A是描述根据本公开实施例的图像处理方法的示意流程图。如图13A所示，在步骤S131，由CT扫描***获取对象的投影数据。在步骤S132，利用卷积神经网络对投影数据进行处理，以获取所述对象的估计图像。
根据本公开实施例的神经网络可以包括投影域网络、解析重建网络层和图像域网络。投影域网络用于处理输入的投影数据，得到估计投影数据。解析重建网络层对估计投影数据进行解析重建，得到重建图像。图像域网络对重建图像进行处理，得到估计图像。根据本公开实施例的神经网络可以包括投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；以及统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性。
根据本公开实施例的图像处理方法还可以包括对神经网络进行训练。图13B是描述根据本公开的实施例的训练神经网络的方法的示意性流程图。
如图13B所示，在步骤S1321，在第一训练过程，将仿真数据集合作为输入的投影数据训练神经网络。第一训练过程是对神经网络进行预训练，可加快训练过程。
在步骤S1322，在第二训练过程，将采集的真实CT数据作为输入的投影数据进一步训练神经网络。在步骤S1322中，可以利用基于估计图像的先验模型代价函数调整图像域网络，并经过解析重建网络层对梯度进行反向传递，以调整投影域网络的卷积核参数。在步骤S1322，利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。更为具体的实现方案参见如上结合图6A和图6B所示的网络结构所描述的实施例，这里不再赘述。
虽然上面给出了角度稀疏采样扫描等方式，但是本领域技术人员可以想到， 本公开的训练方法同样可以用在有限角度CT扫描、内重建扫描、探测器欠采样扫描以及直线轨迹CT扫描***中。
图14是根据本公开另一实施例的实现有限角度CT扫描的扫描装置的示意图。如图14所示，从射线源10发出的X射线透射视野45中的被检查对象40后，被探测器30接收，转换成电信号得到衰减数据，经过预处理操作后得到投影数据，由计算机进行重建。利用上述的方案，即使对被检查对象40进行有限角度的CT扫描(例如130度)，训练得到的神经网络也能够重建得到质量较高的图像。
图15是根据本公开再一实施例的实现内重建扫描方式的扫描装置的示意图。如图15所示，从射线源10发出的X射线透射视野45中的被检查对象40的一部分后，被探测器30接收，转换成电信号并进而转换成表示衰减值的数字信号，作为投影数据，由计算机进行重建。利用上述的方案，即使对被检查对象40进行内重建CT扫描，训练得到的神经网络也能够重建得到质量较高的图像。
图16是根据本公开再一实施例的实现探测器欠采样扫描方式的扫描装置的示意图。如图16所示，从射线源10发出的X射线透射视野45中的被检查对象40后被探测器30接收，转换成电信号并进而转换成表示衰减值的数字信号，作为投影数据，由计算机进行重建。在该例子中，探测器30被设置成欠采样的情形，例如将各个探测器单元间隔预定的距离来实现欠采样。这样，利用上述的方案，即使对被检查对象40进行探测器欠采样CT扫描，训练得到的神经网络也能够重建得到质量较高的图像。
图17示出了根据本公开再一实施例的实现直线轨迹CT扫描的扫描装置的示意图。如图17所示，从射线源10发出的X射线透射视野中的被检查物体40后被探测器30接收，转换成电信号并进而转换成表示衰减值的数字信号，作为投影数据，由计算机进行重建。在该例子中，被检查物体40在与探测器平行的传送带上沿着直线轨迹运动。探测器在水平方向与射线源张角尽可能大，在竖直方向覆盖物体。例如，探测器阵列放置在源的对边，要求射线水平张角θ在90度以上，得到直线轨迹CT扫描投影数据。利用上述的方案，即使对被检查物体40进行直线轨迹CT扫描，训练得到的神经网络也能够重建得到质量较高的图像。
本公开实施例的训练方法可以先采用仿真数据来进行预训练，再利用真实数据进行无监督训练。另外，也可以直接利用大量的真实数据无监督训练。这样深 度挖掘数据信息，形成卷积神经网络和***针对性参数，获得高效的CT图像重建方法。
本公开的方法可以灵活适用于不同的CT扫描模式和***架构，可用于医学诊断、工业无损检测和安检领域。
以上的详细描述通过使用示意图、流程图和/或示例，已经阐述了训练神经网络的方法和设备的众多实施例。在这种示意图、流程图和/或示例包含一个或多个功能和/或操作的情况下，本领域技术人员应理解，这种示意图、流程图或示例中的每一功能和/或操作可以通过各种结构、硬件、软件、固件或实质上它们的任意组合来单独和/或共同实现。在一个实施例中，本公开实施例所述主题的若干部分可以通过专用集成电路(ASIC)、现场可编程门阵列(FPGA)、数字信号处理器(DSP)、或其他集成格式来实现。然而，本领域技术人员应认识到，这里所公开的实施例的一些方面在整体上或部分地可以等同地实现在集成电路中，实现为在一台或多台计算机上运行的一个或多个计算机程序(例如，实现为在一台或多台计算机***上运行的一个或多个程序)，实现为在一个或多个处理器上运行的一个或多个程序(例如，实现为在一个或多个微处理器上运行的一个或多个程序)，实现为固件，或者实质上实现为上述方式的任意组合，并且本领域技术人员根据本公开，将具备设计电路和/或写入软件和/或固件代码的能力。此外，本领域技术人员将认识到，本公开所述主题的机制能够作为多种形式的程序产品进行分发，并且无论实际用来执行分发的信号承载介质的具体类型如何，本公开所述主题的示例性实施例均适用。信号承载介质的示例包括但不限于：可记录型介质，如软盘、硬盘驱动器、紧致盘(CD)、数字通用盘(DVD)、数字磁带、计算机存储器等；以及传输型介质，如数字和/或模拟通信介质(例如，光纤光缆、波导、有线通信链路、无线通信链路等)。
虽然已参照几个典型实施例描述了本公开实施例，但应当理解，所用的术语是说明和示例性、而非限制性的术语。由于本公开实施例能够以多种形式具体实施而不脱离公开实施例的精神或实质，所以应当理解，上述实施例不限于任何前述的细节，而应在随附权利要求所限定的精神和范围内广泛地解释，因此落入权利要求或其等效范围内的全部变化和改型都应为随附权利要求所涵盖。

Claims (20)

  1. 一种用于训练神经网络的方法，所述神经网络包括：
    投影域网络，用于处理输入的投影数据，得到估计投影数据；
    解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；
    图像域网络，用于对重建图像进行处理，得到估计图像；
    投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和
    统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
    其中，所述方法包括：
    利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
  2. 如权利要求1所述的方法，还包括：利用投影层构建与投影一致的代价函数，利用统计模型层构建似然关系代价函数，以及利用与投影一致的代价函数和似然关系代价函数中的至少一个形成所述数据模型的一致性代价函数。
  3. 如权利要求1所述的方法，其中，所述卷积神经网络还包括至少一个先验模型层，所述先验模型层用于基于估计图像的先验模型代价函数调整图像域网络，并经过解析重建网络层对梯度进行反向传递，以调整投影域网络的卷积核参数。
  4. 如权利要求1或3所述的方法，其中，投影域网络、解析重建网络层和图像域网络的正向传递过程包括：
    投影域网络的输入投影数据表示为g＝{g 1，g 2，...，g M}，投影域网络输出的估计投影数据表示为
    Figure PCTCN2018119372-appb-100001
    M′≥M，经过加权后得到
    Figure PCTCN2018119372-appb-100002
    经过斜坡滤波层后得到
    Figure PCTCN2018119372-appb-100003
    经过反投影得到解析重建网络层的输出
    Figure PCTCN2018119372-appb-100004
    令
    Figure PCTCN2018119372-appb-100005
    表示图像域网络的作用函数，则图像域网络输出的估计图像
    Figure PCTCN2018119372-appb-100006
    其中，上标T表示矩阵的转置，h为离散化的斜坡滤波算子，H R是M′×N维重建 用***矩阵，N是重建图像的像素总数，W 1，W 2，……，W M′表示加权系数。
  5. 如权利要求4述的训练方法，其中，将所述数据模型的一致性代价函数表示为
    Figure PCTCN2018119372-appb-100007
    来自数据模型一致性的误差传递关系为：
    Figure PCTCN2018119372-appb-100008
    其中，
    Figure PCTCN2018119372-appb-100009
    为似然关系代价函数，
    Figure PCTCN2018119372-appb-100010
    越小，则投影数据g与估计投影数据
    Figure PCTCN2018119372-appb-100011
    越吻合，
    Figure PCTCN2018119372-appb-100012
    为与投影一致的代价函数，β为拉格朗日乘子参数，H为***投影矩阵。
  6. 如权利要求3或4所述的方法，其中，所述先验模型代价函数
    Figure PCTCN2018119372-appb-100013
    包括局部条件的全变分、马尔科夫场先验、纹理先验、特征空间稀疏度之一或者组合，其中
    Figure PCTCN2018119372-appb-100014
    为所述估计图像。
  7. 如权利要求6所述的方法，还包括利用先验误差ε Pr定义先验模型代价函数
    Figure PCTCN2018119372-appb-100015
    Figure PCTCN2018119372-appb-100016
  8. 如权利要求6所述的方法，其中，根据各个先验模型代价函数
    Figure PCTCN2018119372-appb-100017
    在误差反馈过程中的重要性λ来调整图像域网络。
  9. 如权利要求7所述的方法，其中，在解析重建网络层，按照如下的传递关系实现解析重建网络层的先验误差反向传递：
    Figure PCTCN2018119372-appb-100018
    其中，投影域网络的输入投影数据表示为g＝{g 1，g 2，...，g M}，投影域网络输出的估计投影数据表示为
    Figure PCTCN2018119372-appb-100019
    M′≥M，经过加权后得到
    Figure PCTCN2018119372-appb-100020
    经过斜坡滤波层后得到
    Figure PCTCN2018119372-appb-100021
    经过反投影得到解析重建网络层的输出
    Figure PCTCN2018119372-appb-100022
    其中上标T表示矩阵的转置，h为离散化的斜坡滤波算子，H R是M′×N维重建用***矩阵，N是重建图像的像素总数，W 1，W 2，……，W M′表示加权系数。
  10. 如权利要求9所述的方法，其中，用
    Figure PCTCN2018119372-appb-100023
    表示图像域网络的作用函数，即
    Figure PCTCN2018119372-appb-100024
    则按照如下的传递关系实现先验误差反向传递：
    Figure PCTCN2018119372-appb-100025
  11. 如权利要求10述的方法，所述方法还包括：将
    Figure PCTCN2018119372-appb-100026
    和
    Figure PCTCN2018119372-appb-100027
    共同传递至投影域网络，以便对各层参数进行更新。
  12. 如权利要求1所述的方法，还包括：由CT扫描***获取衰减信号数据，并对衰减信号数据进行预处理得到输入的投影数据。
  13. 如权利要求1所述的方法，还包括由CT扫描***按照如下扫描方式之一来获取对象的投影数据：探测器欠采样扫描、稀疏角度扫描、内重建扫描、有限角扫描、和直线轨迹扫描。
  14. 如权利要求1所述的方法，其中，投影域网络包括多个并行的卷积神经网络支路。
  15. 如权利要求1所述的方法，其中，图像域网络包括U型卷积神经网络。
  16. 如权利要求1所述的方法，还包括：将仿真数据集合作为输入的投影数据，以对所述卷积神经网络进行预训练。
  17. 一种图像处理方法，包括：
    由CT扫描***获取对象的投影数据；以及
    利用卷积神经网络对所述投影数据进行处理，以获取所述对象的估计图像；
    其中，所述卷积神经网络包括：
    投影域网络，用于处理输入的投影数据，得到估计投影数据；
    解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；
    图像域网络，用于对重建图像进行处理，得到估计图像；
    投影层，用于利用所述CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和
    统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
    其中，所述图像处理方法包括训练卷积神经网络，包括：
    利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者 的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
  18. 一种用于训练神经网络的设备，包括：
    存储器，用于存储指令和数据，
    处理器，配置为执行所述指令，以便：
    构建所述神经网络，使其包括：
    投影域网络，用于处理输入的投影数据，得到估计投影数据；
    解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；
    图像域网络，用于对重建图像进行处理，得到估计图像；
    投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和
    统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果这三者基于统计模型的一致性；
    其中，所述处理器还配置为训练所述卷积神经网络，包括利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
  19. 一种图像处理设备，包括：
    存储器，用于存储指令和数据，
    处理器，配置为执行所述指令，以便：
    接收CT扫描***获取的对象的投影数据；以及
    利用卷积神经网络对所述投影数据进行处理，以获取所述对象的估计图像；
    其中，所述处理器还配置为构建所述卷积神经网络，使其包括：
    投影域网络，用于处理输入的投影数据，得到估计投影数据；
    解析重建网络层，用于对估计投影数据进行解析重建，得到重建图像；
    图像域网络，用于对重建图像进行处理，得到估计图像；
    投影层，用于利用CT扫描***的***投影矩阵对估计图像进行投影运算，得到估计图像的投影结果；和
    统计模型层，用于确定输入的投影数据、估计投影数据和估计图像的投影结果 这三者基于统计模型的一致性；
    其中，所述处理器还配置为训练所述卷积神经网络，包括利用基于输入的投影数据、估计投影数据、以及估计图像的投影结果这三者的数据模型的一致性代价函数来调整图像域网络和投影域网络的卷积核参数。
  20. 一种计算机可读存储介质，其中存储有计算机指令，当所述指令被处理器执行时实现如权利要求1-17之一所述的方法。
PCT/CN2018/119372 2017-12-29 2018-12-05 训练神经网络的方法和设备、图像处理方法和设备以及存储介质 WO2019128660A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711498783 2017-12-29
CN201711498783.0 2017-12-29

Publications (1)

Publication Number Publication Date
WO2019128660A1 true WO2019128660A1 (zh) 2019-07-04

Family

ID=65011758

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/119372 WO2019128660A1 (zh) 2017-12-29 2018-12-05 训练神经网络的方法和设备、图像处理方法和设备以及存储介质

Country Status (5)

Country Link
US (1) US10984565B2 (zh)
EP (1) EP3506209B1 (zh)
CN (1) CN110047113B (zh)
RU (1) RU2709437C1 (zh)
WO (1) WO2019128660A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019224800A1 (en) * 2018-05-25 2019-11-28 Mahajan Vidur Method and system for simulating and constructing original medical images from one modality to other modality
CN112926517A (zh) * 2021-03-26 2021-06-08 北京航空航天大学 一种人工智能监控方法

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019033390A1 (en) * 2017-08-18 2019-02-21 Shenzhen United Imaging Healthcare Co., Ltd. SYSTEM AND METHOD FOR IMAGE RECONSTRUCTION
US10949252B1 (en) * 2018-02-13 2021-03-16 Amazon Technologies, Inc. Benchmarking machine learning models via performance feedback
US10977842B2 (en) * 2018-06-04 2021-04-13 Korea Advanced Institute Of Science And Technology Method for processing multi-directional X-ray computed tomography image using artificial neural network and apparatus therefor
US10885277B2 (en) 2018-08-02 2021-01-05 Google Llc On-device neural networks for natural language understanding
US11170542B1 (en) * 2018-10-10 2021-11-09 Lickenbrock Technologies, LLC Beam hardening and scatter removal
CN111368996B (zh) * 2019-02-14 2024-03-12 谷歌有限责任公司 可传递自然语言表示的重新训练投影网络
CN110363826B (zh) * 2019-07-16 2022-11-25 上海联影医疗科技股份有限公司 医学图像重建方法、装置、***及存储介质
CN110415311B (zh) * 2019-07-29 2024-04-16 上海联影医疗科技股份有限公司 Pet图像重建方法、***、可读存储介质和设备
CN110544282B (zh) * 2019-08-30 2022-03-29 清华大学 基于神经网络的三维多能谱ct重建方法和设备及存储介质
WO2021051049A1 (en) * 2019-09-12 2021-03-18 Xie Huidong Few-view ct image reconstruction system
CN110717951B (zh) * 2019-09-12 2021-08-03 浙江大学 一种基于cGANs的PET图像直接重建方法
CN110728729B (zh) * 2019-09-29 2023-05-26 天津大学 一种基于注意机制的无监督ct投影域数据恢复方法
CN112581513B (zh) * 2019-09-29 2022-10-21 北京大学 锥束计算机断层扫描图像特征提取与对应方法
CN112581554B (zh) * 2019-09-30 2024-02-27 中国科学院深圳先进技术研究院 一种ct成像方法、装置、存储设备及医学成像***
CN110742635B (zh) * 2019-10-08 2021-10-08 南京安科医疗科技有限公司 一种复合能谱ct成像方法
CN110751701B (zh) * 2019-10-18 2021-03-30 北京航空航天大学 一种基于深度学习的x射线吸收衬度计算机断层成像不完备数据重建方法
US10936916B1 (en) * 2019-10-31 2021-03-02 Booz Allen Hamilton Inc. System and method for classifying image data
CN112862944B (zh) * 2019-11-09 2024-04-12 无锡祥生医疗科技股份有限公司 人体组织超声建模方法、超声设备及存储介质
US11386592B2 (en) 2019-12-20 2022-07-12 Varian Medical Systems International Ag Tomographic image analysis using artificial intelligence (AI) engines
US11436766B2 (en) 2019-12-20 2022-09-06 Varian Medical Systems International Ag Tomographic image reconstruction using artificial intelligence (AI) engines
EP4078525A1 (en) * 2019-12-20 2022-10-26 Varian Medical Systems International AG Tomographic image processing using artificial intelligence (ai) engines
CN111311531B (zh) * 2020-01-22 2024-03-08 东软医疗***股份有限公司 图像增强方法、装置、控制台设备及医学成像***
CN111652951B (zh) * 2020-05-07 2023-06-06 中国工程物理研究院材料研究所 一种稀疏角度快中子ct成像方法
CN111612719A (zh) * 2020-05-21 2020-09-01 东软医疗***股份有限公司 Ct图像处理方法、装置、ct设备及ct***
CN111950705A (zh) * 2020-08-10 2020-11-17 深圳高性能医疗器械国家研究院有限公司 一种重建神经网络及其应用
WO2022032445A1 (zh) * 2020-08-10 2022-02-17 深圳高性能医疗器械国家研究院有限公司 一种重建神经网络及其应用
CN111932463B (zh) * 2020-08-26 2023-05-30 腾讯科技（深圳）有限公司 图像处理方法、装置、设备及存储介质
CN112017256B (zh) * 2020-08-31 2023-09-15 南京安科医疗科技有限公司 在线ct图像质量自由定制方法及计算机可读存储介质
CN112669401B (zh) * 2020-12-22 2022-08-19 中北大学 基于卷积神经网络的ct图像重建方法及***
CN114764750B (zh) * 2021-01-12 2023-08-18 四川大学 基于自适应一致性先验深度网络的图像去噪方法
CN113192155B (zh) * 2021-02-04 2023-09-26 南京安科医疗科技有限公司 螺旋ct锥束扫描图像重建方法、扫描***及存储介质
CN113012293B (zh) * 2021-03-22 2023-09-29 平安科技（深圳）有限公司 石刻模型构建方法、装置、设备及存储介质
CN112907691A (zh) * 2021-03-26 2021-06-04 深圳安科高技术股份有限公司 基于神经网络的ct图像重建方法、装置、设备及存储介质
CN113096211B (zh) * 2021-04-16 2023-04-18 上海联影医疗科技股份有限公司 一种校正散射的方法和***
WO2023287586A1 (en) * 2021-07-12 2023-01-19 The Regents Of The University Of California Diffractive optical network for seeing through diffusive or scattering media
CN114283235B (zh) * 2021-12-07 2022-08-23 中国科学院国家空间科学中心 一种基于有限角度投影数据的三维磁层重构方法及***
CN114782566B (zh) * 2021-12-21 2023-03-10 首都医科大学附属北京友谊医院 Ct数据重建方法和装置、电子设备和计算机可读存储介质
WO2024108203A1 (en) * 2022-11-18 2024-05-23 Rensselaer Polytechnic Institute Patch-based denoising diffusion probabilistic model for sparse tomographic imaging
CN118154899A (zh) * 2024-05-09 2024-06-07 成都数之联科技股份有限公司 面板边缘识别方法和装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102456227A (zh) * 2010-10-28 2012-05-16 清华大学 Ct图像重建方法及装置
CN102750676A (zh) * 2012-06-07 2012-10-24 南方医科大学 基于余弦变换的x线ct医学影像投影数据自适应阈值滤波重建方法
US20160296193A1 (en) * 2015-04-09 2016-10-13 Christian Hofmann Multi-Cycle Dynamic CT Imaging
CN106530366A (zh) * 2015-09-09 2017-03-22 清华大学 能谱ct图像重建方法及能谱ct成像***
CN106780641A (zh) * 2016-11-14 2017-05-31 西安交通大学 一种低剂量x射线ct图像重建方法
CN107481297A (zh) * 2017-08-31 2017-12-15 南方医科大学 一种基于卷积神经网络的ct图像重建方法

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011010231A1 (en) * 2009-07-20 2011-01-27 Koninklijke Philips Electronics N.V. Anatomy modeling for tumor region of interest defiinition
RU2505800C2 (ru) * 2012-05-10 2014-01-27 Федеральное государственное бюджетное образовательное учреждение высшего профессионального образования "Национальный исследовательский Томский государственный университет" (ТГУ) Способ рентгеновской томографии и устройство для его осуществления
US9700219B2 (en) * 2013-10-17 2017-07-11 Siemens Healthcare Gmbh Method and system for machine learning based assessment of fractional flow reserve
US9808216B2 (en) * 2014-06-20 2017-11-07 Marquette University Material decomposition of multi-spectral x-ray projections using neural networks
WO2017223560A1 (en) * 2016-06-24 2017-12-28 Rensselaer Polytechnic Institute Tomographic image reconstruction via machine learning
US11126914B2 (en) * 2017-10-11 2021-09-21 General Electric Company Image generation using machine learning

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102456227A (zh) * 2010-10-28 2012-05-16 清华大学 Ct图像重建方法及装置
CN102750676A (zh) * 2012-06-07 2012-10-24 南方医科大学 基于余弦变换的x线ct医学影像投影数据自适应阈值滤波重建方法
US20160296193A1 (en) * 2015-04-09 2016-10-13 Christian Hofmann Multi-Cycle Dynamic CT Imaging
CN106530366A (zh) * 2015-09-09 2017-03-22 清华大学 能谱ct图像重建方法及能谱ct成像***
CN106780641A (zh) * 2016-11-14 2017-05-31 西安交通大学 一种低剂量x射线ct图像重建方法
CN107481297A (zh) * 2017-08-31 2017-12-15 南方医科大学 一种基于卷积神经网络的ct图像重建方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019224800A1 (en) * 2018-05-25 2019-11-28 Mahajan Vidur Method and system for simulating and constructing original medical images from one modality to other modality
US12039699B2 (en) 2018-05-25 2024-07-16 Vidur MAHAJAN Method and system for simulating and constructing original medical images from one modality to other modality
CN112926517A (zh) * 2021-03-26 2021-06-08 北京航空航天大学 一种人工智能监控方法
CN112926517B (zh) * 2021-03-26 2022-11-18 北京航空航天大学 一种人工智能监控方法

Also Published As

Publication number Publication date
RU2709437C1 (ru) 2019-12-17
EP3506209B1 (en) 2021-08-18
US10984565B2 (en) 2021-04-20
CN110047113A (zh) 2019-07-23
EP3506209A1 (en) 2019-07-03
CN110047113B (zh) 2021-05-18
US20190206095A1 (en) 2019-07-04

Similar Documents

Publication Publication Date Title
WO2019128660A1 (zh) 训练神经网络的方法和设备、图像处理方法和设备以及存储介质
CN110660123B (zh) 基于神经网络的三维ct图像重建方法和设备以及存储介质
Dong et al. A deep learning reconstruction framework for X-ray computed tomography with incomplete data
JP7202302B2 (ja) 断層撮影再構成に使用するためのデータのディープラーニングに基づく推定
CN109300166B (zh) 重建ct图像的方法和设备以及存储介质
JP7455622B2 (ja) 医用画像処理装置及び学習用画像の取得方法
CN109300167B (zh) 重建ct图像的方法和设备以及存储介质
Thibault et al. A three‐dimensional statistical approach to improved image quality for multislice helical CT
CN110544282B (zh) 基于神经网络的三维多能谱ct重建方法和设备及存储介质
CN107871331B (zh) 用于重构发射活动图像的***和方法
EP2310840B1 (en) High efficiency computed tomography
JP2020516345A (ja) 深層学習に基づくトモグラフィ再構成
US8971599B2 (en) Tomographic iterative reconstruction
WO2020237873A1 (zh) 基于神经网络的螺旋ct图像重建方法和设备及存储介质
KR20190138292A (ko) 뉴럴 네트워크를 이용한 다방향 엑스레이 전산단층 촬영 영상 처리 방법 및 그 장치
US8416914B2 (en) System and method of iterative image reconstruction for computed tomography
US20140363067A1 (en) Methods and systems for tomographic reconstruction
Wu et al. Stabilizing deep tomographic reconstruction networks
US20170340287A1 (en) Method And Apparatus For Motion Correction In CT Imaging
Kelkar et al. Prior image-constrained reconstruction using style-based generative models
KR20190135618A (ko) 뉴럴 네트워크를 이용한 내부 전산단층 촬영 영상 처리 방법 및 그 장치
US9495770B2 (en) Practical model based CT construction
Perelli et al. Regularization by denoising sub-sampled Newton method for spectral CT multi-material decomposition
Wu et al. Low dose CT reconstruction via L 1 norm dictionary learning using alternating minimization algorithm and balancing principle
Langet et al. Compressed‐sensing‐based content‐driven hierarchical reconstruction: Theory and application to C‐arm cone‐beam tomography

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18896472

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18896472

Country of ref document: EP

Kind code of ref document: A1