CN112988382A - Medical image intelligent analysis system based on distributed deep learning - Google Patents
Medical image intelligent analysis system based on distributed deep learning Download PDFInfo
- Publication number
- CN112988382A CN112988382A CN202110268700.9A CN202110268700A CN112988382A CN 112988382 A CN112988382 A CN 112988382A CN 202110268700 A CN202110268700 A CN 202110268700A CN 112988382 A CN112988382 A CN 112988382A
- Authority
- CN
- China
- Prior art keywords
- model
- neural network
- distributed
- server
- medical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 42
- 238000013135 deep learning Methods 0.000 title claims abstract description 42
- 238000012549 training Methods 0.000 claims abstract description 97
- 238000003062 neural network model Methods 0.000 claims abstract description 68
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000004891 communication Methods 0.000 claims abstract description 20
- 238000007781 pre-processing Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 238000013136 deep learning model Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 6
- 230000035945 sensitivity Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000010191 image analysis Methods 0.000 description 3
- 238000004806 packaging method and process Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000003902 lesion Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 206010002091 Anaesthesia Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000037005 anaesthesia Effects 0.000 description 1
- 238000000137 annealing Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000001574 biopsy Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000008733 trauma Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of deep learning and intelligent analysis of medical images, and particularly relates to an intelligent analysis system of medical images based on distributed deep learning, aiming at solving the problems of difficult model training, low classification precision and poor generalization capability caused by the fact that the existing intelligent analysis system of medical images cannot share medical data. The system of the invention comprises: the system comprises service ends arranged in N medical centers and a central client end arranged in a remote server, wherein the service ends are connected with the central client end through a communication link; the server comprises an image acquisition module configured to acquire a medical image and perform image preprocessing; the central client comprises a distributed model training and scheduling module which is configured to train the deep neural network models in a distributed mode and centrally schedule each distributed deep neural network model to perform model iteration. The method avoids medical data sharing, simplifies the difficulty of multi-center model training, and improves the analysis precision and generalization capability of the deep learning model.
Description
Technical Field
The invention belongs to the field of deep learning and intelligent analysis of medical images, and particularly relates to an intelligent analysis system, method and equipment of medical images based on distributed deep learning.
Background
At present, medical imaging technology is used for reflecting focus information of a patient and further realizing detection and classification, and the method is one of common and effective means, and can reduce clinical limitations such as trauma, surgical anesthesia, time waiting and the like brought by traditional means such as tissue biopsy or gene detection and the like. However, analysis of medical images often depends on subjective experience of doctors, and a great amount of quantitative information related to lesions, which is contained in medical images, is urgently needed to be developed. Therefore, the medical image is automatically and individually intelligently analyzed, the medical image is matched with the current precise medical concept, the precision of medical image detection and classification tasks can be effectively improved, and the efficiency of medical image analysis work is improved.
The deep learning technology has potential in tasks such as medical image detection and classification, and can create a feasible solution for solving the problem of automated lesion detection and classification. However, applying deep learning techniques in the medical field often requires a large amount of training data, and when the differences between imaging phenotypes are small or there is large heterogeneity in the population, the sample size of patients is often limited, which may lead to poor generalization ability of deep learning models. To address this challenge, data from multiple medical centers is typically collected to increase sample size and sample diversity, thereby enhancing the analytical effectiveness of deep learning models. On the other hand, however, the image data of the patient may occupy a large storage space and is inconvenient to transmit; even if the transmission requirements are met, sharing of the image data of the patient among the medical centers may involve legal issues such as privacy security, which also results in the hindrance of the analysis research of the medical images.
The combination of distributed technology and deep learning technology can provide a new opportunity for medical image analysis research among a plurality of medical centers. The distributed deep learning method assumes that the medical centers do not need to share the original data and independently execute the deep learning training process, so that the distributed deep learning method has stronger safety and higher efficiency. However, the application of the current distributed deep learning method in medical image analysis is still limited to the theoretical proof and simulation stages, and the development of the distributed deep learning system with the asynchronous communication function and the distributed model training and scheduling function is of great significance to the future intelligent, automatic and individual medical development direction. Aiming at the problems, the invention provides a medical image intelligent analysis system based on distributed deep learning.
Disclosure of Invention
In order to solve the problems in the prior art, namely to solve the problems of difficult model training, low classification precision and poor generalization capability caused by the fact that the existing medical image intelligent analysis system cannot share medical data, the invention provides a medical image intelligent analysis system based on distributed deep learning, which comprises a server side arranged in N medical centers and a center client side arranged in a remote server; each server is connected with the central client through a communication link:
the server comprises an image acquisition module; the image acquisition module is configured to acquire a medical image to be analyzed and perform image preprocessing; associating the preprocessed medical image serving as training sample data with each service terminal;
the central client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode, and the central client side centrally schedules each distributed deep neural network model to perform model iteration;
the server and the central client further comprise an image classification module; the image classification module is configured to obtain the category of the medical image to be analyzed through a trained deep neural network model;
the deep neural network model is trained by the method of;
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
In some preferred embodiments, the communication link connection between each server and the central client is implemented by a communication protocol and a chain model; the chain model is used for integrating a series of operations on the tensor into an operation chain, and each operation can represent the state or the transformation of the tensor; the link structure uses child attribute to access downwards at the head part of the link structure and parent attribute to access upwards, and asynchronous sending and receiving of the directivity tensor between the server side and the central client side are achieved.
In some preferred embodiments, the training sample data is associated with the id index, ip address, and port of each server in a Torch Hook structure.
In some preferred embodiments, the image pre-processing comprises image cropping, scaling, normalization.
In some preferred embodiments, the deep neural network model is constructed based on a densely connected convolutional neural network; wherein each layer of the densely connected convolutional neural network is connected with all layers in front of it in the channel dimension and serves as an input of the next layer.
In some preferred embodiments, the deep neural network model calculates a learning rate during the training process by:
lr′＝max(lr′×0.95，lr0×0.01)
or
Wherein i represents the ith iteration, r represents the total number of iterations, lr0Indicating the initial learning rate and lr' the updated learning rate after the ith iteration.
In some preferred embodiments, the method for acquiring the model training loss value and the updated model weight of each server and performing weighted average processing by the central client includes:
wherein L isi，jRepresents the model training loss value, W, calculated by the jth server side in the ith round of iterationi，jModel weight, W, representing the jth server update of the ith round of iterationi+1，0The model weight after weighted average processing.
The second aspect of the invention provides a medical image intelligent analysis method based on distributed deep learning, which comprises the following steps:
s10, respectively obtaining medical images to be analyzed and preprocessing the images by the server side of each medical center; associating the preprocessed medical image serving as training sample data with each service terminal;
s20, training the deep neural network model by the central client arranged on the remote server in a distributed mode, and scheduling each distributed deep neural network model by the central client in a centralized mode to perform model iteration;
s30, the server side and the central client side obtain the category of the medical image to be analyzed through the trained deep neural network model;
the deep neural network model is trained by the following method:
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
In a third aspect of the present invention, a computer device is provided, including: at least one processor and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used for being executed by the processor to realize the above-mentioned medical image intelligent analysis method based on distributed deep learning.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions for being executed by the computer to implement the above-mentioned method for intelligent analysis of medical images based on distributed deep learning.
The invention has the beneficial effects that:
the invention avoids the transmission and sharing of medical data, simplifies the difficulty of multi-center model training, and improves the analysis precision of medical images and the generalization capability of deep learning models.
1) The invention effectively combines a plurality of single machine resources under the condition of avoiding data sharing based on the flexibility and the expandability of the existing distributed technology. The training data are respectively sent to the service end nodes positioned in each medical center for training, and the distributed training of the deep neural network model can be completed only by transmitting the model and the parameters between each service end and the central client, so that the difficulty of model training is simplified, the analysis precision of the medical image and the generalization capability of the deep learning model are improved, and the method is an effective strategy for analyzing large-scale medical image data sets. Meanwhile, the problem that medical data cannot be shared due to privacy security of the medical data is effectively avoided, and the method has higher security.
2) The invention effectively combines a PyTorch deep learning framework, a PySyft distributed computing framework and a WebSocket communication technology, realizes a distributed computing function based on chain tensor operation and an asynchronous communication function between each medical center server and a center client, and provides a distributed deep learning scheme with simple interface and strong expandability for intelligent analysis of medical images.
3) In addition, the intelligent analysis of medical images by using the distributed deep learning has important significance on future intelligent, automatic and individual medical development directions.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a block diagram of an intelligent analysis system for medical images based on distributed deep learning according to an embodiment of the present invention;
FIG. 2 is a flow diagram of server side image acquisition and processing according to an embodiment of the invention;
FIG. 3 is an exemplary diagram of an acquired medical image in accordance with one embodiment of the invention;
FIG. 4 is a schematic diagram of a training and iteration flow of a deep neural network model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the structure of deep neural network model training and iteration according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a computer system suitable for implementing an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
A medical image intelligent analysis system based on distributed deep learning according to a first embodiment of the present invention, as shown in fig. 1, includes a server installed in N medical centers, and a center client installed in a remote server; each server is connected with the central client through a communication link:
the server comprises an image acquisition module; the image acquisition module is configured to acquire a medical image to be analyzed and perform image preprocessing; associating the preprocessed medical image serving as training sample data with each service terminal;
the central client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode, and the central client side centrally schedules each distributed deep neural network model to perform model iteration;
the server and the central client further comprise an image classification module; the image classification module is configured to obtain the category of the medical image to be analyzed through a trained deep neural network model;
the deep neural network model is trained by the method of;
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
In order to more clearly describe the medical image intelligent analysis system based on distributed deep learning of the present invention, the following will describe each step in an embodiment of the method of the present invention in detail.
In the following embodiments, a distributed acquisition method of a medical image is introduced, a distributed training and iteration method of a deep neural network model is described, and a detailed description is given to a specific process of acquiring an analysis result of the medical image by a medical image intelligent analysis system based on distributed deep learning.
1. Distributed acquisition of medical images, as shown in FIG. 2
Respectively acquiring medical images at the service terminals of N medical centers, packaging training sample data into a Dataset class based on PyTorch API, and transmitting the DataLoader; carrying out image preprocessing on training sample data; further packaging training sample data into a BaseDateset base class based on PySyft API, and respectively associating with each medical center service terminal;
in this embodiment, medical images including modalities such as CT, MRI, X-ray and the like are independently acquired at N medical center service terminals, as shown in fig. 3, and stored in a DICOM format or a common picture format.
The image preprocessing is carried out on the medical image, and comprises image cutting, zooming and standardization, and specifically comprises the following steps: firstly, cutting an image of a medical image, and zooming the cut medical image to a preset size; secondly, converting the medical image into a tensor form; and finally, standardizing the medical image by adopting methods such as z-score or Min-Max standardization and the like.
Defining index ids of N medical center service terminals, and acquiring ip addresses and port; loading training sample data on a local ip address and a preset port of each server, packaging the training sample data by adopting a BaseDateset base class provided by a PySyft frame, and associating the training sample data with a server id index, the ip address and a port by adopting a Torch Hook function.
2. Training and iteration process of deep neural network model, as shown in FIG. 4
A10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
in this embodiment, a deep neural network model is written based on PyTorch at a central client, and the structure of the deep neural network model is specifically designed to be a convolutional neural network that adopts a dense connection mechanism, for example: DenseNet-64, DenseNet-121, etc., each layer accepts all layers before it as its additional input, i.e., each layer will be connected with all layers before it in the channel dimension and serve as the input for the next layer; the nonlinear transformation function includes batch normalization, ReLU and convolution operations.
The weight of the random initialization model is W0，0Meanwhile, setting distributed training parameters for the model, including iteration round number, batch size, learning rate, loss function and optimizer, wherein the loss function can adopt a cross entropy loss function, a log likelihood cost function and the like for classifying tasks, can also adopt L1 loss, L2 loss and the like for detecting the tasks, and can also adopt Dice loss and the like for segmenting the tasks; the optimizer can adopt a random gradient descent optimizer, an adaptive moment estimation optimizer and the like; the setting strategy of the learning rate can optionally select one of the following strategies: constant learning rate lr0Performing learning rate attenuation according to an equation (1), performing learning rate cosine annealing according to an equation (2), wherein i represents the ith iteration, r represents the total iteration number, and lr' represents the updated learning rate after the ith iteration:
lr′＝max(lr′×0.95，lr0×0.01) (1)
each server and the central client can establish persistent connection through WebSocket API only by completing one handshake, and bidirectional byte transmission is carried out. In the embodiment, the connection between a central client and asynchronous communication links of N servers is established through a WebSocket communication protocol of an asynchronous communication module, id indexes, ip addresses and port numbers corresponding to the servers are obtained, and a deep neural network model, weight and training parameters of the central client are encoded into a directivity tensor (PointTensor); and then, the directivity tensor is sent to each server side from the central client side through a chain model (SyftTensor) of the chain coding module. The communication link connection in the invention comprises a chain coding module and an asynchronous communication module; the chain model in the chain coding module is used for integrating a series of operations on the tensor into one operation chain, and each operation can represent the state or the transformation of the tensor; the link structure uses child attribute to access downwards at the head part of the link structure and parent attribute to access upwards, and asynchronous sending and receiving of the directivity tensor between the server side and the central client side are achieved.
A20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
in this embodiment, the N servers respectively receive the deep neural network model, the weights, and the directivity tensors of the training parameters through a chain model (syftsensor), and perform independent training on the locally loaded training sample data. Calculating the model loss value L of the ith iteration by combining the class label of the training sample datai，1，Li，2，Li，3....Li，NAnd calculating the gradient through back propagation to update the model weight of the ith round to Wi，1，Wi，2，Wi，3....Wi，N. And coding the model loss value and the updated weight of the model into a directivity tensor, and sending the directivity tensor to the central client.
A30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
in this embodiment, the central client receives the model loss values and the updated model weights from the N servers, and performs federal calculation, i.e. centralized scheduling on the distributed models, as shown in fig. 5, the average value of the model weights may be calculated according to equation (3) or the weighted average value of the model weights may be calculated according to equation (4):
and taking the weighted average model weight as the initialized model weight of a new iteration, and sending the initialized model weight to the N service terminals.
A40, looping the steps A20 and A30 until the deep neural network model converges.
In the embodiment, the steps a20 and a30 are repeated until the model training termination condition is met, and the optimal deep neural network model is obtained. The model training termination condition is as follows: and the actual training iteration rounds of each server have reached the preset distributed training iteration rounds, or after each server receives the updated model weight, the model precision has reached the expected requirement. And then, the central client transmits the optimal deep neural network model to the N service terminals, and respectively calculates the effectiveness of the model on multi-center data, including evaluation indexes such as AUC, accuracy, sensitivity and specificity.
3. Medical image intelligent analysis system based on distributed deep learning obtains analysis result of medical image
The medical center server of the medical image intelligent analysis system based on the distributed deep learning comprises an image acquisition module; the image acquisition module is configured to acquire a medical image to be analyzed and perform image preprocessing; associating the preprocessed medical image serving as training sample data with each service terminal;
the central client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode, and the central client side centrally schedules each distributed deep neural network model to perform model iteration;
the server and the central client also comprise an image classification module; the image classification module is configured to obtain an analysis result of the medical image to be analyzed, namely the category of the medical image, through the trained deep neural network model;
in this embodiment, a medical image to be analyzed is obtained first as a model input; then, the deep neural network model is trained and iterated in a distributed mode; and finally, obtaining the category of the medical image to be analyzed through the trained deep neural network model.
In addition, in the present invention, it is preferable that the parameters are set as follows:
loading data: 55000 training samples, 10000 testing samples, 5 service terminals of a medical center and 1 central client;
data preprocessing: random resize crop is randomly cut, the image is zoomed to 64 multiplied by 64, and the image is standardized (the mean value is 0 and the standard deviation is 1);
deep neural network model: DenseNet-121;
distributed training parameters: performing iteration for 100 rounds, wherein the batch size is 32, the constant learning rate is 0.01, the cross entropy loss function is adopted, and a random gradient descent optimizer is adopted;
the time consumed for testing the model is as follows: performing model training by using a CPU (Central processing Unit), and completing one round of model training and iteration, wherein the time is about 5 minutes;
testing the efficiency of the model: the AUC of the server 1 reaches 0.87, the accuracy is 0.78, the sensitivity is 0.83, and the specificity is 0.76; the AUC of the server 2 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 3 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 4 reaches 0.87, the accuracy is 0.76, the sensitivity is 0.86, and the specificity is 0.73; the AUC of the server 5 reaches 0.87, the accuracy is 0.74, the sensitivity is 0.86, and the specificity is 0.71.
It should be noted that, the medical image intelligent analysis system based on distributed deep learning provided in the foregoing embodiment is only illustrated by the division of the functional modules, and in practical applications, the functions may be distributed by different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
The medical image intelligent analysis method based on distributed deep learning in the second embodiment of the invention comprises the following steps:
s10, respectively obtaining medical images to be analyzed and preprocessing the images by the server side of each medical center; associating the preprocessed medical image serving as training sample data with each service terminal;
s20, training the deep neural network model by the central client arranged on the remote server in a distributed mode, and scheduling each distributed deep neural network model by the central client in a centralized mode to perform model iteration;
s30, the server side and the central client side obtain the category of the medical image to be analyzed through the trained deep neural network model;
the deep neural network model is trained by the following method:
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the method described above may refer to the corresponding process in the foregoing system embodiment, and are not described herein again.
A third embodiment of the present invention provides a computer device, including: at least one processor and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used for being executed by the processor to realize the above-mentioned medical image intelligent analysis method based on distributed deep learning.
According to a fourth embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions for being executed by the computer to implement the above-mentioned medical image intelligent analysis method based on distributed deep learning.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method examples, and are not described herein again.
Referring now to FIG. 6, there is illustrated a block diagram of a computer system suitable for use as a server in implementing embodiments of the system, method and apparatus of the present application. The server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 6, the computer system includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for system operation are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An Input/Output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a cathode ray tube, a liquid crystal display, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a lan card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program, when executed by the CPU601, performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the C language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network or a wide area network, or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A medical image intelligent analysis system based on distributed deep learning is characterized by comprising service terminals arranged in N medical centers and a center client arranged in a remote server; each server is connected with the central client through a communication link;
the server comprises an image acquisition module; the image acquisition module is configured to acquire a medical image to be analyzed and perform image preprocessing; associating the preprocessed medical image serving as training sample data with each service terminal;
the central client comprises a distributed model training and scheduling module; the distributed model training and scheduling module is configured to train the deep neural network model in a distributed mode, and the central client side centrally schedules each distributed deep neural network model to perform model iteration;
the server and the central client further comprise an image classification module; the image classification module is configured to obtain the category of the medical image to be analyzed through a trained deep neural network model;
the deep neural network model is trained by the following method:
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
2. The medical image intelligent analysis system based on the distributed deep learning of claim 1, wherein the communication link connection between each server and the central client is realized through a communication protocol and a chain model; the chain model is used for integrating a series of operations on the tensor into an operation chain, and each operation can represent the state or the transformation of the tensor; the link structure uses child attribute to access downwards at the head part of the link structure and parent attribute to access upwards, and asynchronous sending and receiving of the directivity tensor between the server side and the central client side are achieved.
3. The medical image intelligent analysis system based on distributed deep learning of claim 1, wherein the training sample data is associated with id index, ip address and port of each server by adopting a Torch Hook structure.
4. The system for intelligently analyzing medical images based on distributed deep learning as claimed in claim 1, wherein the image preprocessing comprises image cropping, scaling and normalization.
5. The medical image intelligent analysis system based on distributed deep learning of claim 1, wherein the deep neural network model is constructed based on a densely connected convolutional neural network; wherein each layer of the densely connected convolutional neural network is connected with all layers in front of it in the channel dimension and serves as an input of the next layer.
6. The medical image intelligent analysis system based on distributed deep learning of claim 1, wherein the deep neural network model is calculated by the learning rate in the training process:
lr′＝max(lr′×0.95，lr0×0.01)
or
Wherein i represents the ith iteration, r represents the total number of iterations, lr0Indicating the initial learning rate and lr' the updated learning rate after the ith iteration.
7. The medical image intelligent analysis system based on distributed deep learning of claim 1, wherein the central client obtains the model training loss value and the updated model weight of each server and performs weighted average processing, and the method comprises:
wherein L isi，jRepresents the model training loss value, W, calculated by the jth server side in the ith round of iterationi，jModel weight, W, representing the jth server update of the ith round of iterationi+1，0The model weight after weighted average processing.
8. A medical image intelligent analysis method based on distributed deep learning is characterized by comprising the following steps:
s10, respectively obtaining medical images to be analyzed and preprocessing the images by the server side of each medical center; associating the preprocessed medical image serving as training sample data with each service terminal;
s20, training the deep neural network model by the central client arranged on the remote server in a distributed mode, and scheduling each distributed deep neural network model by the central client in a centralized mode to perform model iteration;
s30, the server side and the central client side obtain the category of the medical image to be analyzed through the trained deep neural network model;
the deep neural network model is trained by the following method:
a10, the central client encodes the deep neural network model to be trained, the initialized model weight and the model training parameters into directivity tensors, and respectively sends the directivity tensors to the servers in the N medical centers; the directivity tensor is a tensor with a set direction;
a20, training a received deep neural network model based on locally acquired training sample data, calculating a model training loss value, updating a model weight, encoding the loss value and the model weight into a directional vector, and sending the directional vector to a center client side by each server side;
a30, the central client side obtains the model training loss value and the updated model weight of each server side and carries out weighted average processing; taking the weighted and averaged model weight as a new round of initialized model weight, and sending the new round of initialized model weight to each server;
a40, looping the steps A20 and A30 until the deep neural network model converges.
9. A computer device, comprising: at least one processor and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the method for intelligent analysis of medical images based on distributed deep learning of claim 8.
10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions for being executed by the computer to implement the method for intelligent analysis of medical images based on distributed deep learning according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110268700.9A CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110268700.9A CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112988382A true CN112988382A (en) | 2021-06-18 |
| CN112988382B CN112988382B (en) | 2023-09-19 |
Family
ID=76335066
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110268700.9A Active CN112988382B (en) | 2021-03-12 | 2021-03-12 | Medical image intelligent analysis system based on distributed deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988382B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114140478A (en) * | 2022-01-30 | 2022-03-04 | 电子科技大学 | Federal learning method, system, device and medium for medical image segmentation |
| CN114463330A (en) * | 2022-04-12 | 2022-05-10 | 江苏康医通科技有限公司 | CT data collection system, method and storage medium |
| CN115115064A (en) * | 2022-07-11 | 2022-09-27 | 山东大学 | Semi-asynchronous federal learning method and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106297774A (en) * | 2015-05-29 | 2017-01-04 | 中国科学院声学研究所 | The distributed parallel training method of a kind of neutral net acoustic model and system |
| CN106878304A (en) * | 2017-02-15 | 2017-06-20 | 国网天津市电力公司 | A kind of method of the link multiplexing of distributed agent |
| CN109032671A (en) * | 2018-06-25 | 2018-12-18 | 电子科技大学 | A kind of distributed deep learning method and system based on data parallel strategy |
| US20190311298A1 (en) * | 2018-04-09 | 2019-10-10 | Here Global B.V. | Asynchronous parameter aggregation for machine learning |
| US20190325302A1 (en) * | 2018-04-23 | 2019-10-24 | EMC IP Holding Company LLC | Implementing parameter server in networking infrastructure for high-performance computing |
| CN110956202A (en) * | 2019-11-13 | 2020-04-03 | 重庆大学 | Image training method, system, medium and intelligent device based on distributed learning |
| US20200184278A1 (en) * | 2014-03-18 | 2020-06-11 | Z Advanced Computing, Inc. | System and Method for Extremely Efficient Image and Pattern Recognition and Artificial Intelligence Platform |
| CN111444019A (en) * | 2020-03-31 | 2020-07-24 | 中国科学院自动化研究所 | Cloud-end-collaborative deep learning model distributed training method and system |
-
2021
- 2021-03-12 CN CN202110268700.9A patent/CN112988382B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200184278A1 (en) * | 2014-03-18 | 2020-06-11 | Z Advanced Computing, Inc. | System and Method for Extremely Efficient Image and Pattern Recognition and Artificial Intelligence Platform |
| CN106297774A (en) * | 2015-05-29 | 2017-01-04 | 中国科学院声学研究所 | The distributed parallel training method of a kind of neutral net acoustic model and system |
| CN106878304A (en) * | 2017-02-15 | 2017-06-20 | 国网天津市电力公司 | A kind of method of the link multiplexing of distributed agent |
| US20190311298A1 (en) * | 2018-04-09 | 2019-10-10 | Here Global B.V. | Asynchronous parameter aggregation for machine learning |
| US20190325302A1 (en) * | 2018-04-23 | 2019-10-24 | EMC IP Holding Company LLC | Implementing parameter server in networking infrastructure for high-performance computing |
| CN109032671A (en) * | 2018-06-25 | 2018-12-18 | 电子科技大学 | A kind of distributed deep learning method and system based on data parallel strategy |
| CN110956202A (en) * | 2019-11-13 | 2020-04-03 | 重庆大学 | Image training method, system, medium and intelligent device based on distributed learning |
| CN111444019A (en) * | 2020-03-31 | 2020-07-24 | 中国科学院自动化研究所 | Cloud-end-collaborative deep learning model distributed training method and system |
Non-Patent Citations (6)
| Title |
|---|
| CSDN博客: ""PySyft是将隐私引入深度学习模型的框架"", Retrieved from the Internet <URL:https://blog.csdn.net/weixin_26632369/article/details/108174232> * |
| SERGEEV, ALEXANDER, AND MIKE DEL BALSO.: "\"Horovod: fast and easy distributed deep learning in TensorFlow\"", 《ARXIV PREPRINT ARXIV:1802.05799 (2018)》, pages 1 - 10 * |
| TURINA, VALERIA, ET AL.: "\"Combining split and federated architectures for efficiency and privacy in deep learning\"", 《PROCEEDINGS OF THE 16TH INTERNATIONAL CONFERENCE ON EMERGING NETWORKING EXPERIMENTS AND TECHNOLOGIES. 2020》, pages 562 - 563 * |
| VEPAKOMMA, PRANEETH, ET AL.: "\"NoPeek: Information leakage reduction to share activations in distributed deep learning\"", 《2020 INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS (ICDMW)》, pages 933 - 942 * |
| ZHANG, LIWEN , ET AL.: "\"A Deep Learning Risk Prediction Model for Overall Survival in Patients with Gastric Cancer: A Multicenter Study\"", 《 RADIOTHERAPY AND ONCOLOGY》, pages 73 - 80 * |
| 谭作文;张连福;: ""机器学习隐私保护研究综述"", 《软件学报》, pages 2128 - 2150 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114140478A (en) * | 2022-01-30 | 2022-03-04 | 电子科技大学 | Federal learning method, system, device and medium for medical image segmentation |
| CN114463330A (en) * | 2022-04-12 | 2022-05-10 | 江苏康医通科技有限公司 | CT data collection system, method and storage medium |
| CN115115064A (en) * | 2022-07-11 | 2022-09-27 | 山东大学 | Semi-asynchronous federal learning method and system |
| CN115115064B (en) * | 2022-07-11 | 2023-09-05 | 山东大学 | Semi-asynchronous federal learning method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112988382B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112988382B (en) | Medical image intelligent analysis system based on distributed deep learning | |
| US10755411B2 (en) | Method and apparatus for annotating medical image | |
| US10803584B2 (en) | Method and apparatus for acquiring information | |
| CN108520220B (en) | Model generation method and device | |
| CN107729929B (en) | Method and device for acquiring information | |
| CN108197652B (en) | Method and apparatus for generating information | |
| CN109002842A (en) | Image-recognizing method and device | |
| CN110660484B (en) | Bone age prediction method, device, medium, and electronic apparatus | |
| CN109344752B (en) | Method and apparatus for processing mouth image | |
| CN109243600B (en) | Method and apparatus for outputting information | |
| CN110517254A (en) | The automatic delineation method in clinical target area, device and relevant device based on deep learning | |
| US11301995B2 (en) | Feature identification in medical imaging | |
| CN111274490A (en) | Method and device for processing consultation information | |
| EP3907697A1 (en) | Method and apparatus for acquiring information | |
| CN111985197A (en) | Template generation method based on medical information | |
| CN111523593A (en) | Method and apparatus for analyzing medical images | |
| CN114627331A (en) | Model training method and device | |
| CN111340015A (en) | Positioning method and device | |
| CN112397194B (en) | Method, device and electronic equipment for generating patient disease attribution interpretation model | |
| CN113360611A (en) | AI diagnosis method, device, storage medium and equipment based on inspection result | |
| CN111862081A (en) | Image scoring method, and training method and device of score prediction network | |
| CN115205677A (en) | Method, device, electronic equipment and computer readable medium for identifying image information | |
| CN111833993B (en) | AI-based regional image remote quality control management system | |
| CN111798967A (en) | Wisdom ultrasonic testing system | |
| CN112446192A (en) | Method, device, electronic equipment and medium for generating text labeling model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |