CN113298780B - Deep learning-based bone age assessment method and system for children - Google Patents
Deep learning-based bone age assessment method and system for children Download PDFInfo
- Publication number
- CN113298780B CN113298780B CN202110564662.1A CN202110564662A CN113298780B CN 113298780 B CN113298780 B CN 113298780B CN 202110564662 A CN202110564662 A CN 202110564662A CN 113298780 B CN113298780 B CN 113298780B
- Authority
- CN
- China
- Prior art keywords
- hand bone
- image
- bone
- hand
- bone image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000000988 bone and bone Anatomy 0.000 title claims abstract description 94
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 210000002411 hand bone Anatomy 0.000 claims abstract description 259
- 230000011218 segmentation Effects 0.000 claims abstract description 44
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 80
- 238000012795 verification Methods 0.000 claims description 34
- 238000012360 testing method Methods 0.000 claims description 30
- 238000011176 pooling Methods 0.000 claims description 14
- 210000002569 neuron Anatomy 0.000 claims description 9
- 238000010586 diagram Methods 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 230000003213 activating effect Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 3
- 230000002776 aggregation Effects 0.000 claims 2
- 238000004220 aggregation Methods 0.000 claims 2
- 238000011156 evaluation Methods 0.000 abstract description 15
- 230000014461 bone development Effects 0.000 abstract description 8
- 230000002349 favourable effect Effects 0.000 abstract description 5
- 230000000694 effects Effects 0.000 description 6
- 230000005477 standard model Effects 0.000 description 5
- 238000000605 extraction Methods 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000001105 regulatory effect Effects 0.000 description 2
- 238000013077 scoring method Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000037182 bone density Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 230000000399 orthopedic effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a deep learning-based bone age assessment method and system for children. The method comprises the following steps: acquiring a hand bone image to be evaluated; preprocessing a hand bone image to be evaluated; inputting the preprocessed image into a trained hand bone segmentation model to obtain a segmented hand bone region mask; fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed; and inputting the hand bone image with the background information removed into a trained bone age assessment model, and assessing the bone age. The invention solves the problems that the current hand bone X-ray image evaluation efficiency is too low, the hand bone X-ray images cannot be processed in batches, and the like, and relieves the difficulty of reading the film by doctors. The invention can effectively and automatically evaluate the bone age of the X-ray image of the hand bone of the child, and provides favorable support for the subsequent evaluation of the bone development maturity.
Description
Technical Field
The invention relates to the technical field of bone age assessment, in particular to a method and a system for assessing bone age of children based on deep learning.
Background
In the clinical field, bone age assessment is generally calculated from statistical rules and relative quantitative indices by observing the size, structure and degree of closure of bones from X-ray images of non-dominant hands (typically left hand). The traditional bone age assessment method can be divided into: counting, atlas and score methods. Common methods include G & P mapping, TW2 scoring, TW3 scoring, and the like. The method of Li Guozhen percent count, the CHN method of wrist bone development standard of China staff, the She Shi scoring method directly referring to TW2 and the like are proposed in China. Although these methods can all complete the bone age assessment task, they are all excessively dependent on manual operation by a professional orthopedics practitioner, and mainly have the following two problems:
first, the process of evaluation is relatively complex. The bone age assessment process is very complex and difficult whether the counting method, the atlas method or the scoring method is adopted, the requirements on the professional skills of practitioners are very high, and a great number of data indexes are required to be analyzed and compared for a long time by the practitioners to obtain the bone age assessment result.
Second, the influence of subjective factors and random errors is relatively large. Because of the complexity of the bone age assessment process, the two bone age assessments of the same bone age person to be tested by different doctors may have differences, and the subjective factors and the business level of the doctors directly influence the assessment result; the results of two bone age evaluation of the same bone age testee in the same doctor may also be different, and the characteristics of large random error and poor repeatability of the traditional bone age evaluation method are reflected.
Disclosure of Invention
The invention aims to provide a deep learning-based bone age assessment method and system for children, which can effectively and automatically assess the bone age of the hand bone X-ray image of the children and provide favorable support for subsequent bone development maturity assessment.
In order to achieve the above object, the present invention provides the following solutions:
a deep learning-based bone age assessment method for children, comprising:
acquiring a hand bone image to be evaluated;
preprocessing a hand bone image to be evaluated;
inputting the preprocessed image into a trained hand bone segmentation model to obtain a segmented hand bone region mask;
fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed;
and inputting the hand bone image with the background information removed into a trained bone age assessment model, and assessing the bone age.
Further, the preprocessing of the hand bone image to be evaluated specifically includes:
the hand bone image to be evaluated is locally contrast enhanced using histogram equalization.
Further, the training process of the hand bone segmentation model specifically comprises the following steps:
establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples;
processing the hand bone image sample and marking hand bone contours to construct a first data set, wherein the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set;
training the Mask R-CNN model through a hand bone image training set to obtain a hand bone segmentation model;
and optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set.
Further, the training process of the bone age assessment model specifically includes:
dividing the first data set through a hand bone dividing model to obtain a hand bone region mask sample;
fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct a second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information;
training the improved Xreception model through the hand bone image training set with the background information removed to obtain a bone age assessment model;
and optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information.
The invention also provides a bone age assessment system for children based on deep learning, which comprises the following steps:
the image acquisition module is used for acquiring a hand bone image to be evaluated;
the preprocessing module is used for preprocessing the hand bone image to be evaluated;
the first input module is used for inputting the preprocessed image into the trained hand bone segmentation model to obtain a segmented hand bone region mask;
the fusion module is used for fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed;
and the second input module is used for inputting the hand bone image with the background information removed into a trained bone age assessment model to assess the bone age.
Further, the preprocessing module specifically includes:
and the equalization unit is used for enhancing the local contrast by utilizing histogram equalization on the hand bone image to be evaluated.
Further, the method further comprises the following steps:
the sample library establishing module is used for establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples;
the first data set construction module is used for processing the hand bone image sample and marking hand bone contours to construct a first data set, and the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set;
the first training module is used for training the Mask R-CNN model through the hand bone image training set to obtain a hand bone segmentation model;
and the first optimization module is used for optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set.
Further, the method further comprises the following steps:
the segmentation module is used for segmenting the first data set through a hand bone segmentation model to obtain a hand bone region mask sample;
the second data set is constructed and used for fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct the second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information;
the second training module is used for training the improved Xattention model through the hand bone image training set after removing the background information to obtain a bone age assessment model;
and the second optimization module is used for optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention discloses a deep learning-based bone age assessment method for children, which comprises the following steps: acquiring a hand bone image to be evaluated; preprocessing a hand bone image to be evaluated; inputting the preprocessed image into a trained hand bone segmentation model to obtain a segmented hand bone region mask; fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed; and inputting the hand bone image with the background information removed into a trained bone age assessment model, and assessing the bone age. The invention solves the problems that the current hand bone X-ray image evaluation efficiency is too low, the hand bone X-ray images cannot be processed in batches, and the like, and relieves the difficulty of reading the film by doctors. The invention can effectively and automatically evaluate the bone age of the X-ray image of the hand bone of the child, and provides favorable support for the subsequent evaluation of the bone development maturity.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a deep learning based method for assessing bone age of children according to an embodiment of the present invention;
FIG. 2 is a diagram of an improved Xreception architecture of an embodiment of the present invention;
FIG. 3 is a schematic view of a loaded hand bone X-ray image
FIG. 4 is a schematic diagram of the present loading model;
fig. 5 is a schematic diagram showing the evaluation result.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a deep learning-based bone age assessment method and system for children, which can effectively and automatically assess the bone age of the hand bone X-ray image of the children and provide favorable support for subsequent bone development maturity assessment.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
As shown in fig. 1, the method for estimating bone age of children based on deep learning provided by the invention comprises the following steps:
step 101: and acquiring a hand bone image to be evaluated.
Step 102: and preprocessing the hand bone image to be evaluated. The hand bone image to be evaluated is locally contrast enhanced using histogram equalization.
Step 103: and inputting the preprocessed image into a trained hand bone segmentation model to obtain a segmented hand bone region mask.
Step 104: and fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed.
Step 105: and inputting the hand bone image with the background information removed into a trained bone age assessment model, and assessing the bone age.
The training process of the hand bone segmentation model specifically comprises the following steps: establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples; processing the hand bone image sample and marking hand bone contours to construct a first data set, wherein the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set; training the Mask R-CNN model through a hand bone image training set to obtain a hand bone segmentation model; and optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set.
The detailed process is as follows:
(1) Establishing a hand bone image sample library: collecting left hand bone fragments of an X-ray image, converting the left hand bone fragments into PNG format images, and establishing a hand bone image sample library;
(2) Image preprocessing: the image data collected in the step (1) are subjected to histogram equalization to enhance local contrast, a two-dimensional histogram of an original input image is calculated firstly, then gray level distribution of the histogram is regulated through a gray level conversion function to be uniformly distributed in the whole gray level interval range (the gray level interval range is [0, L-1], wherein L is the gray level number of gray level images), finally, the gray level value of the original input image is modified according to the regulated histogram to complete equalization operation, and then a data label tool labelme is used for marking the hand bone outline of the image and dividing the image into a training set, a verification set and a test set;
(3) Hand bone image processing
1) And (3) constructing a hand bone segmentation model: inputting images in a training set into a Mask R-CNN model (existing model), automatically learning the characteristics of a hand bone region by utilizing ResNet-50 and a characteristic pyramid network, correcting and extracting the deviation between the characteristics and the region of interest by utilizing a RoIAlign layer, predicting and segmenting the hand bone Mask by utilizing a full convolution network, continuously adjusting parameters (learning rate and batch size) of the model in the training process, enabling the model to learn the characteristics of more samples by the training model to obtain training models, inputting data of a verification set into each training model, predicting the data in the verification set by utilizing the models, recording the accuracy of each training model, selecting the parameters corresponding to the training model with the best effect, and determining the hand bone segmentation model; inputting the test set data into the hand bone segmentation model for evaluating the performance of the model and the segmentation capability of the target area, adding the images into a training set after the images to be evaluated are segmented, repeating the steps, fine-tuning the training model, continuously optimizing the structure of the training model, improving the generalization capability of the training model, and finally generating the hand bone segmentation model.
2) Hand bone segmentation: inputting the image data preprocessed in the step (2) into a hand bone segmentation model, accurately segmenting a hand bone region mask, and then carrying out image fusion on the hand bone region mask and an original hand bone image, namely after the original hand bone image is segmented by the hand bone segmentation model, the region of interest is white, the pixel values of the region of interest are all non-0, the non-region of interest are all black, the pixel values of the non-region of interest are all 0, and the hand bone region mask and the original hand bone image are subjected to AND operation to obtain an image only comprising the region of interest, namely the hand bone image with background information removed;
the training process of the bone age assessment model specifically comprises the following steps: dividing the first data set through a hand bone dividing model to obtain a hand bone region mask sample; fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct a second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information; training the improved Xreception model through the hand bone image training set with the background information removed to obtain a bone age assessment model; and optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information.
The detailed process is as follows:
the hand bone image with the background information removed is input into an improved Xception model (shown in fig. 2), which is: after the original Xreception network (1) outputs, firstly, the channel information of the feature map is aggregated by using global maximization (2) and global average pooling (3) to generate two different channel features, secondly, the two different channel features are fed into a multi-layer perceptron (4), the outputs (5 and 6) are added element by element, and finally, a channel attention map is generated through activation of a sigmoid function (7), so that a channel attention module (which is used for enhancing the feature extraction capability of channel dimension) is obtained; after inputting the characteristics output by the channel attention module, firstly, carrying out global maximum pooling (8) and global average pooling (9) on a channel shaft to obtain two-dimensional spatial characteristics, splicing the two-dimensional spatial characteristics into a characteristic diagram, secondly, reducing the dimension through a convolution kernel (10) of 7×7, and finally, activating (11) through a sigmoid function to generate channel attention mapping, wherein the spatial attention module is used for enhancing the characteristic extraction capacity of the spatial dimension; the output is connected to a global averaging pooling layer (12) and a gender input module is created, which takes as input binary gender information (13) which is "0" or "1", male is "1", and female is "0". The output image characteristic information and sex characteristic information are connected in series through a dense connection layer (14) with 32 neurons and activated by ReLU, two layers of dense connection layers (15, 16) are connected, each layer is fed by a dense connection layer with 1024 neurons and a dense connection layer with 1 neuron (17) activated by linearity, and the last layer is used for predicting bone age and obtaining a bone age assessment value (18).
Continuously adjusting parameters of the model in the training process, training the model to enable the model to learn the characteristics of more samples to obtain training models, inputting verification set data into each training model, predicting the data in the verification set by using the model, recording the accuracy of each training model, selecting the parameters corresponding to the training model with the best effect, and determining a bone age regression model; inputting test set data into a bone age regression model for evaluating the performance of the model, adding images to be evaluated into a training set after bone age evaluation is carried out on the images to be evaluated, repeating the steps, fine-tuning training model parameters, continuously optimizing the structure of the training model, improving the generalization capability of the training model, and taking the model with the best bone age evaluation effect as a bone age regression standard model;
specific example 1:
1. quantification of bone age X-ray images into digital images
X-ray hand bone images of the left hand between 0 and 18 years old were collected from each hospital. In addition, the later marked images to be evaluated are continuously added into the hand bone image sample library.
2. Pretreatment of
And (3) carrying out contrast enhancement on the bone age picture collected in the step (1), and carrying out contour marking on the hand bone region of the sample by using a tool labelme. After the processing is finished, all the images are divided into a training set, a verification set and a test set according to the proportion of 7:2:1.
3. Hand bone segmentation
And 3.1, inputting the images in the training set processed in the step 2 into a Mask R-CNN frame, automatically learning the characteristics of the hand bone region by utilizing ResNet-50 and a characteristic pyramid network, and correcting and extracting the deviation between the characteristics and the region of interest by using a RoIAlign layer, so that the hand bone region can be accurately segmented by a subsequent full convolution network. In the training process, parameters of the model are required to be continuously adjusted, the structure of the model is optimized, and the model can learn the characteristics of more samples through training.
And 3.2, inputting data of the verification set into each model after training a plurality of models, predicting the data in the verification set by using the models, recording the accuracy of each model, selecting parameters corresponding to the model with the best effect, and determining the hand bone segmentation model.
3.3 inputting the test set data into the hand bone segmentation model for evaluating the performance of the model and the segmentation capability of the target region. After the images to be evaluated are segmented, the images are added into a training set, the work is repeated, the model parameters are finely adjusted, the structure of the model is continuously optimized, the generalization capability of the model is improved, and finally the hand bone segmentation standard model is generated.
And 3.4, performing image fusion on the hand bone mask segmented by the hand bone segmentation standard model and the original hand bone image to generate a hand bone image with background information removed.
4. Bone age assessment
4.1 inputting the hand bone image obtained in the step 3 with the background information removed into an improved Xreception model, and extracting the bottom layer characteristics of the image through an original Xreception network; then refining feature mapping from two independent dimensions of a channel and a space through a channel attention module and a space attention module, and extracting more effective features; and finally, balancing the difference of the hand bone development degrees of different sexes through the sex input module to obtain a bone age evaluation value. In the training process, parameters of the model are required to be continuously adjusted, the structure of the model is optimized, and the model can learn the characteristics of more samples through training.
And 4.2, inputting data of the verification set into each model after training a plurality of models, predicting the data in the verification set by using the models, recording the accuracy of each model, selecting parameters corresponding to the model with the best effect, and determining the bone age regression model.
4.3 the test set data was input into an age regression model to evaluate the performance of the model. After the images to be evaluated are subjected to bone age evaluation, the images are added into a training set, the work is repeated, fine adjustment is carried out on model parameters, the structure of the model is continuously optimized, the generalization capability of the model is improved, and finally a bone age regression standard model is generated.
4.4, performing bone age assessment on the non-assessed image by using a bone age regression standard model to obtain a bone age assessment value.
Specific example 2:
as shown in fig. 3 to 5, the specific steps of the present invention are as follows:
step one: a user firstly shoots left-hand bone images of children by using an X-ray bone density bone age tester, all the images are converted into PNG format on the bone age tester, then the images are imported into the system, the system automatically performs operations such as contrast enhancement, hand bone mask segmentation and image fusion on the images, and hand bone images with background information removed are generated and stored in a specified folder.
Step two: the user clicks the "select" button to load the X-ray image of the hand bone, below which is the file path, as shown in fig. 3.
Step three: the user clicks the "load model" button to load the bone age regression model, and "model loaded is complete.
Step four: the user clicks the "start recognition" button, and the evaluation result is displayed, which is divided into a recognition result and a prediction time, as shown in fig. 5.
The Mask R-CNN is used for dividing the hand area, unnecessary background information can be filtered, noise interference is reduced, bone age assessment technology is combined with deep learning, high-precision low-time-delay bone age assessment is achieved, a channel attention module and a space attention module are embedded behind an original Xattention network, feature mapping can be refined from two independent dimensions of a channel and a space, and more effective features are extracted; the embedded gender information can enable the network to balance the difference of the hand bone development degrees of different sexes, and fine granularity attention is improved. The invention solves the problems that the current hand bone X-ray image evaluation efficiency is too low, the hand bone X-ray images cannot be processed in batches, and the like, and relieves the difficulty of reading the film by doctors. The invention can effectively and automatically evaluate the bone age of the X-ray image of the hand bone of the child, and provides favorable support for the subsequent evaluation of the bone development maturity.
The invention also provides a bone age assessment system for children based on deep learning, which comprises the following steps:
the image acquisition module is used for acquiring a hand bone image to be evaluated;
the preprocessing module is used for preprocessing the hand bone image to be evaluated;
the first input module is used for inputting the preprocessed image into the trained hand bone segmentation model to obtain a segmented hand bone region mask;
the fusion module is used for fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed;
and the second input module is used for inputting the hand bone image with the background information removed into a trained bone age assessment model to assess the bone age.
Wherein, the preprocessing module specifically includes:
and the equalization unit is used for enhancing the local contrast by utilizing histogram equalization on the hand bone image to be evaluated.
The bone age assessment system for children provided by the invention further comprises:
the sample library establishing module is used for establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples;
the first data set construction module is used for processing the hand bone image sample and marking hand bone contours to construct a first data set, and the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set;
the first training module is used for training the Mask R-CNN model through the hand bone image training set to obtain a hand bone segmentation model;
and the first optimization module is used for optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set.
The bone age assessment system for children provided by the invention further comprises:
the segmentation module is used for segmenting the first data set through a hand bone segmentation model to obtain a hand bone region mask sample;
the second data set is constructed and used for fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct the second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information;
the second training module is used for training the improved Xattention model through the hand bone image training set after removing the background information to obtain a bone age assessment model;
and the second optimization module is used for optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (4)
1. A method for estimating bone age of children based on deep learning, which is characterized by comprising the following steps:
acquiring a hand bone image to be evaluated;
preprocessing a hand bone image to be evaluated;
inputting the preprocessed image into a trained hand bone segmentation model to obtain a segmented hand bone region mask;
fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed;
inputting the hand bone image with the background information removed into a trained bone age assessment model, and assessing the bone age;
the training process of the hand bone segmentation model specifically comprises the following steps:
establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples;
processing the hand bone image sample and marking hand bone contours to construct a first data set, wherein the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set;
training the Mask R-CNN model through a hand bone image training set to obtain a hand bone segmentation model;
optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set;
the training process of the bone age assessment model specifically comprises the following steps:
dividing the first data set through a hand bone dividing model to obtain a hand bone region mask sample;
fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct a second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information;
training the improved Xreception model through the hand bone image training set with the background information removed to obtain a bone age assessment model;
optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information;
the improved Xreception model is as follows: after the original Xreception network is output, firstly, channel information of a global maximum pooling and global average pooling aggregation feature diagram is used for generating two different channel features, secondly, the channel features are fed into a multi-layer perceptron, the output of the multi-layer perceptron is added element by element, and finally, channel attention mapping is generated through activating by a sigmoid function, so that a channel attention module is obtained; after inputting the characteristics output by the channel attention module, firstly, carrying out global maximum pooling and global average pooling operation on a channel shaft to obtain two-dimensional space characteristics, splicing the two-dimensional space characteristics into a characteristic diagram, secondly, reducing the dimension by a convolution kernel of 7 multiplied by 7, and finally, activating to generate channel attention mapping by a sigmoid function, wherein the space attention module is used for generating the channel attention map; the output is connected into a global average pooling layer, then a gender input module is created, binary gender information is used as input, the binary gender information is 0 or 1, the male is 1, and the female is 0; the output image characteristic information and sex characteristic information are connected in series through a dense connecting layer with 32 neurons and activated by ReLU, two layers of dense connecting layers are connected, each layer is fed by a ReLU activating layer and a dropout layer which are tightly connected with 1024 neurons, and the last layer is formed by a dense connecting layer with 1 neuron and activated linearly and used for predicting bone age and obtaining a bone age assessment value.
2. The deep learning-based bone age assessment method for children according to claim 1, wherein the preprocessing of the hand bone image to be assessed specifically comprises:
the hand bone image to be evaluated is locally contrast enhanced using histogram equalization.
3. A deep learning based bone age assessment system for children, comprising:
the image acquisition module is used for acquiring a hand bone image to be evaluated;
the preprocessing module is used for preprocessing the hand bone image to be evaluated;
the first input module is used for inputting the preprocessed image into the trained hand bone segmentation model to obtain a segmented hand bone region mask;
the fusion module is used for fusing the segmented hand bone region mask with the preprocessed image to obtain a hand bone image with background information removed;
the second input module is used for inputting the hand bone image with the background information removed into a trained bone age assessment model to assess the bone age;
further comprises:
the sample library establishing module is used for establishing a hand bone image sample library; the hand bone image sample library comprises a plurality of hand bone image samples;
the first data set construction module is used for processing the hand bone image sample and marking hand bone contours to construct a first data set, and the data set comprises a hand bone image training set, a hand bone image verification set and a hand bone image test set;
the first training module is used for training the Mask R-CNN model through the hand bone image training set to obtain a hand bone segmentation model;
the first optimization module is used for optimizing the hand bone segmentation model through the hand bone image test set and the hand bone image verification set;
further comprises:
the segmentation module is used for segmenting the first data set through a hand bone segmentation model to obtain a hand bone region mask sample;
the second data set is constructed and used for fusing the segmented hand bone region mask sample with the preprocessed hand bone image sample to construct the second data set; the second data set comprises a hand bone image training set after removing background information, a hand bone image test set after removing background information and a hand bone image verification set after removing background information;
the second training module is used for training the improved Xattention model through the hand bone image training set after removing the background information to obtain a bone age assessment model;
the second optimization module is used for optimizing the bone age assessment model through the hand bone image test set after removing the background information and the hand bone image verification set after removing the background information;
the improved Xreception model is as follows: after the original Xreception network is output, firstly, channel information of a global maximum pooling and global average pooling aggregation feature diagram is used for generating two different channel features, secondly, the channel features are fed into a multi-layer perceptron, the output of the multi-layer perceptron is added element by element, and finally, channel attention mapping is generated through activating by a sigmoid function, so that a channel attention module is obtained; after inputting the characteristics output by the channel attention module, firstly, carrying out global maximum pooling and global average pooling operation on a channel shaft to obtain two-dimensional space characteristics, splicing the two-dimensional space characteristics into a characteristic diagram, secondly, reducing the dimension by a convolution kernel of 7 multiplied by 7, and finally, activating to generate channel attention mapping by a sigmoid function, wherein the space attention module is used for generating the channel attention map; the output is connected into a global average pooling layer, then a gender input module is created, binary gender information is used as input, the binary gender information is 0 or 1, the male is 1, and the female is 0; the output image characteristic information and sex characteristic information are connected in series through a dense connecting layer with 32 neurons and activated by ReLU, two layers of dense connecting layers are connected, each layer is fed by a ReLU activating layer and a dropout layer which are tightly connected with 1024 neurons, and the last layer is formed by a dense connecting layer with 1 neuron and activated linearly and used for predicting bone age and obtaining a bone age assessment value.
4. The deep learning based bone age assessment system of claim 3, wherein the preprocessing module specifically comprises:
and the equalization unit is used for enhancing the local contrast by utilizing histogram equalization on the hand bone image to be evaluated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110564662.1A CN113298780B (en) | 2021-05-24 | 2021-05-24 | Deep learning-based bone age assessment method and system for children |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110564662.1A CN113298780B (en) | 2021-05-24 | 2021-05-24 | Deep learning-based bone age assessment method and system for children |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113298780A CN113298780A (en) | 2021-08-24 |
| CN113298780B true CN113298780B (en) | 2023-09-12 |
Family
ID=77324126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110564662.1A Active CN113298780B (en) | 2021-05-24 | 2021-05-24 | Deep learning-based bone age assessment method and system for children |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298780B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661052B (en) * | 2022-10-13 | 2023-09-12 | 高峰医疗器械(无锡)有限公司 | Alveolar bone detection method, device, equipment and storage medium |
| TWI838104B (en) * | 2023-02-01 | 2024-04-01 | 宏碁智醫股份有限公司 | X-ray image enhancement system and x-ray image enhancement method |
| CN116433607B (en) * | 2023-03-17 | 2024-03-15 | 四川文理学院 | Bone age assessment method and system for X-ray images of hand bones of children based on double weighted fusion of key region features |
| CN116342516B (en) * | 2023-03-17 | 2024-04-16 | 四川文理学院 | Model integration-based method and system for assessing bone age of X-ray images of hand bones of children |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107767376A (en) * | 2017-11-02 | 2018-03-06 | 西安邮电大学 | X-ray film stone age Forecasting Methodology and system based on deep learning |
| WO2018057714A1 (en) * | 2016-09-21 | 2018-03-29 | The General Hospital Corporation | Systems, methods and media for automatically generating a bone age assessment from a radiograph |
| CN109345508A (en) * | 2018-08-31 | 2019-02-15 | 北京航空航天大学 | A kind of Assessing Standards For Skeletal method based on two stages neural network |
| CN111080579A (en) * | 2019-11-28 | 2020-04-28 | 杭州电子科技大学 | Bone age assessment method for realizing image segmentation and classification based on deep learning |
| CN111161254A (en) * | 2019-12-31 | 2020-05-15 | 上海体育科学研究所 | Bone age prediction method |
| CN111985370A (en) * | 2020-08-10 | 2020-11-24 | 华南农业大学 | Crop pest and disease fine-grained identification method based on improved mixed attention module |
| CN116342516A (en) * | 2023-03-17 | 2023-06-27 | 四川文理学院 | Model integration-based method and system for assessing bone age of X-ray images of hand bones of children |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6999812B2 (en) * | 2018-08-01 | 2022-01-19 | 中國醫藥大學附設醫院 | Bone age evaluation and height prediction model establishment method, its system and its prediction method |
| US11367181B2 (en) * | 2018-12-29 | 2022-06-21 | Shanghai United Imaging Intelligence Co., Ltd. | Systems and methods for ossification center detection and bone age assessment |
-
2021
- 2021-05-24 CN CN202110564662.1A patent/CN113298780B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018057714A1 (en) * | 2016-09-21 | 2018-03-29 | The General Hospital Corporation | Systems, methods and media for automatically generating a bone age assessment from a radiograph |
| CN107767376A (en) * | 2017-11-02 | 2018-03-06 | 西安邮电大学 | X-ray film stone age Forecasting Methodology and system based on deep learning |
| CN109345508A (en) * | 2018-08-31 | 2019-02-15 | 北京航空航天大学 | A kind of Assessing Standards For Skeletal method based on two stages neural network |
| CN111080579A (en) * | 2019-11-28 | 2020-04-28 | 杭州电子科技大学 | Bone age assessment method for realizing image segmentation and classification based on deep learning |
| CN111161254A (en) * | 2019-12-31 | 2020-05-15 | 上海体育科学研究所 | Bone age prediction method |
| CN111985370A (en) * | 2020-08-10 | 2020-11-24 | 华南农业大学 | Crop pest and disease fine-grained identification method based on improved mixed attention module |
| CN116342516A (en) * | 2023-03-17 | 2023-06-27 | 四川文理学院 | Model integration-based method and system for assessing bone age of X-ray images of hand bones of children |
Non-Patent Citations (1)
| Title |
|---|
| adapting mask-rcnn for automatic nucleus segmentation;jeremiah W.johnson;arXiv;1-7 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113298780A (en) | 2021-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113298780B (en) | Deep learning-based bone age assessment method and system for children | |
| CN112132817B (en) | Retina blood vessel segmentation method for fundus image based on mixed attention mechanism | |
| CN110399929B (en) | Fundus image classification method, fundus image classification apparatus, and computer-readable storage medium | |
| CN111784671B (en) | Pathological image focus region detection method based on multi-scale deep learning | |
| WO2020024127A1 (en) | Bone age assessment and height prediction model, system thereof and prediction method therefor | |
| CN108335303B (en) | Multi-scale palm skeleton segmentation method applied to palm X-ray film | |
| TWI684997B (en) | Establishing method of bone age assessment and height prediction model, bone age assessment and height prediction system, and bone age assessment and height prediction method | |
| CN109948522B (en) | X-ray hand bone maturity interpretation method based on deep neural network | |
| CN110503635B (en) | Hand bone X-ray film bone age assessment method based on heterogeneous data fusion network | |
| CN112508953B (en) | Meningioma rapid segmentation qualitative method based on deep neural network | |
| CN112819821B (en) | Cell nucleus image detection method | |
| CN110731773A (en) | abnormal electrocardiogram screening method based on fusion of global and local depth features of electrocardiogram | |
| CN112489029A (en) | Medical image segmentation method and device based on convolutional neural network | |
| CN113643297B (en) | Computer-aided age analysis method based on neural network | |
| CN111383222A (en) | Intervertebral disc MRI image intelligent diagnosis system based on deep learning | |
| Gulati et al. | Comparative analysis of deep learning approaches for the diagnosis of diabetic retinopathy | |
| CN116452592B (en) | Method, device and system for constructing brain vascular disease AI cognitive function evaluation model | |
| CN113052227A (en) | Pulmonary tuberculosis identification method based on SE-ResNet | |
| CN116778587A (en) | Living body age deducing method, equipment and storage medium based on deep learning | |
| CN116993639A (en) | Visible light and infrared image fusion method based on structural re-parameterization | |
| Xian et al. | Automatic tongue image quality assessment using a multi-task deep learning model | |
| CN116071373A (en) | Automatic U-net model tongue segmentation method based on fusion PCA | |
| CN112614092A (en) | Spine detection method and device | |
| CN112365474A (en) | Blood vessel extraction method, device, electronic equipment and storage medium | |
| CN115019045B (en) | Small data thyroid ultrasound image segmentation method based on multi-component neighborhood |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |