CN109544518A - A kind of method and its system applied to the assessment of skeletal maturation degree - Google Patents
A kind of method and its system applied to the assessment of skeletal maturation degree Download PDFInfo
- Publication number
- CN109544518A CN109544518A CN201811319500.6A CN201811319500A CN109544518A CN 109544518 A CN109544518 A CN 109544518A CN 201811319500 A CN201811319500 A CN 201811319500A CN 109544518 A CN109544518 A CN 109544518A
- Authority
- CN
- China
- Prior art keywords
- skeleton character
- bone
- feature
- assessment
- skeleton
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to bone discrimination technology field, in particular to a kind of method and its system applied to the assessment of skeletal maturation degree；The present invention is by carrying out feature extraction to bone image using transfer learning method, and using being generated based on skeleton character and the semi-supervised generation confrontation network of feature identification identifies solve the problems such as excessive parameter amount in the classification and identification of high-resolution bone input picture and over-fitting to skeletal maturation degree.
Description
Technical field
The present invention relates to bone discrimination technology field, in particular to it is a kind of applied to skeletal maturation degree assessment method and its
System.
Background technique
Bone growth Stage Classification is one of the important indicator for measuring growth in humans's maturity, can be by few to blueness
The measurement prediction its growth and development potentiality of year skeletal maturation degree or to the operating time of specified disease and disease development into
Row auxiliary prediction.Traditional skeletal maturation degree appraisal procedure is that expert is compared by read tablet with map or expert is to each
The development degree of bone epiphysis is given a mark, and obtains the prediction of skeletal maturation degree by superposition metering.Conventional method height according to
Rely the experience in specialist, and carry out artificial read tablet between different physicians with very big error, artificial read tablet when
Between cost and human cost it is very high.
The Accessory Diagnostic Model Based of bone growth Stage evaluation is carried out dependent on a large amount of using existing depth learning technology
There is mark training data, and extremely rely on effective label of high quality, carrying out the assessment of skeletal maturation degree using deep learning needs
Largely to have mark bone image sample, and data mark need of work specialist team spend flood tide it is artificial at
This, the label of high quality relies on the abundant degree of doctor's team experience, and needs to the mark of bone image costly
Energy, so that effective training sample be made to be difficult to obtain.In addition, the mark need of work consumption for huge data set is huge
Time cost, and mark quality and restricted vulnerable to doctor's energy and physical factor, for the mark work of a large amount of sample bones
The mark quality of sample is difficult to ensure for work.
Summary of the invention
The invention mainly solves the technical problem of providing a kind of methods applied to the assessment of skeletal maturation degree, pass through utilization
Transfer learning method carries out feature extraction to bone image, and semi-supervised with feature identification using being generated based on skeleton character
It generates confrontation network and identifies solve the ginseng in the classification and identification of high-resolution bone input picture to skeletal maturation degree
The problems such as quantity is excessive and over-fitting also provides a kind of system applied to the assessment of skeletal maturation degree.
In order to solve the above technical problems, one technical scheme adopted by the invention is that: it provides a kind of applied to skeletal maturation
Spend the method for assessment, wherein include the following steps:
Step S1, the intensive connection network model with image characteristics extraction ability is established, to intensive connection network model
After carrying out pre-training, the intensive link block in the intensive connection network model is subjected to module transfer, generates and can get bone
The skeleton character extraction module of image primary features；
Step S2, by the one-dimensional noise inputs with Gaussian Profile into multi-channel feature generator, the multichannel is special
It levies and generates bone identical with the characteristic size size that the skeleton character extraction module extracts in generator in each channel
Characteristic pattern, then each skeleton character figure in the multi-channel feature generator in each channel is subjected to cascade splicing, splice
Dimension skeleton character figure identical with the characteristic dimension that the skeleton character extraction module extracts is obtained afterwards, and the bone is special
Sign figure output；
Step S3, true picture is obtained into bone image primary features and described more by the skeleton character extraction module
The skeleton character figure that channel characteristics generator generates is input in capsule skeleton character arbiter, to obtain retaining space position
The bone instantiated feature vector of confidence breath, obtains classification corresponding to the bone instantiated feature vector, then knows that this is true
The skeletal maturation degree assessment result of image.
As an improvement of the present invention, further include step S4, construct bone spy in the multi-channel feature generator
The loss function for levying figure, is then defined the loss function using cross entropy in the capsule skeleton character arbiter,
The bone instantiated feature vector is extracted in the loss function, then and is obtained corresponding to the bone instantiated feature vector
Classification then knows the skeletal maturation degree assessment result of the true picture.
As a further improvement of the present invention, step S1 includes:
Step S11, intensive connection network model is established, using the training set image of pre-training data set as input picture；
Step S12, input picture is subjected to feature extraction, output prediction knot by several intensive link blocks and transition zone
Fruit.
As a further improvement of the present invention, step S1 further include:
Step S13, loss letter is calculated using the label of the prediction result and the true picture of the pre-training data set
Number, then backpropagation optimize network parameter, are trained by the network parameter in the intensive connection network model, make described
Intensive connection network model has image characteristics extraction ability；
Step S14, the pre-training data set is subjected to module migration, generates the bone that can get bone image primary features
The true picture of bone is input in the skeleton character extraction module by bone characteristic extracting module, and it is primary to obtain bone image
Feature.
As a further improvement of the present invention, step S2 includes:
Step S21, by the one-dimensional noise inputs with Gaussian Profile into the multi-channel feature generator；
Step S22, for the feature generator in each channel in the multi-channel feature generator, by the noise of input
By several warp laminations, the skeleton character figure of generation is successively amplified after the deconvolution of warp lamination, makes generation
Every skeleton character figure is identical as the characteristic dimension that the skeleton character extraction module extracts.
As a further improvement of the present invention, step S2 further include:
Step S23, the skeleton character for generating the feature generator in each channel of the multi-channel feature generator
Figure carries out cascade splicing, and dimension bone identical with the characteristic dimension that the skeleton character extraction module extracts is obtained after splicing
Characteristic pattern；
Step S24, spliced skeleton character figure is input in capsule skeleton character arbiter.
As a further improvement of the present invention, step S3 includes:
Step S31, the bone image primary features obtained true picture by the skeleton character extraction module and institute
The skeleton character figure for stating the generation of multi-channel feature generator is input in the capsule skeleton character arbiter；
Step S32, spy is extracted to the bone image primary features and skeleton character figure by several convolutional layers
Levy vector.
As a further improvement of the present invention, step S3 further include:
Step S33, the feature vector of extraction is inputted into capsule layer, to obtain the bone example of retaining space location information
Change feature vector；
Step S34, obtain classification corresponding to the bone instantiated feature vector, then known to the true picture bone at
Ripe degree assessment result.
A kind of system applied to the assessment of skeletal maturation degree, wherein include:
Pre-training bone image characteristic module, for establishing intensive connection network model, to the intensive connection network mould
Type carries out pre-training to generate image characteristics extraction ability；
Skeleton character extraction module, for obtaining bone image primary features by true picture；
Multi-channel feature generator, for generating the characteristic size size phase extracted with the skeleton character extraction module
Same skeleton character figure simultaneously carries out cascade splicing, and the feature that dimension is extracted with the skeleton character extraction module is obtained after splicing
The identical skeleton character figure of dimension；
Capsule skeleton character arbiter, for obtaining the bone instantiated feature vector of retaining space location information.
It as an improvement of the present invention, further include compensating module, for being constructed in the multi-channel feature generator
The loss function of skeleton character figure then carries out the loss function using cross entropy in the capsule skeleton character arbiter
Definition.
The beneficial effects of the present invention are: compared with prior art, the present invention is by utilizing transfer learning method to skeletal graph
As carry out feature extraction, and using based on skeleton character generate and feature identification semi-supervised generation confrontation network to bone at
Ripe degree identified, solves that parameter amount in the classification and identification of high-resolution bone input picture is excessive and over-fitting etc.
Problem.
Detailed description of the invention
Fig. 1 is the step block diagram of the method for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 2 is the embodiment one of the method for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 3 is the step block diagram of the step S1 of the method for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 4 is the step block diagram of the step S2 of the method for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 5 is the step block diagram of the step S3 of the method for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 6 is the structural schematic diagram of ulna and radius；
Fig. 7 is that each skeletal maturation of ulna and radius spends stage schematic diagram in the present invention；
Fig. 8 is the structural block diagram of the system for being applied to the assessment of skeletal maturation degree of the invention；
Fig. 9 is the calculation mechanism block diagram of the capsule layer of capsule skeleton character arbiter in embodiment two in the present invention；
Figure 10 is the route selection algorithm block diagram of capsule skeleton character arbiter in embodiment two in the present invention.
Specific embodiment
In the method for existing depth learning technology processing bone growth Stage evaluation problem, Most models are sorted
The mode that bone image feature is extracted in journey generallys use convolution and pondization operation, but convolutional neural networks are in processing image
When feature, the specific directional information of feature can not be effectively detected, so that it is lost the spatial information of feature, this space
The missing of information converting finally influences whether the performance of skeletal maturation degree prediction model.So carrying out ulna using convolutional network
Radius growth phase classification task in, only to the skeleton data of input using data enhance etc. modes, to input data into
The spatial alternation of row multiplicity, can just promote convolutional network to the study of spatial variations, accomplish to positions change informations such as rotations more
Good identification, but convolutional network its substantially there are problems that the spatial level information of image easy to be lost and directional information so that
It is insensitive to the spatial transformation information identification of same semantic content, to influence the precision of classification.
The good behaviour of deep learning skeletal maturation degree assessment models depend on a large amount of high quality training sample, bone at
The accuracy of ripe degree prediction model relies on a large amount of training data, if possessing enough training datas is influence depth study mould
Can type obtain an important factor for satisfactory result.When being predicted using traditional convolutional neural networks the bone growth stage,
The pondization of use operates spatial positional information easy to be lost and loses, and reaches high-precision result and generally requires training set with abundant
Training sample so that neural network sufficiently learns various diversified shape posture features, and appoints the assessment of skeletal maturation degree
Business, image pattern is due to patient privacy etc., it is difficult to obtain large batch of training sample, therefore utilize general convolutional network
When model carries out prediction task, nicety of grading is influenced since training samples number is limited.
A kind of existing stone age automatic identification system (Chinese patent application based on improved residual error network
CN201711274742.3), this method takes pretreated bone image using sliding window progress frames images, defeated to obtain
Enter image.Hereafter the multiple images that sliding window takes same frames images are separately input to the residual error bone age classification net trained
In network, which carries out feature extraction to input picture by several residual error characteristic extracting modules.Finally by softmax
Function classifies to each image that sliding window frame takes, and obtains multiple classification results, finally takes the highest classification knot of frequency
Fruit is predicted as the stone age of the original image.
Also with other similar schemes of the invention: Chinese patent application CN201711125692.2, the invention propose one
Kind contains the automation bone age assessment method of bone interest region detection and bony areas classification.This method passes through training
RPN network and Fast R-CNN network extract the interest region of target bone, the bone in the interest region that target detection is obtained
Image is input in disaggregated model, and disaggregated model is extracted and adopted to input picture feature by using convolution and pond
Sample, final output image belong to the final stone age prediction of probability final output of all categories.
In addition, to propose a kind of X-ray film stone age based on deep learning pre- by Chinese patent application CN201711065065.4
Survey method, after this method is by carrying out pretreatment and data enhancing for hand x-ray picture, by utilizing neural network opponent
Bone region is split, by X-ray film image palm area and background area be labeled as image positive sample block and the negative sample of image
Sample after label is input to neural network and is trained by this block, using the result of acquisition as segmentation template.To segmentation template
After being handled, surrounding small area interference region is rejected, the musculature area of palm segmentation template area is rejected, is gone
Except the hand bone X-ray film image after noise.The neural networks such as GoogLeNet or Alexnet are used followed by transfer learning method
Model carries out bone age classification to image after denoising.
As shown in Figures 1 to 10, the present invention provides a kind of method applied to the assessment of skeletal maturation degree, including walks as follows
It is rapid:
Step S1, the intensive connection network model with image characteristics extraction ability is established, to intensive connection network model
After carrying out pre-training, the intensive link block in the intensive connection network model is subjected to module transfer, generates and can get bone
The skeleton character extraction module of image primary features；
Step S2, by the one-dimensional noise inputs with Gaussian Profile into multi-channel feature generator, the multichannel is special
It levies and generates bone identical with the characteristic size size that the skeleton character extraction module extracts in generator in each channel
Characteristic pattern, then each skeleton character figure in the multi-channel feature generator in each channel is subjected to cascade splicing, splice
Dimension skeleton character figure identical with the characteristic dimension that the skeleton character extraction module extracts is obtained afterwards, and the bone is special
Sign figure output；
Step S3, true picture is obtained into bone image primary features and described more by the skeleton character extraction module
The skeleton character figure that channel characteristics generator generates is input in capsule skeleton character arbiter, to obtain retaining space position
The bone instantiated feature vector of confidence breath, obtains classification corresponding to the bone instantiated feature vector, then knows that this is true
The skeletal maturation degree assessment result of image.
Wherein, as shown in figure 3, step S1 includes:
Step S11, intensive connection network model is established, using the training set image of pre-training data set as input picture；
Step S12, input picture is subjected to feature extraction, output prediction knot by several intensive link blocks and transition zone
Fruit；
Step S13, loss letter is calculated using the label of the prediction result and the true picture of the pre-training data set
Number, then backpropagation optimize network parameter, are trained by the network parameter in the intensive connection network model, make described
Intensive connection network model has image characteristics extraction ability；
Step S14, the pre-training data set is subjected to module migration, generates the bone that can get bone image primary features
The true picture of bone is input in the skeleton character extraction module by bone characteristic extracting module, and it is primary to obtain bone image
Feature.
In the present invention, as shown in figure 4, step S2 includes:
Step S21, by the one-dimensional noise inputs with Gaussian Profile into the multi-channel feature generator；
Step S22, for the feature generator in each channel in the multi-channel feature generator, by the noise of input
By several warp laminations, the skeleton character figure of generation is successively amplified after the deconvolution of warp lamination, makes generation
Every skeleton character figure is identical as the characteristic dimension that the skeleton character extraction module extracts；
Step S23, the skeleton character for generating the feature generator in each channel of the multi-channel feature generator
Figure carries out cascade splicing, and dimension bone identical with the characteristic dimension that the skeleton character extraction module extracts is obtained after splicing
Characteristic pattern；
Step S24, spliced skeleton character figure is input in capsule skeleton character arbiter.
In the present invention, as shown in figure 5, step S3 includes:
Step S31, the bone image primary features obtained true picture by the skeleton character extraction module and institute
The skeleton character figure for stating the generation of multi-channel feature generator is input in the capsule skeleton character arbiter；
Step S32, spy is extracted to the bone image primary features and skeleton character figure by several convolutional layers
Levy vector；
Step S33, the feature vector of extraction is inputted into capsule layer, to obtain the bone example of retaining space location information
Change feature vector；
Step S34, obtain classification corresponding to the bone instantiated feature vector, then known to the true picture bone at
Ripe degree assessment result.
The present invention provides one embodiment one, as shown in Fig. 2, the embodiment it is a kind of applied to skeletal maturation degree assessment
Method includes the following steps:
Step S1, the intensive connection network model with image characteristics extraction ability is established, to intensive connection network model
After carrying out pre-training, the intensive link block in the intensive connection network model is subjected to module transfer, generates and can get bone
The skeleton character extraction module of image primary features；
Step S2, by the one-dimensional noise inputs with Gaussian Profile into multi-channel feature generator, the multichannel is special
It levies and generates bone identical with the characteristic size size that the skeleton character extraction module extracts in generator in each channel
Characteristic pattern, then each skeleton character figure in the multi-channel feature generator in each channel is subjected to cascade splicing, splice
Dimension skeleton character figure identical with the characteristic dimension that the skeleton character extraction module extracts is obtained afterwards, and the bone is special
Sign figure output；
Step S3, true picture is obtained into bone image primary features and described more by the skeleton character extraction module
The skeleton character figure that channel characteristics generator generates is input in capsule skeleton character arbiter, to obtain retaining space position
The bone instantiated feature vector of confidence breath, obtains classification corresponding to the bone instantiated feature vector, then knows that this is true
The skeletal maturation degree assessment result of image；
Step S4, the loss function that skeleton character figure is constructed in the multi-channel feature generator, then in the glue
The loss function is defined using cross entropy in capsule skeleton character arbiter, it is real that the bone is extracted in the loss function
Exampleization feature vector, then and obtain classification corresponding to the bone instantiated feature vector, then known to the true picture bone
Maturity assessment result.
The present invention provides embodiment two, is the maturity appraisal procedure for ulna and radius, currently, skeletal maturation degree is commented
Estimate an important indicator as growth and development degree and the auxiliary identification of growth and development relevant issues, has in clinical medicine
Important function, in the treatment for some related diseases such as idiopathic scoliosis and associated bone malformation disorders, to bone
Bone, which carries out maturity assessment, can assist doctor to identify subjects bones' growth phase, and doctor is made to determine golden hour, and
Clinical observation interval.For needing to carry out the patient of operative treatment, auxiliary doctor determine right times be supported treatment or
Terminate support treatment.Identification for skeletal maturation degree, since wrist and palm part bone block number are more in skeleton, and
And bone connection growth plate is obvious, include contains much information, and very convenient for the acquisition of skeletal image, leads in the world
It is often used palm part and carries out the assessment of skeletal maturation degree, as shown in Figure 7.
Each skeletal maturation of radius spends the stage and is described as follows table at present:
Each skeletal maturation of ulna spends the stage and is described as follows table at present:
Skeletal maturation degree growth phase classification provides children and Juvenile development summit of growth phase and withholding period
Relationship, carrying out clinical decision for doctor has important meaning, and the Stage Classification framework elaborates each growth phase bone
The correlation properties and age and sexual development degree of the corresponding skeletal form of bone maturity and subject.
As shown in figure 8, embodiment two provides a kind of system applied to the assessment of skeletal maturation degree, comprising:
Pre-training bone image characteristic module, for establishing intensive connection network model, to the intensive connection network mould
Type carries out pre-training to generate image characteristics extraction ability；
Skeleton character extraction module, for obtaining bone image primary features by true picture；
Multi-channel feature generator, for generating the characteristic size size phase extracted with the skeleton character extraction module
Same skeleton character figure simultaneously carries out cascade splicing, and the feature that dimension is extracted with the skeleton character extraction module is obtained after splicing
The identical skeleton character figure of dimension；
Capsule skeleton character arbiter, for obtaining the bone instantiated feature vector of retaining space location information；
Compensating module then exists for constructing the loss function of skeleton character figure in the multi-channel feature generator
The loss function is defined using cross entropy in the capsule skeleton character arbiter.
Compared with prior art, present invention advantage specific as follows:
1, semi-supervised learning method is introduced into skeletal maturation degree classification task, it is only necessary to which fraction sample is concentrated to data
Originally it is labeled, reduces the demand to mark training sample, reduce mark many and diverse to training data in traditional algorithm
The workload of work reduces the duty cycle of skeletal maturation degree identification mission, improves the whole of skeletal maturation degree identification mission
Body efficiency.
2, capsule network structure is introduced into stone age identification technology, improves skeleton character arbiter to skeleton character sky
Between information identify susceptibility, can identify the angle of bone image feature, the deformation datas such as position, to improve skeletal maturation
Spend accuracy of identification.
3, efficiently image characteristics extraction mode can be passed through based on the improved capsule skeleton character arbiter of capsule thought
And retain image space change in location information, relative to traditional skeletal maturation degree disaggregated model, present invention reduces nerves
To the demand of training sample amount when network processes skeletal maturation degree identification mission, pass through relatively small number of sample compared to lower
Reach corresponding degree of precision.
4, semi-supervised generation is input to by authentic specimen by the way of skeleton character extraction to high-resolution bone image
Fight network in, while by multi-channel feature generator substitute tradition generate confrontation network directly generate image in the way of, subtract
The problem of small high-definition picture is to traditional network bring senior staff officer's quantity, and do not need relative to general network to original
Beginning image carries out scaling, avoids the loss of input picture feature.
Meanwhile embodiment two provides a kind of method applied to the assessment of skeletal maturation degree,
(1), the building of pre-training bone image characteristic extracting module
Step a1: intensive connection network model is established, using the training set image of pre-training data set as input；
Step a2: by input picture by several intensive link blocks and transition zone (as shown in figure 8, the intensive connection of application
Block 1, intensive link block 2, intensive link block 3, convolutional layer, transition zone) feature extraction is carried out, export prediction result；
Step a3: it calculates and damages using the prediction result and pre-training data set true picture label of intensive connection network model
Function is lost, loss function is minimized, backpropagation optimizes network parameter, and training network makes intensively connect network model and passes through in advance
It first trains, makes intensively to connect network model with good image characteristics extraction ability；
Step a4: by the intensive link block in part (intensive link block 1 and the intensive link block of intensive connection network model front layer
2) parameter carries out module migration, uses it as skeleton character extraction module, ulna/radius true picture is input to bone
In characteristic extracting module, ulna/radius image primary features are obtained, skeleton character extraction module has filtered in bone image sample
To the lesser feature of maturity classification image and noise characteristic, and reduce the characteristic size for being input to feature decision device.
(2), multi-channel feature generator generates characteristics of image
Step b1: it using the one-dimensional random noise with Gaussian Profile as input, is input in multi-channel feature generator；
Step b2: for each multi-channel feature generator, the random noise of input is by several warp laminations, wherein instead
Convolutional layer activation primitive uses ReLU function, successively amplifies the skeleton character figure of generation after multilayer deconvolution, generation
Every skeleton character figure is identical as the characteristic size size that skeleton character extraction module extracts；
Step b3: the skeleton character figure that the feature generator in each channel generates is subjected to cascade splicing, is obtained after splicing
Dimension is consistent with the characteristic dimension that skeleton character extraction module extracts；
Step b4: the feature that multi-channel feature generator generates is input to capsule skeleton character arbiter, skeleton character
Arbiter classifies to the multichannel skeleton character of generation, obtains and differentiates result.
(3), capsule skeleton character arbiter carries out classification and prediction result
Step c1: the ulna that true picture is obtained by skeleton character extraction module/radius primary features or multichannel
The skeleton character that feature generator generates is input in capsule skeleton character arbiter as input；
Step c2: input further extracts ulna/radius low-level features of input by several convolutional layers,
Wherein convolutional layer activation primitive uses ReLU function；
Step c3: ulna/radius feature that step c2 is extracted is input to capsule layer, obtains retaining space location information
Ulna radius feature vector；
Step c4: the ulna/connection type of radius instantiated feature vector between the layers is used and utilizes routing
Selection algorithm currently is best able to characterize the bone instantiated feature vector of all input feature vectors by routing mechanism building,
Such as it represents the instantiated feature vector of bone width information or represents the instantiation vector of bone texture information, and its is defeated
It is input to next layer out；
Step c5: after several capsule layers carry out the extraction of ulna radius instantiated feature, several bones instantiations are obtained
Feature vector, for the classification task of radius, wherein first four vector field homoemorphism is long indicates that current input belongs to each radius maturation
Spend the probability in stage, the long input for representing current skeleton character of another vector field homoemorphism whether be generate the probability of feature, and
For ulna classification task, wherein first three vector field homoemorphism is long indicates that current input belongs to the general of each ulna maturity stage
Rate, whether the long input for representing current skeleton character of another vector field homoemorphism is the probability for generating feature, and modulus length is maximum
It is current ulna/radius input feature vector classification results that bone, which instantiates classification corresponding to vector,.
Wherein, in above-mentioned steps, bone capsule interlayer calculation described in step c4 specifically calculate as shown in figure 9,
If the ulna radius instantiated feature vector u of upper layer capsule layeriIt is input in this layer, uiWith optimizable transition matrix WijIt is multiplied
The predicted vector obtainedVector weighted sum S is wherein obtained by linear combination between predicted vectorj, linear combination is
Number size is by cijIt determines, is obtaining SjLater, by compression function by SjVector length limits, and obtains output vector vj, coefficient
cijFor constant.Constant cijBy bijIt calculates and determines, in interative computation, constantly update bijTo update coefficient of coup cij。
The above process is the mathematical description of routing mechanism, in our skeleton character arbiter between capsule network layer
Mathematical operation be of virtually corresponding instantiation meaning, for the linear combination mechanism and router of capsule interlayer operation
The instantiation explanation of system is as shown in Figure 10.
Three, the building of loss function
Above-mentioned steps b and step c describes the propagated forward of capsule skeleton character arbiter Yu multichannel generator respectively
Process is approached in the present embodiment two by ulna/radius feature that the building of loss function generates skeleton character generator
The feature distribution of true bone image makes it by constructing polytypic loss function similarly for capsule skeleton character arbiter
Can classification corresponding to each ulna radius input feature vector of good discrimination, steps are as follows:
Step d1: for the training process of skeleton character arbiter, when model prediction goes out current input ulna radius feature
After corresponding classification, by the loss of skeletal maturation degree computational discrimination device corresponding to prediction result and current input feature, sentence
The loss of other device is made of 3 parts, may be expressed as:
Skeleton character arbiter total losses=LReally there is the loss of label bone input feature vector+LReally lost without label bone input feature vector+LGenerate the loss of skeleton characterFor having
The loss function of the feature of the true bone image of label, by taking radius is classified as an example, feature decision device carries out four classification to it, each
Class represents a radius maturity stage.We calculate its loss function by losing using edge penalty Margin.We
The class probability of each skeletal maturation degree is predicted using the output of first four capsule vector.And bone true for no label
The loss function of the feature of bone image, ulna/radius feature of input only would know that its label be true skeleton character or
The skeleton character of generation, therefore whether false skeleton character is belonged to current input using an output vector in feature decision device
Probability differentiated, probability, which more levels off to, 0 represents the image as true bone input feature vector, and probability more levels off to 1 representative should
Image is the skeleton character generated.We define the loss of the authentic specimen of no label using the cross entropy of sigmoid function
Function.For generating the loss of skeleton character, we equally use same mode and construct loss function, lose letter
Number equally uses the cross entropy formal definition of sigmoid function.
Step d2: for multi-channel feature generator, the purpose is to generate by simulating true ulna/radius feature
The skeleton character image that feature decision device can be cheated, for our multi-channel feature generator, its loss includes two altogether
Point, it may be expressed as:
Multi-channel feature generator total losses=LGenerate special feature decision loss+LGenerate feature multichannel match penalties
For generating the differentiation loss of skeleton character, the skeleton character of generation wants to imitate true ulna/radius spy
The distribution of sign is to cheat feature decision device, so differentiating that result is true with it to it by arbiter to the skeleton character of generation
Label is compared acquisition loss function, for generator loss we calculated using the cross entropy of sigmoid.For
The multichannel match penalties for generating feature, when skeleton character information of the feature decision device to input carries out further feature extraction
When, feature generator wishes skeleton character that it is generated when discrimination model further extracts feature, and output result can be with
The true skeleton character intermediate result that arbiter extracts is matched.We are using real features in arbiter intermediate result and life
At feature arbiter intermediate result squared difference as loss function.
The present embodiment two by carrying out feature extraction to high-resolution ulna/radius image using transfer learning method, and
And ulna radius maturity is identified using the semi-supervised generation confrontation network generated based on skeleton character and feature identifies,
Solves the problems such as excessive parameter amount in the classification and identification of high-resolution bone input picture and over-fitting, and using more
Channel characteristics generator generates ulna, radius characteristic information figure, using capsule skeleton character arbiter to the bone for extracting acquisition
Sample input feature vector is differentiated, skeletal maturation degree stage forecast is obtained, and capsule reality is utilized in capsule skeleton character arbiter
Exampleization feature propagation mechanism generates confrontation network arbiter by vector characteristics representation method and routing mechanism substitution tradition
The convolution operation and pondization of model operate, by connecting the capsule of different levels step by step in capsule skeleton character arbiter to mention
It takes bone image to instantiate vector, enhances the ability in feature extraction of skeletal maturation degree discrimination model and the reservation to spatial information
Property, while network is reduced to the demand of bone training sample, which uses semi-supervised learning mode, weakens network
To the demand property of bone image markup information, and then realize that there is high-precision, the skeletal maturation degree of high robust automates assessment
Model.
It is had the advantages that in the present embodiment two
1, the semi-supervised generation identified based on feature is fought into intensive connection network application and assesses task in skeletal maturation degree
In, the multichannel skeleton character of proposition generates mode, and the spy of true ulna/radius is imitated using multichannel skeleton character generator
Sign generates skeleton character figure and carries out multichannel image generation, and the superposition calculation of loss is matched and differentiated by using multi-channel feature
Loss function makes the skeleton character generated that can more approach the distribution with true skeleton character, by dual training, promotes bone
Recognition capability and anti-noise ability of the maturity discrimination model to feature.
2, intensive connection network is fought using semi-supervised generation and carries out the identification of skeletal maturation degree, have label using fraction
Ulna/radius true picture is learnt by the confrontation of skeleton character generator and capsule skeleton character arbiter, makes capsule bone
Feature decision device sufficiently learns the characteristics of image of each stage ulna/radius, reduces model to the demand for having mark bone image,
It only needs that high-precision skeletal maturation degree classification results can be obtained using true picture label a small amount of in data set.
3, using transfer learning mode, network is intensively connected by using other data set pre-training, and will intensively connect
There is network the module of feature extraction functions to migrate to generating in the intensive connection network of confrontation, using it to true ulna/radius
Input picture carries out feature extraction, reduces high-resolution bone image and is directly inputted into the parameter amount that confrontation network occurs that generates
Problems of too reduces the generation of over-fitting, provides solution for the input of high-definition picture, avoids input picture
The loss for carrying out feature when size change over remains all features of input bone image.
4, in capsule skeleton character arbiter, capsule layer can retain the extraction of feature the space of bone image feature
The instantiated feature of different bones is combined screening by routing mechanism by change in location information, to improve to ruler
Bone/radius image feature extraction efficiency.
5, using capsule network optimization capsule skeleton character arbiter, enable capsule skeleton character arbiter extract containing
The feature vector of image space positions information, substitution tradition carry out the process of feature extraction using convolution operation to bone image,
Make capsule skeleton character arbiter by that just can identify same or similar feature to the study of a small amount of ulna/radius feature
Variant reduces the demand in skeletal maturation degree task to training sample.
6, ulna/radius image is using characteristic formp as the input and multi-channel feature generator for generating confrontation network
Network is improved using ulna/radius characteristics of image as the discriminant approach of i/o base with capsule skeleton character arbiter
Operational efficiency improves the output of multi-channel feature generator to the interference of capsule skeleton character arbiter, while improving and sentencing
Discriminating power of the other device to skeleton character.
7, the semi-supervised generation based on feature identification fights network, obtains the feature decision mould trained by dual training
Type, and by feature extraction network and the feature decision model transplantations trained to hardware platform, it can be achieved that ulna/radius bone at
Ripe degree assessment task, and facilitate subsequent system upgrade and update.
The appraisal procedure of skeletal maturation degree of the invention can be applied to the classification problem of the same race with other industry background,
Such as other Medical Images Classification task dispatchings, that is, meet the task of general deep learning classification task standard, difference is only
It needs to replace with corresponding training sample in the training stage and be learnt using this system.
Mode the above is only the implementation of the present invention is not intended to limit the scope of the invention, all to utilize this
Equivalent structure or equivalent flow shift made by description of the invention and accompanying drawing content, it is relevant to be applied directly or indirectly in other
Technical field is included within the scope of the present invention.
Claims (10)
1. a kind of method applied to the assessment of skeletal maturation degree, which comprises the steps of:
Step S1, the intensive connection network model with image characteristics extraction ability is established, intensive connection network model is carried out
After pre-training, the intensive link block in the intensive connection network model is subjected to module transfer, generates and can get bone image
The skeleton character extraction module of primary features；
Step S2, by the one-dimensional noise inputs with Gaussian Profile into multi-channel feature generator, the multi-channel feature is raw
Skeleton character identical with the characteristic size size that the skeleton character extraction module extracts is generated in growing up to be a useful person in each channel
Figure, then each skeleton character figure in the multi-channel feature generator in each channel is subjected to cascade splicing, it is obtained after splicing
Dimension skeleton character figure identical with the characteristic dimension that the skeleton character extraction module extracts, and by the skeleton character figure
Output；
Step S3, true picture is obtained into bone image primary features and the multichannel by the skeleton character extraction module
The skeleton character figure that feature generator generates is input in capsule skeleton character arbiter, to obtain retaining space position letter
The bone instantiated feature vector of breath obtains classification corresponding to the bone instantiated feature vector, then knows the true picture
Skeletal maturation degree assessment result.
2. a kind of method applied to the assessment of skeletal maturation degree according to claim 1, which is characterized in that further include step
S4, the loss function that skeleton character figure is constructed in the multi-channel feature generator, then sentence in the capsule skeleton character
The loss function is defined using cross entropy in other device, extracted in the loss function bone instantiated feature to
Amount, then and obtain classification corresponding to the bone instantiated feature vector, then known to the true picture skeletal maturation degree assessment
As a result.
3. a kind of method applied to the assessment of skeletal maturation degree according to claim 1 or 2, which is characterized in that step S1
Include:
Step S11, intensive connection network model is established, using the training set image of pre-training data set as input picture；
Step S12, input picture is subjected to feature extraction by several intensive link blocks and transition zone, exports prediction result.
4. a kind of method applied to the assessment of skeletal maturation degree according to claim 3, which is characterized in that step S1 is also wrapped
It includes:
Step S13, loss function is calculated using the label of the prediction result and the true picture of the pre-training data set, then
Backpropagation optimizes network parameter, is trained by the network parameter in the intensive connection network model, makes described intensive
Connecting network model has image characteristics extraction ability；
Step S14, the pre-training data set is subjected to module migration, generates the bone spy that can get bone image primary features
Extraction module is levied, the true picture of bone is input in the skeleton character extraction module, obtains bone image primary features.
5. a kind of method applied to the assessment of skeletal maturation degree according to claim 1 or 4, which is characterized in that step S2
Include:
Step S21, by the one-dimensional noise inputs with Gaussian Profile into the multi-channel feature generator；
Step S22, for the feature generator in each channel in the multi-channel feature generator, the noise of input is passed through
Several warp laminations successively amplify the skeleton character figure of generation after the deconvolution of warp lamination, make every generated
Skeleton character figure is identical as the characteristic dimension that the skeleton character extraction module extracts.
6. a kind of method applied to the assessment of skeletal maturation degree according to claim 5, which is characterized in that step S2 is also wrapped
It includes:
Step S23, by each channel of the multi-channel feature generator feature generator generate skeleton character figure into
Row cascade splicing, obtains dimension skeleton character identical with the characteristic dimension that the skeleton character extraction module extracts after splicing
Figure；
Step S24, spliced skeleton character figure is input in capsule skeleton character arbiter.
7. a kind of method applied to the assessment of skeletal maturation degree according to claim 1 or 6, which is characterized in that step S3
Include:
Step S31, the bone image primary features that obtain true picture by the skeleton character extraction module and described more
The skeleton character figure that channel characteristics generator generates is input in the capsule skeleton character arbiter；
Step S32, by several convolutional layers to the bone image primary features and skeleton character figure extract feature to
Amount.
8. a kind of method applied to the assessment of skeletal maturation degree according to claim 7, which is characterized in that step S3 is also wrapped
It includes:
Step S33, the feature vector of extraction is inputted into capsule layer, so that the bone instantiation for obtaining retaining space location information is special
Levy vector；
Step S34, classification corresponding to the bone instantiated feature vector is obtained, then knows the skeletal maturation degree of the true picture
Assessment result.
9. a kind of system applied to the assessment of skeletal maturation degree characterized by comprising
Pre-training bone image characteristic module, for establishing intensive connection network model, to the intensive connection network model into
Row pre-training is to generate image characteristics extraction ability；
Skeleton character extraction module, for obtaining bone image primary features by true picture；
Multi-channel feature generator, the characteristic size size for generating with the skeleton character extraction module extracts are identical
Skeleton character figure simultaneously carries out cascade splicing, and the characteristic dimension that dimension is extracted with the skeleton character extraction module is obtained after splicing
Identical skeleton character figure；
Capsule skeleton character arbiter, for obtaining the bone instantiated feature vector of retaining space location information.
10. a kind of system applied to the assessment of skeletal maturation degree according to claim 9, which is characterized in that further include mending
Module is repaid, for constructing the loss function of skeleton character figure in the multi-channel feature generator, then in the capsule bone
The loss function is defined using cross entropy in bone feature decision device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811319500.6A CN109544518B (en) | 2018-11-07 | 2018-11-07 | Method and system applied to bone maturity assessment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811319500.6A CN109544518B (en) | 2018-11-07 | 2018-11-07 | Method and system applied to bone maturity assessment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109544518A true CN109544518A (en) | 2019-03-29 |
| CN109544518B CN109544518B (en) | 2020-11-03 |
Family
ID=65844644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811319500.6A Active CN109544518B (en) | 2018-11-07 | 2018-11-07 | Method and system applied to bone maturity assessment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109544518B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110009034A (en) * | 2019-04-02 | 2019-07-12 | 中南大学 | Air-conditioner set type identifier method |
| CN110009097A (en) * | 2019-04-17 | 2019-07-12 | 电子科技大学 | The image classification method of capsule residual error neural network, capsule residual error neural network |
| CN110022422A (en) * | 2019-04-19 | 2019-07-16 | 吉林大学 | A kind of sequence of frames of video generation method based on intensive connection network |
| CN110322432A (en) * | 2019-05-27 | 2019-10-11 | 上海联影医疗科技有限公司 | Medical image processing method, device, computer equipment and readable storage medium storing program for executing |
| CN110648317A (en) * | 2019-09-18 | 2020-01-03 | 上海交通大学 | Quality classification method and system suitable for spine metastasis tumor bone |
| CN110879985A (en) * | 2019-11-18 | 2020-03-13 | 西南交通大学 | Anti-noise data face recognition model training method |
| CN111539963A (en) * | 2020-04-01 | 2020-08-14 | 上海交通大学 | Bone scan image hot spot segmentation method, system, medium and device |
| CN111553412A (en) * | 2020-04-27 | 2020-08-18 | 广州市妇女儿童医疗中心（广州市妇幼保健院、广州市儿童医院、广州市妇婴医院、广州市妇幼保健计划生育服务中心） | Method, device and equipment for training precocious puberty classification model |
| CN112102285A (en) * | 2020-09-14 | 2020-12-18 | 辽宁工程技术大学 | Bone age detection method based on multi-modal confrontation training |
| CN112365553A (en) * | 2019-07-24 | 2021-02-12 | 北京新唐思创教育科技有限公司 | Human body image generation model training, human body image generation method and related device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004112580A2 (en) * | 2003-06-19 | 2004-12-29 | Compumed, Inc. | Method and system for analyzing bone conditions using dicom compliant bone radiographic image |
| CN107767376A (en) * | 2017-11-02 | 2018-03-06 | 西安邮电大学 | X-ray film stone age Forecasting Methodology and system based on deep learning |
| CN108052940A (en) * | 2017-12-17 | 2018-05-18 | 南京理工大学 | SAR remote sensing images waterborne target detection methods based on deep learning |
| CN108510004A (en) * | 2018-04-04 | 2018-09-07 | 深圳大学 | A kind of cell sorting method and system based on depth residual error network |
| CN108564611A (en) * | 2018-03-09 | 2018-09-21 | 天津大学 | A kind of monocular image depth estimation method generating confrontation network based on condition |
-
2018
- 2018-11-07 CN CN201811319500.6A patent/CN109544518B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004112580A2 (en) * | 2003-06-19 | 2004-12-29 | Compumed, Inc. | Method and system for analyzing bone conditions using dicom compliant bone radiographic image |
| CN107767376A (en) * | 2017-11-02 | 2018-03-06 | 西安邮电大学 | X-ray film stone age Forecasting Methodology and system based on deep learning |
| CN108052940A (en) * | 2017-12-17 | 2018-05-18 | 南京理工大学 | SAR remote sensing images waterborne target detection methods based on deep learning |
| CN108564611A (en) * | 2018-03-09 | 2018-09-21 | 天津大学 | A kind of monocular image depth estimation method generating confrontation network based on condition |
| CN108510004A (en) * | 2018-04-04 | 2018-09-07 | 深圳大学 | A kind of cell sorting method and system based on depth residual error network |
Non-Patent Citations (4)
| Title |
|---|
| 朱应钊 等: "胶囊网络技术及发展趋势研究", 《广东通信技术》 * |
| 杜宇: "基于深度机器学习的体态与手势感知计算关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
| 胡婷鸿 等: "深度学习在图像识别及骨龄评估中的优势及应用前景", 《法医学杂志》 * |
| 薛迪秀: "基于卷积神经网络的医学图像癌变识别研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110009034A (en) * | 2019-04-02 | 2019-07-12 | 中南大学 | Air-conditioner set type identifier method |
| CN110009097A (en) * | 2019-04-17 | 2019-07-12 | 电子科技大学 | The image classification method of capsule residual error neural network, capsule residual error neural network |
| CN110009097B (en) * | 2019-04-17 | 2023-04-07 | 电子科技大学 | Capsule residual error neural network and image classification method of capsule residual error neural network |
| CN110022422A (en) * | 2019-04-19 | 2019-07-16 | 吉林大学 | A kind of sequence of frames of video generation method based on intensive connection network |
| CN110022422B (en) * | 2019-04-19 | 2020-02-07 | 吉林大学 | Video frame sequence generation method based on dense connection network |
| CN110322432A (en) * | 2019-05-27 | 2019-10-11 | 上海联影医疗科技有限公司 | Medical image processing method, device, computer equipment and readable storage medium storing program for executing |
| CN110322432B (en) * | 2019-05-27 | 2021-11-23 | 上海联影医疗科技股份有限公司 | Medical image processing method, apparatus, computer device and readable storage medium |
| CN112365553A (en) * | 2019-07-24 | 2021-02-12 | 北京新唐思创教育科技有限公司 | Human body image generation model training, human body image generation method and related device |
| CN110648317A (en) * | 2019-09-18 | 2020-01-03 | 上海交通大学 | Quality classification method and system suitable for spine metastasis tumor bone |
| CN110648317B (en) * | 2019-09-18 | 2023-06-30 | 上海交通大学 | Quality classification method and system suitable for spine metastasis tumor bone |
| CN110879985A (en) * | 2019-11-18 | 2020-03-13 | 西南交通大学 | Anti-noise data face recognition model training method |
| CN110879985B (en) * | 2019-11-18 | 2022-11-11 | 西南交通大学 | Anti-noise data face recognition model training method |
| CN111539963B (en) * | 2020-04-01 | 2022-07-15 | 上海交通大学 | Bone scanning image hot spot segmentation method, system, medium and device |
| CN111539963A (en) * | 2020-04-01 | 2020-08-14 | 上海交通大学 | Bone scan image hot spot segmentation method, system, medium and device |
| CN111553412A (en) * | 2020-04-27 | 2020-08-18 | 广州市妇女儿童医疗中心（广州市妇幼保健院、广州市儿童医院、广州市妇婴医院、广州市妇幼保健计划生育服务中心） | Method, device and equipment for training precocious puberty classification model |
| CN112102285A (en) * | 2020-09-14 | 2020-12-18 | 辽宁工程技术大学 | Bone age detection method based on multi-modal confrontation training |
| CN112102285B (en) * | 2020-09-14 | 2024-03-12 | 辽宁工程技术大学 | Bone age detection method based on multi-modal countermeasure training |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109544518B (en) | 2020-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109544518A (en) | A kind of method and its system applied to the assessment of skeletal maturation degree | |
| CN112150478B (en) | Method and system for constructing semi-supervised image segmentation framework | |
| CN108717568B (en) | A kind of image characteristics extraction and training method based on Three dimensional convolution neural network | |
| CN111047594B (en) | Tumor MRI weak supervised learning analysis modeling method and model thereof | |
| CN108021916B (en) | Deep learning diabetic retinopathy sorting technique based on attention mechanism | |
| CN105787439B (en) | A kind of depth image human synovial localization method based on convolutional neural networks | |
| CN109635835A (en) | A kind of breast lesion method for detecting area based on deep learning and transfer learning | |
| CN106296699A (en) | Cerebral tumor dividing method based on deep neural network and multi-modal MRI image | |
| CN107481188A (en) | A kind of image super-resolution reconstructing method | |
| CN108510194A (en) | Air control model training method, Risk Identification Method, device, equipment and medium | |
| CN110853009B (en) | Retina pathology image analysis system based on machine learning | |
| CN109522973A (en) | Medical big data classification method and system based on production confrontation network and semi-supervised learning | |
| CN108364006A (en) | Medical Images Classification device and its construction method based on multi-mode deep learning | |
| CN107909566A (en) | A kind of image-recognizing method of the cutaneum carcinoma melanoma based on deep learning | |
| CN108460726A (en) | A kind of magnetic resonance image super-resolution reconstruction method based on enhancing recurrence residual error network | |
| CN110490242B (en) | Training method of image classification network, fundus image classification method and related equipment | |
| CN107516312B (en) | A kind of Chinese medicine complexion automatic classification method using shallow-layer neural network | |
| CN107610123A (en) | A kind of image aesthetic quality evaluation method based on depth convolutional neural networks | |
| CN107169974A (en) | It is a kind of based on the image partition method for supervising full convolutional neural networks more | |
| CN109711426A (en) | A kind of pathological picture sorter and method based on GAN and transfer learning | |
| CN103699904B (en) | The image computer auxiliary judgment method of multisequencing nuclear magnetic resonance image | |
| CN107529650A (en) | The structure and closed loop detection method of network model, related device and computer equipment | |
| CN112784879A (en) | Medical image segmentation or classification method based on small sample domain self-adaption | |
| CN110490880A (en) | A kind of hip joint x-ray image dividing method and system based on local visual clue | |
| CN109086660A (en) | Training method, equipment and the storage medium of multi-task learning depth network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |