CN114359133A - Hand bone image analysis method based on feature extraction and related equipment - Google Patents

Hand bone image analysis method based on feature extraction and related equipment Download PDF

Info

Publication number
CN114359133A
CN114359133A CN202111382606.2A CN202111382606A CN114359133A CN 114359133 A CN114359133 A CN 114359133A CN 202111382606 A CN202111382606 A CN 202111382606A CN 114359133 A CN114359133 A CN 114359133A
Authority
CN
China
Prior art keywords
neural network
sample
diagnosis
global
hand bone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111382606.2A
Other languages
Chinese (zh)
Inventor
贺志强
牛凯
乔永康
王璐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202111382606.2A priority Critical patent/CN114359133A/en
Publication of CN114359133A publication Critical patent/CN114359133A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Analysis (AREA)

Abstract

The application provides a hand bone image analysis method and related equipment based on feature extraction, firstly, a trained main neural network extracts global features of a hand bone image of a patient to obtain global feature vectors, key region features are extracted through a frame selection key region to obtain key region feature vectors, and then the global feature vectors and the key region feature vectors are spliced to obtain spliced vectors; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.

Description

Hand bone image analysis method based on feature extraction and related equipment
Technical Field
The application relates to the technical field of neural networks, in particular to a hand bone image analysis method based on feature extraction and related equipment.
Background
The Kaschin-Beck disease is an endemic and deformed joint bone disease, and is also called dwarf disease, abacus bead disease and the like in China. The distribution range is large in China, the distribution is mostly in provinces such as black, Ji, Liao, jin, Shaan and the like, and the distribution is common in mountainous areas and semi-mountainous areas, and is rarely seen in plains. Patients are mostly children and teenagers, and the patients are accompanied with the clinical characteristics of arthralgia, deformity, dwarfism, dyskinesia and the like.
X-ray images of the Kaschin-Beck disease under different conditions (different onset ages, different parts and different lesion degrees of patients) generally have different manifestations, wherein the condition of an early patient can be reflected from the X-ray images of hands, so that the patient can be screened by checking the X-ray images of the hands in a large scale in an easily-attacked area, the patient can be found as early as possible, and the treatment is provided in time.
For large-scale screening of patients with the Kaschin-Beck disease, the speed of manually identifying the image features is relatively slow, and a Kaschin-Beck disease diagnosis method capable of efficiently meeting the large-scale screening requirement is needed.
Disclosure of Invention
In view of the above, an object of the present application is to provide a hand bone image analysis method and related apparatus based on feature extraction.
Based on the above purpose, the present application provides a hand bone image analysis method based on feature extraction, including: preprocessing an initial hand bone image of a patient to obtain a hand bone image; the master neural network extracts and obtains global features of the hand bone image and generates a global feature vector of the hand bone image, the master neural network carries out global diagnosis on the global feature vector to obtain a global diagnosis result, wherein the master neural network is obtained by training an initial master neural network by utilizing an acquired data set; the area frame selection neural network frames a plurality of key areas of the hand bone image to obtain a plurality of key area images, wherein the area frame selection neural network is obtained by training an initial area frame selection neural network by using the acquired data set; the master neural network extracts the characteristics of the key area images to obtain key area characteristics of the hand bone images and generates key area characteristic vectors of the hand bone images, and the master neural network carries out key area diagnosis on the key area characteristic vectors to obtain a plurality of key area diagnosis results; the master neural network splices the global feature vector and the key region feature vector to obtain a spliced vector, and the master neural network splices and diagnoses the spliced vector to obtain a spliced diagnosis result; and the master neural network carries out comprehensive diagnosis on the hand bone image according to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
Optionally, the method further comprises: constructing the initial main neural network and the initial region frame selection neural network; acquiring the data set, wherein the data set comprises a plurality of initial sample hand bone images, a diagnosis result obtained by manually carrying out comprehensive diagnosis on the plurality of initial sample hand bone images, and coordinates of a key area manually framed by the plurality of initial sample hand bone images; the initial main neural network is trained based on the data set to obtain the main neural network; and training the initial region frame selection neural network based on the data set to obtain the region frame selection neural network.
Optionally, the training of the initial master neural network based on the data set to obtain the master neural network includes: acquiring the initial sample hand bone image from the data set, and preprocessing the initial sample hand bone image to obtain a sample hand bone image; the initial main neural network is paired with theCarrying out global diagnosis processing on the hand bone image of the sample to obtain a global diagnosis result of the sample, and calculating to obtain a global loss L by using the global diagnosis result of the sampleg(ii) a The initial main neural network carries out key region diagnosis processing on the sample hand bone image to obtain a plurality of sample key region diagnosis results, and key region loss L is calculated by utilizing the plurality of sample key region diagnosis resultsp(ii) a The initial main neural network carries out splicing diagnosis processing on the hand bone images of the samples to obtain a plurality of sample splicing diagnosis results, and the splicing loss L is calculated by utilizing the plurality of sample splicing diagnosis results*(ii) a The initial master neural network utilizes the global loss LgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss Ltotal(ii) a Based on overall network loss LtotalAdjusting parameters of the initial main neural network to finish one training of the initial main neural network; in response to determining the overall network loss LtotalAnd finishing training on all the data sets of the initial main neural network or less than or equal to a preset threshold value, and taking the trained initial main neural network as the main neural network.
Optionally, the method further comprises: the initial main neural network carries out global diagnosis processing on the sample hand bone image to obtain a sample global diagnosis result, and the method comprises the following steps: the initial master neural network extracts and obtains sample global features of the sample hand bone image and generates a sample global feature vector of the sample hand bone image, and the initial master neural network carries out global diagnosis on the sample global feature vector to obtain a sample global diagnosis result; the initial main neural network carries out key region diagnosis processing on the sample hand bone image to obtain a plurality of sample key region diagnosis results, and the method comprises the following steps: the initial area framing neural network frames a plurality of key areas of the sample hand bone image to obtain a plurality of sample key area images; the initial master neural network extracts the characteristics of the sample key area images to obtain the sample key area characteristics of the sample hand bone images and generates a plurality of sample key area characteristic vectors of the sample hand bone images, and the initial master neural network carries out key area diagnosis on the sample key area characteristic vectors to obtain a plurality of sample key area diagnosis results; the initial main neural network carries out splicing diagnosis processing on the sample hand bone images to obtain a plurality of sample splicing diagnosis results, and the method comprises the following steps: and the initial master neural network splices the sample global feature vector and the sample key region feature vector to obtain a sample splicing vector, and the initial master neural network splices and diagnoses the sample splicing vector to obtain a sample splicing diagnosis result.
Optionally, the initial master neural network utilizes the global penalty LgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss LtotalThe method comprises the following steps:
the initial main neural network calculates the overall network loss L by using the following formulatotal：
Ltotal＝L*+λLg+βLp
Wherein the parameters λ and β are the global loss LgWith the critical area loss LpThe weight of (c).
Optionally, the performing, by the master neural network, a comprehensive diagnosis on the hand bone image according to the global diagnosis result, the key region diagnosis result, and the stitching diagnosis result to obtain a final diagnosis result of whether the patient suffers from a Kaschin-Beck disease, including: the main neural network gives different weights to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result, and a result numerical value is obtained through calculation; and the main neural network obtains the final diagnosis result by comparing the result value with a preset diagnosis threshold value.
Optionally, the global features include distribution of the hand bones in the hand bone image and texture of the bones; the key areas comprise middle and base epiphysis of ring finger, middle finger and index finger, end epiphysis of thumb and proximal end of third metacarpal bone; the critical area features include metaphyseal pits, hardening, asperities.
Based on the above-mentioned purpose, this application still provides a hand bone image analysis device based on feature extraction, includes: the preprocessing module is configured to preprocess the initial hand bone image of the patient to obtain a hand bone image; the global diagnosis module is configured to extract and obtain global features of the hand bone image by a main neural network and generate global feature vectors of the hand bone image, and the main neural network carries out global diagnosis on the global feature vectors to obtain a global diagnosis result; a key area framing module configured to frame a plurality of key areas of the hand bone image by using an area framing neural network to obtain a plurality of key area images; a key region diagnosis module configured to extract features of the plurality of key region images by the master neural network, obtain key region features of the plurality of hand bone images, and generate key region feature vectors of the plurality of hand bone images, wherein the master neural network performs key region diagnosis on the plurality of key region feature vectors to obtain a plurality of key region diagnosis results; the splicing diagnosis module is configured to splice the global feature vector and the key region feature vector by the main neural network to obtain a spliced vector, and the main neural network performs splicing diagnosis on the spliced vector to obtain a splicing diagnosis result; and the final result diagnosis module is configured to perform comprehensive diagnosis on the hand bone image by the master neural network according to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
In view of the above, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes any one of the methods for analyzing a hand bone image based on feature extraction.
In view of the above, the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause a computer to execute any one of the methods for analyzing hand bones based on feature extraction.
From the above, according to the hand bone image analysis method based on feature extraction and the related device provided by the application, firstly, the trained main neural network extracts the global features of the hand bone image of the patient to obtain the global feature vector, then the key region features are extracted through the frame selection of the key region to obtain the key region feature vector, and then the global feature vector and the key region feature vector are spliced to obtain the spliced vector; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.
Drawings
In order to more clearly illustrate the technical solutions in the present application or the related art, the drawings needed to be used in the description of the embodiments or the related art will be briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart illustrating a hand bone image analysis method based on feature extraction according to an embodiment of the present disclosure;
FIG. 2 is a block diagram of a sample hand bone image preprocessing process according to an embodiment of the present disclosure;
FIG. 3 is a block flow diagram of global diagnostics according to an embodiment of the present application;
FIG. 4 is a block diagram of a process of selecting key zones and diagnosing key zones according to an embodiment of the present disclosure;
FIG. 5 is a block flow diagram of a stitching diagnostic according to an embodiment of the present application;
FIG. 6 is a block diagram of an initial master neural network training process according to an embodiment of the present application;
fig. 7 is a block diagram of a hand bone image analysis apparatus based on feature extraction according to an embodiment of the present application;
fig. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below with reference to the accompanying drawings in combination with specific embodiments.
It should be noted that technical terms or scientific terms used in the embodiments of the present application should have a general meaning as understood by those having ordinary skill in the art to which the present application belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the present application do not denote any order, quantity, or importance, but rather the terms are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
For convenience of understanding, terms referred to in the embodiments of the present disclosure are explained below:
ResNet: the Residual error Network is easy to optimize, and the accuracy can be improved by increasing the equivalent depth; the inner residual block uses jump connection, so that the problem of gradient disappearance caused by depth increase in a deep neural network is solved;
ResNeXt: combining a residual error network with a grouping convolution strategy to obtain a network;
f1 index: a comprehensive evaluation (namely a harmonic mean value of the accuracy and the recall ratio) is given according to the accuracy and the recall ratio in the information retrieval and the natural language processing.
As described in the background art, for large-scale screening of patients with Kaschin-Beck disease, the speed of manually identifying image features is relatively slow, and a Kaschin-Beck disease diagnosis method capable of efficiently meeting the large-scale screening requirement is needed.
In view of the above, the invention provides a hand bone image analysis method based on feature extraction by using a deep learning neural network technology, which automatically and respectively extracts features of an overall image and a common lesion area through a neural network, performs feature splicing, and realizes feature expression of higher coverage of the image by using the spliced features.
For convenience of understanding, terms referred to in the embodiments of the present disclosure are explained below:
in view of the above, an embodiment of the present application provides a hand bone image analysis method based on feature extraction, as shown in fig. 1, including:
s101, preprocessing the initial hand bone image of the patient to obtain a hand bone image.
S102, extracting and obtaining the global features of the hand bone image by a master neural network, generating a global feature vector of the hand bone image, and carrying out global diagnosis on the global feature vector by the master neural network to obtain a global diagnosis result, wherein the master neural network is obtained by training an initial master neural network by using an acquired data set.
S103, the area frame selection neural network frames a plurality of key areas of the hand bone image to obtain a plurality of key area images, wherein the area frame selection neural network is obtained by training the initial area frame selection neural network by using the acquired data set.
And S104, extracting the features of the plurality of key area images by the main neural network to obtain the key area features of the plurality of hand bone images and generate a plurality of key area feature vectors of the hand bone images, and performing key area diagnosis on the plurality of key area feature vectors by the main neural network to obtain a plurality of key area diagnosis results.
And S105, the master neural network splices the global feature vector and the key region feature vector to obtain a spliced vector, and the master neural network splices and diagnoses the spliced vector to obtain a spliced and diagnosed result.
And S106, the master neural network carries out comprehensive diagnosis on the hand bone image according to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
From the above, according to the hand bone image analysis method based on feature extraction provided by the application, firstly, the trained main neural network extracts the global features of the hand bone image of the patient to obtain the global feature vector, then key region features are extracted through frame selection of the key region to obtain the key region feature vector, and then the global feature vector and the key region feature vector are spliced to obtain the spliced vector; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.
In some embodiments, the preprocessing includes performing segmentation rotation on the initial hand bone image, removing background and other hand tissues, and obtaining a uniform-size hand bone image with a forward orientation maintained.
In a specific embodiment, the segmentation of the skeleton and the background can be performed by using the gray scale difference between the skeleton part and the background part according to the gray scale feature of the image. From the gray histogram of the initial hand bone image, a peak appears at the lower gray, where nearby pixels are marked as background and other tissues, and at the higher gray, the histogram is relatively flat, where the pixels are the hand bone part, and the segmentation of the tissues is completed according to the logic.
In some embodiments, the method further comprises:
s107, constructing the initial main neural network and the initial region frame selection neural network.
And S108, acquiring the data set, wherein the data set comprises a plurality of initial sample hand bone images, a diagnosis result obtained by manually carrying out comprehensive diagnosis on the plurality of initial sample hand bone images, and coordinates of key areas manually framed by the plurality of initial sample hand bone images.
S109, the initial main neural network is trained based on the data set to obtain the main neural network.
S110, the initial area frame selection neural network is trained based on the data set to obtain the area frame selection neural network.
The initial sample hand bone image in the above-mentioned data set of this application is mainly X line image, and data acquisition is comparatively convenient, and the cost is lower, and initial major neural net and initial regional frame select neural network train through the coordinate of diagnostic result and key region simultaneously, have guaranteed the reliability of model at the data aspect.
In a specific embodiment, the data set comprises images of teenager's hand taken at areas with an epidemic of Kaschin-Beck disease; a professional physician manually marks whether the image is ill or not according to the clinical diagnosis standard of the Kaschin-Beck disease, and the mark is used as a label for online learning; and the specialist carries out manual positioning on common lesion occurrence areas of the partial images and records the midpoint coordinates of the areas.
The initial area frame selection neural network is trained to extract the point coordinates in the key area, so that frame selection is more accurate, and meanwhile, the key area can be selected in different frame selection ranges according to different areas, and further feature extraction is carried out.
In one embodiment, the initial master neural network is constructed using a ResNet network and the initial zone-boxed neural network is constructed using a ResNeXt network.
In a specific embodiment, the data set is divided into two sets, namely a training set and a test set, and the test set data does not participate in the training process of the neural network, and only data augmentation is performed in the training set. After the neural network is trained, the stored network model with the best performance is subjected to image classification on a test set, classification results are compared with labels, the classification correct number of each condition is counted to calculate common parameters of the measurement model, such as accuracy, precision and recall, and the F1 index is used for neutralizing the indexes to visually evaluate the performance of the model.
In some embodiments, S109 comprises:
s901, acquiring the initial sample hand bone image from the data set, and preprocessing the initial sample hand bone image to obtain a sample hand bone image.
S902, the initial master neural network carries out global diagnosis processing on the sample hand bone image to obtain a sample global diagnosis result, and a global loss L is calculated by utilizing the sample global diagnosis resultg。
S903, the initial main neural network carries out key region diagnosis processing on the sample hand bone image to obtain a plurality of sample key region diagnosis results, and key region loss L is calculated by utilizing the plurality of sample key region diagnosis resultsp。
S904, the initial main neural network carries out splicing diagnosis processing on the hand bone images of the samples to obtain a plurality of sample splicing diagnosis results, and the splicing loss L is obtained by utilizing the plurality of sample splicing diagnosis results to calculate*。
S905, the initial main neural network utilizes the global loss LgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss Ltotal。
S906, based on the overall network loss LtotalAdjusting the parameters of the initial main neural network to finish the adjustment of the initial main neural networkOne training of the initial master neural network.
S907, in response to determining the overall network loss LtotalAnd finishing training on all the data sets of the initial main neural network or less than or equal to a preset threshold value, and taking the trained initial main neural network as the main neural network.
The method adopts the transfer learning method to effectively train the data set, and ensures that the initial neural network can reliably extract the features under the condition of relatively less sample number.
In a specific embodiment, the preprocessing the initial sample hand bone image to obtain a sample hand bone image further includes: the initial sample hand bone image is obtained by splicing the hand bone images of four examined persons, 4 independent sample hand bone images with different directions are obtained after cross segmentation, then straight line detection is used for finding out the straight line formed by the finger bone parts, the straight line direction is taken as the central axis direction of the hand bone, correction is carried out according to the vertical direction of the picture, and the upright sample hand bone image is guaranteed to be obtained. And finally, carrying out size normalization processing on the obtained image to obtain a sample hand bone image which is finally used for training in the neural network. Meanwhile, the large difference of the number of positive samples and negative samples in the samples is considered, the hand bone images of the samples are subjected to proper image augmentation and balance, and the model expression capacity is improved.
In a specific embodiment, the global loss is used for feeding back and improving the capability of the neural network for extracting global features, and the key area loss is used for improving the capability of the neural network for acquiring the features of the lesion area. And the initial main neural network uses a gradient descent algorithm with momentum as an optimization algorithm, minimizes the loss function, iteratively updates parameters in the neural network by an error back propagation method until the error is smaller than a preset threshold value or reaches a preset iteration number, and stores a group of parameters which best appear in the training process as the main neural network. In the process, error minimization processing is carried out on the global loss and the key area loss, and meanwhile, parameter adjustment is carried out on the corresponding key area feature extraction network and the global feature extraction network by using an error back propagation method, so that the feature extraction capability of the corresponding network is improved.
In a specific embodiment, in the specific calculation of the loss function, the error of the neural network diagnosis result and the data set is calculated by using a cross entropy loss function, and the calculation formula of the cross entropy function is as follows:
Figure BDA0003364511280000091
wherein i represents yiRepresenting the classification number of the exemplars, i is 2 in one embodiment, the positive exemplar label is 0, the negative exemplar label is 1, piCounts are the probability that a sample is predicted to be a positive example.
In some embodiments, the method further comprises:
the S902 includes: the initial master neural network extracts and obtains sample global features of the sample hand bone image and generates a sample global feature vector of the sample hand bone image, and the initial master neural network carries out global diagnosis on the sample global feature vector to obtain a sample global diagnosis result.
S903 includes: the initial area framing neural network frames a plurality of key areas of the sample hand bone image to obtain a plurality of sample key area images; the initial master neural network extracts the characteristics of the sample key area images to obtain the sample key area characteristics of the sample hand bone images and generates a plurality of sample key area characteristic vectors of the sample hand bone images, and the initial master neural network carries out key area diagnosis on the sample key area characteristic vectors to obtain a plurality of sample key area diagnosis results.
S904 includes: and the initial master neural network splices the sample global feature vector and the sample key region feature vector to obtain a sample splicing vector, and the initial master neural network splices and diagnoses the sample splicing vector to obtain a sample splicing diagnosis result.
In some embodiments, S905 includes: the above-mentionedThe initial main neural network calculates the overall network loss L by using the following formulatotal：
Ltotal＝L*+λLg+βLp
Wherein the parameters λ and β are the global loss LgWith the critical area loss LpThe weight of (c).
In one specific embodiment, the parameters λ and β are preset to 1.
In some embodiments, S106 comprises:
s601, the main neural network gives different weights to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result, and a result numerical value is obtained through calculation.
S602, the main neural network obtains the final diagnosis result by comparing the result value with a preset diagnosis threshold value.
The main neural network simultaneously uses the features of multiple scales to jointly judge, and the final diagnosis result is jointly obtained by combining the three diagnosis results, so that the accuracy of the final diagnosis result is ensured.
In some embodiments, the global features include distribution of the hand bones in the hand bone image and texture of the bones; the key areas comprise middle and base epiphysis of ring finger, middle finger and index finger, end epiphysis of thumb and proximal end of third metacarpal bone; the critical area features include metaphyseal pits, hardening, asperities.
A specific embodiment a is provided below for the hand bone image analysis method based on feature extraction of the present application.
Step 1: data pre-processing
The hand X-ray images used in the invention as an initial training set are acquired from Tibetan Kaschin-Beck disease epidemic areas by a team, the marking of original sample images is completed by a doctor, data are provided for the training of a network later, meanwhile, each sample image is obtained by splicing the hand X-ray images of four detected persons, in addition, in order to ensure that the network can be concentrated on the hand skeleton structure, the sample images are preprocessed, an independent hand X-ray image only comprising a palm and finger bone structure is obtained, and the flow of data preprocessing is shown in figure 2.
Step 1.1: initial sample image annotation
The labeling of the hand initial sample image used by the invention is carried out by a professional doctor according to the current diagnosis standard of the Kaschin-Beck disease in China, the labeling is used as a label for network learning, and meanwhile, the Kaschin-Beck disease is positive, namely the image label of a patient is marked as 1, otherwise, the label is marked as 0.
After classification marking whether the initial sample images are ill or not, randomly dividing part of the initial sample images as training samples, carrying out manual frame selection of common lesion parts on the initial sample images, recording central coordinates of the areas, and using the central coordinates for training the area frame selection neural network for automatic feature extraction of a subsequent key area.
Step 1.2: initial sample image preprocessing
In order to remove the background of the image and other tissues besides the hand skeleton, the initial sample image needs to be segmented. The invention starts from the property that different organization structures in the X-ray image have different gray levels, and can complete the segmentation of the skeleton and the background by means of the gray level difference value of the skeleton part and the background part according to the gray level characteristics of the X-ray image. From the gray histogram of the original sample image, a peak appears at the lower gray, where nearby pixels are marked as background and other tissues, and at the higher gray, the histogram is relatively flat, where the pixels are the skeleton part of the hand, according to which the segmentation of the tissues can be completed.
The initial sample image is obtained by splicing X-ray images of hands of four examined persons, 4 independent hand images in different directions are obtained after cross segmentation, then straight line detection is used for finding out straight lines formed by finger bone parts, the straight line direction is taken as the direction of a central axis of the hand, correction is carried out according to the vertical direction of a picture, and the fact that an upright sample image is obtained is guaranteed.
And finally, carrying out size normalization processing on the obtained sample image to obtain sample image data finally used for training in the network. Meanwhile, the large difference of the number of positive samples and negative samples in the samples is considered, the image amplification of the samples is properly carried out, the samples are properly balanced, and the expression capacity of the model is improved.
Step 2: image global feature extraction
The method utilizes the main neural network to extract the global features of the image, selects ResNet as the main network to build an integral model, and respectively extracts the global features of the integral image and the features of a key area in two parts. The operation flow chart of extracting the global features of the hand X-ray image is shown in fig. 3.
The neural network for extracting the global feature part is composed of a convolutional layer, a pooling layer, a full-connection layer and the like, the network extracts features and feature maps by using the convolutional layer, then the pooling layer compresses the features and the feature maps, main features are extracted, meanwhile, the complexity of the network is simplified, and finally, the full-connection layer outputs the features to a classifier of the network for extracting the global feature part to perform global diagnosis.
The network input of the feature extraction part is the preprocessed hand bone X-ray image with uniform size obtained in the step 1, the output is the global feature vector of the image, and the size of the global feature vector is T multiplied by 1.
In the part, a network classified by using global feature vectors is a single-layer fully-connected neural network, an activation function is selected as a ReLU function, a fully-connected layer converts a fully-connected layer weight matrix, multiplies an input vector, adds a bias term to obtain an output, at the moment, the fully-connected layer finishes mapping the global feature vectors to a sample mark space, and obtains the category information of the image.
And step 3: automatic frame selection and feature extraction of common lesion areas
Some orthopedic disease patients often have obvious lesions at certain positions in X-ray images, taking early-stage patients with Kaschin-Beck disease as an example, one obvious expression on the hand X-ray images refers to lesion features such as depression and hardening of epiphysis, and due to the size of a feature region and sample resolution, the information can not be fully utilized in feature extraction in the whole picture scale, so that a key region in the X-ray images needs to be selected, key region features are further extracted through a convolutional neural network to assist diagnosis, and meanwhile, as mentioned in step 2, the extracted features of the key region are also subjected to a classification network to obtain a key region diagnosis result, and the part of result is compared with a known label, a loss function is calculated and fed back to a corresponding feature extraction network to correct parameters, in addition, the method can also be applied to the calculation of the overall network loss function later, and the overall flow is shown in FIG. 4.
Step 3.1: selection of common lesion area
In the related art, the framing of the key region is performed in a fixed region according to experience to extract the features of the key region, but the method is limited by the condition that the posture of a palm and the size of the palm are not fixed in different images, and the original method may fail to correctly frame and select the key region, so that the key region feature acquisition fails, and the final diagnosis result is influenced.
According to the current diagnosis standard of the major arthropathy, the positions of common examined lesions are key positions including middle and base epiphysis of ring finger, middle finger and index finger, tail epiphysis of thumb and proximal end of third metacarpal bone, so that the method firstly labels the midpoint coordinates of the area in the partial data manually, stores the coordinate data as a training set, trains a neural network selected from an initial area frame, and hopefully positions the key points. The convolutional neural network used in the part is a ResNeXt network, after the key points are arranged according to the sequence numbers, different frame selection ranges are selected according to the difference of corresponding positions of the key points, network training is started, the obtained model is evaluated in the training process, and the size of a frame selection area is optimized until a more accurate model is obtained.
And accurately positioning the key area of the sample image by using the trained model, completing key point labeling, and performing frame selection on the key area according to the optimized area size obtained before to obtain a key area image with normalized size for subsequent feature extraction.
Step 3.2: lesion region feature extraction
And (3) extracting the characteristics of the R key areas which are selected, wherein the value of R is 8, the structure of the characteristic extraction network is similar to that of the convolutional neural network for extracting the global characteristics in the step (2), the partial network also comprises a convolutional layer, a pooling layer, a full connection layer and other structures, the range of the frame selection area is checked before the characteristic extraction is carried out, and the characteristic extraction cannot be carried out under the condition that the frame selection area is out of range. The network extracts the feature vectors and feature maps of the R regions respectively by using the convolutional layers, and compresses the R regions by using the pooling layers to obtain the feature vectors of the R lesion regions, wherein the size of each vector is represented as T multiplied by 1. And inputting the obtained feature vector into a classification network with a structure as shown in step 2, and mapping the feature vector to a mark space by using a full connection layer to obtain a prediction label for calculating a loss function later, feeding back and adjusting network parameters, and improving the extraction effect of the network on features.
Step 3.3: vector stitching
And splicing the obtained global feature vector and the key region feature vector to obtain a one-dimensional spliced vector. The obtained one-dimensional vector inherits the global features from the whole image, integrates the features from the key area, supplements information on a smaller scale, enables the splicing features to feed back information from the whole image and the local image, gives consideration to the integrity and the specificity, uses the splicing vector for classification, combines the label calculation loss, corrects the network parameters, and can further improve the overall performance of the network.
And (3) similarly using a single-layer fully-connected neural network for the network for classifying the splicing vectors, and mapping the splicing characteristic vectors which are obtained by splicing and have the size of (T + T multiplied by R) multiplied by 1 to a sample label space to finish a classification task. The classification result can relatively effectively reflect the feature extraction capability of the network, and simultaneously participate in the calculation of the loss function, feed back the neural network and adjust the parameters. The flow of vector stitching is shown in fig. 5.
And 4, step 4: neural network training
The training process of the initial master neural network model in the present invention is shown in fig. 6.
In the process of training the neural network, considering the problem of the size of a data set, the invention uses a transfer learning method to pre-train an initial main neural network on a large-scale public data set, and presets network parameters on the basis of the pre-training, so that the training cost can be obviously reduced, the performance of the overall model is improved, and other networks in the overall model are set by adopting a network parameter random initialization method. In addition, in consideration of the size of the data set used by the method, a batch standardization module is added in the network structure, so that the convergence speed of the whole network is accelerated, the requirement of the model on initialization is reduced, the network can accelerate convergence with a higher learning rate, the generalization performance of the whole network is further improved, and the overfitting phenomenon caused by the size of the data set is avoided as much as possible.
For the calculation of the network loss function, the loss function is defined in the present invention as:
Ltotal＝L*+λLg+βLp
wherein L istotalError between identification and tag for global network characteristics, L*For losses diagnosed in the spliced feature network, LgFor global losses, for feedback improving the ability of the network to extract global features, LpFor key area loss, the ability of acquiring the characteristics of lesion areas by a network is improved, parameters lambda and beta are used for adjusting the weight of global loss and key area loss in the calculation of a loss function, 1 is preset in the training process of the model, the overall model uses a gradient descent algorithm with momentum as an optimization algorithm, the loss function is minimized, network parameters are iteratively updated by an error back propagation method until the error is smaller than a preset threshold value or reaches a preset iteration number, a group of parameters which are best in performance in the training process are stored as a network model, and the network model waits for the test setAnd performing performance verification. In this process, L will also be alignedp、LgAnd performing error minimization on the two errors, and simultaneously performing parameter adjustment on the corresponding key region feature extraction network and global feature extraction network by using an error back propagation method to improve the feature extraction capability of the corresponding networks.
In the network training process, the final diagnosis result of whether the patient is ill or not is given by a multi-decision mode, different weights are given to the diagnosis results of the global feature vector, the key region feature vector and the splicing feature vector, the decision threshold values of the three results are specified, the final score is obtained through calculation and then is compared with the threshold values, and the diagnosis of whether the patient is ill or not is completed.
In the concrete calculation of the network loss function, the invention adopts the cross entropy loss function to calculate the error between the network diagnosis result and the label, and the calculation formula of the cross entropy function is as follows:
Figure BDA0003364511280000141
wherein i represents yiRepresents the classification number of samples, where i is 2, the positive case sample label is 0, the negative case sample label is 1, piCounts are the probability that a sample is predicted to be a positive example.
And 5: network performance verification
All data sets are divided into two sets, namely a training set and a test set, data of the two sets are labeled, but data of the test set do not participate in the training process of the network, and meanwhile data augmentation is only carried out in the training set. And (5) classifying the images on the test set by using the network model with the best performance, which is stored after being trained in the step 5, comparing the classification result with the label, counting the classification correct number of each condition for calculating common parameters of the measurement model, such as accuracy, precision and recall rate, and using an F1 index to neutralize the indexes so as to visually evaluate the performance of the model.
It should be noted that the method of the embodiment of the present application may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment, and the multiple devices interact with each other to complete the method.
It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, the present application further provides a hand bone image analysis apparatus based on feature extraction, as shown in fig. 7, including:
the device comprises a preprocessing module 10, a display module and a display module, wherein the preprocessing module is configured to preprocess an initial hand bone image of a patient to obtain a hand bone image;
the global diagnosis module 20 is configured to extract and obtain global features of the hand bone image by a master neural network and generate global feature vectors of the hand bone image, and the master neural network performs global diagnosis on the global feature vectors to obtain a global diagnosis result;
a key area framing module 30 configured to frame a plurality of key areas of the hand bone image by using an area framing neural network to obtain a plurality of key area images;
a key region diagnosis module 40 configured to extract features of the plurality of key region images by the primary neural network, obtain key region features of the plurality of hand bone images, and generate key region feature vectors of the plurality of hand bone images, wherein the primary neural network performs key region diagnosis on the plurality of key region feature vectors to obtain a plurality of key region diagnosis results;
a splicing diagnosis module 50 configured to splice the global feature vector and the key region feature vector by the master neural network to obtain a spliced vector, and perform splicing diagnosis on the spliced vector by the master neural network to obtain a splicing diagnosis result;
and a final result diagnosis module 60 configured to perform comprehensive diagnosis on the hand bone image by the master neural network according to the global diagnosis result, the key region diagnosis result and the stitching diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
The device comprises a preprocessing module, a global diagnosis module, a key region framing module, a key region diagnosis module, a splicing diagnosis module and a final result diagnosis module, wherein firstly, a trained main neural network extracts global features of a hand bone image of a patient to obtain a global feature vector, then extracts key region features through a framing key region to obtain a key region feature vector, and then splices the global feature vector and the key region feature vector to obtain a spliced vector; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.
In some embodiments, the apparatus further comprises:
a construction module configured to construct the initial master neural network and the initial region boxed neural network.
An acquisition module configured to acquire the data set, wherein the data set comprises a plurality of initial sample hand bone images, a diagnosis result obtained by manually performing comprehensive diagnosis on the plurality of initial sample hand bone images, and coordinates of a key area manually framed by the plurality of initial sample hand bone images.
And the main neural network training module is configured to train the initial main neural network based on the data set to obtain the main neural network.
And the area frame selection neural network training module is configured to train the initial area frame selection neural network based on the data set to obtain the area frame selection neural network.
In some embodiments, the master neural network training module comprises:
and the preprocessing submodule is configured to acquire the initial sample hand bone image from the data set, and preprocess the initial sample hand bone image to obtain a sample hand bone image.
A global diagnosis sub-module configured to perform global diagnosis processing on the sample hand bone image by the initial master neural network to obtain a sample global diagnosis result, and calculate a global loss L by using the sample global diagnosis resultg。
A key region diagnosis sub-module configured to perform key region diagnosis processing on the sample hand bone image by the initial main neural network to obtain a plurality of sample key region diagnosis results, and calculate a key region loss L by using the plurality of sample key region diagnosis resultsp。
A splicing diagnosis submodule configured to perform splicing diagnosis processing on the sample hand bone image by the initial main neural network to obtain a plurality of sample splicing diagnosis results, and calculate a splicing loss L by using the plurality of sample splicing diagnosis results*。
A first computation submodule configured to utilize the global loss L by the initial master neural networkgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss Ltotal。
A training submodule configured to base overall network loss L ontotalAdjusting the parameters of the initial main neural network to complete the initializationAnd (5) one training of the main neural network.
A response submodule configured to respond to the determination of the overall network loss LtotalAnd finishing training on all the data sets of the initial main neural network or less than or equal to a preset threshold value, and taking the trained initial main neural network as the main neural network.
In some embodiments, the apparatus further comprises:
the global diagnostic submodule is further configured to: the initial master neural network extracts and obtains sample global features of the sample hand bone image and generates a sample global feature vector of the sample hand bone image, and the initial master neural network carries out global diagnosis on the sample global feature vector to obtain a sample global diagnosis result.
The critical region diagnostic sub-module is further configured to: the initial area framing neural network frames a plurality of key areas of the sample hand bone image to obtain a plurality of sample key area images; the initial master neural network extracts the characteristics of the sample key area images to obtain the sample key area characteristics of the sample hand bone images and generates a plurality of sample key area characteristic vectors of the sample hand bone images, and the initial master neural network carries out key area diagnosis on the sample key area characteristic vectors to obtain a plurality of sample key area diagnosis results.
The splice diagnostic submodule is further configured to: and the initial master neural network splices the sample global feature vector and the sample key region feature vector to obtain a sample splicing vector, and the initial master neural network splices and diagnoses the sample splicing vector to obtain a sample splicing diagnosis result.
In some embodiments, the first computation submodule comprises: the initial main neural network calculates the overall network loss L by using the following formulatotal：
Ltotal＝L*+λLg+βLp
Wherein the parameters λ and β are the global loss LgWith the critical area loss LpThe weight of (c).
In some embodiments, the end result diagnostic module comprises:
and the second calculation submodule is configured to endow different weights to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result by the main neural network, and obtain a result numerical value through calculation.
And the final result diagnosis sub-module is configured to obtain the final diagnosis result by comparing the result value with a preset diagnosis threshold value through the main neural network.
In some embodiments, the global features include distribution of the hand bones in the hand bone image and texture of the bones; the key areas comprise middle and base epiphysis of ring finger, middle finger and index finger, end epiphysis of thumb and proximal end of third metacarpal bone; the critical area features include metaphyseal pits, hardening, asperities.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations as the present application.
The device of the above embodiment is used to implement the corresponding hand bone image analysis method based on feature extraction in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to the method of any embodiment described above, the present disclosure further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the hand bone image analysis method based on feature extraction described in any embodiment above is implemented.
Fig. 8 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
Bus 1050 includes a path that transfers information between various components of the device, such as processor 1010, memory 1020, input/output interface 1030, and communication interface 1040.
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the above embodiment is used to implement the corresponding hand bone image analysis method based on feature extraction in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the hand bone image analysis method based on feature extraction as described in any of the above embodiments.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the hand bone image analysis method based on feature extraction according to any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described herein again.
The embodiment corresponding to the application mainly realizes the following technical effects:
1) the hand bone image analysis method based on feature extraction comprises the steps that firstly, a trained main neural network extracts global features of a hand bone image of a patient to obtain global feature vectors, key region features are extracted through a frame selection key region to obtain key region feature vectors, and then the global feature vectors and the key region feature vectors are spliced to obtain spliced vectors; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.
2) The device comprises a preprocessing module, a global diagnosis module, a key region framing module, a key region diagnosis module, a splicing diagnosis module and a final result diagnosis module, wherein firstly, a trained main neural network extracts global features of a hand bone image of a patient to obtain a global feature vector, then extracts key region features through a framing key region to obtain a key region feature vector, and then splices the global feature vector and the key region feature vector to obtain a spliced vector; the trained main neural network diagnoses the global feature vector, the key region feature vector and the splicing vector respectively to obtain a global diagnosis result, a key region diagnosis result and a splicing diagnosis result, and then comprehensively diagnoses the hand bone image according to the three diagnosis results to judge whether the patient suffers from the Kaschin-Beck disease. The method provided by the application ensures the efficiency and accuracy of diagnosis of the Kaschin-Beck disease, meets the requirement of large-scale screening of Kaschin-Beck disease patients, and solves the problem of low efficiency of manually judging whether the image is ill or not in large-scale screening in the related technology.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the context of the present application, features from the above embodiments or from different embodiments may also be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the application. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the application, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the application are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the application, it should be apparent to one skilled in the art that the embodiments of the application can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The present embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present application are intended to be included within the scope of the present application.

Claims (10)

1. A hand bone image analysis method based on feature extraction is characterized by comprising the following steps:
preprocessing an initial hand bone image of a patient to obtain a hand bone image;
the master neural network extracts and obtains global features of the hand bone image and generates a global feature vector of the hand bone image, the master neural network carries out global diagnosis on the global feature vector to obtain a global diagnosis result, wherein the master neural network is obtained by training an initial master neural network by utilizing an acquired data set;
the area frame selection neural network frames a plurality of key areas of the hand bone image to obtain a plurality of key area images, wherein the area frame selection neural network is obtained by training an initial area frame selection neural network by using the acquired data set;
the master neural network extracts the characteristics of the key area images to obtain key area characteristics of the hand bone images and generates key area characteristic vectors of the hand bone images, and the master neural network carries out key area diagnosis on the key area characteristic vectors to obtain a plurality of key area diagnosis results;
the master neural network splices the global feature vector and the key region feature vector to obtain a spliced vector, and the master neural network splices and diagnoses the spliced vector to obtain a spliced diagnosis result;
and the master neural network carries out comprehensive diagnosis on the hand bone image according to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
2. The hand bone image analysis method based on feature extraction as claimed in claim 1, further comprising:
constructing the initial main neural network and the initial region frame selection neural network;
acquiring the data set, wherein the data set comprises a plurality of initial sample hand bone images, a diagnosis result obtained by manually carrying out comprehensive diagnosis on the plurality of initial sample hand bone images, and coordinates of a key area manually framed by the plurality of initial sample hand bone images;
the initial main neural network is trained based on the data set to obtain the main neural network;
and training the initial region frame selection neural network based on the data set to obtain the region frame selection neural network.
3. The method for analyzing hand bone image based on feature extraction of claim 2, wherein the training of the initial master neural network based on the data set to obtain the master neural network comprises:
acquiring the initial sample hand bone image from the data set, and preprocessing the initial sample hand bone image to obtain a sample hand bone image;
the initial main neural network carries out global diagnosis processing on the hand bone image of the sample to obtain a sample global diagnosis result, and the sample global diagnosis result is utilized to calculate and obtain a global loss Lg；
The initial main neural network carries out key region diagnosis processing on the sample hand bone image to obtain a plurality of sample key region diagnosis results, and key region loss L is calculated by utilizing the plurality of sample key region diagnosis resultsp；
The initial main neural network carries out splicing diagnosis processing on the hand bone images of the samples to obtain a plurality of sample splicing diagnosis results, and the splicing loss L is calculated by utilizing the plurality of sample splicing diagnosis results*；
The initial master neural network utilizes the global loss LgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss Ltotal；
Based on the overall network loss LtotalAdjusting parameters of the initial main neural network to finish one training of the initial main neural network;
in response to determining the overall network loss LtotalAnd finishing training on all the data sets of the initial main neural network or less than or equal to a preset threshold value, and taking the trained initial main neural network as the main neural network.
4. The hand bone image analysis method based on feature extraction as claimed in claim 3, further comprising:
the initial main neural network carries out global diagnosis processing on the sample hand bone image to obtain a sample global diagnosis result, and the method comprises the following steps:
the initial master neural network extracts and obtains sample global features of the sample hand bone image and generates a sample global feature vector of the sample hand bone image, and the initial master neural network carries out global diagnosis on the sample global feature vector to obtain a sample global diagnosis result;
the initial main neural network carries out key region diagnosis processing on the sample hand bone image to obtain a plurality of sample key region diagnosis results, and the method comprises the following steps:
the initial area framing neural network frames a plurality of key areas of the sample hand bone image to obtain a plurality of sample key area images;
the initial master neural network extracts the characteristics of the sample key area images to obtain the sample key area characteristics of the sample hand bone images and generates a plurality of sample key area characteristic vectors of the sample hand bone images, and the initial master neural network carries out key area diagnosis on the sample key area characteristic vectors to obtain a plurality of sample key area diagnosis results;
the initial main neural network carries out splicing diagnosis processing on the sample hand bone images to obtain a plurality of sample splicing diagnosis results, and the method comprises the following steps:
and the initial master neural network splices the sample global feature vector and the sample key region feature vector to obtain a sample splicing vector, and the initial master neural network splices and diagnoses the sample splicing vector to obtain a sample splicing diagnosis result.
5. The method of claim 3, wherein the initial autonomic neural network utilizes the global loss LgThe critical area loss LpAnd the splice loss L*Calculating the overall network loss LtotalThe method comprises the following steps:
the initial main neural network calculates the overall network loss L by using the following formulatotal：
Ltotal＝L*+λLg+βLp
Wherein the parameters λ and β are the global loss LgWith the critical area loss LpThe weight of (c).
6. The hand bone image analysis method based on feature extraction as claimed in claim 1, wherein the master neural network performs comprehensive diagnosis on the hand bone image according to the global diagnosis result, the key region diagnosis result and the stitching diagnosis result to obtain a final diagnosis result of whether the patient suffers from Kaschin-Beck disease, and comprises:
the main neural network gives different weights to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result, and a result numerical value is obtained through calculation;
and the main neural network obtains the final diagnosis result by comparing the result value with a preset diagnosis threshold value.
7. The hand bone image analysis method based on feature extraction as claimed in claim 1, wherein the global features include distribution of hand bones and textures of bones in the hand bone image; the key areas comprise middle and base epiphysis of ring finger, middle finger and index finger, end epiphysis of thumb and proximal end of third metacarpal bone; the critical area features include metaphyseal pits, hardening, asperities.
8. A hand bone image analysis device based on feature extraction is characterized by comprising:
the preprocessing module is configured to preprocess the initial hand bone image of the patient to obtain a hand bone image;
the global diagnosis module is configured to extract and obtain global features of the hand bone image by a main neural network and generate global feature vectors of the hand bone image, and the main neural network carries out global diagnosis on the global feature vectors to obtain a global diagnosis result;
a key area framing module configured to frame a plurality of key areas of the hand bone image by using an area framing neural network to obtain a plurality of key area images;
a key region diagnosis module configured to extract features of the plurality of key region images by the master neural network, obtain key region features of the plurality of hand bone images, and generate key region feature vectors of the plurality of hand bone images, wherein the master neural network performs key region diagnosis on the plurality of key region feature vectors to obtain a plurality of key region diagnosis results;
the splicing diagnosis module is configured to splice the global feature vector and the key region feature vector by the main neural network to obtain a spliced vector, and the main neural network performs splicing diagnosis on the spliced vector to obtain a splicing diagnosis result;
and the final result diagnosis module is configured to perform comprehensive diagnosis on the hand bone image by the master neural network according to the global diagnosis result, the key region diagnosis result and the splicing diagnosis result to obtain a final diagnosis result of whether the patient suffers from the Kaschin-Beck disease.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the program.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.
CN202111382606.2A 2021-11-19 2021-11-19 Hand bone image analysis method based on feature extraction and related equipment Pending CN114359133A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111382606.2A CN114359133A (en) 2021-11-19 2021-11-19 Hand bone image analysis method based on feature extraction and related equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111382606.2A CN114359133A (en) 2021-11-19 2021-11-19 Hand bone image analysis method based on feature extraction and related equipment

Publications (1)

Publication Number Publication Date
CN114359133A true CN114359133A (en) 2022-04-15

Family

ID=81095361

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111382606.2A Pending CN114359133A (en) 2021-11-19 2021-11-19 Hand bone image analysis method based on feature extraction and related equipment

Country Status (1)

Country Link
CN (1) CN114359133A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117853432A (en) * 2023-12-26 2024-04-09 北京长木谷医疗科技股份有限公司 Hybrid model-based osteoarthropathy identification method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117853432A (en) * 2023-12-26 2024-04-09 北京长木谷医疗科技股份有限公司 Hybrid model-based osteoarthropathy identification method and device

Similar Documents

Publication Publication Date Title
US11288550B2 (en) Data processing apparatus and method, recognition apparatus, learning data storage apparatus, machine learning apparatus, and program
WO2020215984A1 (en) Medical image detection method based on deep learning, and related device
CN109035187B (en) Medical image labeling method and device
US10733727B2 (en) Application of deep learning for medical imaging evaluation
CN112233117A (en) New coronary pneumonia CT detects discernment positioning system and computing equipment
CN109191451B (en) Abnormality detection method, apparatus, device, and medium
CN112614133B (en) Three-dimensional pulmonary nodule detection model training method and device without anchor point frame
CN111612756B (en) Coronary artery specificity calcification detection method and device
CN113487656B (en) Image registration method and device, training method and device, control method and device
Ankenbrand et al. Deep learning‐based cardiac cine segmentation: Transfer learning application to 7T ultrahigh‐field MRI
CN110969623A (en) Lung CT multi-symptom automatic detection method, system, terminal and storage medium
US20220374768A1 (en) Apparatus of machine learning, machine learning method, and inference apparatus
CN114387317A (en) CT image and MRI three-dimensional image registration method and device
CN109147927B (en) Man-machine interaction method, device, equipment and medium
CN110570425A (en) Lung nodule analysis method and device based on deep reinforcement learning algorithm
CN114359133A (en) Hand bone image analysis method based on feature extraction and related equipment
Jiménez-Sánchez et al. Detecting shortcuts in medical images-a case study in chest x-rays
CN109214451A (en) A kind of classification method and equipment of brain exception
Pandey et al. An extended deep learning based solution for screening COVID-19 CT-scans
EP4327333A1 (en) Methods and systems for automated follow-up reading of medical image data
CN113112463A (en) Medical image quality evaluation method, electronic device, and storage medium
Monroy et al. Automated chronic wounds medical assessment and tracking framework based on deep learning
CN111326227A (en) Case report generation method, case report generation device, case report generation equipment and storage medium
CN116071372B (en) Knee joint segmentation method, knee joint segmentation device, electronic equipment and storage medium
KR102374275B1 (en) Method and apparatus for measuring body size by matching medical images with different modalities

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination