CN112733708A - Hepatic portal vein detection positioning method and system based on semi-supervised learning - Google Patents

Hepatic portal vein detection positioning method and system based on semi-supervised learning Download PDF

Info

Publication number
CN112733708A
CN112733708A CN202110023112.9A CN202110023112A CN112733708A CN 112733708 A CN112733708 A CN 112733708A CN 202110023112 A CN202110023112 A CN 202110023112A CN 112733708 A CN112733708 A CN 112733708A
Authority
CN
China
Prior art keywords
hepatic portal
portal vein
attention
semi
supervised learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110023112.9A
Other languages
Chinese (zh)
Inventor
李克峰
刘泉凯
张广渊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Jiaotong University
Original Assignee
Shandong Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Jiaotong University filed Critical Shandong Jiaotong University
Priority to CN202110023112.9A priority Critical patent/CN112733708A/en
Publication of CN112733708A publication Critical patent/CN112733708A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/251—Fusion techniques of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30056—Liver; Hepatic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/14—Vascular patterns

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a hepatic portal vein detection positioning method and a hepatic portal vein detection positioning system based on semi-supervised learning, model training is carried out by using a model of semi-supervised learning, feature extraction is carried out on unlabelled data, and the probability of whether the feature is hepatic portal vein is taken as a soft label, so that model training can be carried out by using a small amount of label samples and a large amount of unlabelled sample data, and the manual labeling cost is greatly reduced; in addition, according to the position characteristics of the hepatic portal vein structure, focus attention is formed by the serial connection of channel attention and space attention, so that priori knowledge is input into a convolutional neural network to guide model training, and the defects of the existing hepatic portal vein detection are overcome; and replacing the loss function, selecting the focal local loss function as the loss function of the classifier, and normally training under the condition of unbalance of positive and negative samples.

Description

Hepatic portal vein detection positioning method and system based on semi-supervised learning
Technical Field
The invention relates to the technical field of medical image processing, in particular to a hepatic portal vein detection positioning method and system based on semi-supervised learning.
Background
The hepatic portal vein is used as a very important vein of a human body, provides very important support for normal life activities of the human body, and is often required to detect and position the hepatic portal vein of a CT image in clinical medical treatment.
The detection and location of the hepatic portal vein in the CT image in the past are usually manually identified by a professional physician, the working efficiency is low, the accuracy completely depends on the working experience of the physician, and the problem of location error is inevitable. Nowadays, with the development of deep learning technology, the application of the deep learning technology to the target detection and positioning of CT images is in the trend. The target detection algorithms are various and mainly comprise two detection modes, namely single-stage detection and double-stage detection, wherein the single-stage detection mainly comprises a YOLO series, an SSD series, RetinaNet and the like, the double-stage detection mainly comprises Faster R-CNN, Mask R-CNN, SSP-Net and the like, the single-stage detection has the advantage of high real-time detection speed, and the double-stage detection has the advantage of high detection precision.
Most target detection algorithms are full-supervised learning at present, although the detection accuracy is high, a large amount of manual labeling data are needed to support the training of the models, the manual labeling is tedious and time-consuming, and the economic cost of the models is greatly improved. In addition, most target detection algorithms are not strong in pertinence to hepatic portal vein detection, which is reflected in that no pertinence improvement is made on hepatic portal vein detection, and most target detection algorithm training has certain requirements on the distribution of positive and negative samples of data, and if the distribution is unbalanced, the model cannot be trained ideally.
Disclosure of Invention
The invention aims to provide a hepatic portal vein detection and positioning method and system based on semi-supervised learning, and aims to solve the problems that manual labeling is long in time consumption and the hepatic portal vein detection and positioning pertinence is not strong in the prior art, reduce the manual labeling cost and improve the positioning accuracy pertinently.
In order to achieve the technical purpose, the invention provides a hepatic portal vein detection and positioning method based on semi-supervised learning, which comprises the following operations:
inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability as a soft label into the image;
carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and taking the fusion data set as a training set, inputting a convolutional neural network added with priori knowledge for feature extraction, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and performing coordinate regression to obtain target positioning.
Preferably, the softmax function formula is as follows:
Figure BDA0002889380850000021
wherein Z is the prediction category, j is the total category number, and i is the current category index.
Preferably, the formula of the MixUp fusion is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
the lambda obeys Beta distribution, the maximum value of the lambda is compared, if the lambda is larger than 0.5, the lambda value is kept unchanged, and if the lambda is not larger than 0.5, 1-lambda is selected as a new lambda value; x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p.
Preferably, the a priori knowledge is added in the form of focus attention, which consists of a concatenation of channel attention and spatial attention;
Figure BDA0002889380850000022
wherein, AttspatialFor spatial attention, AttchannelAttention is paid to the channel;
and respectively inputting the feature maps into the multilayer perceptron after maximum pooling and average pooling, and adding the output results after activation to obtain channel attention:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is average pooling, Maxpool (F) is maximum pooling;
after the characteristic diagram is subjected to maximum pooling and average pooling, activation is performed firstly, and after activation, selective weight change is performed on the image according to the structural characteristics of the hepatic portal vein, so that the spatial attention is obtained, wherein the formula is as follows:
Figure BDA0002889380850000031
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
The invention also provides a hepatic portal vein detection and positioning system based on semi-supervised learning, which comprises:
the soft label adding module is used for inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability into the image as a soft label;
the data fusion module is used for carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and the training identification module is used for taking the fusion data set as a training set, inputting the convolution neural network added with priori knowledge to extract features, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and coordinate regression to obtain target positioning.
Preferably, the softmax function formula is as follows:
Figure BDA0002889380850000032
wherein Z is the prediction category, j is the total category number, and i is the current category index.
Preferably, the formula of the MixUp fusion is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
the lambda obeys Beta distribution, the maximum value of the lambda is compared, if the lambda is larger than 0.5, the lambda value is kept unchanged, and if the lambda is not larger than 0.5, 1-lambda is selected as a new lambda value; x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p.
Preferably, the a priori knowledge is added in the form of focus attention, which consists of a concatenation of channel attention and spatial attention;
Figure BDA0002889380850000041
wherein, AttspatialFor spatial attention, AttchannelAttention is paid to the channel;
and respectively inputting the feature maps into the multilayer perceptron after maximum pooling and average pooling, and adding the output results after activation to obtain channel attention:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is average pooling, Maxpool (F) is maximum pooling;
after the characteristic diagram is subjected to maximum pooling and average pooling, activation is performed firstly, and after activation, selective weight change is performed on the image according to the structural characteristics of the hepatic portal vein, so that the spatial attention is obtained, wherein the formula is as follows:
Figure BDA0002889380850000042
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
The invention also provides a hepatic portal vein detection positioning device based on semi-supervised learning, which comprises:
a memory for storing a computer program;
and the processor is used for executing the computer program to realize the hepatic portal vein detection and positioning method based on semi-supervised learning.
The invention also provides a readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the hepatic portal vein detection and positioning method based on semi-supervised learning.
The effect provided in the summary of the invention is only the effect of the embodiment, not all the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
compared with the prior art, the method has the advantages that the model training is carried out by using the semi-supervised learning model, the characteristic extraction is carried out on the label-free data, and the probability of whether the characteristic is the hepatic portal vein is taken as a soft label, so that the model training can be carried out by using a small amount of label samples and a large amount of label-free sample data, and the manual labeling cost is greatly reduced; in addition, according to the position characteristics of the hepatic portal vein structure, focus attention is formed by the serial connection of channel attention and space attention, so that priori knowledge is input into a convolutional neural network to guide model training, and the defects of the existing hepatic portal vein detection are overcome; and replacing the loss function, selecting the focal local loss function as the loss function of the classifier, and normally training under the condition of unbalance of positive and negative samples.
Drawings
Fig. 1 is a flowchart of a hepatic portal vein detection and positioning method based on semi-supervised learning according to an embodiment of the present invention;
fig. 2 is an overall structural diagram of a neural network provided in the embodiment of the present invention;
FIG. 3 is a schematic view of a channel attention structure provided in an embodiment of the present invention;
FIG. 4 is a schematic view of a spatial attention structure provided in an embodiment of the present invention;
fig. 5 is a block diagram of a hepatic portal vein detection and localization system based on semi-supervised learning according to an embodiment of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
The following describes a hepatic portal vein detection and location method and system based on semi-supervised learning in detail with reference to the accompanying drawings.
As shown in fig. 1 and 2, an embodiment of the present invention discloses a hepatic portal vein detection and positioning method based on semi-supervised learning, which includes the following operations:
inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability as a soft label into the image;
carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and taking the fusion data set as a training set, inputting a convolutional neural network added with priori knowledge for feature extraction, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and performing coordinate regression to obtain target positioning.
Inputting label-free data into a network, obtaining augmentation data through vertical turnover and random shearing, extracting features through convolution operation of the data through a convolution neural network added with priori knowledge, calculating the probability of whether the extracted features are hepatic portal veins by using a softmax function during prediction, and using the probability as a soft label for subsequent model training. Wherein, the formula of the softmax function is as follows:
Figure BDA0002889380850000061
wherein Z is the prediction category, j is the total category number, and i is the current category index.
Carrying out MixUp fusion on the picture with the soft label and the picture with the label to obtain a fusion data set, wherein the MixUp fusion formula is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
and (4) the lambda obeys Beta distribution, the maximum value comparison is carried out on the lambda, if the lambda is larger than 0.5 lambda value, the lambda value is kept unchanged, and if the lambda is not larger than 0.5 lambda value, 1-lambda is selected as a new lambda value. x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p. And (4) fusing data as a training set, inputting a feature extraction network with the added prior knowledge, and extracting features.
For the construction of the feature extraction network added with the prior knowledge, the prior knowledge is added in a focus attention mode, and the focus attention is formed by connecting channel attention and space attention in series.
As shown in fig. 3, for the channel attention, the feature map is input to the multi-layered sensor after being subjected to maximum pooling and average pooling simultaneously, the output results are activated and added to obtain the channel attention, and the channel attention is then subjected to inner product with the original image to obtain the feature map with the channel attention, wherein the formula is as follows:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is the average pooling, and Maxpool (F) is the maximum pooling.
As shown in fig. 4, for spatial attention, after the feature map is maximally pooled and averagely pooled, the feature map is activated first, and after activation, the image is selectively weighted according to the structural features of the hepatic portal vein, so as to obtain spatial attention, wherein the formula is as follows:
Figure BDA0002889380850000071
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
And (3) performing inner product on the feature map with the channel attention and the space attention to finish final focus attention addition, wherein the formula is as follows:
Figure BDA0002889380850000072
after CT image feature extraction, performing logistic regression through a softmax classifier, completing target detection and coordinate regression, and completing target positioning, thereby identifying the hepatic portal vein. The focal loss is selected from the classifier as the loss function of the classifier, so that the neural network can still be normally trained under the condition of unbalance of positive and negative samples.
Model training is carried out by using a semi-supervised learning model, and model training can be carried out by using a small amount of label samples and a large amount of label-free sample data, so that the manual labeling cost is greatly reduced, and in addition, according to the position characteristics of the hepatic portal vein structure, a design guidance module inputs priori knowledge into a convolutional neural network to guide model training, so that the defects of the existing hepatic portal vein detection are overcome.
As shown in fig. 5, the embodiment of the present invention further discloses a hepatic portal vein detection and localization system based on semi-supervised learning, wherein the system includes:
the soft label adding module is used for inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability into the image as a soft label;
the data fusion module is used for carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and the training identification module is used for taking the fusion data set as a training set, inputting the convolution neural network added with priori knowledge to extract features, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and coordinate regression to obtain target positioning.
And adding the image soft label through a soft label adding module. Inputting label-free data into a network, obtaining augmentation data through vertical turnover and random shearing, extracting features through convolution operation of the data through a convolution neural network added with priori knowledge, calculating the probability of whether the extracted features are hepatic portal veins by using a softmax function during prediction, and using the probability as a soft label for subsequent model training. Wherein, the formula of the softmax function is as follows:
Figure BDA0002889380850000081
wherein Z is the prediction category, j is the total category number, and i is the current category index.
Carrying out MixUp fusion on the picture with the soft label and the picture with the label through a data fusion module to obtain a fusion data set, wherein a MixUp fusion formula is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
and (4) the lambda obeys Beta distribution, the maximum value comparison is carried out on the lambda, if the lambda is larger than 0.5 lambda value, the lambda value is kept unchanged, and if the lambda is not larger than 0.5 lambda value, 1-lambda is selected as a new lambda value. x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p. And (4) fusing data as a training set, inputting a feature extraction network with the added prior knowledge, and extracting features.
For the construction of the feature extraction network added with the prior knowledge, the prior knowledge is added in a focus attention mode, and the focus attention is formed by connecting channel attention and space attention in series.
For the channel attention, the feature map is input into the multilayer perceptron after maximum pooling and average pooling simultaneously, the output result is added after activation to obtain the channel attention, and the channel attention is then subjected to inner product with the original image to obtain the feature map with the channel attention, wherein the formula is as follows:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is the average pooling, and Maxpool (F) is the maximum pooling.
For spatial attention, after the characteristic diagram is subjected to maximum pooling and average pooling, activation is firstly carried out, and after activation, selective weight change is carried out on the image according to the structural characteristics of the hepatic portal vein, so that the spatial attention is obtained, wherein the formula is as follows:
Figure BDA0002889380850000091
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
And (3) performing inner product on the feature map with the channel attention and the space attention to finish final focus attention addition, wherein the formula is as follows:
Figure BDA0002889380850000092
after the CT image features are extracted, performing logistic regression through a softmax classifier, completing the detection of the target and the positioning of the target through coordinate regression, thereby identifying the hepatic portal vein and completing the positioning of the training identification module on the hepatic portal vein. The focal loss is selected from the classifier as the loss function of the classifier, so that the neural network can still be normally trained under the condition of unbalance of positive and negative samples.
The embodiment of the invention also discloses hepatic portal vein detection and positioning equipment based on semi-supervised learning, which comprises:
a memory for storing a computer program;
and the processor is used for executing the computer program to realize the hepatic portal vein detection and positioning method based on semi-supervised learning.
The embodiment of the invention also discloses a readable storage medium for storing a computer program, wherein the computer program is used for realizing the hepatic portal vein detection and positioning method based on semi-supervised learning when being executed by a processor.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (10)

1. A hepatic portal vein detection and positioning method based on semi-supervised learning is characterized by comprising the following operations:
inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability as a soft label into the image;
carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and taking the fusion data set as a training set, inputting a convolutional neural network added with priori knowledge for feature extraction, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and performing coordinate regression to obtain target positioning.
2. The hepatic portal vein detection and positioning method based on semi-supervised learning of claim 1, wherein the softmax function is formulated as follows:
Figure FDA0002889380840000011
wherein Z is the prediction category, j is the total category number, and i is the current category index.
3. The hepatic portal vein detection and localization method based on semi-supervised learning of claim 1, wherein the formula of MixUp fusion is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
the lambda obeys Beta distribution, the maximum value of the lambda is compared, if the lambda is larger than 0.5, the lambda value is kept unchanged, and if the lambda is not larger than 0.5, 1-lambda is selected as a new lambda value; x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p.
4. The hepatic portal vein detection and positioning method based on semi-supervised learning as recited in claim 1, wherein the a priori knowledge is added in the form of focus attention, and the focus attention is composed of channel attention and spatial attention in series;
Figure FDA0002889380840000012
wherein, AttspatialFor spatial attention, AttchannelAttention is paid to the channel;
and respectively inputting the feature maps into the multilayer perceptron after maximum pooling and average pooling, and adding the output results after activation to obtain channel attention:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is average pooling, Maxpool (F) is maximum pooling;
after the characteristic diagram is subjected to maximum pooling and average pooling, activation is performed firstly, and after activation, selective weight change is performed on the image according to the structural characteristics of the hepatic portal vein, so that the spatial attention is obtained, wherein the formula is as follows:
Figure FDA0002889380840000021
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
5. A hepatic portal vein detection and location system based on semi-supervised learning, the system comprising:
the soft label adding module is used for inputting label-free image data into a convolutional neural network added with priori knowledge for feature extraction, calculating the probability that the extracted features are hepatic portal veins by utilizing a softmax function, and adding the probability into the image as a soft label;
the data fusion module is used for carrying out MixUp fusion on the soft label image and the labeled image to obtain a fusion data set;
and the training identification module is used for taking the fusion data set as a training set, inputting the convolution neural network added with priori knowledge to extract features, performing logistic regression on the extracted features through a softmax classifier, and identifying a hepatic portal vein target and coordinate regression to obtain target positioning.
6. The semi-supervised learning based hepatic portal vein detection and localization system according to claim 5, wherein the softmax function is formulated as follows:
Figure FDA0002889380840000022
wherein Z is the prediction category, j is the total category number, and i is the current category index.
7. The semi-supervised learning based hepatic portal vein detection and localization system according to claim 5, wherein the formula of MixUp fusion is as follows:
λ-Beta(α,α)
λ＝max(λ,1-λ)
x＝λx1+(1-λ)x2
p＝λp1+(1-λ)p2
lambda obeys Beta distributionAnd comparing the maximum value of the lambda, if the lambda is larger than 0.5, keeping the lambda value unchanged, and if not, selecting 1-lambda as a new lambda value; x is the number of1For labeled CT picture sets, x2For label-free CT picture sets, p1For sets of labels with labelled pictures, p2And respectively fusing the soft label sets generated for the non-labeled pictures to obtain a fused CT picture set x and a fused label set p.
8. The hepatic portal vein detection and positioning system based on semi-supervised learning of claim 5, wherein the a priori knowledge is added in the form of focus attention, and the focus attention is composed of channel attention and spatial attention in series;
Figure FDA0002889380840000031
wherein, AttspatialFor spatial attention, AttchannelAttention is paid to the channel;
and respectively inputting the feature maps into the multilayer perceptron after maximum pooling and average pooling, and adding the output results after activation to obtain channel attention:
Attchannel＝σ(MLP(Avgpool(F))+MLP(Maxpool(F)))
wherein Avgpool (F) is average pooling, Maxpool (F) is maximum pooling;
after the characteristic diagram is subjected to maximum pooling and average pooling, activation is performed firstly, and after activation, selective weight change is performed on the image according to the structural characteristics of the hepatic portal vein, so that the spatial attention is obtained, wherein the formula is as follows:
Figure FDA0002889380840000032
Wspatial＝σ(f7×7([Avgpool(F)；Maxpool(F)]))
wherein w is the picture width, α and γ are enhancement factor and attenuation factor, respectively, m is a parameter for dividing attenuation and enhancement regions, and σ is an activation function.
9. A hepatic portal vein detection positioning device based on semi-supervised learning, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the hepatic portal vein detection localization method based on semi-supervised learning according to any one of claims 1-4.
10. A readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the hepatic portal vein detection localization method based on semi-supervised learning according to any one of claims 1-4.
CN202110023112.9A 2021-01-08 2021-01-08 Hepatic portal vein detection positioning method and system based on semi-supervised learning Pending CN112733708A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110023112.9A CN112733708A (en) 2021-01-08 2021-01-08 Hepatic portal vein detection positioning method and system based on semi-supervised learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110023112.9A CN112733708A (en) 2021-01-08 2021-01-08 Hepatic portal vein detection positioning method and system based on semi-supervised learning

Publications (1)

Publication Number Publication Date
CN112733708A true CN112733708A (en) 2021-04-30

Family

ID=75591262

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110023112.9A Pending CN112733708A (en) 2021-01-08 2021-01-08 Hepatic portal vein detection positioning method and system based on semi-supervised learning

Country Status (1)

Country Link
CN (1) CN112733708A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113379773A (en) * 2021-05-28 2021-09-10 陕西大智慧医疗科技股份有限公司 Dual attention mechanism-based segmentation model establishing and segmenting method and device

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107092918A (en) * 2017-03-29 2017-08-25 太原理工大学 It is a kind of to realize that Lung neoplasm sign knows method for distinguishing based on semantic feature and the image retrieval for having supervision Hash
CN108764281A (en) * 2018-04-18 2018-11-06 华南理工大学 A kind of image classification method learning across task depth network based on semi-supervised step certainly
CN110074809A (en) * 2019-06-17 2019-08-02 祁小龙 The hepatic vein pressure gradient classification method and computer equipment of CT image
CN111626116A (en) * 2020-04-21 2020-09-04 泉州装备制造研究所 Video semantic analysis method based on fusion of multi-attention mechanism and Graph
CN111723209A (en) * 2020-06-28 2020-09-29 上海携旅信息技术有限公司 Semi-supervised text classification model training method, text classification method, system, device and medium
CN112132815A (en) * 2020-09-25 2020-12-25 广州视源电子科技股份有限公司 Pulmonary nodule detection model training method, detection method and device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107092918A (en) * 2017-03-29 2017-08-25 太原理工大学 It is a kind of to realize that Lung neoplasm sign knows method for distinguishing based on semantic feature and the image retrieval for having supervision Hash
CN108764281A (en) * 2018-04-18 2018-11-06 华南理工大学 A kind of image classification method learning across task depth network based on semi-supervised step certainly
CN110074809A (en) * 2019-06-17 2019-08-02 祁小龙 The hepatic vein pressure gradient classification method and computer equipment of CT image
CN111626116A (en) * 2020-04-21 2020-09-04 泉州装备制造研究所 Video semantic analysis method based on fusion of multi-attention mechanism and Graph
CN111723209A (en) * 2020-06-28 2020-09-29 上海携旅信息技术有限公司 Semi-supervised text classification model training method, text classification method, system, device and medium
CN112132815A (en) * 2020-09-25 2020-12-25 广州视源电子科技股份有限公司 Pulmonary nodule detection model training method, detection method and device

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113379773A (en) * 2021-05-28 2021-09-10 陕西大智慧医疗科技股份有限公司 Dual attention mechanism-based segmentation model establishing and segmenting method and device

Similar Documents

Publication Publication Date Title
CN109166094B (en) Insulator fault positioning and identifying method based on deep learning
CN113642431B (en) Training method and device of target detection model, electronic equipment and storage medium
WO2022007193A1 (en) Weak supervision video behavior detection method and system based on iterative learning
CN112418330A (en) Improved SSD (solid State drive) -based high-precision detection method for small target object
CN111507965A (en) Novel coronavirus pneumonia focus detection method, system, device and storage medium
CN111243729B (en) Automatic generation method of lung X-ray chest radiography examination report
CN111080624B (en) Sperm movement state classification method, device, medium and electronic equipment
CN112581450B (en) Pollen detection method based on expansion convolution pyramid and multi-scale pyramid
CN112163530B (en) SSD small target detection method based on feature enhancement and sample selection
CN113487610A (en) Herpes image recognition method and device, computer equipment and storage medium
CN113762269A (en) Chinese character OCR recognition method, system, medium and application based on neural network
CN113159134A (en) Intelligent diagnosis evaluation method based on mammary gland structural report
CN115187772A (en) Training method, device and equipment of target detection network and target detection method, device and equipment
CN112991280B (en) Visual detection method, visual detection system and electronic equipment
CN112991281B (en) Visual detection method, system, electronic equipment and medium
CN115019133A (en) Method and system for detecting weak target in image based on self-training and label anti-noise
CN112733708A (en) Hepatic portal vein detection positioning method and system based on semi-supervised learning
Huang et al. Improved blood cell detection method based on YOLOv5 algorithm
CN109977400B (en) Verification processing method and device, computer storage medium and terminal
CN112818946A (en) Training of age identification model, age identification method and device and electronic equipment
CN115546735B (en) System and method for detecting and identifying icing of cooling tower and storage medium
CN116721291A (en) Metal surface defect detection method based on improved YOLOv7 model
CN116580232A (en) Automatic image labeling method and system and electronic equipment
CN114783597B (en) Method and device for diagnosing multi-class diseases, electronic equipment and storage medium
CN113111804B (en) Face detection method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210430