CN116934754B - Liver image identification method and device based on graph neural network - Google Patents

Liver image identification method and device based on graph neural network Download PDF

Info

Publication number
CN116934754B
CN116934754B CN202311197432.1A CN202311197432A CN116934754B CN 116934754 B CN116934754 B CN 116934754B CN 202311197432 A CN202311197432 A CN 202311197432A CN 116934754 B CN116934754 B CN 116934754B
Authority
CN
China
Prior art keywords
super
fusion
image
edge
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311197432.1A
Other languages
Chinese (zh)
Other versions
CN116934754A (en
Inventor
廖怡
曲海波
李学胜
贾凤林
马鑫茂
罗乐凯
宁刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
West China Second University Hospital of Sichuan University
Original Assignee
West China Second University Hospital of Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by West China Second University Hospital of Sichuan University filed Critical West China Second University Hospital of Sichuan University
Priority to CN202311197432.1A priority Critical patent/CN116934754B/en
Publication of CN116934754A publication Critical patent/CN116934754A/en
Application granted granted Critical
Publication of CN116934754B publication Critical patent/CN116934754B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30056—Liver; Hepatic
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a liver image identification method and a device based on a graph neural network, which relate to the technical field of medical image analysis and comprise the steps of acquiring first information and second information; according to a preset multi-mode attention mathematical model, carrying out fusion processing on the first information to obtain a fusion feature set; performing feature mapping according to the fusion feature set, and performing super-pixel segmentation processing on the multi-channel image to construct a graph data representation set; training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model; and carrying out recognition processing on the second information according to the liver image recognition model to obtain a recognition result. The multi-channel image is subjected to super-pixel segmentation processing, and a graph data representation set is constructed based on the spatial relationship and the modal characteristics among the super-pixel blocks, so that the graph neural network can better capture the structure and characteristic information of the liver image, and the sensitivity to the potential lesion area is enhanced.

Description

Liver image identification method and device based on graph neural network
Technical Field
The application relates to the technical field of medical image analysis, in particular to a liver image identification method and device based on a graph neural network.
Background
In the field of medical imaging today, image recognition technology plays an increasingly important role in disease diagnosis and therapy planning. The liver image recognition is taken as a part of the liver image recognition, and has important significance in the aspects of early detection of liver diseases, lesion type classification, lesion position calibration and the like. However, liver image recognition still has challenges, and multi-modal data, heterogeneous lesions, and large-scale data processing of liver images increase difficulty. The existing liver image analysis and identification mainly adopts a manual rule method. These methods use rules and logic based on manual design to process liver images, detect liver lesions by setting thresholds and shape features, and thereby determine the location and size of the lesions. However, due to the complexity and diversity of liver images, it is often difficult to design appropriate rules and features, resulting in insufficient accuracy and robustness of identification.
Based on the above-mentioned drawbacks of the prior art, a method and a device for identifying liver images based on a neural network are needed.
Disclosure of Invention
The application aims to provide a liver image recognition method and a liver image recognition system based on a graph neural network so as to solve the problems. In order to achieve the above purpose, the technical scheme adopted by the application is as follows:
in one aspect, the application provides a liver image recognition method based on a graph neural network, which comprises the following steps:
acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified;
fusing the first information according to a preset multi-mode attention mathematical model, and performing convolution extraction processing on the input image of each mode through an attention introducing mechanism to obtain a fused feature set;
performing feature mapping according to the fusion feature set, and obtaining a graph data representation set by performing super-pixel segmentation processing on the multi-channel image and constructing based on spatial relations and modal features among super-pixel blocks;
training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model;
and carrying out recognition processing on the second information according to the liver image recognition model to obtain a recognition result, wherein the recognition result comprises a lesion type, a lesion position calibration and credibility.
On the other hand, the application also provides a liver image recognition device based on the graph neural network, which comprises:
the acquisition module is used for acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified;
the fusion module is used for fusing the first information according to a preset multi-mode attention mathematical model, and carrying out convolution extraction processing on the input image of each mode through an attention introducing mechanism to obtain a fusion feature set;
the mapping module is used for carrying out feature mapping according to the fusion feature set, and obtaining a graph data representation set by carrying out super-pixel segmentation processing on the multi-channel image and constructing based on the spatial relationship and the modal feature among the super-pixel blocks;
the construction module is used for training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model;
the identification module is used for carrying out identification processing on the second information according to the liver image identification model to obtain an identification result, wherein the identification result comprises a lesion type, a lesion position calibration and credibility.
The beneficial effects of the application are as follows:
according to the application, the multi-mode image data sets under different lesion types are fused through the multi-mode attention mathematical model, an attention mechanism is introduced to carry out convolution extraction processing on the input image of each mode, so that a fusion feature set is obtained, and the characteristics of images of different modes can be comprehensively utilized by the fusion mode, so that the comprehensiveness and accuracy of liver image information are improved. The multi-channel image is subjected to super-pixel segmentation processing, and a graph data representation set is constructed based on the spatial relationship and the modal characteristics among the super-pixel blocks, so that the graph neural network can better capture the structure and characteristic information of the liver image, and the sensitivity to the potential lesion area is enhanced.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the embodiments of the application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a liver image recognition method based on a graph neural network according to an embodiment of the application;
fig. 2 is a schematic diagram of a liver image recognition device based on a neural network according to an embodiment of the present application.
The marks in the figure: 1. an acquisition module; 2. a fusion module; 21. a first extraction unit; 22. a first detection unit; 23. a first distribution unit; 24. a first fusion unit; 3. a mapping module; 31. a first dividing unit; 311. a second dividing unit; 312. a second detection unit; 313. a second calculation unit; 314. a first merging unit; 32. a first building unit; 33. a second construction unit; 34. a first calculation unit; 35. a third construction unit; 4. constructing a module; 41. a third calculation unit; 42. a first processing unit; 43. a fourth construction unit; 5. and an identification module.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. Meanwhile, in the description of the present application, the terms "first", "second", and the like are used only to distinguish the description, and are not to be construed as indicating or implying relative importance.
Example 1
The embodiment provides a liver image identification method based on a graph neural network.
Referring to fig. 1, the method is shown to include steps S100, S200, S300, S400, and S500.
Step S100, acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified.
It will be appreciated that the first information in this step is composed of multi-modal image datasets for at least two different lesion types, which datasets may contain liver images from different imaging devices, such as CT and MRI. These multi-modality image datasets can provide rich image information covering features of different structures and lesion types of the liver. The second information refers to liver image data of the patient to be identified, typically an image such as CT or MRI. These patient images to be identified are targets for lesion type and location calibration, and are input data for identification processing in the present application.
Step 200, fusing first information according to a preset multi-mode attention mathematical model, and performing convolution extraction processing on the input image of each mode through an attention introducing mechanism to obtain a fused feature set, wherein the multi-mode attention mathematical model is a model which automatically gives different weights to image data of different modes, and focuses on important mode information through the attention introducing mechanism and performs feature extraction and fusion on the input image of each mode through a convolution neural network.
It can be understood that, in this step, different weights are given to the image data of different modes according to a preset multi-mode attention mathematical model. The weights reflect the importance degree of each mode in liver image recognition and can be flexibly adjusted according to actual conditions. By means of the attention mechanism, the model can pay more attention to modal information which is helpful to liver image analysis and lesion detection, and the sensitivity and accuracy of the model are improved. Then, convolution extraction processing is carried out on the input image of each mode, and features are extracted from the image by utilizing deep learning technologies such as a convolution neural network and the like. These features enable capturing local and global information in the image, including features of the anatomy of the liver and the morphology of lesions. Finally, the features of different modes are fused into a unified fused feature set. The fusion feature set integrates information of multiple modes, has higher characterization capability, and is beneficial to improving the performance and stability of the liver image recognition model. The step S200 includes a step S210, a step S220, a step S230, and a step S240.
And step S210, extracting features according to the images of each mode in the first information to obtain a mode feature set.
It can be understood that this step performs feature extraction by using a deep learning technique such as convolutional neural network for the image data of each modality. Convolutional neural networks can effectively learn features with characterization capability from images, which can capture local and global information in the images, including features such as anatomical structures of livers and morphologies of lesions.
And step S220, performing similarity detection according to all the modal features in the same lesion type in the modal feature set to obtain a similarity detection result.
It will be appreciated that this step performs similarity calculations on liver image data of different modalities to measure their similarity in the feature representation. The step S220 includes a step S221, a step S222, and a step S223.
Step S221, normalization processing is carried out on the mode feature set.
It will be appreciated that for each modality feature set we need to normalize it. Normalization can scale the feature data of different modalities to the same extent so that they have consistent importance in the similarity calculation.
Step S222, calculating the modal weight.
For each mode in the mode characteristic set, different weights are given according to the characteristics of the mode and the importance of the corresponding task. For example, CT and MRI may contribute to the description of a lesion region to varying degrees when identifying a certain lesion type. Preferably, in this embodiment, each modality is given a different weight by means of some a priori knowledge or data analysis.
Step S223, calculating the weighted cosine similarity.
In computing the similarity we will consider the weight of each modality to reflect its importance in the similarity computation. And (4) calculating the weighted cosine similarity of each mode to obtain a similarity value between each pair of modes. The related formulas are as follows:
;
wherein Weighted Cosine Similarity is weighted cosine similarity, representing the calculated similarity value; n is the number of modes in the mode feature set; i is the index value of the mode;feature vectors of the ith modality; />Is the i-th weight in the weight set.
And step S230, introducing an attention mechanism according to the modal feature set and the similarity detection result, and obtaining a weight set by learning the feature contribution degree of each mode and performing weight distribution processing.
And step S240, carrying out weighted fusion processing on each mode characteristic under the same lesion type in the mode characteristic set according to the weight set to obtain a fusion characteristic set.
It can be understood that the similarity matrix is obtained through similarity detection, and the similarity value between each pair of modes is recorded by the matrix, so that the relation and the difference between different modes can be reflected. Attention vectors are then drawn and the similarity matrix is processed using a Softmax function to calculate the attention weight for each modality. The specific calculation formula is as follows:
；
wherein,is a similarity matrix; a is an attention vector; i and j are index values of modes; n is the total number of features in the feature set; />Attention weight for the ith modality; />Representing similarity matrix->Elements of row i and column i;is the similarity between the ith modality and the jth modality. By doing so, each element in the attention vector a represents the characteristic contribution of the corresponding modality, and the sum of all elements is 1. And finally, carrying out weighted fusion on the model feature set according to the attention vector A. For each modality feature, we multiply it with the corresponding attention weight and then accumulate all weighted features to get a fusion feature. The specific calculation formula is as follows:
；
wherein i is the index value of the mode; n is the total number of features in the feature set;is a fusion feature; />Is a modal feature set; />Is the attention weight of the ith modality. The attention mechanism weight distribution method can automatically learn the characteristic contribution degree of each mode, and adjust the weight according to the similarity and the difference between the modes. The processing mode can better combine the characteristic information of each mode, so that important characteristics are focused in the fusion process, and the representation capability of the network on different areas in the liver image is improved. At the same time, the method avoids manually setting weightsAnd the complex process ensures that the network construction is more automatic and flexible.
And step S300, performing feature mapping according to the fusion feature set, and obtaining a graph data representation set by performing super-pixel segmentation processing on the multi-channel image and constructing based on spatial relations and modal features among super-pixel blocks.
It will be appreciated that this step converts the multi-channel image into a form of a graph, each super-pixel block being considered a node in the graph, and the spatial relationship and modal characteristics between the super-pixel blocks being used to construct the edges of the graph, thereby forming a graph data representation set. The process can better capture the information of different areas in the image, and can perform more accurate and fine-grained analysis and identification in the subsequent graph neural network. The step S300 includes step S310, step S320, step S330, step S340, and step S340.
And step S310, carrying out segmentation processing and edge fusion on each channel image in the fusion feature set based on a super-pixel segmentation algorithm to obtain a super-pixel block fusion result.
It can be understood that the super-pixel block in this step is to divide the image into higher-level regions, and can combine adjacent pixels into a larger block, so as to reduce redundant information in the image. The adoption of superpixel segmentation can convert an image into a group of more compact and representative regional blocks, which is helpful for extracting the characteristic with more distinguishing degree, thereby improving the liver image recognition performance. The super-pixel segmentation may cause some incomplete edges or over-segmentation conditions, so that in the process of edge fusion, the edges of adjacent super-pixel blocks are combined by adopting morphological operation, edge connection algorithm and other technologies to form smoother and more accurate super-pixel block boundaries. The step S310 includes a step S311, a step S312, a step S313, and a step S314.
Step S311, dividing the images of each channel in the fusion feature set based on a super-pixel segmentation algorithm, and dividing the images into a plurality of super-pixel blocks to obtain a preliminary segmentation result, wherein the preliminary segmentation result comprises the liver structure and the image of the lesion area.
It will be appreciated that the clustering-based superpixel segmentation of the present step for each channel image (e.g., liver CT and MRI images) in the multi-modality image dataset may take full advantage of the characteristics of the different modality images, thereby better capturing the characteristics of liver structures and lesion areas. For liver CT images, the pixel values of the CT images reflect the differences in X-ray absorption of different tissues. Since liver tissue and lesion areas have different densities in a CT image, the CT image is segmented using a gray value based clustering method. In the clustering process, pixel points with similar gray values are divided into a super pixel block, so that a liver structure and a lesion area are effectively separated. Whereas for liver MRI images, MRI images reflect the signal strength of tissue in the magnetic field. Different sequences in the MRI images (e.g., T1 weighted and T2 weighted images) correspond to different tissue contrasts. The MRI image is segmented using a clustering method based on sequence and texture features. In the clustering process, pixel points with similar sequences and texture features are divided into a super-pixel block so as to better distinguish liver structures from lesion areas. Meanwhile, for clustering segmentation of liver CT and MRI images, spatial information is combined, so that the super-pixel block can better capture structure and morphological characteristics in the image. Specifically, in the cluster segmentation process, adjacent pixel points are preferentially divided into the same super-pixel block by considering the spatial relationship among the pixel points, so that the continuity of the image is maintained. By adopting different clustering methods for liver CT and MRI images and combining pixel values, sequences, textures and spatial information, the liver structure and lesion areas in the images can be effectively segmented, and super-pixel blocks with representativeness and continuity can be generated.
Step S312, edge detection is carried out according to all super pixel blocks of each channel image under the same type in the preliminary segmentation result to obtain an edge set.
Preferably, this step employs a modified Canny edge detection algorithm. Canny edge detection is a classical edge detection method that can accurately find edge points in an image and mark these edge points with the location where the brightness change is greatest. In this embodiment, the Canny algorithm is adjusted according to the characteristics of the liver CT and MRI images, so as to improve the edge detection effect on the liver structure and the lesion area. Firstly, two important parameters, namely the standard deviation of Gaussian filtering and the edge gradient threshold value, exist in the Canny algorithm, and the parameters are adjusted according to the characteristics of liver images. In particular, liver images typically have a high noise level, so the standard deviation of gaussian filtering can be appropriately increased to smooth the image; and meanwhile, according to the gray level change condition of the liver image, the edge gradient threshold value is adjusted, so that the edges of the liver structure and the lesion area can be detected more accurately. Next, in the step of super-pixel segmentation, the image has been divided into a plurality of super-pixel blocks. The superpixel blocks contain some structural information in the image, and the edge information of the superpixel blocks can be used to guide Canny edge detection, in particular, edge points of the superpixel blocks can be used as input of a Canny algorithm, and a proper gradient threshold value can be set so that the Canny algorithm is more prone to detect the edge points.
And step S313, performing confidence calculation on the edge of each super pixel block in the edge set to obtain an edge confidence calculation result.
It will be appreciated that there may be some uncertainty and error in the extracted edge points due to factors such as image noise, super pixel block size, etc. Thus, in this step, confidence calculations need to be performed on the edges of each super pixel block to determine which edges are more reliable, so that these edge information are better utilized in subsequent steps. Edge confidence calculations include edge intensity calculations, edge connectivity calculations, and intra-superpixel block consistency calculations. Edge intensity, i.e. the gradient of pixel values or the degree of brightness variation of edge points. Edge points with greater edge strength are generally more clear and accurate and thus may give higher confidence. Edge connectivity refers to detecting, for each edge of a super-pixel block, whether it is connected to the edge of an adjacent super-pixel block. An edge is also more trustworthy if it has better connectivity throughout the image. Intra-super-pixel block consistency refers to detecting, for edge points within each super-pixel block, whether there is a consistent direction and arrangement between them. If edge points within a super-pixel block exhibit some regularity, then the edges of the super-pixel block are also more trusted.
And step S314, carrying out weighted combination processing on the edge set according to the edge confidence coefficient calculation result, obtaining a super-pixel block fusion result by reserving the super-pixel block edge with high confidence coefficient in each channel, and constructing to obtain a node set according to the super-pixel block fusion result.
The step S314 includes a step S3141, a step S3142, and a step S3143.
In step S3141, a weight is given to the edge point of each super pixel block, and the weight value is determined according to the edge confidence. The higher the edge confidence, the greater the weight value of the edge point, which indicates the importance of the edge point in the fusion process.
Step S3142, for the overlapped super pixel blocks, performing weighted average processing on the edges between them. This allows more trusted edge information to be retained and reduces the uncertainty introduced by the overlap.
In step S3143, each super pixel block edge in the fusion result is a combination of edges with high confidence in the corresponding channel. Therefore, in the weighted merging process, the edge with the highest confidence in each channel is selected and reserved so as to ensure the reliability and accuracy of the fusion result.
And step 320, constructing a node set according to the liver structure and the simplified image block of the lesion area in the super pixel block fusion result.
And S330, constructing an edge set according to the spatial relation and the modal characteristics among the super pixel blocks in the super pixel block fusion result.
And S340, performing similarity calculation according to the gray value, the texture feature and the position information of the super pixel block in the super pixel block fusion result to obtain an edge weight set.
And step 350, constructing and obtaining a graph data representation set according to the node set, the edge set and the edge weight set.
It will be appreciated that the graph data representation set consists of a set of nodes corresponding to simplified image blocks of liver structures and lesion areas in the super-pixel block fusion result and a set of edges representing spatial relationships and modal features between the super-pixel blocks. The edge weight set assigns a similarity value to each edge in the edge set for describing the similarity degree between the super pixel blocks. By the graph data representation, the characteristic information of the liver image can be more comprehensively captured, and the relevance and the similarity between the super pixel blocks are fully utilized, so that the identification accuracy and the robustness of the liver image are improved.
And step 400, training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model.
It can be understood that the graph neural network has better graph structure modeling capability, can effectively capture the information of nodes and edges in the graph, and realizes information transmission and feature fusion between the nodes. In the step, the graph data representation set is input into a preset graph neural network mathematical model, and network parameters are gradually updated through calculation and optimization of the graph neural network, so that the network can better understand and represent the characteristics of liver images. Through continuous training and optimization, the graph neural network gradually learns a more accurate characteristic representation mode, so that the identification performance of liver images is improved. Finally, a complete liver image recognition model is obtained through training and optimization. The step S400 includes a step S410, a step S420, and a step S430.
Step S410, performing self-adaptive kernel density estimation according to the graph data representation set, and calculating to obtain the kernel density information of each super-pixel block.
It will be appreciated that kernel density estimation is a non-parametric statistical method for estimating probability density functions, describing the probability distribution of random variables. In this embodiment, we apply this to the superpixel blocks in the graph data representation set to estimate the probability density distribution of each superpixel block, thereby measuring its importance in the image. Specifically, for each super-pixel block, the distances from other super-pixel blocks within a certain distance range around the super-pixel block are calculated first, and the kernel density value of each super-pixel block is calculated according to the distances. The kernel density value may be regarded as an indicator of the importance of the super-pixel block, with larger kernel density values indicating that the super-pixel block is more significant and important in the image. The kernel density information of each super pixel block, namely the importance degree of each super pixel block in the image, can be obtained through the self-adaptive kernel density estimation.
And step S420, carrying out characteristic weighting processing on the characteristics in the graph data representation set according to the kernel density information to obtain a weighted data set.
It will be appreciated that this step multiplies each super-pixel block in the representation set of map data by its characteristic with the corresponding kernel density value to obtain a weighted characteristic. The aim of the method is to strengthen the characteristic expression of the important super-pixel blocks, so that the characteristic expression of the important super-pixel blocks can influence the learning and representing capacity of the model more prominently in the subsequent graphic neural network training process. The feature of each super-pixel block is adjusted and enhanced by a feature weighting process to obtain a weighted data set. The weighted data set can better guide the model to learn the characteristics of the important region in the training of the graph neural network, and is beneficial to improving the identification accuracy and the robustness of the model to the liver image in the subsequent identification process.
And S430, according to a preset graph neural network mathematical model and a weighted data set, adopting a propagation mode based on a probability graph, and carrying out iterative propagation by taking a super-pixel block with the maximum density in the nuclear density information as a seed node, so as to construct and obtain a liver image recognition model.
It will be appreciated that this step treats each superpixel block in the weighted dataset as a node in the graph neural network and the edges in the graph dataset as edges in the graph neural network. Then, the information is propagated to other nodes step by a propagation method based on a probability map, starting from the super-pixel block (seed node) with the highest density. During propagation, each node will update its own feature representation based on the features and edge weights of its neighboring nodes. Such iterative propagation processes may convey information in the graph data representation set and affect the direction and strength of information propagation based on edge weights and the characteristic contribution of the nodes. Through iterative propagation, the feature representation of each node will be continually updated and optimized, gradually learning into the complex structure and feature information in the graph data representation set. Finally, a trained and optimized liver image recognition model is obtained, the feature contribution degrees of different modes in the multi-mode image dataset can be comprehensively considered, and information transmission is carried out according to the spatial relationship and the mode features among the super-pixel blocks, so that accurate recognition and analysis of liver images are realized. The graph neural network model has good fitting capacity and characterization capacity, information in multi-mode image data can be effectively utilized, and accuracy and robustness of identifying and analyzing liver images are improved. Meanwhile, the propagation mode and the nuclear density information based on the probability map are used as initial seed nodes, so that the training process of the model is more stable and efficient.
And S500, carrying out recognition processing on the second information according to the liver image recognition model to obtain a recognition result, wherein the recognition result comprises a lesion type, a lesion position calibration and credibility.
It can be appreciated that the present step can determine the type of lesion, such as tumor, cyst, steatosis, etc., present in the liver image of the patient to be identified, by means of the liver image identification model. According to the multi-mode image data set during model training, the model can compare the image of the patient with various lesion types and determine the most matched lesion type. In addition to identifying the lesion type, the liver image identification model can also accurately mark the location of the lesion in the image. This means that the model can identify the specific location and extent of the lesion in the liver, providing important reference information to the physician during diagnosis and treatment. The recognition results also include a confidence assessment for each lesion type and location. The model will give a value representing the confidence level reflecting the accuracy and reliability of the recognition result. The doctor can judge the credibility of the identification result according to the credibility and further confirm and verify if necessary. Through the liver image recognition model, the automatic analysis and recognition can be effectively carried out on the image data of the patient, so that the lesion type and the position information can be rapidly and accurately obtained, and diagnostic references can be provided for doctors. The automatic identification process not only saves time and energy of doctors, but also greatly improves the accuracy and efficiency of liver image identification.
Example 2
As shown in fig. 2, this embodiment provides a liver image recognition device based on a graph neural network, where the device includes:
the acquisition module 1 is used for acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified.
The fusion module 2 is configured to fuse the first information according to a preset multi-modal attention mathematical model, and perform convolution extraction processing on the input image of each modality by introducing an attention mechanism to obtain a fusion feature set, where the multi-modal attention mathematical model is a model that focuses on important modality information by introducing the attention mechanism and performs feature extraction and fusion on the input image of each modality by using a convolutional neural network by automatically giving different weights to image data of different modalities.
And the mapping module 3 is used for carrying out feature mapping according to the fusion feature set, carrying out super-pixel segmentation processing on the multi-channel image, and constructing a graph data representation set based on the spatial relationship and the modal features among the super-pixel blocks.
And the construction module 4 is used for training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing and obtaining a liver image recognition model.
And the identification module 5 is used for carrying out identification processing on the second information according to the liver image identification model to obtain an identification result, wherein the identification result comprises a lesion type, a lesion position calibration and credibility.
In one embodiment of the present disclosure, the fusion module 2 includes:
the first extracting unit 21 is configured to perform feature extraction according to the image of each mode in the first information to obtain a mode feature set.
The first detecting unit 22 is configured to perform similarity detection according to all the modal features in the same lesion type in the modal feature set to obtain a similarity detection result.
The first allocation unit 23 is configured to introduce an attention mechanism according to the feature set of the modes and the similarity detection result, and perform weight allocation processing by learning the feature contribution of each mode to obtain a weight set.
The first fusion unit 24 is configured to perform weighted fusion processing on each of the modal features in the same lesion type in the modal feature set according to the weight set to obtain a fused feature set.
In one embodiment of the present disclosure, the mapping module 3 includes:
the first segmentation unit 31 performs segmentation processing and edge fusion on each channel image in the fusion feature set based on a super-pixel segmentation algorithm to obtain a super-pixel block fusion result.
A first construction unit 32, configured to construct a node set according to the simplified image blocks of the liver structure and the lesion area in the super pixel block fusion result.
The second construction unit 33 is configured to construct an edge set according to the spatial relationship and the modal characteristics between the super pixel blocks in the super pixel block fusion result.
The first calculating unit 34 is configured to calculate a side weight set according to the gray value, the texture feature and the position information of the super pixel block in the super pixel block fusion result.
A third construction unit 35 is configured to construct a graph data representation set according to the node set, the edge set and the edge weight set.
In one embodiment of the present disclosure, the first dividing unit 31 includes:
the second segmentation unit 311 performs segmentation processing on each channel image in the fusion feature set based on a superpixel segmentation algorithm, and divides the image into a plurality of superpixel blocks to obtain a preliminary segmentation result, where the preliminary segmentation result includes images of a liver structure and a lesion region.
The second detecting unit 312 is configured to perform edge detection according to all the super pixel blocks of each channel image under the same type in the preliminary segmentation result to obtain an edge set.
The second calculating unit 313 is configured to calculate the confidence coefficient of the edge of each super pixel block in the edge set to obtain an edge confidence coefficient calculation result.
The first merging unit 314 is configured to perform weighted merging processing on the edge set according to the edge confidence coefficient calculation result, obtain a super-pixel block fusion result by retaining the super-pixel block edge with high confidence coefficient in each channel, and construct a node set according to the super-pixel block fusion result.
In one embodiment of the present disclosure, the building block 4 comprises:
a third calculation unit 41, configured to perform adaptive kernel density estimation according to the graph data representation set, and calculate kernel density information of each superpixel block.
The first processing unit 42 is configured to perform feature weighting processing on features in the graph data representation set according to the kernel density information, so as to obtain a weighted data set.
And a fourth construction unit 43, configured to perform iterative propagation by using a propagation mode based on a probability map and using a superpixel block with the maximum density in the kernel density information as a seed node according to a preset graph neural network mathematical model and a weighted data set, so as to construct and obtain a liver image recognition model.
It should be noted that, regarding the apparatus in the above embodiments, the specific manner in which the respective modules perform the operations has been described in detail in the embodiments regarding the method, and will not be described in detail herein.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims (4)

1. The liver image recognition method based on the graph neural network is characterized by comprising the following steps of:
acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified;
the first information is fused according to a preset multi-modal attention mathematical model, convolution extraction processing is carried out on the input image of each mode through an attention introducing mechanism, and a fusion feature set is obtained, wherein the multi-modal attention mathematical model is a model which is obtained by automatically giving different weights to image data of different modes, paying attention to important mode information through the attention introducing mechanism and carrying out feature extraction and fusion on the input image of each mode through a convolution neural network;
performing feature mapping according to the fusion feature set, and obtaining a graph data representation set by performing super-pixel segmentation processing on the multi-channel image and constructing based on spatial relations and modal features among super-pixel blocks;
training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model;
performing recognition processing on the second information according to the liver image recognition model to obtain a recognition result, wherein the recognition result comprises a lesion type, a lesion position calibration and credibility;
the method for obtaining the fusion feature set includes the steps of:
carrying out feature extraction according to the image of each mode in the first information to obtain a mode feature set;
performing similarity detection according to all the modal features in the same lesion type in the modal feature set to obtain a similarity detection result;
introducing an attention mechanism according to the modal feature set and the similarity detection result, and carrying out weight distribution processing by learning the feature contribution degree of each modal to obtain a weight set;
weighting and fusing all the modal features under the same lesion type in the modal feature set according to the weight set to obtain a fused feature set;
the feature mapping is performed according to the fusion feature set, and a graph data representation set is obtained by performing super-pixel segmentation processing on a multi-channel image and constructing based on spatial relations and modal features among super-pixel blocks, including:
dividing and edge fusion are carried out on each channel image in the fusion feature set based on a super-pixel dividing algorithm to obtain a super-pixel block fusion result;
constructing a node set according to the simplified image blocks of the liver structure and the lesion area in the super pixel block fusion result;
constructing an edge set according to the spatial relation and modal characteristics among the super pixel blocks in the super pixel block fusion result;
performing similarity calculation according to the gray value, texture characteristics and position information of the super pixel blocks in the super pixel block fusion result to obtain an edge weight set;
constructing a graph data representation set according to the node set, the edge set and the edge weight set;
the method for obtaining the super pixel block fusion result based on the super pixel segmentation algorithm for carrying out segmentation processing and edge fusion on each channel image in the fusion feature set comprises the following steps:
dividing each channel image in the fusion feature set based on a super-pixel segmentation algorithm, dividing the image into a plurality of super-pixel blocks to obtain a preliminary segmentation result, wherein the preliminary segmentation result comprises images of a liver structure and a lesion area;
performing edge detection according to all super-pixel blocks of each channel image under the same type in the preliminary segmentation result to obtain an edge set;
performing confidence calculation on the edge of each super pixel block in the edge set to obtain an edge confidence calculation result;
and carrying out weighted combination processing on the edge set according to the edge confidence coefficient calculation result, obtaining a super-pixel block fusion result by reserving the super-pixel block edge with high confidence coefficient in each channel, and constructing and obtaining a node set according to the super-pixel block fusion result.
2. The liver image recognition method based on the graph neural network according to claim 1, wherein training and optimizing are performed according to the graph data representation set and a preset graph neural network mathematical model, and the liver image recognition model is constructed and obtained, and the method comprises the following steps:
performing self-adaptive kernel density estimation according to the graph data representation set, and calculating to obtain kernel density information of each super-pixel block;
performing feature weighting processing on the features in the graph data representation set according to the kernel density information to obtain a weighted data set;
and according to a preset graph neural network mathematical model and the weighted data set, adopting a propagation mode based on a probability graph, and carrying out iterative propagation by taking a super-pixel block with the maximum density in the nuclear density information as a seed node, so as to construct and obtain a liver image recognition model.
3. A liver image recognition device based on a graph neural network, comprising:
the acquisition module is used for acquiring first information and second information, wherein the first information is a multi-mode image data set under at least two lesion types, and the second information is liver image data of a patient to be identified;
the fusion module is used for fusing the first information according to a preset multi-mode attention mathematical model, and carrying out convolution extraction processing on the input image of each mode through an attention introducing mechanism to obtain a fusion feature set, wherein the multi-mode attention mathematical model is a model which is used for focusing on important mode information through the attention introducing mechanism and carrying out feature extraction and fusion on the input image of each mode by utilizing a convolution neural network through automatically giving different weights to the image data of different modes;
the mapping module is used for carrying out feature mapping according to the fusion feature set, and obtaining a graph data representation set by carrying out super-pixel segmentation processing on the multi-channel image and constructing based on the spatial relationship and the modal feature among the super-pixel blocks;
the construction module is used for training and optimizing according to the graph data representation set and a preset graph neural network mathematical model, and constructing to obtain a liver image recognition model;
the identification module is used for carrying out identification processing on the second information according to the liver image identification model to obtain an identification result, wherein the identification result comprises a lesion type, a lesion position calibration and credibility;
wherein, the fusion module includes:
the first extraction unit is used for extracting features according to the images of each mode in the first information to obtain a mode feature set;
the first detection unit is used for carrying out similarity detection according to all the modal features in the same lesion type in the modal feature set to obtain a similarity detection result;
the first distribution unit is used for introducing an attention mechanism according to the modal feature set and the similarity detection result, and carrying out weight distribution processing by learning the feature contribution of each modal to obtain a weight set;
the first fusion unit is used for carrying out weighted fusion processing on each modal feature under the same lesion type in the modal feature set according to the weight set to obtain a fusion feature set;
wherein the mapping module comprises:
the first segmentation unit is used for carrying out segmentation processing and edge fusion on each channel image in the fusion feature set based on a super-pixel segmentation algorithm to obtain a super-pixel block fusion result;
the first construction unit is used for constructing a node set according to the liver structure and the simplified image block of the lesion area in the super pixel block fusion result;
the second construction unit is used for constructing an edge set according to the spatial relationship and the modal characteristics among the super pixel blocks in the super pixel block fusion result;
the first calculation unit is used for carrying out similarity calculation according to the gray value, the texture feature and the position information of the super pixel block in the super pixel block fusion result to obtain an edge weight set;
the third construction unit is used for constructing a graph data representation set according to the node set, the edge set and the edge weight set;
wherein the first dividing unit includes:
the second segmentation unit is used for carrying out segmentation processing on each channel image in the fusion feature set based on a super-pixel segmentation algorithm, and dividing the image into a plurality of super-pixel blocks to obtain a preliminary segmentation result, wherein the preliminary segmentation result comprises an image of a liver structure and a lesion area;
the second detection unit is used for carrying out edge detection on all super pixel blocks of each channel image under the same type in the preliminary segmentation result to obtain an edge set;
the second computing unit is used for computing the confidence coefficient of the edge of each super pixel block in the edge set to obtain an edge confidence coefficient computing result;
the first merging unit is used for carrying out weighted merging processing on the edge set according to the edge confidence coefficient calculation result, obtaining a super-pixel block fusion result by reserving the super-pixel block edge with high confidence coefficient in each channel, and constructing a node set according to the super-pixel block fusion result.
4. The liver image recognition device based on a graph neural network of claim 3, wherein the construction module comprises:
the third calculation unit is used for carrying out self-adaptive kernel density estimation according to the graph data representation set and calculating to obtain the kernel density information of each super pixel block;
the first processing unit is used for carrying out characteristic weighting processing on the characteristics in the graph data representation set according to the kernel density information to obtain a weighted data set;
and the fourth construction unit is used for carrying out iterative propagation by adopting a propagation mode based on a probability map and taking the super-pixel block with the maximum density in the nuclear density information as a seed node according to a preset graph neural network mathematical model and the weighted data set, so as to construct and obtain a liver image identification model.
CN202311197432.1A 2023-09-18 2023-09-18 Liver image identification method and device based on graph neural network Active CN116934754B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311197432.1A CN116934754B (en) 2023-09-18 2023-09-18 Liver image identification method and device based on graph neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311197432.1A CN116934754B (en) 2023-09-18 2023-09-18 Liver image identification method and device based on graph neural network

Publications (2)

Publication Number Publication Date
CN116934754A CN116934754A (en) 2023-10-24
CN116934754B true CN116934754B (en) 2023-12-01

Family

ID=88388259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311197432.1A Active CN116934754B (en) 2023-09-18 2023-09-18 Liver image identification method and device based on graph neural network

Country Status (1)

Country Link
CN (1) CN116934754B (en)

Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110503649A (en) * 2019-07-05 2019-11-26 陕西科技大学 One kind is based on Multi scale U-net and the modified liver segmentation method of super-pixel
WO2020244434A1 (en) * 2019-06-03 2020-12-10 腾讯科技（深圳）有限公司 Method and apparatus for recognizing facial expression, and electronic device and storage medium
CN112347970A (en) * 2020-11-18 2021-02-09 江苏海洋大学 Remote sensing image ground object identification method based on graph convolution neural network
EP3812926A1 (en) * 2020-01-15 2021-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Multimodal content processing method, apparatus, device and storage medium
CN113223028A (en) * 2021-05-07 2021-08-06 西安智诊智能科技有限公司 Multi-modal liver tumor segmentation method based on MR and CT
CN113313164A (en) * 2021-05-27 2021-08-27 复旦大学附属肿瘤医院 Digital pathological image classification method and system based on superpixel segmentation and image convolution
CN113591629A (en) * 2021-07-16 2021-11-02 深圳职业技术学院 Finger three-mode fusion recognition method, system, device and storage medium
WO2021233112A1 (en) * 2020-05-20 2021-11-25 腾讯科技（深圳）有限公司 Multimodal machine learning-based translation method, device, equipment, and storage medium
CN113723255A (en) * 2021-08-24 2021-11-30 中国地质大学（武汉） Hyperspectral image classification method and storage medium
CN114266726A (en) * 2021-11-22 2022-04-01 中国科学院深圳先进技术研究院 Medical image segmentation method, system, terminal and storage medium
CN114581773A (en) * 2022-02-28 2022-06-03 西安电子科技大学 Multi-mode remote sensing data classification method based on graph convolution network
WO2023036159A1 (en) * 2021-09-07 2023-03-16 Huawei Technologies Co., Ltd. Methods and devices for audio visual event localization based on dual perspective networks
CN115984555A (en) * 2022-12-13 2023-04-18 哈尔滨医科大学附属第一医院 Coronary artery stenosis identification method based on depth self-encoder composition
CN116128898A (en) * 2023-02-17 2023-05-16 重庆邮电大学 Skin lesion image segmentation method based on transducer double-branch model
CN116468995A (en) * 2022-07-21 2023-07-21 西北工业大学深圳研究院 Sonar image classification method combining SLIC super-pixel and graph annotation meaning network
CN116524369A (en) * 2023-04-18 2023-08-01 中国地质大学（武汉） Remote sensing image segmentation model construction method and device and remote sensing image interpretation method
CN116543227A (en) * 2023-05-22 2023-08-04 北京数慧时空信息技术有限公司 Remote sensing image scene classification method based on graph convolution network
WO2023159073A1 (en) * 2022-02-15 2023-08-24 Currus Ai Inc. Methods and systems of sensor fusion in cooperative perception systems
CN116645579A (en) * 2023-05-22 2023-08-25 广东工业大学 Feature fusion method based on heterogeneous graph attention mechanism
CN116664954A (en) * 2023-06-30 2023-08-29 西安电子科技大学 Hyperspectral ground object classification method based on graph convolution and convolution fusion
CN116703744A (en) * 2023-04-18 2023-09-05 二十一世纪空间技术应用股份有限公司 Remote sensing image dodging and color homogenizing method and device based on convolutional neural network
CN116740418A (en) * 2023-05-22 2023-09-12 广东工业大学 Target detection method based on graph reconstruction network
CN116740419A (en) * 2023-05-22 2023-09-12 广东工业大学 Target detection method based on graph regulation network

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3433816A1 (en) * 2016-03-22 2019-01-30 URU, Inc. Apparatus, systems, and methods for integrating digital media content into other digital media content
US11328172B2 (en) * 2020-08-24 2022-05-10 Huawei Technologies Co. Ltd. Method for fine-grained sketch-based scene image retrieval
US11688061B2 (en) * 2020-11-20 2023-06-27 International Business Machines Corporation Interpretation of whole-slide images in digital pathology
US20230230408A1 (en) * 2022-01-14 2023-07-20 Nielsen Consumer Llc Methods, systems, articles of manufacture, and apparatus for decoding images
US20230252644A1 (en) * 2022-02-08 2023-08-10 Ping An Technology (Shenzhen) Co., Ltd. System and method for unsupervised superpixel-driven instance segmentation of remote sensing image

Patent Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020244434A1 (en) * 2019-06-03 2020-12-10 腾讯科技（深圳）有限公司 Method and apparatus for recognizing facial expression, and electronic device and storage medium
CN110503649A (en) * 2019-07-05 2019-11-26 陕西科技大学 One kind is based on Multi scale U-net and the modified liver segmentation method of super-pixel
EP3812926A1 (en) * 2020-01-15 2021-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Multimodal content processing method, apparatus, device and storage medium
WO2021233112A1 (en) * 2020-05-20 2021-11-25 腾讯科技（深圳）有限公司 Multimodal machine learning-based translation method, device, equipment, and storage medium
CN112347970A (en) * 2020-11-18 2021-02-09 江苏海洋大学 Remote sensing image ground object identification method based on graph convolution neural network
CN113223028A (en) * 2021-05-07 2021-08-06 西安智诊智能科技有限公司 Multi-modal liver tumor segmentation method based on MR and CT
CN113313164A (en) * 2021-05-27 2021-08-27 复旦大学附属肿瘤医院 Digital pathological image classification method and system based on superpixel segmentation and image convolution
CN113591629A (en) * 2021-07-16 2021-11-02 深圳职业技术学院 Finger three-mode fusion recognition method, system, device and storage medium
CN113723255A (en) * 2021-08-24 2021-11-30 中国地质大学（武汉） Hyperspectral image classification method and storage medium
WO2023036159A1 (en) * 2021-09-07 2023-03-16 Huawei Technologies Co., Ltd. Methods and devices for audio visual event localization based on dual perspective networks
CN114266726A (en) * 2021-11-22 2022-04-01 中国科学院深圳先进技术研究院 Medical image segmentation method, system, terminal and storage medium
WO2023159073A1 (en) * 2022-02-15 2023-08-24 Currus Ai Inc. Methods and systems of sensor fusion in cooperative perception systems
CN114581773A (en) * 2022-02-28 2022-06-03 西安电子科技大学 Multi-mode remote sensing data classification method based on graph convolution network
CN116468995A (en) * 2022-07-21 2023-07-21 西北工业大学深圳研究院 Sonar image classification method combining SLIC super-pixel and graph annotation meaning network
CN115984555A (en) * 2022-12-13 2023-04-18 哈尔滨医科大学附属第一医院 Coronary artery stenosis identification method based on depth self-encoder composition
CN116128898A (en) * 2023-02-17 2023-05-16 重庆邮电大学 Skin lesion image segmentation method based on transducer double-branch model
CN116524369A (en) * 2023-04-18 2023-08-01 中国地质大学（武汉） Remote sensing image segmentation model construction method and device and remote sensing image interpretation method
CN116703744A (en) * 2023-04-18 2023-09-05 二十一世纪空间技术应用股份有限公司 Remote sensing image dodging and color homogenizing method and device based on convolutional neural network
CN116543227A (en) * 2023-05-22 2023-08-04 北京数慧时空信息技术有限公司 Remote sensing image scene classification method based on graph convolution network
CN116645579A (en) * 2023-05-22 2023-08-25 广东工业大学 Feature fusion method based on heterogeneous graph attention mechanism
CN116740418A (en) * 2023-05-22 2023-09-12 广东工业大学 Target detection method based on graph reconstruction network
CN116740419A (en) * 2023-05-22 2023-09-12 广东工业大学 Target detection method based on graph regulation network
CN116664954A (en) * 2023-06-30 2023-08-29 西安电子科技大学 Hyperspectral ground object classification method based on graph convolution and convolution fusion

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Weighted Feature Fusion of Convolutional Neural Network and Graph Attention Network for Hyperspectral Image Classification;Yanni Dong等;《IEEE Transactions on Image Processing》;第31卷;1559-1572 *
图卷积神经网络及其在图像识别领域的应用综述;李文静等;《计算机工程与应用》;第1-26页 *

Also Published As

Publication number Publication date
CN116934754A (en) 2023-10-24

Similar Documents

Publication Publication Date Title
CN106682435B (en) System and method for automatically detecting lesion in medical image through multi-model fusion
US10176408B2 (en) Systems and methods for analyzing pathologies utilizing quantitative imaging
El-Baz et al. 3D shape analysis for early diagnosis of malignant lung nodules
US7876938B2 (en) System and method for whole body landmark detection, segmentation and change quantification in digital images
US6125194A (en) Method and system for re-screening nodules in radiological images using multi-resolution processing, neural network, and image processing
TWI496112B (en) Cell image segmentation method and a nuclear-to-cytoplasmic ratio evaluation method using the same
US9230320B2 (en) Computer aided diagnostic system incorporating shape analysis for diagnosing malignant lung nodules
US8175351B2 (en) Computer-aided detection and classification of suspicious masses in breast imagery
JP5851160B2 (en) Image processing apparatus, operation method of image processing apparatus, and image processing program
El-Baz et al. Three-dimensional shape analysis using spherical harmonics for early assessment of detected lung nodules
Ashwin et al. Efficient and reliable lung nodule detection using a neural network based computer aided diagnosis system
Shakibapour et al. An unsupervised metaheuristic search approach for segmentation and volume measurement of pulmonary nodules in lung CT scans
CN112614133B (en) Three-dimensional pulmonary nodule detection model training method and device without anchor point frame
CN112308846B (en) Blood vessel segmentation method and device and electronic equipment
Rani et al. Superpixel with nanoscale imaging and boosted deep convolutional neural network concept for lung tumor classification
US20230005140A1 (en) Automated detection of tumors based on image processing
Sirjani et al. Automatic cardiac evaluations using a deep video object segmentation network
Wang Automatic and optimal segmentation of the left ventricle in cardiac magnetic resonance images independent of the training sets
Shan et al. Lung cancer diagnosis based on an ann optimized by improved teo algorithm
Faisal et al. Computer assisted diagnostic system in tumor radiography
CN103839048A (en) Stomach CT image lymph gland recognition system and method based on low-rank decomposition
CN116934754B (en) Liver image identification method and device based on graph neural network
Gordon et al. Evaluation of uterine cervix segmentations using ground truth from multiple experts
Wieclawek 3D marker-controlled watershed for kidney segmentation in clinical CT exams
Shetty et al. Water Cycle bat algorithm and dictionary-based deformable model for lung tumor segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant