CN111932538B - Method, device, computer equipment and storage medium for analyzing thyroid gland atlas - Google Patents

Method, device, computer equipment and storage medium for analyzing thyroid gland atlas Download PDF

Info

Publication number
CN111932538B
CN111932538B CN202011077645.7A CN202011077645A CN111932538B CN 111932538 B CN111932538 B CN 111932538B CN 202011077645 A CN202011077645 A CN 202011077645A CN 111932538 B CN111932538 B CN 111932538B
Authority
CN
China
Prior art keywords
thyroid
data
topological
map
labeling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011077645.7A
Other languages
Chinese (zh)
Other versions
CN111932538A (en
Inventor
陈超
蔡洵
侯怡卿
詹维伟
黄凌云
刘玉宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN202011077645.7A priority Critical patent/CN111932538B/en
Publication of CN111932538A publication Critical patent/CN111932538A/en
Application granted granted Critical
Publication of CN111932538B publication Critical patent/CN111932538B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Geometry (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the field of artificial intelligence and discloses a method for analyzing a thyroid map, which comprises the following steps: obtaining a thyroid map to be analyzed; inputting a thyroid atlas to be analyzed into a texture analysis network to obtain a first characteristic map; labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed; superposing the first characteristic diagram with a topological attention matrix corresponding to topological data to obtain a second characteristic diagram; and obtaining an analysis result of the thyroid gland map to be analyzed according to the characteristic data in the first characteristic diagram and the second characteristic diagram. By additionally arranging the topological branch in the deep learning network and by the topological data structure corresponding to the marking frame data in the training data, the positions and sizes of the marking frames of the organs detected by the deep learning network in the first characteristic diagram are corrected, the condition that the carotid artery or the jugular vein and the like are easily judged as the thyroid gland by mistake when the deep learning network is independently used for analysis is overcome, and the accuracy of analysis and positioning is improved.

Description

Method, device, computer equipment and storage medium for analyzing thyroid gland atlas
Technical Field
The present application relates to the field of artificial intelligence, and in particular, to a method, an apparatus, a computer device, and a storage medium for analyzing a thyroid map.
Background
National cancer reports that thyroid cancer is a common tumor ranked 7, and the incidence rate tends to increase year by year. Clinical practice shows that early discovery, early diagnosis and early treatment are the key points for reducing the incidence and mortality of thyroid cancer. At present, in the aspect of thyroid diagnosis and treatment, ultrasonic examination is a more effective clinical screening means, and doctors combine ultrasonic examination results to locate the thyroid of patients and perform subsequent inquiry work. However, the thyroid gland of a patient is positioned according to the ultrasonic examination result, so that the thyroid gland not only needs abundant medical experience and has large workload, but also has the defects of long time consumption for accurate diagnosis, poor measurement precision and the like, and the working quality and the efficiency of doctors are influenced. Moreover, with the further enhancement of health consciousness of people, the requirement for regular examination of each body part is further increased, the analysis work of the ultrasonic image completely depends on the analysis method of medical experience of medical personnel, and is not practical, and the artificial intelligence algorithm is used for the analysis of the ultrasonic image, for example, the thyroid gland automatic detection and positioning is realized by analyzing the thyroid gland atlas, but the organ positioning in the thyroid gland atlas is still inaccurate by the existing artificial intelligence algorithm.
Disclosure of Invention
The application mainly aims to provide a data processing method and aims to solve the technical problem that the existing artificial intelligence algorithm is still inaccurate in organ positioning in a thyroid gland atlas.
The present application provides a method of analyzing a thyroid map comprising:
acquiring a thyroid map to be analyzed, wherein the thyroid map comprises a B ultrasonic image obtained by ultrasonic scanning;
inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and sizes of the organ labeling boxes detected by the texture analysis network;
labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed;
superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
and obtaining an analysis result of the thyroid atlas to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map, wherein the analysis result comprises the relative position of each organ.
Preferably, the step of labeling the first feature map according to a preset mode to obtain the topological data of the thyroid map to be analyzed includes:
equally dividing the thyroid map to be analyzed into map matrixes formed by a plurality of grids;
judging whether a coincidence region exists between the designated grid and a labeling frame of the designated organ in the first characteristic diagram, wherein the designated grid is any grid in the atlas matrix;
if so, calculating the area of the overlapped area;
judging whether the area proportion of the area of the overlapped area occupying the designated grid exceeds a preset proportion or not;
if so, setting the label value corresponding to the labeling frame of the specified organ as topological data corresponding to the specified grid;
and according to the labeling process of the topological data corresponding to the specified grid, labeling the topological data of all grids in the thyroid map to be analyzed.
Preferably, the step of setting the label data corresponding to the labeling box of the specified organ as the topology data corresponding to the specified mesh includes:
acquiring label data of a plurality of specified organ labeling boxes;
calculating the sum of the label data of a plurality of the specified organ labeling boxes;
and taking the added value as the topology data corresponding to the specified grid.
Preferably, the step of calculating a sum of the labeling data of the plurality of specified organ labeling boxes includes:
acquiring the types and the number of all organ images included in the thyroid atlas to be analyzed;
normalizing the labeling data of each designated organ labeling frame according to the number of the types of all organ images included in the thyroid atlas to be analyzed and the label values respectively corresponding to the types to obtain normalized values respectively corresponding to the designated organ labeling frames;
and summing the normalized numerical values respectively corresponding to the plurality of specified organ labeling boxes covering the specified grid to obtain the summed numerical value of the labeling data of the plurality of specified organ labeling boxes.
Preferably, the step of adding a topology branch in the deep learning network, and superimposing the first feature map on the topology attention matrix corresponding to the topology data to obtain a second feature map includes:
inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics;
multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram;
and taking the feature map of the updated assignment as the second feature map.
Preferably, after the step of labeling the first feature map according to a preset manner to obtain the topology data of the thyroid map to be analyzed, the method includes:
obtaining classifiers of two classes in the topological branches, and calculating a confidence value of the topological data of the thyroid atlas to be analyzed;
judging whether the confidence value exceeds a preset confidence threshold value;
and if so, generating instruction information for superposing the first characteristic diagram on a topology attention matrix corresponding to the topology data.
Preferably, after the step of obtaining the analysis result of the thyroid map to be analyzed according to the feature data in the first feature map and the second feature map, the method further includes:
acquiring a thyroid nodule monitoring network, and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability and the estimated incidence position of thyroid cancer;
judging whether the morbidity probability exceeds a preset probability;
and if so, determining coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed.
The present application also provides an apparatus for analyzing thyroid map, comprising:
the first acquisition module is used for acquiring a thyroid atlas to be analyzed, wherein the thyroid atlas comprises a B ultrasonic image obtained by ultrasonic scanning;
the input module is used for inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and the sizes of the organ labeling boxes detected by the texture analysis network;
the labeling module is used for labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid map to be analyzed;
the superposition module is used for superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
and the obtaining module is used for obtaining an analysis result of the thyroid atlas to be analyzed according to the feature data in the first feature map and the second feature map, wherein the analysis result comprises the relative position of each organ.
The present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the method as described above.
According to the method, the topological branch is additionally arranged in the deep learning network, the position and the size of each organ marking frame detected by the deep learning network in the first characteristic diagram are corrected through the topological data structure corresponding to the marking frame data in the training data, the condition that the carotid artery or the jugular vein is misjudged as the thyroid gland easily when the deep learning network is independently used for analysis is overcome, and the accuracy of analysis and positioning is improved.
Drawings
FIG. 1 is a schematic flow chart of a method for analyzing a thyroid map according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an apparatus for analyzing thyroid maps according to an embodiment of the present application;
fig. 3 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Referring to fig. 1, a method of analyzing a thyroid map according to an embodiment of the present application includes:
s1: acquiring a thyroid map to be analyzed, wherein the thyroid map comprises a B ultrasonic image obtained by ultrasonic scanning;
s2: inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and sizes of the organ labeling boxes detected by the texture analysis network;
s3: labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed;
s4: superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
s5: and obtaining an analysis result of the thyroid atlas to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map, wherein the analysis result comprises the relative position of each organ.
In the embodiment of the application, the thyroid map is a B-mode ultrasonic image obtained by ultrasonic scanning, and preliminary image identification is performed on the thyroid map on the basis of a deep learning network so as to obtain the positions and sizes of the marking frames of various organs. The above thyroid maps are transected images. There are many anatomical structures in the transected image, such as carotid arteries, trachea, thyroid glands, and sometimes jugular veins, esophagus, etc. Whereas a longitudinal cut image generally has only thyroid glands, a transverse cut image and a longitudinal cut image can be distinguished by the number of organs included in the image. Because the data volume of the thyroid atlas data marked by the standard cannot reach the standard of accurate image identification, and because the thyroid ultrasound atlas has larger ultrasonic noise influence and is easy to have particularity such as artifacts during scanning, after the thyroid atlas is subjected to preliminary image identification only through the deep learning network, the positions and the sizes of marking frames of various organs are obtained, larger data errors can exist, a large number of false detection and missing detection conditions are caused, data obtained by image identification of the thyroid atlas through the deep learning network is caused, the requirement on data precision cannot be met, and the accuracy is poor.
According to the method, the topological branch is additionally arranged in the deep learning network, so that a deep learning network analysis model based on a topological expression mode is obtained. The deep learning network includes, but is not limited to, the Faster R-CNN, which is composed of a FPN layer and an RPN layer. According to the method, a thyroid atlas is obtained through ultrasonic scanning, the thyroid atlas is input into an FPN layer, an alternative organ labeling frame is determined by combining non-maximum inhibition of a threshold value, and then the thyroid atlas is input into an RPN layer to obtain a first feature map with uniform size. And then, adding coordinate values and label values to the organ labeling boxes obtained from the first characteristic diagram to obtain labeled topological data. And combining a coordinate value convolution process to form input data of the topology branches, obtaining the topology characteristics extracted through the topology branches after convolution and pooling, obtaining a topology attention matrix corresponding to the topology data, and obtaining a second characteristic diagram after superposing the topology attention matrix on the basis of the first characteristic diagram. And then obtaining an analysis result of the thyroid gland map to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map. For example, the feature data of the first feature map shows that the current map includes the texture of the thyroid gland and the texture of the carotid artery, the second feature map shows that the probability of including the thyroid gland labeling box is 90%, the probability of the esophagus labeling box is 86%, and the probability of the carotid artery labeling box is 40%, then the analysis result is that the current thyroid map to be analyzed scans the area compatible with the thyroid gland and the esophagus instead of the area compatible with the thyroid gland and the carotid artery, and the current thyroid map to be analyzed shows the position relationship between the thyroid gland and the esophagus.
In the training process, because the training samples are marked by the marking boxes in advance, the topological matrix corresponding to each training sample is generated through the marking boxes. And after the coordinate value convolution (CoordConv) is changed into a new feature map of 3 channels, the dimension is raised to a topological attention matrix of 256 channels through operations of convolution calculation of 1-1 convolution kernel and 3-3 convolution kernel, average pooling and the like. The coefficients of the convolution kernels are obtained by learning through a back propagation algorithm.
According to the method and the device, the texture data obtained by the texture detection head through the first feature map and the topological features of the topology branch detection head in the second feature map are combined, so that the accurate positioning analysis structure of the anatomical structure included in the thyroid gland map is realized, and the accurate positioning analysis structure comprises organs adjacent to the thyroid gland, relative position relations between the organs and the thyroid gland and the like in the thyroid gland map. The method can be expressed as T = M (P (I) and T (P (I)), wherein T represents an edge detection result, M is a topological model, P represents a deep learning network, I is an input image, and T represents a topological representation mode.
Further, the step S3 of labeling the first feature map according to a preset manner to obtain the topology data of the thyroid map to be analyzed includes:
s31: equally dividing the thyroid map to be analyzed into map matrixes formed by a plurality of grids;
s32: judging whether a coincidence region exists between the designated grid and a labeling frame of the designated organ in the first characteristic diagram, wherein the designated grid is any grid in the atlas matrix;
s33: if so, calculating the area of the overlapped area;
s34: judging whether the area proportion of the area of the overlapped area occupying the designated grid exceeds a preset proportion or not;
s35: if so, setting the label value corresponding to the labeling frame of the specified organ as topological data corresponding to the specified grid;
s36: and according to the labeling process of the topological data corresponding to the specified grid, labeling the topological data of all grids in the thyroid map to be analyzed.
In the embodiment of the application, during model training, the topological matrix realizes training of a topological data structure in a thyroid map through known labeled box data of each organ, and the topological data structure comprises various organ types adjacent to the thyroid and a relative position relationship network between each organ and the thyroid. During detection, the input image is averagely divided into N × N grids, for example, N is set to be 8, and the input image is divided into 8 × 8 grids. Each grid is used as an element in the topological matrix, and the matrix elements corresponding to the grid are assigned according to the position relation of the organ labeling frame in the grid to form topological data. For example, organs adjacent to the thyroid in a thyroid map include, but are not limited to, the trachea, carotid artery, jugular vein, esophagus. For example, the tag value corresponding to the thyroid is set to 1, the tracheal tag value is set to 2, the carotid tag value is set to 3, the jugular tag value is set to 4, and the esophageal tag value is set to 5. If the current grid is occupied by an organ marking frame, preliminarily estimating a label value corresponding to the marking frame through a deep learning network, and then judging whether the area proportion of the organ marking frame occupying the grid reaches or exceeds a preset proportion so as to determine the assignment of the grid. For example, the tag value corresponding to the tag frame is preliminarily estimated to be the trachea tag value 2 through the deep learning network, and the trachea tag frame exceeds 30% of the area of the grid, then the grid is assigned to be 2. And (3) assigning each grid in the atlas matrix to obtain a topological expression mode in the thyroid atlas.
Further, the step S35 of setting the label data corresponding to the labeling boxes of the specified organ as the topology data corresponding to the specified mesh includes:
s351: acquiring label data of a plurality of specified organ labeling boxes;
s352: calculating the sum of the label data of a plurality of the specified organ labeling boxes;
s353: and taking the added value as the topology data corresponding to the specified grid.
In the embodiment of the present application, if a mesh is simultaneously covered by the labeling boxes of multiple organs, and the area proportion of the mesh occupied by each organ labeling box exceeds 30%, the sum of the label values of the multiple organs is used as the assignment of the mesh. For example, if the current mesh is occupied by the labeled boxes corresponding to the trachea and the esophagus at the same time, and the occupied area exceeds 30% of the area of the mesh, the mesh is assigned as "2 +5= 7".
Further, the step S352 of calculating a sum of the labeling data of the plurality of specified organ labeling boxes includes:
s3521: acquiring the types and the number of all organ images included in the thyroid atlas to be analyzed;
s3522: normalizing the labeling data of each designated organ labeling frame according to the number of the types of all organ images included in the thyroid atlas to be analyzed and the label values respectively corresponding to the types to obtain normalized values respectively corresponding to the designated organ labeling frames;
s3523: and summing the normalized numerical values respectively corresponding to the plurality of specified organ labeling boxes covering the specified grid to obtain the summed numerical value of the labeling data of the plurality of specified organ labeling boxes.
In order to improve the consistency in the data processing process, the label values are normalized so as to be compatible with the data unification when the labeling frames of a plurality of organs occupy one grid and the labeling frame of one organ occupies one or more grids. For example, the number of organ types in the present application is 5, and the values of the organ labels are 1, 2, 3, 4 and 5 in sequence. The assignment is normalized to be between 0 and 1 by the normalization process. For example, if the current mesh is identified as occupied by a carotid label box, the corresponding assignment for the current mesh is 3/15 (or 1/5). For another example, if the current mesh is occupied by the labeling boxes corresponding to the trachea and the esophagus, respectively, the assignment value corresponding to the current mesh is 7/15.
Further, step S4, in which a topology branch is added in the deep learning network, and the first feature map is superimposed on the topology attention matrix corresponding to the topology data to obtain a second feature map, includes:
s41: inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics;
s42: multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram;
s43: and taking the feature map of the updated assignment as the second feature map.
In the embodiment of the application, the topological data is changed into a new characteristic diagram of 3 channels through the coordinate value convolution (CoordConv), and the dimension is increased to the topological attention matrix of 256 channels through the convolution calculation of 1 × 1 convolution kernel and 3 × 3 convolution kernel, the average pooling and other operations. The topological attention matrix and the first feature map output by the RPN have the same dimension, which is 256 × 14 in the map, so that the first feature map is in proportion to the corresponding position of the topological attention moment array, that is, the topological attention matrix can be completely and correspondingly overlaid on the first feature map, and since the attention value in the topological attention matrix is a real number from 0 to 1, by multiplying the feature value of each dimension position of the first feature map by the attention value of the corresponding dimension position in a one-to-one correspondence manner, it is equivalent to performing weighting processing on each position of each channel of the first feature map output by the RPN, and obtaining the weighted second feature map.
Further, after the step S3 of labeling the first feature map according to a preset manner to obtain the topology data of the thyroid map to be analyzed, the method includes:
s301: obtaining classifiers of two classes in the topological branches, and calculating a confidence value of the topological data of the thyroid atlas to be analyzed;
s302: judging whether the confidence value exceeds a preset confidence threshold value;
s303: and if so, generating instruction information for superposing the first characteristic diagram on a topology attention matrix corresponding to the topology data.
In the embodiment of the application, the RPN network has a plurality of output combined results for the same input data, and the combined results conforming to the human thyroid gland anatomy structure rule need to be screened through confidence. The method and the device realize confidence estimation on input data by presetting a classifier of two classifications in a newly added topology branch and adding an output branch. The classification recognition capability of the classifier of the second classification is trained according to the pre-labeled training data during model training. As the training data are reasonable data which accord with the anatomical structure rule of the thyroid of the human body, the confidence coefficient is marked to be 1. Meanwhile, assignment of the topological matrix in the training data is disturbed, and the greater the disturbance degree is, the closer the confidence coefficient is to 0, so that the classification recognition capability of the classifier of the second classification is trained. During detection, the topological data corresponding to the thyroid map to be analyzed is used for calculating a confidence coefficient through the topological branch, and the relation between the score of the confidence coefficient and a preset confidence coefficient threshold value is judged. If the preset confidence threshold is 0.5, for example, and the confidence is less than 0.5, the current first feature map is considered to be not in accordance with the anatomical structure rule of the thyroid of the human body, the first feature map is discarded, the subsequent operation of overlaying the topological attention matrix is not performed, the candidate results meeting the conditions are continuously screened from other combined results of the RPN, and then the calculation is performed.
Further, after the step S5 of obtaining an analysis result of the thyroid map to be analyzed according to the feature data in the first feature map and the second feature map, the method further includes:
s51: acquiring a thyroid nodule monitoring network, and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability and the estimated incidence position of thyroid cancer;
s52: judging whether the morbidity probability exceeds a preset probability;
s53: and if so, determining coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed.
The embodiment of the application combines the detection data that current node detection evaluation system obtained, and accurate location position of onset provides effective, accurate data basis.
Further, after the step S1 of acquiring a thyroid map to be analyzed, the method includes:
s101: judging whether the picture size of the thyroid atlas to be analyzed is the same as a preset size or not;
s102: if not, normalizing the picture size of the thyroid atlas to be analyzed to be the same as the preset size.
The mesh equal division in the thyroid gland atlas to be analyzed is changed based on the floating of the picture size of an input image, the fixed size is realized through the normalization operation of the picture size for the thyroid gland atlases with different picture sizes obtained by different ultrasonic scanning machine types, the data amplification and the data reduction of the same thyroid gland atlas are carried out in the same proportion in the subsequent topology attention moment array correction process through the fixed size, the original anatomical structure of the thyroid gland atlas is not influenced, and the positioning analysis accuracy of the thyroid gland atlas obtained by different machine types is improved.
The process of normalizing the picture size of the thyroid map to be analyzed to be the same as the preset size comprises the following steps: defining each pixel point (x, y) as the probability function d of thyroid atlas pixel in x, y directionsx(x, y) and dy(x, y), forming the mapping of the thyroid gland atlas to be analyzed in the histogram with preset sizes to realize normalization. The normalized pixel is expressed as (x ', y'), then
Figure 822003DEST_PATH_IMAGE002
And the thyroid maps with different sizes are normalized into images with fixed sizes through mapping pixel points one by one, and the relative position relation of each organ in the thyroid maps is not changed. Wherein, W1Representing the mapping coefficient, W, in the x-direction2The mapping coefficient representing the y-direction,
Figure 166572DEST_PATH_IMAGE004
,
Figure 883992DEST_PATH_IMAGE006
,p x (x)to representxThe way the grey density in the direction is calculated,p y (y)to representyThe way the grey density in the direction is calculated,h x (x)all representing thyroid map correspondencesxThe gray-scale density of the values is,h y (y)and (3) a calculation mode of gray density of all y values corresponding to the thyroid gland atlas.
Referring to fig. 2, an apparatus for analyzing a thyroid map according to an embodiment of the present application includes:
the first acquisition module 1 is used for acquiring a thyroid atlas to be analyzed, wherein the thyroid atlas comprises a B ultrasonic image obtained by ultrasonic scanning;
the input module 2 is configured to input the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, where the texture analysis network is a deep learning network, and the first feature map includes positions and sizes of organ labeling boxes detected by the texture analysis network;
the labeling module 3 is used for labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid map to be analyzed;
the superposition module 4 is used for superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
and the obtaining module 5 is configured to obtain an analysis result of the thyroid map to be analyzed according to the feature data in the first feature map and the second feature map, where the analysis result includes a relative position between each organ.
In the embodiment of the application, the thyroid map is a B-mode ultrasonic image obtained by ultrasonic scanning, and preliminary image identification is performed on the thyroid map on the basis of a deep learning network so as to obtain the positions and sizes of the marking frames of various organs. The above thyroid maps are transected images. There are many anatomical structures in the transected image, such as carotid arteries, trachea, thyroid glands, and sometimes jugular veins, esophagus, etc. Whereas a longitudinal cut image generally has only thyroid glands, a transverse cut image and a longitudinal cut image can be distinguished by the number of organs included in the image. Because the data volume of the thyroid atlas data marked by the standard cannot reach the standard of accurate image identification, and because the thyroid ultrasound atlas has larger ultrasonic noise influence and is easy to have particularity such as artifacts during scanning, after the thyroid atlas is subjected to preliminary image identification only through the deep learning network, the positions and the sizes of marking frames of various organs are obtained, larger data errors can exist, a large number of false detection and missing detection conditions are caused, data obtained by image identification of the thyroid atlas through the deep learning network is caused, the requirement on data precision cannot be met, and the accuracy is poor.
According to the method, the topological branch is additionally arranged in the deep learning network, so that a deep learning network analysis model based on a topological expression mode is obtained. The deep learning network includes, but is not limited to, the Faster R-CNN, which is composed of a FPN layer and an RPN layer. According to the method, a thyroid atlas is obtained through ultrasonic scanning, the thyroid atlas is input into an FPN layer, an alternative organ labeling frame is determined by combining non-maximum inhibition of a threshold value, and then the thyroid atlas is input into an RPN layer to obtain a first feature map with uniform size. And then, adding coordinate values and label values to the organ labeling boxes obtained from the first characteristic diagram to obtain labeled topological data. And combining a coordinate value convolution process to form input data of the topology branches, obtaining the topology characteristics extracted through the topology branches after convolution and pooling, obtaining a topology attention matrix corresponding to the topology data, and obtaining a second characteristic diagram after superposing the topology attention matrix on the basis of the first characteristic diagram. And then obtaining an analysis result of the thyroid gland map to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map. For example, the feature data of the first feature map shows that the current map includes the texture of the thyroid gland and the texture of the carotid artery, the second feature map shows that the probability of including the thyroid gland labeling box is 90%, the probability of the esophagus labeling box is 86%, and the probability of the carotid artery labeling box is 40%, then the analysis result is that the current thyroid map to be analyzed scans the area compatible with the thyroid gland and the esophagus instead of the area compatible with the thyroid gland and the carotid artery, and the current thyroid map to be analyzed shows the position relationship between the thyroid gland and the esophagus.
In the training process, because the training samples are marked by the marking boxes in advance, the topological matrix corresponding to each training sample is generated through the marking boxes. And after the coordinate value convolution (CoordConv) is changed into a new feature map of 3 channels, the dimension is raised to a topological attention matrix of 256 channels through operations of convolution calculation of 1-1 convolution kernel and 3-3 convolution kernel, average pooling and the like. The coefficients of the convolution kernels are obtained by learning through a back propagation algorithm.
According to the method and the device, the texture data obtained by the texture detection head through the first feature map and the topological features of the topology branch detection head in the second feature map are combined, so that the accurate positioning analysis structure of the anatomical structure included in the thyroid gland map is realized, and the accurate positioning analysis structure comprises organs adjacent to the thyroid gland, relative position relations between the organs and the thyroid gland and the like in the thyroid gland map. The method can be expressed as T = M (P (I) and T (P (I)), wherein T represents an edge detection result, M is a topological model, P represents a deep learning network, I is an input image, and T represents a topological representation mode.
Further, the labeling module 3 includes:
the equipartition submodule is used for evenly dividing the thyroid atlas to be analyzed into an atlas matrix formed by a plurality of grids;
the first judgment submodule is used for judging whether a specified grid and a labeling frame of a specified organ in the first characteristic diagram have a superposition area, wherein the specified grid is any grid in the atlas matrix;
the calculation submodule is used for calculating the area of the overlapping area if the overlapping area exists;
the second judgment submodule is used for judging whether the area of the overlapped area occupies the area proportion of the specified grid and exceeds a preset proportion or not;
setting a submodule, and if the preset proportion is exceeded, setting a label value corresponding to a labeling frame of the specified organ as topological data corresponding to the specified grid;
and the standard submodule is used for labeling the topological data of all the grids in the thyroid atlas to be analyzed according to the labeling process of the topological data corresponding to the specified grid.
In the embodiment of the application, during model training, the topological matrix realizes training of a topological data structure in a thyroid map through known labeled box data of each organ, and the topological data structure comprises various organ types adjacent to the thyroid and a relative position relationship network between each organ and the thyroid. During detection, the input image is averagely divided into N × N grids, for example, N is set to be 8, and the input image is divided into 8 × 8 grids. Each grid is used as an element in the topological matrix, and the matrix elements corresponding to the grid are assigned according to the position relation of the organ labeling frame in the grid to form topological data. For example, organs adjacent to the thyroid in a thyroid map include, but are not limited to, the trachea, carotid artery, jugular vein, esophagus. For example, the tag value corresponding to the thyroid is set to 1, the tracheal tag value is set to 2, the carotid tag value is set to 3, the jugular tag value is set to 4, and the esophageal tag value is set to 5. If the current grid is occupied by an organ marking frame, preliminarily estimating a label value corresponding to the marking frame through a deep learning network, and then judging whether the area proportion of the organ marking frame occupying the grid reaches or exceeds a preset proportion so as to determine the assignment of the grid. For example, the tag value corresponding to the tag frame is preliminarily estimated to be the trachea tag value 2 through the deep learning network, and the trachea tag frame exceeds 30% of the area of the grid, then the grid is assigned to be 2. And (3) assigning each grid in the atlas matrix to obtain a topological expression mode in the thyroid atlas.
Further, the specified mesh is occupied by labeled boxes of a plurality of specified organs, and the setting submodule comprises:
an obtaining unit, configured to obtain tag data of a plurality of the specified organ labeling boxes;
a calculation unit, configured to calculate an addition value of the tag data of the plurality of specified organ labeling boxes;
and the unit is used for taking the addition value as the topological data corresponding to the specified grid.
In the embodiment of the present application, if a mesh is simultaneously covered by the labeling boxes of multiple organs, and the area proportion of the mesh occupied by each organ labeling box exceeds 30%, the sum of the label values of the multiple organs is used as the assignment of the mesh. For example, if the current mesh is occupied by the labeled boxes corresponding to the trachea and the esophagus at the same time, and the occupied area exceeds 30% of the area of the mesh, the mesh is assigned as "2 +5= 7".
Further, a computing unit comprising:
the acquisition subunit is used for acquiring the types and the quantity of all organ images included in the thyroid atlas to be analyzed;
a normalization subunit, configured to perform normalization processing on the labeling data of each designated organ labeling box according to the number of types of all organ images included in the thyroid map to be analyzed and the label values corresponding to the types, respectively, to obtain a normalization value corresponding to each designated organ labeling box;
and the adding subunit is configured to add the normalized numerical values corresponding to the multiple specified organ labeling boxes covering the specified grid, so as to obtain a sum numerical value of labeling data of the multiple specified organ labeling boxes.
In order to improve the consistency in the data processing process, the label values are normalized so as to be compatible with the data unification when the labeling frames of a plurality of organs occupy one grid and the labeling frame of one organ occupies one or more grids. For example, the number of organ types in the present application is 5, and the values of the organ labels are 1, 2, 3, 4 and 5 in sequence. The assignment is normalized to be between 0 and 1 by the normalization process. For example, if the current mesh is identified as occupied by a carotid label box, the corresponding assignment for the current mesh is 3/15 (or 1/5). For another example, if the current mesh is occupied by the labeling boxes corresponding to the trachea and the esophagus, respectively, the assignment value corresponding to the current mesh is 7/15.
Further, the superposition module 4 includes:
the convolution submodule is used for inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics;
the multiplication submodule is used for multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram;
and the sub-module is used for taking the feature map of the updated assignment as the second feature map.
In the embodiment of the application, the topological data is changed into a new characteristic diagram of 3 channels through the coordinate value convolution (CoordConv), and the dimension is increased to the topological attention matrix of 256 channels through the convolution calculation of 1 × 1 convolution kernel and 3 × 3 convolution kernel, the average pooling and other operations. The topological attention matrix and the first feature map output by the RPN have the same dimension, which is 256 × 14 in the map, so that the first feature map is in proportion to the corresponding position of the topological attention moment array, that is, the topological attention matrix can be completely and correspondingly overlaid on the first feature map, and since the attention value in the topological attention matrix is a real number from 0 to 1, by multiplying the feature value of each dimension position of the first feature map by the attention value of the corresponding dimension position in a one-to-one correspondence manner, it is equivalent to performing weighting processing on each position of each channel of the first feature map output by the RPN, and obtaining the weighted second feature map.
Further, an apparatus for analyzing a thyroid map, comprising:
the second acquisition module is used for acquiring classifiers of the second classification in the topological branch and calculating a confidence value of the topological data of the thyroid atlas to be analyzed;
the first judgment module is used for judging whether the confidence value exceeds a preset confidence threshold value;
and the generating module is used for generating instruction information for superposing the first feature map on the topology attention matrix corresponding to the topology data if the preset confidence threshold is exceeded.
In the embodiment of the application, the RPN network has a plurality of output combined results for the same input data, and the combined results conforming to the human thyroid gland anatomy structure rule need to be screened through confidence. The method and the device realize confidence estimation on input data by presetting a classifier of two classifications in a newly added topology branch and adding an output branch. The classification recognition capability of the classifier of the second classification is trained according to the pre-labeled training data during model training. As the training data are reasonable data which accord with the anatomical structure rule of the thyroid of the human body, the confidence coefficient is marked to be 1. Meanwhile, assignment of the topological matrix in the training data is disturbed, and the greater the disturbance degree is, the closer the confidence coefficient is to 0, so that the classification recognition capability of the classifier of the second classification is trained. During detection, the topological data corresponding to the thyroid map to be analyzed is used for calculating a confidence coefficient through the topological branch, and the relation between the score of the confidence coefficient and a preset confidence coefficient threshold value is judged. If the preset confidence threshold is 0.5, for example, and the confidence is less than 0.5, the current first feature map is considered to be not in accordance with the anatomical structure rule of the thyroid of the human body, the first feature map is discarded, the subsequent operation of overlaying the topological attention matrix is not performed, the candidate results meeting the conditions are continuously screened from other combined results of the RPN, and then the calculation is performed.
Further, a device for analyzing thyroid map, further comprising:
the third acquisition module is used for acquiring a thyroid nodule monitoring network and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability of thyroid cancer and an estimated incidence position;
the second judgment module is used for judging whether the morbidity probability exceeds a preset probability;
and the determining module is used for determining the coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed if the probability exceeds the preset probability.
The embodiment of the application combines the detection data that current node detection evaluation system obtained, and accurate location position of onset provides effective, accurate data basis.
Further, an apparatus for analyzing a thyroid map, comprising:
the third judging module is used for judging whether the picture size of the thyroid atlas to be analyzed is the same as the preset size;
and the normalization module is used for normalizing the picture size of the thyroid atlas to be analyzed to be the same as the preset size if the picture size is not the same as the preset size.
The mesh equal division in the thyroid gland atlas to be analyzed is changed based on the floating of the picture size of an input image, the fixed size is realized through the normalization operation of the picture size for the thyroid gland atlases with different picture sizes obtained by different ultrasonic scanning machine types, the data amplification and the data reduction of the same thyroid gland atlas are carried out in the same proportion in the subsequent topology attention moment array correction process through the fixed size, the original anatomical structure of the thyroid gland atlas is not influenced, and the positioning analysis accuracy of the thyroid gland atlas obtained by different machine types is improved.
The process of normalizing the picture size of the thyroid map to be analyzed to be the same as the preset size comprises the following steps: defining each pixel point (x, y) as the probability function d of thyroid atlas pixel in x, y directionsx(x, y) and dy(x, y), forming the mapping of the thyroid gland atlas to be analyzed in the histogram with preset sizes to realize normalization. The normalized pixel is expressed as (x ', y'), then
Figure 216885DEST_PATH_IMAGE008
And the thyroid maps with different sizes are normalized into images with fixed sizes through mapping pixel points one by one, and the relative position relation of each organ in the thyroid maps is not changed. Wherein, W1Representing the mapping coefficient, W, in the x-direction2The mapping coefficient representing the y-direction,
Figure DEST_PATH_IMAGE010
,
Figure DEST_PATH_IMAGE012
,p x (x)to representxThe way the grey density in the direction is calculated,p y (y)to representyThe way the grey density in the direction is calculated,h x (x)all representing thyroid map correspondencesxThe gray-scale density of the values is,h y (y)and (3) a calculation mode of gray density of all y values corresponding to the thyroid gland atlas.
According to the embodiment of the application, the data required to be called in the thyroid gland atlas is analyzed from the data prestored in the block chain, the model analysis result is stored in the block chain, and data storage and sharing are achieved based on the advantages of the block chain. The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.
The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node equipment and used for verifying the validity of the service request, recording the service request to storage after consensus on the valid request is completed, for a new service request, the basic service firstly performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, transmits the service information to a shared account (network communication) completely and consistently after encryption, and performs recording and storage; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.
Referring to fig. 3, a computer device, which may be a server and whose internal structure may be as shown in fig. 3, is also provided in the embodiment of the present application. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the computer designed processor is used to provide computational and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store all the data required for the process of analyzing thyroid maps. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of analyzing a thyroid map.
The processor performs the method for analyzing a thyroid map, comprising: acquiring a thyroid map to be analyzed, wherein the thyroid map comprises a B ultrasonic image obtained by ultrasonic scanning; inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and sizes of the organ labeling boxes detected by the texture analysis network; labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed; superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram; and obtaining an analysis result of the thyroid atlas to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map, wherein the analysis result comprises the relative position of each organ.
According to the computer equipment, the topological branch is additionally arranged in the deep learning network, and the position and the size of each organ marking frame detected by the deep learning network in the first characteristic diagram are corrected through the topological data structure corresponding to the marking frame data in the training data, so that the condition that the carotid artery or the jugular vein and the like are misjudged as the thyroid gland when the deep learning network is independently used for analysis is overcome, and the accuracy of analysis and positioning is improved.
In an embodiment, the step of labeling, by the processor, the first feature map according to a preset mode to obtain the topology data of the thyroid map to be analyzed includes: equally dividing the thyroid map to be analyzed into map matrixes formed by a plurality of grids; judging whether a coincidence region exists between the designated grid and a labeling frame of the designated organ in the first characteristic diagram, wherein the designated grid is any grid in the atlas matrix; if so, calculating the area of the overlapped area; judging whether the area proportion of the area of the overlapped area occupying the designated grid exceeds a preset proportion or not; if so, setting the label value corresponding to the labeling frame of the specified organ as topological data corresponding to the specified grid; and according to the labeling process of the topological data corresponding to the specified grid, labeling the topological data of all grids in the thyroid map to be analyzed.
In one embodiment, the step of setting, by the processor, label data corresponding to a labeling box of the specified organ as topology data corresponding to the specified mesh includes: acquiring label data of a plurality of specified organ labeling boxes; calculating the sum of the label data of a plurality of the specified organ labeling boxes; and taking the added value as the topology data corresponding to the specified grid.
In one embodiment, the step of calculating the sum of the labeling data of the plurality of designated organ labeling boxes by the processor comprises: acquiring the types and the number of all organ images included in the thyroid atlas to be analyzed; normalizing the labeling data of each designated organ labeling frame according to the number of the types of all organ images included in the thyroid atlas to be analyzed and the label values respectively corresponding to the types to obtain normalized values respectively corresponding to the designated organ labeling frames; and summing the normalized numerical values respectively corresponding to the plurality of specified organ labeling boxes covering the specified grid to obtain the summed numerical value of the labeling data of the plurality of specified organ labeling boxes.
In an embodiment, the step of adding a topology branch in the deep learning network, and the step of superimposing the first feature map on the topology attention matrix corresponding to the topology data by the processor to obtain a second feature map includes: inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics; multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram; and taking the feature map of the updated assignment as the second feature map.
In an embodiment, after the step of labeling, by the processor, the first feature map according to a preset manner to obtain the topology data of the thyroid map to be analyzed, the method includes: obtaining classifiers of two classes in the topological branches, and calculating a confidence value of the topological data of the thyroid atlas to be analyzed; judging whether the confidence value exceeds a preset confidence threshold value; and if so, generating instruction information for superposing the first characteristic diagram on a topology attention matrix corresponding to the topology data.
In an embodiment, after the step of obtaining the analysis result of the thyroid map to be analyzed according to the feature data in the first feature map and the second feature map, the processor further includes: acquiring a thyroid nodule monitoring network, and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability and the estimated incidence position of thyroid cancer; judging whether the morbidity probability exceeds a preset probability; and if so, determining coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is only a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects may be applied.
An embodiment of the present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs a method of analyzing a thyroid map, comprising: acquiring a thyroid map to be analyzed, wherein the thyroid map comprises a B ultrasonic image obtained by ultrasonic scanning; inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and sizes of the organ labeling boxes detected by the texture analysis network; labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed; superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram; and obtaining an analysis result of the thyroid atlas to be analyzed according to the characteristic data in the first characteristic map and the second characteristic map, wherein the analysis result comprises the relative position of each organ.
According to the computer-readable storage medium, the topological branch is additionally arranged in the deep learning network, and the position and the size of each organ marking frame detected by the deep learning network in the first characteristic diagram are corrected through the topological data structure corresponding to the marking frame data in the training data, so that the condition that the carotid artery or the jugular vein and the like are misjudged as the thyroid gland when the deep learning network is independently used for analysis is overcome, and the accuracy of analysis and positioning is improved.
In an embodiment, the step of labeling, by the processor, the first feature map according to a preset mode to obtain the topology data of the thyroid map to be analyzed includes: equally dividing the thyroid map to be analyzed into map matrixes formed by a plurality of grids; judging whether a coincidence region exists between the designated grid and a labeling frame of the designated organ in the first characteristic diagram, wherein the designated grid is any grid in the atlas matrix; if so, calculating the area of the overlapped area; judging whether the area proportion of the area of the overlapped area occupying the designated grid exceeds a preset proportion or not; if so, setting the label value corresponding to the labeling frame of the specified organ as topological data corresponding to the specified grid; and according to the labeling process of the topological data corresponding to the specified grid, labeling the topological data of all grids in the thyroid map to be analyzed.
In one embodiment, the step of setting, by the processor, label data corresponding to a labeling box of the specified organ as topology data corresponding to the specified mesh includes: acquiring label data of a plurality of specified organ labeling boxes; calculating the sum of the label data of a plurality of the specified organ labeling boxes; and taking the added value as the topology data corresponding to the specified grid.
In one embodiment, the step of calculating the sum of the labeling data of the plurality of designated organ labeling boxes by the processor comprises: acquiring the types and the number of all organ images included in the thyroid atlas to be analyzed; normalizing the labeling data of each designated organ labeling frame according to the number of the types of all organ images included in the thyroid atlas to be analyzed and the label values respectively corresponding to the types to obtain normalized values respectively corresponding to the designated organ labeling frames; and summing the normalized numerical values respectively corresponding to the plurality of specified organ labeling boxes covering the specified grid to obtain the summed numerical value of the labeling data of the plurality of specified organ labeling boxes.
In an embodiment, the step of adding a topology branch in the deep learning network, and the step of superimposing the first feature map on the topology attention matrix corresponding to the topology data by the processor to obtain a second feature map includes: inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics; multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram; and taking the feature map of the updated assignment as the second feature map.
In an embodiment, after the step of labeling, by the processor, the first feature map according to a preset manner to obtain the topology data of the thyroid map to be analyzed, the method includes: obtaining classifiers of two classes in the topological branches, and calculating a confidence value of the topological data of the thyroid atlas to be analyzed; judging whether the confidence value exceeds a preset confidence threshold value; and if so, generating instruction information for superposing the first characteristic diagram on a topology attention matrix corresponding to the topology data.
In an embodiment, after the step of obtaining the analysis result of the thyroid map to be analyzed according to the feature data in the first feature map and the second feature map, the processor further includes: acquiring a thyroid nodule monitoring network, and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability and the estimated incidence position of thyroid cancer; judging whether the morbidity probability exceeds a preset probability; and if so, determining coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium provided herein and used in the examples may include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double-rate SDRAM (SSRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

Claims (9)

1. A method of analyzing a thyroid map comprising:
acquiring a thyroid map to be analyzed, wherein the thyroid map comprises a B ultrasonic image obtained by ultrasonic scanning;
inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and sizes of the organ labeling boxes detected by the texture analysis network;
labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid atlas to be analyzed;
superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
obtaining an analysis result of the thyroid atlas to be analyzed according to the feature data in the first feature map and the second feature map, wherein the analysis result comprises the relative position of each organ;
adding a topology branch in the deep learning network, and superposing the first feature map on a topology attention matrix corresponding to the topology data to obtain a second feature map, wherein the step comprises the following steps of:
inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics;
multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram;
and taking the feature map of the updated assignment as the second feature map.
2. The method for analyzing a thyroid map according to claim 1, wherein the step of labeling the first feature map in a preset manner to obtain the topological data of the thyroid map to be analyzed comprises:
equally dividing the thyroid map to be analyzed into map matrixes formed by a plurality of grids;
judging whether a coincidence region exists between the designated grid and a labeling frame of the designated organ in the first characteristic diagram, wherein the designated grid is any grid in the atlas matrix;
if so, calculating the area of the overlapped area;
judging whether the area proportion of the area of the overlapped area occupying the designated grid exceeds a preset proportion or not;
if so, setting the label value corresponding to the labeling frame of the specified organ as topological data corresponding to the specified grid;
and according to the labeling process of the topological data corresponding to the specified grid, labeling the topological data of all grids in the thyroid map to be analyzed.
3. The method of analyzing a thyroid map according to claim 2, wherein the designated mesh is occupied by labeling boxes of a plurality of designated organs, and the step of setting label data corresponding to the labeling boxes of the designated organs as topology data corresponding to the designated mesh comprises:
acquiring label data of a plurality of specified organ labeling boxes;
calculating the sum of the label data of a plurality of the specified organ labeling boxes;
and taking the added value as the topology data corresponding to the specified grid.
4. The method of analyzing a thyroid map according to claim 3, wherein the step of calculating a sum value of labeling data of a plurality of the specified organ labeling boxes comprises:
acquiring the types and the number of all organ images included in the thyroid atlas to be analyzed;
normalizing the labeling data of each designated organ labeling frame according to the number of the types of all organ images included in the thyroid atlas to be analyzed and the label values respectively corresponding to the types to obtain normalized values respectively corresponding to the designated organ labeling frames;
and summing the normalized numerical values respectively corresponding to the plurality of specified organ labeling boxes covering the specified grid to obtain the summed numerical value of the labeling data of the plurality of specified organ labeling boxes.
5. The method for analyzing a thyroid map according to claim 1, wherein the step of labeling the first feature map in a predetermined manner to obtain the topology data of the thyroid map to be analyzed comprises:
obtaining classifiers of two classes in the topological branches, and calculating a confidence value of the topological data of the thyroid atlas to be analyzed;
judging whether the confidence value exceeds a preset confidence threshold value;
and if so, generating instruction information for superposing the first characteristic diagram on a topology attention matrix corresponding to the topology data.
6. The method for analyzing a thyroid map according to claim 1, wherein the step of obtaining the analysis result of the thyroid map to be analyzed from the feature data in the first feature map and the second feature map further comprises:
acquiring a thyroid nodule monitoring network, and detecting a nodule detection result of the thyroid atlas to be analyzed, wherein the nodule detection result comprises the incidence probability and the estimated incidence position of thyroid cancer;
judging whether the morbidity probability exceeds a preset probability;
and if so, determining coordinate data corresponding to the estimated disease position according to the analysis result of the thyroid atlas to be analyzed.
7. An apparatus for analyzing a thyroid map, comprising:
the first acquisition module is used for acquiring a thyroid atlas to be analyzed, wherein the thyroid atlas comprises a B ultrasonic image obtained by ultrasonic scanning;
the input module is used for inputting the thyroid atlas to be analyzed into a texture analysis network to obtain a first feature map, wherein the texture analysis network is a deep learning network, and the first feature map comprises the positions and the sizes of the organ labeling boxes detected by the texture analysis network;
the labeling module is used for labeling the first characteristic diagram according to a preset mode to obtain topological data of the thyroid map to be analyzed;
the superposition module is used for superposing the first characteristic diagram on a topological attention matrix corresponding to the topological data to obtain a second characteristic diagram;
an obtaining module, configured to obtain an analysis result of the thyroid map to be analyzed according to feature data in the first feature map and the second feature map, where the analysis result includes a relative position between each organ;
the deep learning network is additionally provided with topology branches and an overlapping module, and the topology branches and the overlapping module comprise:
the convolution submodule is used for inputting the first characteristic diagram marked with the topological data into a topological branch, and performing convolution operation to obtain a topological attention matrix corresponding to the topological data, wherein the topological attention matrix and the first characteristic diagram have the same dimensional characteristics;
the multiplication submodule is used for multiplying the characteristic value in the first characteristic diagram and the characteristic value in the topological attention matrix according to the one-to-one correspondence of the dimension positions to obtain an updated and assigned characteristic diagram;
and the sub-module is used for taking the feature map of the updated assignment as the second feature map.
8. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
CN202011077645.7A 2020-10-10 2020-10-10 Method, device, computer equipment and storage medium for analyzing thyroid gland atlas Active CN111932538B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011077645.7A CN111932538B (en) 2020-10-10 2020-10-10 Method, device, computer equipment and storage medium for analyzing thyroid gland atlas

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011077645.7A CN111932538B (en) 2020-10-10 2020-10-10 Method, device, computer equipment and storage medium for analyzing thyroid gland atlas

Publications (2)

Publication Number Publication Date
CN111932538A CN111932538A (en) 2020-11-13
CN111932538B true CN111932538B (en) 2021-01-15

Family

ID=73333723

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011077645.7A Active CN111932538B (en) 2020-10-10 2020-10-10 Method, device, computer equipment and storage medium for analyzing thyroid gland atlas

Country Status (1)

Country Link
CN (1) CN111932538B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112419396B (en) * 2020-12-03 2024-04-26 前线智能科技(南京)有限公司 Automatic thyroid ultrasonic video analysis method and system
CN112633299B (en) * 2020-12-30 2024-01-16 深圳市优必选科技股份有限公司 Target detection method, network, device, terminal equipment and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108504555A (en) * 2018-04-16 2018-09-07 清华大学 The device and method for differentiating and evaluating tumour progression
CN108830157A (en) * 2018-05-15 2018-11-16 华北电力大学（保定） Human bodys' response method based on attention mechanism and 3D convolutional neural networks

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9980099B2 (en) * 2016-02-12 2018-05-22 International Business Machines Corporation Indoor location detection using ceiling patterns
EP3551052A4 (en) * 2016-12-06 2020-05-06 Darmiyan, Inc. Methods and systems for identifying brain disorders
CN106919689B (en) * 2017-03-03 2018-05-11 中国科学技术信息研究所 Professional domain knowledge mapping dynamic fixing method based on definitions blocks of knowledge
EP3833779A4 (en) * 2018-08-08 2022-05-11 Indian Institute Of Science Education And Research Combined expression pattern of satb family chromatin organizers as improved biomarker tool for cancer prognosis
CN109767842B (en) * 2018-12-13 2023-08-22 平安科技（深圳）有限公司 Disease early warning method, disease early warning device and computer readable storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108504555A (en) * 2018-04-16 2018-09-07 清华大学 The device and method for differentiating and evaluating tumour progression
CN108830157A (en) * 2018-05-15 2018-11-16 华北电力大学（保定） Human bodys' response method based on attention mechanism and 3D convolutional neural networks

Also Published As

Publication number Publication date
CN111932538A (en) 2020-11-13

Similar Documents

Publication Publication Date Title
CN111402228B (en) Image detection method, device and computer readable storage medium
CN109872333B (en) Medical image segmentation method, medical image segmentation device, computer equipment and storage medium
CN108968991B (en) Hand bone X-ray film bone age assessment method, device, computer equipment and storage medium
US11715203B2 (en) Image processing method and apparatus, server, and storage medium
CN111862044A (en) Ultrasonic image processing method and device, computer equipment and storage medium
CN111932538B (en) Method, device, computer equipment and storage medium for analyzing thyroid gland atlas
KR20190028422A (en) Systems and methods for automatic detection, localization, and semantic segmentation of anatomical objects
CN109377500B (en) Image segmentation method based on neural network and terminal equipment
CN111310841A (en) Medical image classification method, apparatus, device, computer device and storage medium
CN111524137A (en) Cell identification counting method and device based on image identification and computer equipment
CN110334722A (en) Image classification method, device, computer equipment and storage medium
CN110717905B (en) Brain image detection method, computer device, and storage medium
CN110866909B (en) Training method of image generation network, image prediction method and computer equipment
WO2022178946A1 (en) Melanoma image recognition method and apparatus, computer device, and storage medium
CN110781976A (en) Extension method of training image, training method and related device
CN110555860A (en) Method, electronic device and storage medium for marking rib region in medical image
WO2022179046A1 (en) Facial recognition method and apparatus, computer device, and storage medium
CN112241952A (en) Method and device for recognizing brain central line, computer equipment and storage medium
CN113705685A (en) Disease feature recognition model training method, disease feature recognition device and disease feature recognition equipment
CN111241331A (en) Image searching method, device, equipment and medium based on artificial intelligence
CN112200780B (en) Bone tissue positioning method, device, computer equipment and storage medium
CN110490841B (en) Computer-aided image analysis method, computer device and storage medium
CN111160441B (en) Classification method, computer device, and storage medium
CN117036305A (en) Image processing method, system and storage medium for throat examination
CN115860067B (en) Method, device, computer equipment and storage medium for generating countermeasure network training

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant