CN113223671B - Microvascular tree generation method based on conditional generation countermeasure network and constraint rule - Google Patents
Microvascular tree generation method based on conditional generation countermeasure network and constraint rule Download PDFInfo
- Publication number
- CN113223671B CN113223671B CN202110540501.9A CN202110540501A CN113223671B CN 113223671 B CN113223671 B CN 113223671B CN 202110540501 A CN202110540501 A CN 202110540501A CN 113223671 B CN113223671 B CN 113223671B
- Authority
- CN
- China
- Prior art keywords
- bifurcation
- branch
- vessel
- sub
- microvascular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Epidemiology (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Apparatus For Radiation Diagnosis (AREA)
Abstract
The invention provides a microvascular tree generation method based on a condition generation countermeasure network and a constraint rule. The mode can generate the microvascular tree by only giving an entrance bifurcation and growing step by step. Meanwhile, the optimized conditions of blood vessel density and bifurcation angle are added in the growth process, the growth angle can be finely corrected, and the micro blood vessel tree with more uniform blood vessel density can be obtained in the growth process, so that the physiological requirements of the micro blood vessel network in the process of delivering oxygen and metabolizing nutrients are better met, and the generated micro blood vessel tree not only meets the morphological characteristics, but also indirectly meets partial physiological characteristics.
Description
Technical Field
The invention relates to a microvascular tree generation method based on a condition generation countermeasure network and a constraint rule, and belongs to the field of biomedicine.
Background
Angiogenesis (Angiogenesis) is an important link in the development of vascular networks. The process is that the blood vessels grow and develop into a blood vessel tree and a blood vessel network from the absence to the existence even under the action of a series of physiological requirements such as blood flow, metabolism, substance exchange and the like, and is also an important process for providing oxygen and nutrient substances and discharging metabolic products for the further development and growth of organisms. The research on the angiogenesis mechanism has important significance in a plurality of research fields. But angiogenesis is a physiological process involving multiple time scales, it is difficult for animal experiments to observe physiological processes over short and long time courses simultaneously by controlling experimental conditions, and the process of obtaining a true microvascular network is difficult and complicated. The traditional method is mostly based on statistics and fractal theory to generate the microvascular tree, and the generated microvascular tree has higher homogeneity and cannot reflect the real microvascular tree structure with high heterogeneity. Therefore, researchers have attempted to mimic the generation of microvascular tree structures. How to generate the microvascular tree which accords with the actual physiological structure and topological characteristics has important research value.
Disclosure of Invention
A conditional generation countermeasure network (CGAN) is an extension of the original GAN, and both the generator and the arbiter are conditioned on adding additional information y, which may be any information, such as category information, or data of other modalities. The conditional GAN is implemented by feeding the extra information y to the discriminant model and the generative model as part of the input layer. The invention is based on the condition generation countermeasure network, uses the real rat mesenteric vascular network as a data set, generates a microvascular tree neogenesis model established by the countermeasure network through the condition, and generates the next-stage bifurcation by using the condition of weakening the previous-stage bifurcation. Aiming at the defects of the traditional method, the mode only needs to give entrance bifurcation and then generates the microvascular tree which has high heterogeneity and conforms to the morphological structure through the restriction rule limitations of blood vessel density, bifurcation angle, pipe diameter and the like.
The technical scheme adopted by the invention for solving the technical problem is as follows:
a microvascular tree generation method for generating an antagonistic network and constraint rules based on conditions comprises the following steps:
a. and extracting a microvascular bifurcation pair from the vascular network, wherein the microvascular bifurcation pair is a group of maternal and daughter vessel bifurcations, and the daughter vessel bifurcation is a vessel bifurcation with one branch of the maternal vessel bifurcation as a main branch. And classifying the pipe diameters and lengths of all common vessel branches of the bifurcation of the maternal vessel and the daughter vessel, and using the classification information of each common vessel branch as a bifurcation condition label of the maternal vessel corresponding to the bifurcation of the daughter vessel. And forming a real sample by the branch data of the sub-system blood vessel and the branch condition label of the main system blood vessel, wherein the branch data of the sub-system blood vessel comprises the coordinates and the pipe diameter information of the main branch node of the sub-system, the coordinates and the pipe diameter information of the left branch node of the sub-system, and the coordinates and the pipe diameter information of the right branch node of the sub-system.
b. Constructing a conditional depth convolution containing a generator and a discriminator to generate a countermeasure network; the generator adopts a deep convolutional neural network structure, inputs uniformly distributed noise which is acquired as a parent vessel bifurcation condition label and randomly, and outputs a group of generated branch vessel bifurcation data of a plurality of subsystems; the discriminator also adopts a deep convolutional neural network for distinguishing whether the input data is from a real sample or a false sample generated by the generator, and the input of the discriminator is the real sample corresponding to the real maternal microvascular bifurcation condition label and the false sample of the real maternal vascular bifurcation condition label and the daughter vessel bifurcation combination generated by the generator.
c. Training a real sample with a condition of the antagonistic network generated by utilizing the condition deep convolution, and finally obtaining a generator model capable of generating the vessel bifurcation of the subordinate subsystem through the superior maternal vessel bifurcation condition label through the game of the loss function; generator
d. And c, generating the branch vessel bifurcation step by step according to the trained generator model obtained in the step c, simultaneously growing the microvascular bifurcation tree, and limiting by using the vessel density, the bifurcation angle, the vessel diameter and the like in the growing process according to a constraint rule, so that the growing process of the microvascular tree is more reasonable. The method specifically comprises the following substeps:
d1 obtaining 1 microvascular bifurcation with the largest main branch caliber as the inlet bifurcation of the vascular tree to be generated, namely the 0 th level bifurcation of the microvascular tree, and taking the inlet bifurcation as the superior maternal vascular bifurcation;
d2 sorting the left and right branches of the maternal vessel bifurcation according to the size of the vessel diameter value, and the growing sequence of the bifurcation is from thick to thin.
d3 selecting the branch with the thickest vessel diameter, judging whether the branch grows, if the diameter of the branch is larger than the growth vessel diameter threshold, then the branch can grow, using the branch as the growth branch, classifying according to the diameter and length, using the branch as the parent vessel bifurcation condition label, inputting the label and the noise z after cascading into the trained generator, and operating the O wheel by the generator to obtain the sub vessel bifurcation library containing O multiplied by P sub vessel bifurcations, wherein P is the batch size of the generator output layer.
D4, calculating the tube diameters D, the branch lengths L and the crimpness T of all the branch branches of the branch vessel bifurcation in the branch vessel bifurcation library generated in the step D3, and calculating the difference between the branch diameter D1, the branch length L1 and the crimpness T1 of each branch of the branch vessel bifurcation and the corresponding growing branch of the branch vessel bifurcation of the parent system to form a similarity difference set F:
F＝{F1,F2,F3,...,Fi,...,FI},Fi＝λ1(D1-Di)+λ2(L1-Li)+λ3(T1-Ti)
wherein I represents an index of bifurcation of a sub-vascular system in the sub-vascular bifurcation library, and I is O × P; lambda [ alpha ]1，λ2，λ3As a weighting coefficient, λ1+λ2+λ 31. The smaller the similarity difference in the sub-branch library, that is, the more similar branch.
d5 using the branch of the sub-system blood vessel with the minimum difference obtained from d4 as the growth branch and translating the end point of the branch main branch to the branch node position of the upper-level mother system blood vessel, then using the branch node position of the upper-level mother system blood vessel as the original point, rotating the branch of the sub-system blood vessel according to the optimized growth angle, and finally replacing the branch of the mother system blood vessel with the main branch of the rotated branch of the sub-system blood vessel to complete the splicing. Wherein, the optimized growth angle is optimized and corrected by the following method:
calculating the included angle theta between the growth branch and the positive half axis of the x axisaAnd judging according to the set C of the direction angles suitable for growth:
if the angle theta is includedaWithin the range of the set C, the angle is the final growth angle; if the angle theta is includedaIf the angle is not in the range of the set C, the correction is carried out until the corrected angle is in the range of the set C, and the correction formula is as follows: thetar＝θaThe ± random (30 ° to 60 °) set C is expressed as: c { (δ -60 °, δ +60 °) { (ε -90 °, ε +90 °) }, U360 °
Wherein epsilon is the included angle between the main branch and the positive semi-axis in the bifurcation of the maternal blood vessel, and delta is the initial microvascular tree diagram before the parent vessel is grownIn the growth node, rotate by thetaaAnd then extracting the included angle between the centroid point of the blood vessel density distribution in the rectangular ROI area extracted by the central point and the positive half axis of the x axis.
d6, detecting whether all branches of the spliced microvascular tree are intersected, if not, the spliced sub-vascular bifurcation is the sub-vascular bifurcation which is finally matched with and grows, otherwise, selecting sub-vascular bifurcations with gradually increased differences from the sub-vascular bifurcation library obtained in the step d3 one by one, and repeating the step d5 until no collision occurs among the branches of the grown microvascular tree;
d7 replacing the branch of parent system capillary bifurcation with the branch with the second largest caliber, repeating d3-d6 steps to complete the growth of the branch of the capillary tree of the child system based on the branch, and completing the growth process of the first-stage capillary tree.
d8, taking the branch of the parent vessel obtained by growth as the branch of the parent vessel for the growth of the next-stage microvascular tree, repeating the steps d2-d7 to finish the growth of the next-stage microvascular tree, and stopping the growth step by step until the vessel diameters of all the growing branches of the microvascular tree are smaller than the set minimum growth vessel diameter threshold value, thus obtaining the complete vascular tree.
Further, the step a specifically includes the following sub-steps:
a1 separates the arteriolar tree from the original microvascular network. And selecting a microvascular bifurcation pair from the microvascular bifurcations of the arteriolar tree data. The micro-vessel bifurcation pair is a group of parent-system and sub-system vessel bifurcations, wherein the parent-system vessel bifurcation data comprises parent-system main branch node coordinates and pipe diameter information, parent-system left branch node coordinates and pipe diameter information, and parent-system right branch node coordinates and pipe diameter information of the parent-system vessel bifurcation, and the sub-system vessel bifurcation data is the vessel bifurcation node coordinates and pipe diameter information of a branch in the parent-system vessel bifurcation data, wherein the vessel bifurcation node coordinates and pipe diameter information comprises sub-system main branch node coordinates and pipe diameter information, sub-system left branch node coordinates and pipe diameter information, and sub-system right branch node coordinates and pipe diameter information.
a2, storing the extracted node coordinates and tube diameters of each microvascular bifurcation pair into a two-dimensional array, and uniformly forming a 20 × N matrix. N is the number of nodes which are mostly contained in the microvascular branches, and the number of columns of each row in the array is not uniform due to different numbers of sub-nodes contained in each section of microvascular branches, so that 0 is supplemented after the coordinate and caliber data of each row in the array, zero is filled in the array [9] and the array [19], and the position where 0 is supplemented is marked.
a3 normalizing the coordinates and diameter of the bifurcation of capillary vessel to 0-1 respectively to retain the relative size information of the bifurcation of capillary vessel.
a4 classifying the tube diameters and lengths of all common blood vessel branches of the maternal and sub-blood vessel branches in the normalized 20 XN matrix, taking the classification information of each common blood vessel branch as the label condition of the corresponding sub-blood vessel branch, combining the sub-blood vessel branch data and the maternal blood vessel branch condition label into a real sample, wherein the form of each sample is 10 XN sub-blood vessel branch data, and each sub-blood vessel branch is attached with a 1X 2-dimensional maternal blood vessel branch condition label.
Further, step a3 is preceded by a data amplification step, which specifically comprises the following steps:
in the range of 0-360 degrees, the bifurcation data stored in the two-dimensional array is amplified by increasing alpha-degree rotation each time, and the rotation formula is as follows:
x1＝x0 cosα-y0 sinα
y1＝x0 sinα+y0 cosα
wherein x is0、y0Coordinates of nodes, x, of the bifurcation of the microvessels, respectively1、y1Respectively are node coordinates after the micro-vessel bifurcation rotates clockwise by alpha degrees around the origin.
Further, in the step b, the neural network structure of the generator includes 5 layers, where the first layer is a fully connected layer, the last 4 layers are deconvolution layers, and the size of the convolution kernel is 3 × 3. Performing example normalization processing after each deconvolution layer; ReLU activation functions are used for all the other layers except the last output layer which is a Tanh activation function.
The generator network structure is constructed on the basis of the generator of the deep convolutional neural production confrontation network, the original batch normalization is replaced by the example normalization, and the generation quality of the microvascular bifurcation is greatly improved.
The network structure of the discriminator comprises 5 layers, wherein the first 4 layers are convolution layers, the last layer is a full connection layer, the number of nodes of the output layer is 1, and the nodes are scalars in the range of (0, 1).
Furthermore, in the discriminator, the spectrum regularization and the instance normalization processing are sequentially performed after each convolution layer. The use of the frequency spectrum regularization mode can increase the constraint force on the model and stabilize the training of the model.
Further, in the step c, the training process specifically includes the following sub-steps:
c1 generator receives the cascade of random noise z and maternal vessel bifurcation condition label c, wherein the maternal vessel bifurcation condition label c guides the generator to generate the bifurcation of the daughter vessel;
c2, cascading the generated branch vessel branch of the daughter vessel and the real branch vessel branch condition label c to form a false sample, and inputting the false sample into a discriminator;
c3 inputting the real sample processed in step a into the discriminator, comparing the true and false samples by the discriminator, and outputting the probability of whether the sample is true or false. The discriminator is essentially a two-classifier, and the goal is not only to determine whether the generated data is authentic, but also to determine whether the data is under specified conditions.
c4 through the judgment result of the discriminator, through back propagation algorithm, and then feedback to the generator, to guide the generator to generate more real micro-vessel bifurcation, at the same time, the discriminator also improves the discrimination ability.
c5 repeating steps c2-c4, carrying out constraint reciprocating training through the loss function, and enabling the two to resist each other, and finally enabling the generator to generate the branch of the daughter blood vessel which accords with the original data distribution according to the branch condition label of the mother blood vessel, so that the discriminator cannot distinguish true from false, and the state of Nash equilibrium is achieved. Wherein the loss function is expressed as:
wherein V (D, G) represents an objective function to be optimized; z represents random noise, k represents real sub-vascular bifurcation data, and c represents a maternal vascular bifurcation condition label; k to Pdata(k)Representing obedience to the original data distribution Pdata(k)，
Is shown at Pdata(k)Distributing and calculating expectations; z to Pz(z) denotes obedience to the prior distribution Pz，PzIs [ -1,1 [ ]]The inner part of the inner part is evenly distributed,
is shown at Pz(z) distributing and calculating the expectation; g (z | c) represents the bifurcation of the daughter vessel where the input vector z is generated by the generator under the constraint of condition c; d (k | c) is represented as the output of a set of joined real samples through the discriminator D.
As can be seen from the above equation, this is a binary infinitesimal maximum game, where the first term of the loss function indicates that, when the input is a real microvascular bifurcation, the discriminator enlarges the objective function as much as possible and judges it as real data; the second term states that when the input is the data G (z | c) generated by the generator, the generator tries to make the objective function small, i.e. it is a small one
Through the steps, the generated microvascular bifurcation and the real microvascular bifurcation are similar as much as possible, at the moment, the generator deceives the discriminator and wrongly considers that the input is real data, at the same time, the discriminator tries to identify the input as false data, and the two models resist each other until Nash equilibrium is finally reached.
Further, rotating theta according to the growth nodes in the initial microvascular tree graph before growthαAfter extraction of the centre pointThe method for acquiring the centroid point of the blood vessel density distribution in the rectangular ROI area comprises the following specific steps:
(1) acquiring a picture of an initial microvascular tree before growth, extracting a rectangular ROI (region of interest) of the picture by taking a growth node as a central point, wherein the pixel of the region is H x H, and converting the region into a single-channel gray-scale image;
(2) rotating the growth node clockwise by theta with the middle point of the lower edge of the ROI region as the originαUpdating the pixel matrix of the region by taking the rotated coordinate point as the center point of the ROI region;
(3) performing sliding window processing on the region, calculating the ratio of pixel points covered by blood vessels of unit pixels of each sub-window as the blood vessel density of the pixel point at the center point of the sub-window, converting the density value of all the blood vessels extracted to the center point of the sub-window into a pixel point matrix of h x h, and converting the matrix into a single-channel gray-scale image, wherein the brighter the image, the higher the blood vessel density is; and performing OTSU self-adaptive binarization processing on the gray scale image, and then calculating a centroid point of a bright color area in the image.
The invention has the following beneficial effects: the conditional generation type countermeasure network can generate a sample of the next-stage bifurcation which accords with the original data distribution according to the given previous-stage bifurcation condition, calculate the blood vessel density of the microvascular tree by adopting a sliding window, and optimize the growth process of the microvascular tree by the constraint conditions of the blood vessel density, the bifurcation angle, the pipe diameter and the like. By means of an iterative processing mode of weakening growth branches into conditional labels to generate branch of the microvessels of the subsystems and splicing the grown microvessel trees, the vascular tree with extremely small errors in the form and the topological structure with the real microvessel tree can be generated finally. The method is greatly different from a microvascular tree generation method for generating an antagonistic network based on U-Net conditions, and is mainly embodied as follows: by weakening the upper bifurcation and generating the lower bifurcation by using the upper bifurcation condition, the diversity of the generated lower bifurcation forms is greatly improved, and the microvascular tree with diversity and specificity in form and high heterogeneity can be grown. Meanwhile, the optimal conditions of blood vessel density and bifurcation angle are added in the growth process, the growth angle can be finely corrected, the micro-blood vessel tree with more uniform blood vessel density can be obtained in the growth process, the physiological requirements of the micro-blood vessel network in the process of delivering oxygen and metabolizing nutrients are better met, and the generated micro-blood vessel tree not only meets the morphological characteristics, but also indirectly meets partial physiological characteristics.
Drawings
FIG. 1 is a schematic illustration of a bifurcated pair.
FIG. 2 is a flow chart of the present invention.
Fig. 3 is a generator and discriminator input/output visualization diagram.
Fig. 4 is a diagram of a generator network architecture.
Fig. 5 is a diagram of a discriminator network.
Fig. 6 is a schematic diagram of the overall structure of the generation countermeasure network.
Fig. 7 is a flow chart for generating a first-level microvascular tree.
Fig. 8 is a visualization of the real microvascular tree and the microvascular tree generated by this method, where a is the arteriolar tree and B is the venular tree.
Detailed Description
For the purpose of better explaining the present invention and to facilitate understanding, the present invention will be described in detail by way of specific embodiments with reference to the accompanying drawings. In the embodiment of the invention, a mouse mesenteric vascular network data set is selected. However, the present invention is not limited to this data set and can be applied to different vascular networks.
For example, in this example, the mesenteric vascular network of the mouse is selected to be composed of arteriolar tree, venule tree and capillary vessel, and the venule tree is formed by the growth of many venule branches. Each microvascular bifurcation has three branches, and each branch is composed of different amounts of node coordinates and caliber information.
Referring to fig. 1 to 8, a microvascular tree generation method for generating a countermeasure network and constraint rules based on conditions includes the following steps:
a. the method comprises the steps of extracting microvascular bifurcation pair data from a vascular network, wherein the microvascular bifurcation pair data is a group of main system and sub-system vascular bifurcation data, the main system vascular bifurcation data comprises main system main branch node coordinate and pipe diameter information of main system vascular bifurcation, main system left branch node coordinate and pipe diameter information, and main system right branch node coordinate and pipe diameter information, the sub-system vascular bifurcation data is vessel bifurcation node coordinate and pipe diameter information with one branch being a main branch in the main system vascular bifurcation data, and the sub-system blood vessel bifurcation data comprises sub-system main branch node coordinate and pipe diameter information, sub-system left branch node coordinate and pipe diameter information, and sub-system right branch node coordinate and pipe diameter information.
As a preferable scheme, in the step a, the data preprocessing includes the following steps:
a1 separates the arteriolar tree from the original microvascular network. Selecting paired data from microvascular bifurcations of the arteriolar tree data, taking parent vessel bifurcations as matched bifurcations, uniformly placing the matched branches at the left branch position, matching the branches in a data set, judging that a sub-vessel bifurcations are formed if the matched branches contain the same branch in the other bifurcation and the bifurcation is different from the matched bifurcation, and combining the two bifurcations to form a complete bifurcation pair; fig. 1 shows a visualized image of a bifurcation pair.
a2, storing the extracted node coordinates and pipe diameters of each microvascular bifurcation pair into a two-dimensional array, wherein one bifurcation pair comprises a parent bifurcation and a child bifurcation, and the number of columns of each row in the array is not uniform due to different numbers of child nodes contained in each section of microvascular bifurcation, so that 0 is supplemented after each row of coordinates and pipe diameter data in the array to uniformly form a 9 × N matrix, and N is the number of nodes most contained in the microvascular bifurcation; wherein array [0 ] stores parent system main branch node x coordinates, array [1 ], stores parent system left branch node x coordinates, array [2 ], stores parent system right branch node x coordinates, array [3 ], stores parent system main branch node y coordinates, array [4 ], [ storage parent system left branch node y coordinates, array [5 ], [ storage parent system right branch node y coordinates, array [6 ], [ storage parent system main branch caliber information, array [7 ], [ storage parent system left branch caliber information, array [8 ], [ storage parent system right branch caliber information, array [9], [ all filled with 0 ], array [10 ], [ storage subsystem main branch node x coordinates, array [11 ], [ storage subsystem left branch node x coordinates, array [12 ], right branch node x coordinates, array [13 ], [ branch coordinates, array [14 ] stores the y coordinates of the left branch node of the subsystem, array [15 ], stores the y coordinates of the right branch node of the subsystem, array [16 ], [ main branch pipe diameter information of the subsystem, array [17 ], [ left branch pipe diameter information of the subsystem, and array [18 ], [ right branch pipe diameter information of the subsystem, wherein the arrays [19], [ all ] are filled with 0, the array [ i ], [ i ] represents the ith row of the array, all the rows form a 20 XN matrix in a unified way, and N is the maximum number of the sub-nodes contained in the branch of the micro-blood vessel. Wherein, for ease of processing, common branches of the parent and child are stored in a left branch array, array [1:7 ].
a3 normalizing coordinate value and caliber value of the microvascular bifurcation respectively to make their values between 0-1 so as to retain the information of the relative size of the microvascular bifurcation, wherein the normalization formula is as follows:
normValue＝(value-min)/(max-min)
wherein value represents a pipe diameter value or a coordinate node, min represents the minimum value of the pipe diameter or the coordinate of the node in the data set, and max represents the maximum value of the pipe diameter or the coordinate of the node in the data set;
in addition, the data can be amplified by increasing alpha degree rotation in the range of 0 to 360 degrees for the microvascular bifurcation tree stored in the array, and the rotation formula is as follows:
x1＝x0 cosα-y0 sinα
y1＝x0 sinα+y0 cosα
wherein x is0、y0Respectively node coordinates, x, of the bifurcation of the microvessels1、y1Respectively are node coordinates after the micro-vessel bifurcation rotates clockwise by alpha degrees around the origin.
A4 records the index of the zero-filling start position in array [0 ],/array [5 ],/the corresponding index recorded in array [6 ],/array [8 ],/the index 0 is set to 1, and simultaneously records the index of the zero-filling start position in array [10 ],/array [15 ],/the corresponding index recorded in array [16 ],/array [18 ],/the index 0 is set to 1, and the array [9] and array [19] are filled with zeros. Through the steps, the original microvascular bifurcation data can be processed into a matrix form of 20 × N for data reprocessing.
b. And c, reprocessing the bifurcation data obtained in the step a, weakening the parent bifurcation in the bifurcation pair into a label condition, and thus obtaining the branch vessel bifurcation data with the parent vessel bifurcation condition label as a real sample, wherein the real sample is a group of branch vessel bifurcation data, and the vessel bifurcation is marked with the corresponding parent vessel bifurcation condition label, and the parent vessel bifurcation condition label contains the class information of the vessel branch common to the parent vessel bifurcation and the branch vessel bifurcation.
Preferably, in the step b, the parent vessel bifurcation in the bifurcation pair data obtained in the step a is weakened into a label condition, so as to obtain the child vessel bifurcation data with the parent vessel bifurcation condition label, which specifically includes the following steps:
b1, extracting the pipe diameter information of the left branch of the parent branch, namely the main branch of the bifurcation of the daughter vessel, in the bifurcation pair data obtained in the step a, sorting the pipe diameter information according to the size, uniformly dividing the corresponding bifurcation of the daughter vessel into five categories from small to large, and respectively marking labels of-1, -0.5, 0, 0.5 and 1.
b2, extracting node coordinate information of the left branch of the parent system bifurcation, namely the main branch of the bifurcation of the sub-system blood vessel, in the bifurcation pair data obtained in the step a, calculating the sum of straight line distances between every two nodes, namely the total length of the blood vessel bifurcation, sorting according to the length values of the blood vessel, uniformly dividing the lengths of the corresponding parent system blood vessel bifurcation into three categories from small to large, and respectively marking labels of-1, 0 and 1.
b3 extracting the branch condition labels of the sub-system blood vessel and the mother system blood vessel in the branch pair data obtained in the above steps, combining two by two to be used as a real sample, obtaining branch data of the sub-system blood vessel with the form of 10 multiplied by N, and attaching a branch condition label of the mother system blood vessel of 1 multiplied by 2 dimension to each branch of the sub-system blood vessel.
c. Combining the CGAN, the DCGAN and the SNGAN to construct a conditional deep convolution generation countermeasure network; the generator adopts a deep convolutional neural network structure, inputs parent vessel bifurcation condition labels and random noise and outputs generated daughter vessel bifurcation data; and c, adopting a deep convolutional neural network in the discriminator, simultaneously increasing the constraint force on the model by using a frequency spectrum regularization mode, and stabilizing the training of the model, wherein the input of the discriminator is the real parent capillary bifurcation condition label obtained in the step b, the false sample of the parent vascular bifurcation combination generated by the generator and the real sample of the real parent vascular bifurcation combination corresponding to the real parent capillary bifurcation condition label. The generator generates a vessel bifurcation by a condition, and the discriminator is used for distinguishing whether the input data is from a real sample or a false sample generated by the generator.
As a preferred scheme, in the step c, based on the deep convolutional neural network, a conditional deep convolutional generation countermeasure network composed of a generator and a discriminator is constructed, which specifically includes the following steps:
the c1 generator network structure is constructed on the basis of the generator of the deep convolutional neural production confrontation network, original Batch Normalization (BN) is replaced by Instance Normalization (IN), and the generation quality of the microvascular bifurcation is greatly improved. The input is the combination of maternal vessel bifurcation condition labels and random noise, the number of layers of the set deep convolutional neural network model is 5, wherein the first layer is a full-connected layer, the last 4 layers are deconvolution layers, and the size of a convolutional kernel is 3 multiplied by 3. The activation function ReLU activation function is used for each layer except the output layer is Tanh activation function. The final output is generated false sample data of the bifurcation of the vessel in the subsystem, and the number of output nodes is 10 multiplied by N; the batchsize is set to 128.
The network of the c2 discriminator is constructed by using a deep convolutional neural network model, and meanwhile, the constraint force on the model is increased by using a frequency spectrum regularization mode, so that the training of the model is stabilized. The input of the method is the sub-vascular bifurcation true sample with the maternal bifurcation label obtained in the step b and a false sample generated by the generator after the cascade connection of the sub-vascular bifurcation and the true label bifurcation. The number of layers of the deep convolutional neural network model is set to be 5, wherein the first 4 layers are convolutional layers, the last layer is a full-link layer, the number of nodes of the output layer is 1, and the node is a scalar in a (0,1) range, the output not only judges whether generated data is real, but also judges whether the data is data under a specified condition, and finally the probability that a sample is true or false is output. The generator G and the discriminator D designed in this embodiment have the visualization diagrams and the network structures shown in fig. 3, 4 and 5, respectively.
d. Training a generator and a discriminator according to the real samples and the false samples, and obtaining a generator model capable of generating lower-level forks through upper-level conditions through the game of the loss function;
further, the training process specifically comprises the following steps:
d1 generator receives the cascade of random noise z and parent vessel bifurcation condition label c, then generates false data G (z | c), wherein the parent vessel bifurcation condition label c guides the generation of the generator, and generates false samples of the branch vessel bifurcation;
d2 cascading the generated sub-system blood vessel bifurcation sample with the real maternal blood vessel bifurcation condition, and inputting the generated sub-system blood vessel bifurcation sample as a false blood vessel bifurcation sample into a discriminator;
d3 inputting the data set obtained from step b as the real sub-blood vessel bifurcation sample into the discriminator, comparing the real and false sub-blood vessel bifurcation samples by the discriminator, wherein the discriminator is essentially a two-classifier, the object is not only to determine whether the generated data is real, but also to determine whether the data is under the specified condition, and finally outputting the probability that the sample is true or false.
d4 the result judged by the discriminator is fed back to the generator through a back propagation algorithm to guide the generator to generate more real microvascular bifurcation, and meanwhile, the discriminator also improves the discrimination capability of the discriminator.
d5 repeating steps d2-d4, repeating the training, and finally generating the branch microvascular bifurcation generated according to the parent vessel bifurcation condition, wherein the branch microvascular bifurcation is very similar to the real branch microvascular bifurcation corresponding to the parent vessel bifurcation condition, so that the discriminator can not distinguish true from false, and the Nash equilibrium state is reached. Wherein the loss function of the generator is:
the loss function of the discriminator is to discriminate whether the input data is a real microvascular bifurcation or a generator-generated microvascular bifurcation, which is defined as follows:
the overall loss function for this condition to generate a competing network can be expressed as:
wherein V (D, G) represents an objective function to be optimized; z represents random noise, k represents the branch of the daughter microvasculature corresponding to the branch of the real maternal microvasculature, and c represents the branch condition label of the real maternal microvasculature in the data set; k to Pdata(k)Representing obedience to the original data distribution Pdata(k)，
Is shown at Pdata(k)Distributing and calculating expectations; z to Pz(z) denotes obedience to the prior distribution Pz，PzIs [ -1,1 [ ]]The inner part of the inner part is evenly distributed,
is shown at Pz(z) distributing and calculating the expectation; g (z | c) represents that the input vector z generates a false inferior daughter vessel bifurcation through the generator under the constraint of the maternal vessel bifurcation condition label c; d (k | c) is represented as the output of a set of joined real data samples through the discriminator D. A schematic diagram of the training body frame is shown in fig. 6.
As can be seen from the above equation, this is a binary infinitesimal maximum game, where the first term of the loss function indicates that, when the input is a real microvascular bifurcation, the discriminator enlarges the objective function as much as possible and judges it as real data; the second term states that when the input is the data G (z | c) generated by the generator, the generator tries to make the objective function small, i.e. it is a small one
Through the steps, the generated microvascular bifurcation and the real microvascular bifurcation are similar as much as possible, at the moment, the generator deceives the discriminator and wrongly considers that the input is real data, meanwhile, the discriminator tries to identify the input as false data, and the two models resist each other until Nash equilibrium is finally reached.
e. And d, generating the branch vessel bifurcation step by step and simultaneously growing the microvascular bifurcation tree according to the trained generator model obtained in the step d, and limiting by using the vessel density, the bifurcation angle, the vessel diameter and the like in the growing process according to a constraint rule, so that the growing process of the microvascular tree is more reasonable. The flow chart for generating the primary microvascular tree is shown in fig. 7, and comprises the following steps:
e1 extracting 1 microvascular bifurcation with the largest main branch diameter from the microvascular tree bifurcation library as the inlet bifurcation of the to-be-generated vascular tree, namely the 0 th level bifurcation of the microvascular tree, and taking the inlet bifurcation as the superior maternal vessel bifurcation;
e2 sorting the sub-branches of the parent vessel bifurcation according to the vessel diameter value, and the bifurcation growing sequence is from thick to thin.
e3 selecting the branch with the thickest vessel diameter to judge if it grows, if the diameter of the branch is less than the growth tube diameter threshold, the branch will not allow to grow, and the following steps will not be performed, and the growth cycle will be skipped. Otherwise, the branch can grow. Wherein the growth tube diameter threshold is typically set at 0.1. And c, weakening corresponding branches of the superior parent vessel bifurcation into condition labels by the method of the step b, cascading the condition labels with noise z, inputting the labels into a trained generator, and operating the generator for 5 rounds to obtain a sub-vessel bifurcation library containing 5 multiplied by 128 sub-vessel bifurcations.
e4 calculating the tube diameter D1, branch length L1 and crimpness T1 of the corresponding growing branch of the parent system bifurcation, and calculating the tube diameters D of all main bifurcation branches in the blood vessel bifurcation library of the subsystem generated in step e3iLength of branch LiAnd degree of curling TiThen, constructing the following cost function through a similarity difference calculation formula, and calculating similarity difference sets F of corresponding growing branches and each branch in the generated sub-branch library in sequence:
F＝{F1,F2,F3,...,Fi,...,FI},Fi＝λ1(D1-Di)+λ2(L1-Li)+λ3(T1-Ti)
wherein, { F1,F2,F3,...,Fi,...,FIThe corresponding growth branch and the set of similarity difference of each bifurcation main branch in the generation sub-bifurcation library are represented, I represents the index of bifurcation of the sub-vessels in the sub-vessel bifurcation library, and I is 5 multiplied by 128; lambda [ alpha ]1，λ2，λ3As a weighting coefficient, λ1+λ2+λ3＝1。θrFor an optimized growth angle, θaIs the original growth angle.
Degree of curling TiThe calculation method of (2) is a classical measurement method (DM), and the calculation formula is as follows:
Ti＝C/L
wherein C represents the linear distance between two end points of the blood vessel branch, and L represents the total length of the blood vessel branch.
By sorting the calculated similarity difference set F from small to large and simultaneously re-sorting the corresponding sub-bifurcation libraries according to the order, the earlier-sorted branches in the sub-bifurcation library are considered as branches with smaller similarity difference, i.e. branches with higher similarity.
e5 extracting the branch of sub-blood vessel arranged in the 1 st antegrade branch of sub-blood vessel in the branch library obtained from e4 as growth branch, translating the end point of the branch main branch of the branch to the branch node position of the upper-grade mother blood vessel, and then taking the branch node position of the upper-grade mother blood vessel as the origin.
The coordinate translation calculation mode is as follows:
newArray[0:3,:]＝array[0:3,:]-(endX-nodeX)
newArray[3:5,:]＝array[3:5,:]-(endY-nodeY)
wherein array [0:3 ] represents the x-coordinate of the main branch and the sub-branch of the bifurcation of the daughter vessel before translation, and newArray [0:3 ] represents the x-coordinate of the main branch and the sub-branch of the bifurcation of the daughter vessel after translation; array [3:5 ]: represents the y coordinates of the bifurcation main branch and the bifurcation of the daughter vessel before translation, newArray [3:5 ]: represents the y coordinates of the bifurcation main branch and the bifurcation of the daughter vessel after translation, (endX, endY) represents the end point of the bifurcation main branch of the daughter vessel, and (nodeX, nodeY) represents the node of the bifurcation of the parent vessel;
e6 rotating the bifurcation of the daughter vessel according to the optimized growth angle. The angle is obtained by calculating the growth density of the region to obtain a proper growth angle, namely a rotation angle thetar. The specific optimization steps are as follows:
1) storing the picture of the initial microvascular tree before the growth of the round, wherein the growth nodes of the round are marked by red points, and the vascular tree is depicted in green;
2) by identifying red in the picture, a growth node A can be extracted, a rectangular ROI area of the picture is extracted by taking the growth node A as a central point, pixels of the area are H x H, and the area is converted into a single-channel gray-scale image;
3) calculating the included angle between the branch as the growth branch and the positive half axis of the x axis in the maternal vessel bifurcation, wherein the angle is the original growth angle thetaa. Rotating the growth node clockwise by theta with the middle point of the lower edge of the ROI region as the originaUpdating the pixel matrix of the region by taking the rotated coordinate point B as the central point of the ROI region, wherein the purpose of the step is to extract a more important blood vessel density region biased to the original growth direction by rotating the extracted ROI region matrix;
4) performing sliding window processing on the region by using the window width H/5 and the step length (2 x 2), and calculating the whole blood vessel density of each sub-window as the blood vessel density of the pixel at the center point of the sub-window, wherein the blood vessel density calculation formula of each sub-window w is as follows:
Denw＝Mw/H2
wherein M iswRepresenting the number of pixel points covered by blood vessels in the sub-window, wherein mu represents the gray value of the pixel point, MwThe calculation formula is as follows:
5) and after extracting the list of the blood vessel density values of the center point of the sub-window, converting the list into a pixel point matrix of h x h, and converting the matrix into a single-channel gray image, namely a growth density image, wherein the blood vessel density is higher in an area with higher brightness in the image, and the blood vessel density is lower in an area with lower brightness. Performing OTSU self-adaptive binarization processing on the gray scale image, and extracting a centroid point of a white area in the growth density image after acquiring the outline of the high-density white area in the growth density image, wherein the calculation formula of the centroid point is as follows:
in which the spatial moment m is calculatedjiThe formula (u) and (v) represent coordinates of pixel points in the gray-scale image, and array (u, v) is a pixel point matrix:
6) calculating an included angle delta between the centroid point of the high-density region and the positive half axis of the x axis by taking the centroid point as a starting point and the growth node as an origin, wherein the angular direction region of (delta-60 degrees and delta +60 degrees) is considered as a growth direction unsuitable due to high blood vessel density;
7) and calculating an included angle epsilon between the main branch of the bus-branch fork and the positive half shaft. Counting the distribution of included angles of a main branch and a left branch and a right branch in an original bifurcation database, and finding that the size of the included angles is basically distributed within the range of 100 degrees and 180 degrees, so that the angle direction region of (epsilon-90 degrees and epsilon +90 degrees) is considered as a direction which does not accord with the growth trend;
8) the set of direction angles C suitable for growth is then:
C＝{(δ-60°，δ+60°)∩(ε-90°，ε+90°)}∪360°
if the original growth angle thetaaWithin the range of the set C, the angle is the final growth angle;
if the original growth angle thetaaIf the angle is not in the range of the set C, the angle is considered to need to be corrected, and if the corrected angle is in the range of the set C, the correction is considered to be successful, wherein the correction formula is as follows:
θr＝θa±random(30°～60°)
e7 rotating the branch of the micro-vessel clockwise around the growth node, the rotation angle is the optimized growth angle theta obtained in step e6rThe rotation calculation formula is as follows:
x”＝(x'-nodeX)cosθr+(y'-nodeY)sinθr+nodeX
y”＝(y'-nodeY)cosθr-(x'-nodeX)sinθr+nodeY
wherein x 'and y' are coordinates of bifurcation of the daughter vessel after translation, and x "and y" are coordinates of bifurcation of the microvasculature after x 'and y' rotate clockwise around (nodeX, nodeY), respectively;
e8 splicing the sub-vascular bifurcation after translation and rotation to the microvascular tree, and replacing the growing branch of the main vascular bifurcation with the main branch of the sub-vascular bifurcation after rotation.
e9 performing collision detection on the spliced and grown microvascular tree. Firstly, each branch node of the spliced micro-vessel tree needs to be stored in a two-dimensional array, then whether three branches of the lower-level sub-vessel bifurcation primarily spliced in the process are intersected with all branches of the spliced micro-vessel tree or not is calculated, if the branches are not intersected, the final matched and connected micro-vessel bifurcation is obtained, and the branch nodes are stored in the two-dimensional array. The formula for determining whether intersections are as follows:
min(x”1,a,x”1,b)＜＝max(x”2,a,x”2,b)
min(x”2,a,x”2,b)＜＝max(x”1,a,x”1,b)
min(y”1,a,y”1,b)＜＝max(y”2,a,y”2,b)
min(y”2,a,y”2,b)＜＝max(y”1,a,y”1,b)
wherein x "1,a,x”1,b,y”1,a,y”1,bRespectively are the coordinates of two end points, x, of the 1 st branch in the two compared branches "1,a,x”1,b,y”2,a,y”2,bRespectively is the node coordinates of two end points of the other branch, if the above conditions are not met, the collision does not occur between the branches, and the wheel branches grow successfully;
if the conditions are met simultaneously, judging that the two branches are intersected, indicating that collision occurs between branches, failing to grow the branch, selecting the sub-branches from the sub-branch library obtained in the step e3 again, adding 1 to the selected sub-branch order, repeating the steps e5-e9 until collision does not occur between branches of the grown microvascular tree, and judging that the growth is successful;
if the growth is successful, the successfully grown sub-vessel bifurcation is stored in an array to be grown for the growth of the next wheel vessel bifurcation.
e10, replacing the branch of the parent system capillary bifurcation with the branch with the second largest caliber, repeating the steps e2-e8 to finish the growth of the branch of the parent system capillary bifurcation, and finishing the growth process of the first-stage capillary tree.
e11 using the bifurcation array of the first-level blood vessel tree obtained in the above steps as the maternal bifurcation array for the growth of the next-level microvascular tree, and repeating the steps e2-e10 to complete the growth of the next-level microvascular tree.
e12 when the vessel diameter of all growth branches of the microvascular tree is less than the set minimum growth vessel diameter threshold value, the growth of the vascular tree is completed.
Through the steps, the microvascular tree similar to the real microvascular tree in shape and topology can be generated.
The conditional generation type countermeasure network can generate a false sample of the next-level bifurcation which accords with the original data distribution according to the given previous-level bifurcation condition, calculate the blood vessel density of the microvascular tree by adopting a sliding window, and optimize the growth process of the microvascular tree through the constraint conditions of the blood vessel density, the bifurcation angle, the pipe diameter and the like. By means of an iterative processing mode of weakening growth branches into conditional label generation subsystem capillary bifurcation and splicing growth capillary trees, a blood vessel tree with extremely small errors in shape and topological structure with a real capillary tree can be generated finally, and the growth effects of the arteriolar tree and the venular tree of the network constructed by DCGAN are respectively shown as A, B in fig. 8.
TABLE 1 statistical comparison of data indices of real and generated vessel bifurcations
Respectively randomly selecting 500 microvascular bifurcation data from the actual microvascular bifurcation and the generated microvascular bifurcation, calculating each quantitative analysis index of the microvascular bifurcation data, and then carrying out statistical test. As shown in table one. When p >0.05, the two parts of data have no significant difference and conform to the same distribution. As can be seen from the table, the average value and the standard deviation of each quantitative analysis index of the microvascular bifurcation data generated by the method can be basically consistent with the standard deviation of the real microvascular bifurcation data, and the morphological characteristics of the microvascular tree generated by the method are proved to be consistent with the real microvascular tree.
The invention weakens the upper-level bifurcation and utilizes the upper-level bifurcation condition to generate the lower-level bifurcation, thereby greatly improving the diversity of the generated lower-level bifurcation forms and growing the microvascular tree with diversity and specificity in form and high heterogeneity. Meanwhile, the optimized conditions of blood vessel density and bifurcation angle are added in the growth process, the growth angle can be finely corrected, and the micro blood vessel tree with more uniform blood vessel density can be obtained in the growth process, so that the physiological requirements of the micro blood vessel network in the process of delivering oxygen and metabolizing nutrients are better met, and the generated micro blood vessel tree not only meets the morphological characteristics, but also indirectly meets partial physiological characteristics.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.
Claims (7)
1. A microvascular tree generation method based on a conditional generation countermeasure network and constraint rules is characterized by comprising the following steps:
a. extracting a micro-vessel bifurcation pair from a vessel network, wherein the micro-vessel bifurcation pair is a group of parent vessel and sub-vessel bifurcations, the sub-vessel bifurcation is a vessel bifurcation with one branch of the parent vessel bifurcation as a main branch, the vessel diameters and the lengths of the common vessel branches of all the parent vessel and sub-vessel bifurcations are classified, classification information of each common vessel branch is used as a parent vessel bifurcation condition label of the corresponding sub-vessel bifurcation, and sub-vessel bifurcation data and the parent vessel bifurcation condition label form a real sample, wherein the sub-vessel bifurcation data comprises the coordinates and the vessel diameters of the main branch nodes of the sub-system, the coordinates and the vessel diameters of the left branch nodes of the sub-system, and the coordinates and the vessel diameters of the right branch nodes of the sub-system;
b. constructing a conditional depth convolution containing a generator and a discriminator to generate a countermeasure network; the generator adopts a deep convolutional neural network structure, inputs uniformly distributed noise which is acquired as a parent vessel bifurcation condition label and randomly, and outputs a group of generated branch vessel bifurcation data of a plurality of subsystems; the discriminator also adopts a deep convolutional neural network and is used for distinguishing whether input data come from a real sample or a false sample generated by the generator, and the input of the discriminator is the real sample corresponding to the real maternal microvascular bifurcation condition label and the false sample of the real maternal vascular bifurcation condition label and the daughter vessel bifurcation combination generated by the generator;
c. training a real sample with a condition of the antagonistic network generated by utilizing the condition deep convolution, and finally obtaining a generator model capable of generating the vessel bifurcation of the subordinate subsystem through the superior maternal vessel bifurcation condition label through the game of the loss function;
d. and c, according to the trained generator model obtained in the step c, generating the branch vessel bifurcation step by step and simultaneously growing the branch vessel bifurcation tree, and specifically comprising the following substeps:
d1 obtaining 1 microvascular bifurcation with the largest main branch caliber as the inlet bifurcation of the vascular tree to be generated, namely the 0 th level bifurcation of the microvascular tree, and taking the inlet bifurcation as the superior maternal vascular bifurcation;
d2 sorting the left and right branches of the maternal vessel bifurcation according to the size of the vessel diameter value, and the bifurcation growth sequence is from thick to thin;
d3, selecting the branch with the thickest vessel diameter, judging whether the branch grows, if the vessel diameter of the branch is larger than the growth vessel diameter threshold, growing, using the branch as a growth branch, classifying according to the vessel diameter and the length, using the branch as a parent vessel bifurcation condition label, inputting the label and noise z after cascading into a trained generator, and operating an O wheel by the generator to obtain a subsystem vessel bifurcation library containing O multiplied by P subsystem vessel bifurcations, wherein P is the batch size of the generator output layer;
d4, calculating the tube diameters D, the branch lengths L and the crimpness T of all the branch branches of the branch vessel bifurcation in the branch vessel bifurcation library generated in the step D3, and calculating the difference between the branch diameter D1, the branch length L1 and the crimpness T1 of each branch of the branch vessel bifurcation and the corresponding growing branch of the branch vessel bifurcation of the parent system to form a similarity difference set F:
F＝{F1,F2,F3,...,Fi,...,FI},Fi＝λ1(D1-Di)+λ2(L1-Li)+λ3(T1-Ti)
wherein, i tableAn index indicating the bifurcation of the sub-vascular from the sub-vascular bifurcation library, I ═ O × P; lambda1，λ2，λ3As a weighting coefficient, λ1+λ2+λ3＝1；
d5 using the branch of the sub-system blood vessel with the minimum difference obtained from d4 as a growing branch and translating the end point of the branch main branch to the branch node position of the upper-level mother system blood vessel, then using the branch node position of the upper-level mother system blood vessel as the original point, rotating the branch of the sub-system blood vessel according to the optimized growing angle, and finally replacing the growing branch of the mother system blood vessel with the main branch of the rotated branch of the sub-system blood vessel to complete the splicing; wherein, the optimized growth angle is optimized and corrected by the following method:
calculating the included angle theta between the growth branch and the positive half shaft of the x axisaAnd judging according to the set C of direction angles suitable for growth:
if the angle theta is includedaWithin the range of the set C, the angle is the final growth angle; if the angle theta is includedaIf the angle is not in the range of the set C, the angle after correction is corrected to be in the range of the set C, and the correction formula is as follows: thetar＝θa±random(30°～60°)
Set C is represented as: c { (δ -60 °, δ +60 °) (ε -90 °, ε +90 °) }, U360 °
Wherein epsilon is the included angle between the main branch and the positive half axis in the bifurcation of the maternal blood vessel, and delta is the rotation theta of the growth node in the initial microvascular tree diagram before the growthaThen extracting the included angle between the centroid point of the blood vessel density distribution in the rectangular ROI area extracted for the central point and the positive half axis of the x axis;
d6, detecting whether all branches of the spliced microvascular tree are intersected, if not, the spliced sub-vascular bifurcation is the sub-vascular bifurcation which is finally matched with and grows, otherwise, selecting sub-vascular bifurcations with gradually increased differences from the sub-vascular bifurcation library obtained in the step d3 one by one, and repeating the step d5 until no collision occurs among the branches of the grown microvascular tree;
d7 replacing the branch of the parent system capillary bifurcation with the branch with the second largest caliber, repeating the steps d3-d6 to finish the growth of the branch of the capillary tree of the child system based on the branch, and finishing the growth process of the first-stage capillary tree;
d8, taking the branch of the parent vessel obtained by growth as the branch of the parent vessel for the growth of the next-stage microvascular tree, repeating the steps d2-d7 to finish the growth of the next-stage microvascular tree, and stopping the growth step by step until the vessel diameters of all the growing branches of the microvascular tree are smaller than the set minimum growth vessel diameter threshold value, thus obtaining the complete vascular tree.
2. The microvascular tree generation method based on conditional generation countermeasure networks and constraint rules according to claim 1, wherein the step a specifically comprises the following sub-steps:
a1 separating the arteriolar tree from the original microvascular network, selecting microvascular bifurcation pairs from the microvascular bifurcations of the arteriolar tree data, wherein the microvascular bifurcation pairs are a group of maternal and sub-vascular bifurcations, the maternal vascular bifurcation data comprises maternal main branch node coordinates and pipe diameter information of the maternal vascular bifurcation, maternal left branch node coordinates and pipe diameter information, maternal right branch node coordinates and pipe diameter information, the sub-vascular bifurcation data is the vascular bifurcation node coordinates and pipe diameter information of one branch in the maternal vascular bifurcation data, which is a main branch, and comprises sub-branch node coordinates and pipe diameter information, sub-left branch node coordinates and pipe diameter information, and sub-right branch node coordinates and pipe diameter information;
a2, storing the extracted node coordinates and pipe diameters of each capillary bifurcation pair into a two-dimensional array, and uniformly forming a 20 multiplied by N matrix, wherein N is the number of nodes most contained in the capillary bifurcation, columns with data less than N are complemented by 0, the position where 0 is supplemented is marked, and rows without data are complemented by 0;
a3 respectively carrying out normalization processing on the bifurcation coordinate and the caliber of the micro blood vessel;
a4 classifying the tube diameters and lengths of all common blood vessel branches of the maternal and sub-blood vessel branches in the normalized 20 XN matrix, taking the classification information of each common blood vessel branch as the label condition of the corresponding sub-blood vessel branch, combining the sub-blood vessel branch data and the maternal blood vessel branch condition label into a real sample, wherein the form of each sample is 10 XN sub-blood vessel branch data, and each sub-blood vessel branch is attached with a 1X 2-dimensional maternal blood vessel branch condition label.
3. The method for generating a microvascular tree based on a conditional generation countermeasure network and constraint rules according to claim 2, wherein the step a3 is preceded by a data amplification step, specifically as follows:
in the range of 0-360 degrees, the bifurcation data stored in the two-dimensional array is amplified by increasing alpha-degree rotation each time, and the rotation formula is as follows:
x1＝x0cosα-y0sinα
y1＝x0sinα+y0cosα
wherein x is0、y0Respectively node coordinates, x, of the bifurcation of the microvessels1、y1Respectively are node coordinates after the micro-vessel bifurcation rotates clockwise by alpha degrees around the origin.
4. The method for generating a microvascular tree based on conditional generation countermeasure networks and constraint rules according to claim 1, wherein in the step b, the neural network structure of the generator comprises 5 layers, wherein the first layer is a fully connected layer, the last 4 layers are deconvolution layers, the size of a convolution kernel is 3 x 3, and an example normalization process is adopted after each deconvolution layer; the other layers use the ReLU activation function except the last output layer is the Tanh activation function;
the network structure of the discriminator comprises 5 layers, wherein the first 4 layers are convolution layers, the last layer is a full connection layer, the number of nodes of the output layer is 1, and the nodes are scalars in the range of (0, 1).
5. The method according to claim 4, wherein the discriminator sequentially performs spectral regularization and case normalization after each convolution layer.
6. The microvascular tree generation method based on conditional generation countermeasure networks and constraint rules according to claim 1, wherein in the step c, the training process specifically comprises the following sub-steps:
c1 generator receives the cascade of random noise z and maternal vessel bifurcation condition label c, wherein the maternal vessel bifurcation condition label c guides the generator to generate the bifurcation of the daughter vessel;
c2, cascading the generated branch vessel branch of the daughter vessel and the real branch vessel branch condition label c to form a false sample, and inputting the false sample into a discriminator;
c3 inputting the real sample processed in step a into a discriminator, comparing the true and false samples by the discriminator, and outputting the probability of whether the sample is true or false;
c4, the judgment result of the discriminator is fed back to the generator through a back propagation algorithm to guide the generator to generate more real microvascular bifurcation, and meanwhile, the discriminator also improves the discrimination capability of the discriminator;
c5 repeating steps c2-c4, carrying out constraint reciprocating training through a loss function, wherein the constraint reciprocating training and the constraint reciprocating training are mutually confronted, and finally enabling the generator to generate the branch vessel bifurcation conforming to the original data distribution according to the parent vessel bifurcation condition label, so that the discriminator cannot distinguish true from false, and a Nash equilibrium state is achieved, wherein the loss function is expressed as:
wherein V (D, G) represents an objective function to be optimized; z represents random noise, k represents real sub-vascular bifurcation data, and c represents a maternal vascular bifurcation condition label; k to Pdata(k)Representing obedience to the original data distribution Pdata(k)，
Is shown at Pdata(k)Distributing and calculating expectations; z to Pz(z) denotes obedience to the prior distribution Pz，PzIs [ -1,1 [ ]]The inner part of the inner part is uniformly distributed,
is shown at Pz(z) distributing and calculating the expectation; g (z | c) represents the bifurcation of the daughter vessel where the input vector z is generated by the generator under the constraint of condition c; d (k | c) is represented as the output of a set of joined real samples through the discriminator D.
7. The method of claim 1, wherein θ is rotated according to the growth node in the original microvascular tree graph before growingαThe method for acquiring the centroid point of the blood vessel density distribution in the rectangular ROI extracted from the central point specifically comprises the following steps:
(1) acquiring a picture of an initial microvascular tree before growing, extracting a rectangular ROI (region of interest) of the picture by taking a growing node as a central point, wherein the pixel of the region is H x H, and converting the region into a single-channel grey-scale map;
(2) rotating the growth node clockwise by theta by taking the middle point below the ROI as an originαUpdating the pixel matrix of the region by taking the rotated coordinate point as the center point of the ROI region;
(3) performing sliding window processing on the region, calculating the ratio of pixel points covered by blood vessels of unit pixels of each sub-window as the blood vessel density of the pixel point at the center point of the sub-window, converting the density value of all the blood vessels extracted to the center point of the sub-window into a pixel point matrix of h x h, and converting the matrix into a single-channel gray-scale image, wherein the brighter the image, the higher the blood vessel density is; and performing OTSU self-adaptive binarization processing on the gray-scale image, and calculating a centroid point of a bright color area in the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110540501.9A CN113223671B (en) | 2021-05-18 | 2021-05-18 | Microvascular tree generation method based on conditional generation countermeasure network and constraint rule |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110540501.9A CN113223671B (en) | 2021-05-18 | 2021-05-18 | Microvascular tree generation method based on conditional generation countermeasure network and constraint rule |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113223671A CN113223671A (en) | 2021-08-06 |
| CN113223671B true CN113223671B (en) | 2022-05-27 |
Family
ID=77092624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110540501.9A Active CN113223671B (en) | 2021-05-18 | 2021-05-18 | Microvascular tree generation method based on conditional generation countermeasure network and constraint rule |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113223671B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980899A (en) * | 2017-04-01 | 2017-07-25 | 北京昆仑医云科技有限公司 | The deep learning model and system of flow characteristic on prediction vascular tree blood flow paths |
| GB201811610D0 (en) * | 2018-07-16 | 2018-08-29 | Medsolve Ltd | An intracoronary wire, system and method for evaluating intracoronary flow |
| CN108596840A (en) * | 2018-03-28 | 2018-09-28 | 浙江工业大学 | A kind of data set Enhancement Method for deep learning evaluation blood vessel network developmental level |
| CN109871630A (en) * | 2019-02-28 | 2019-06-11 | 浙江工业大学 | A kind of manually generated method of capilary tree construction |
| CN110599444A (en) * | 2018-08-23 | 2019-12-20 | 深圳科亚医疗科技有限公司 | Device, system and non-transitory readable storage medium for predicting fractional flow reserve of a vessel tree |
| CN112070767A (en) * | 2020-09-10 | 2020-12-11 | 哈尔滨理工大学 | Micro-vessel segmentation method in microscopic image based on generating type countermeasure network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3749956A4 (en) * | 2018-02-06 | 2022-01-05 | The Regents Of The University Of Michigan | Systems and methods for analysis and remote interpretation of optical histologic images |
-
2021
- 2021-05-18 CN CN202110540501.9A patent/CN113223671B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980899A (en) * | 2017-04-01 | 2017-07-25 | 北京昆仑医云科技有限公司 | The deep learning model and system of flow characteristic on prediction vascular tree blood flow paths |
| CN108596840A (en) * | 2018-03-28 | 2018-09-28 | 浙江工业大学 | A kind of data set Enhancement Method for deep learning evaluation blood vessel network developmental level |
| GB201811610D0 (en) * | 2018-07-16 | 2018-08-29 | Medsolve Ltd | An intracoronary wire, system and method for evaluating intracoronary flow |
| CN110599444A (en) * | 2018-08-23 | 2019-12-20 | 深圳科亚医疗科技有限公司 | Device, system and non-transitory readable storage medium for predicting fractional flow reserve of a vessel tree |
| CN109871630A (en) * | 2019-02-28 | 2019-06-11 | 浙江工业大学 | A kind of manually generated method of capilary tree construction |
| CN112070767A (en) * | 2020-09-10 | 2020-12-11 | 哈尔滨理工大学 | Micro-vessel segmentation method in microscopic image based on generating type countermeasure network |
Non-Patent Citations (2)
| Title |
|---|
| RV-GAN: Segmenting Retinal Vascular Structure in Fundus Photographs using a Novel Multi-scale Generative Adversarial Network;Sharif Amit Kamran et al;《arXiv》;20210103;全文 * |
| 基于生成对抗网络的人工微血管树生成方法研究;蒋阿康;《中国优秀博硕士学位论文全文数据库(硕士)基础科学辑》;20200815;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113223671A (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | DENSE-INception U-net for medical image segmentation | |
| Cruz et al. | Detection of grapevine yellows symptoms in Vitis vinifera L. with artificial intelligence | |
| Elnashef et al. | Tensor-based classification and segmentation of three-dimensional point clouds for organ-level plant phenotyping and growth analysis | |
| Zieliński et al. | Deep learning approach to bacterial colony classification | |
| Abdel-Basset et al. | HSMA_WOA: A hybrid novel Slime mould algorithm with whale optimization algorithm for tackling the image segmentation problem of chest X-ray images | |
| CN107292330B (en) | Iterative label noise identification algorithm based on double information of supervised learning and semi-supervised learning | |
| Hu et al. | Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems | |
| US8831327B2 (en) | Systems and methods for tissue classification using attributes of a biomarker enhanced tissue network (BETN) | |
| Kaur et al. | An approach for characterization of infected area in tomato leaf disease based on deep learning and object detection technique | |
| CN107730542B (en) | Cone beam computed tomography image correspondence and registration method | |
| Chin-Wei et al. | Multiobjective optimization approaches in image segmentation–the directions and challenges | |
| Shao et al. | Automatic microarray image segmentation with clustering-based algorithms | |
| Zhang et al. | Triplet attention and dual-pool contrastive learning for clinic-driven multi-label medical image classification | |
| CN112560710A (en) | Method for constructing finger vein recognition system and finger vein recognition system | |
| Rani et al. | New approaches in machine-based image analysis for medical oncology | |
| Dang et al. | Two-layer Ensemble of Deep Learning Models for Medical Image Segmentation | |
| CN113223671B (en) | Microvascular tree generation method based on conditional generation countermeasure network and constraint rule | |
| Azam et al. | Using feature maps to unpack the CNN ‘Black box’theory with two medical datasets of different modality | |
| Ghosh et al. | Combining neural network models for blood cell classification | |
| Sai et al. | Flower Identification and Classification applying CNN through Deep Learning Methodologies | |
| Canales et al. | Optimal segmentation of image datasets by genetic algorithms using color spaces | |
| Adaïmé et al. | Deep learning approaches to the phylogenetic placement of extinct pollen morphotypes | |
| Mulyana et al. | Gender Classification for Anime Character Face Image Using Random Forest Classifier Method and GLCM Feature Extraction | |
| Hamila et al. | Fusarium head blight detection, spikelet estimation, and severity assessment in wheat using 3d convolutional neural networks | |
| Gachoki et al. | Features Selection in Statistical Classification of High Dimensional Image Derived Maize (Zea Mays L.) Phenomic Data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |