CN111986137B

CN111986137B - Biological organ lesion detection method, apparatus, device, and readable storage medium

Info

Publication number: CN111986137B
Application number: CN201910425238.1A
Authority: CN
Inventors: 梁红霞; 赵丽俊; 张晓雅; 董爱莲
Original assignee: Individual
Current assignee: Individual
Priority date: 2019-05-21
Filing date: 2019-05-21
Publication date: 2024-06-28
Anticipated expiration: 2039-05-21
Also published as: CN111986137A

Abstract

The invention discloses a method, a device, equipment and a readable storage medium for detecting biological organ lesions. The method comprises the following steps: acquiring a medical image of an organ, the medical image comprising a plurality of voxels; identifying an organ image from the medical image; carrying out lesion recognition on a blood vessel region obtained from an organ image to obtain a first lesion region in the organ image; performing lesion pre-detection on the organ image through a trained deep learning model to obtain a pre-detection lesion area image in the organ image; judging the pre-detected lesion area image to obtain a second lesion area in the organ image; and obtaining a final lesion region of the organ image according to the first lesion region and the second lesion region. The method provides a method for indirectly detecting the organ lesion, which is combined with the existing direct method, and the organ lesion is detected in an omnibearing way from a wider angle, so that the detection result is greatly improved.

Description

Biological organ lesion detection method, apparatus, device, and readable storage medium

Technical Field

The invention relates to the technical field of medical image processing and application, in particular to a method, a device and equipment for detecting biological organ lesions and a readable storage medium.

Background

CT (Computed Tomography, computerized tomography) is increasingly being used as a powerful means of detecting early lesions in organs.

Currently, the processing method for CT scan results mainly focuses on the following aspects: firstly, preprocessing a CT image by using a computer image processing method such as image segmentation and the like, and further segmenting and enhancing a lesion part so as to improve the display effect of the CT image; and secondly, preprocessing the CT image by using a traditional computer image processing method, and processing and analyzing the preprocessed organ image.

However, the above method for identifying lesions of CT scan results is to process the shape of the lesions (such as nodules) to achieve more complete segmentation and more effective feature extraction and classification, and belongs to a direct detection method. Such methods have a natural dependence on the shape that the node itself presents in the image. The method is particularly suitable for treating small nodules (about 2 mm), special-shaped nodules and the like, and further limits the application range of the detection of the nodes. In actual clinic, the small nodules and the irregularly shaped nodules are more common causes of false negatives and false positives, and are difficult to find by medical staff in actual work.

The above information disclosed in the background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

In view of this, the present invention provides a method, apparatus, device and readable storage medium for detecting a biological organ lesion.

Other features and advantages of the invention will be apparent from the following detailed description, or may be learned by the practice of the invention.

According to an aspect of the present invention, there is provided a method for detecting a biological organ lesion, comprising: acquiring a medical image of an organ, the medical image comprising a plurality of voxels; identifying an organ image from the medical image; performing lesion recognition on a blood vessel region acquired from the organ image to acquire a first lesion region in the organ image; performing lesion pre-detection on the organ image through a trained deep learning model to obtain a pre-detection lesion area image in the organ image; judging the pre-detected lesion area image to obtain a second lesion area in the organ image; and obtaining a final lesion region of the organ image according to the first lesion region and the second lesion region.

According to another aspect of the present invention, there is provided a biological organ lesion detection device comprising: the image acquisition module is used for acquiring medical images of organs, wherein the medical images comprise a plurality of voxels; the image identification module is used for identifying an organ image from the medical image; the lesion recognition module is used for recognizing lesions of the organ image to obtain a first lesion area in the organ image; the lesion pre-detection module is used for pre-detecting the lesions of the organ images through a trained deep learning model to obtain pre-detected lesion area images in the organ images; the lesion research and judgment module is used for researching and judging the pre-detected lesion area image to obtain a second lesion area in the organ image; and a lesion determining module, configured to obtain a final lesion area of the organ image according to the first lesion area and the second lesion area.

According to still another aspect of the present invention, there is provided a computer apparatus comprising: the system comprises a memory, a processor and executable instructions stored in the memory and executable in the processor, wherein the processor implements any one of the methods when executing the executable instructions.

According to yet another aspect of the present invention, there is provided a computer readable storage medium having stored thereon computer executable instructions which when executed by a processor implement a method as any one of the above.

According to the biological organ lesion detection method provided by the embodiment of the invention, an indirect organ lesion detection method is provided, and the method is combined with the existing direct method to carry out omnibearing detection on organ lesions from a wider angle, so that the detection result is greatly improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

Drawings

The above and other objects, features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

Fig. 1 is a flowchart illustrating a method for detecting a biological organ lesion according to an exemplary embodiment.

Fig. 2 is a flow chart illustrating another method of detecting a biological organ lesion according to an exemplary embodiment.

Fig. 3 is a cross-sectional view of a vascular region at a layer, according to an example.

Fig. 4 and 5 show a vessel tree generated according to the vessel tree generation method described above based on scanned retinal images and lung images, respectively.

Fig. 6 is a flowchart illustrating yet another method of detecting a biological organ lesion according to an exemplary embodiment.

Fig. 7 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment.

Fig. 8 shows a cross-sectional view of the three-dimensional skeleton generated for the vascular region shown in fig. 3 in one layer.

Fig. 9 is a schematic diagram showing a process of generating a blood vessel segment according to an example. Fig. 9 (a) shows the respective vessel bifurcation points s of the skeleton.

Fig. 10 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment.

Fig. 11 shows a process of generating a vessel tree from the vessel segments as shown in fig. 9.

Fig. 12 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment.

Fig. 13 shows a schematic representation of a lung lesion identified according to the identification rule.

Fig. 14 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment.

Fig. 15 is a block diagram illustrating a biological organ lesion detection device according to an exemplary embodiment.

Fig. 16 is a schematic diagram showing a structure of an electronic device according to an exemplary embodiment.

FIG. 17 is a schematic diagram of a computer-readable storage medium according to an example embodiment.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known structures, methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

Furthermore, in the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise. "and/or" describes an association relationship of an associated object, meaning that there may be three relationships, e.g., a and/or B, and that there may be a alone, B alone, and both a and B. The symbol "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature.

Referring to fig. 1, a biological organ lesion detection method 10 includes:

in step S102, a medical image of an organ is acquired.

The medical images may include, for example, CT medical images, MRI (Magnetic Resonance Imaging, nuclear magnetic resonance) medical images, ultrasound medical images, etc.; in addition, the organs collected in the medical image can be lung, liver, brain, retina, etc., and the invention is not limited thereto.

Taking a common axial CT scan image as an example, the medical image may be a three-dimensional tensor Q, where each element Q _i,j,k is a non-negative integer, which may also be referred to as a voxel or a pixel. Wherein subscript i=0, 1,2,3, … …, row index in a layer for CT scan; subscript j=0, 1,2,3, … …, column index in a layer for CT scan; k=0, 1,2,3, … … is the layer index of the CT scan.

In step S104, an organ image is identified from the medical image.

For example, the sizes of all elements Q _i,j,k in the three-dimensional tensor Q may be clustered by using an existing clustering method, so as to identify the organ region of each layer, thereby identifying the organ image Q _l. In the organ image Q _l, the voxel values belonging to the organ region are the same as those in the medical image Q, and the voxel values of the other regions are set to 0.

In some embodiments, prior to step S106 or step S108 described below, the method 10 may further include: and removing the interference image in the organ image to obtain the organ image from which the interference image is removed. The interference image may include, for example, a tracheal image and/or a diaphragmatic top muscle image. For the identified organ image, the trachea is sometimes identified as an organ region, so a thresholding method can be adopted, the trachea is identified by using connectivity of the trachea on the three-dimensional tensor Q, and the trachea in the organ image is removed by using a mask (mask), so that the organ image Q _l with the trachea removed is obtained. Similarly, the topmost muscle is also present as a disturbance in the organ image, so that the topmost muscle portion may be identified by thresholding, and the organ image Q _l after removal of the topmost muscle may be obtained by using the topmost muscle in the mask region organ image.

In step S106, lesion recognition is performed on a blood vessel region acquired from an organ image, and a first lesion region in the organ image is obtained.

For example, lesion recognition may be performed on the organ images based on the created vessel tree according to predefined lesion rules. The method for identifying lesions based on the vascular tree will be described below.

In addition, in some embodiments, lesion recognition may be performed on the organ image according to the following method, to obtain a first lesion region in the organ image:

1) A plurality of vascular regions are segmented from the organ image.

The description of how to divide the plurality of blood vessel regions from the organ image is the same as the following step S204 in the method 20, and in particular, refer to the following description of step S204.

2) For each vessel region, a rectangular grid of 2r x 2r is selected, and slides layer by layer on the organ image Q _l, with a sliding step size of, for example, 1.

A) Each time the number of voxels falling within the matrix network above the threshold thresh is calculated, denoted n_thresh.

B) If n_thresh > =4r ² -1, then the corresponding region of the 2r x 2r rectangular grid is determined to be the first lesion region.

In step S108, the organ image is subjected to lesion pre-detection by the trained deep learning model, and a pre-detection lesion region image in the organ image is obtained.

The deep learning model may be, for example, resNet neural network models. The main idea of ResNet (Residual Neural Network) neural networks is to add a direct channel, i.e. the idea of Highway Network, in the Network. ResNet can accelerate the training of the neural network very quickly, and the accuracy of the model is also improved greatly. Meanwhile, resNet has very good popularization and can be even directly used in InceptionNet networks.

In some embodiments, the method 10 may further comprise: and (3) inputting a training set which is manually marked with a lesion region (for example, a supervision tag marked with a lesion center point and a radius) into the ResNet neural network model by adopting an Adam algorithm for learning training so as to obtain a trained deep learning model. Adam's algorithm is an algorithm that optimizes a random objective function based on a first order gradient. Adam's algorithm dynamically adjusts the learning rate for each parameter based on the first and second moment estimates of the gradient of the loss function for each parameter.

The pre-detection method aims at pre-judging the lesions of the organ image so as to reduce the range of the area to be judged later, for example, the lesions with the probability of more than 0.95 can be pre-judged.

In step S110, the pre-detected lesion area image is studied and judged to obtain a second lesion area in the organ image.

For example, pre-detected lesion area images may be studied based on the created vessel tree according to predefined lesion rules. The method for determining lesions based on the vascular tree will be described below.

In addition, in some embodiments, the pre-detected lesion region image may be further studied to obtain a second lesion region in the organ image according to the following method:

for each pre-detected lesion area W e W (W is the pre-detected lesion area image):

1) Calculating a principal direction vector of a midpoint coordinate of the pre-detected lesion region w by using a morphological calculation method: θ _w =pca (C), where C is the coordinates of all points in the pre-detected lesion area w. PCA (principal component analysis) method returns the first three largest characteristic values

\theta＝{\theta_1,\theta_2,\theta_3}。

2) If θ _min/θ_max≥θ_th, this pre-detected region w is determined as the second lesion region.

Wherein, θ _min＝min(θ),θ_max＝max(θ),θ_th can take a value between 0.8 and 0.9, for example. The satisfaction of the condition indicates that the region is approximately "spherical".

The processing in steps S106 to S110 does not indicate or limit the time sequence of these processing. That is, there is no time sequence in execution between step S106 and steps S108 to S110, and step S106 may be executed first, and then steps S108 to S110 may be executed; steps S108 to S110 may be performed first, and step S106 may be performed later; step S106 and steps S108 to S110 may be performed simultaneously. The specific order of execution is determined by the specific application, and the invention is not limited thereto.

In the method for detecting a biological organ lesion according to the embodiment of the present invention, on the one hand, the lesion recognition is performed on the organ image identified in step S104 through step S106, so as to identify the first lesion region; on the other hand, the lesion pre-detection and lesion research are performed on the organ images identified in step S104 through steps S108 and S110, so as to identify the second lesion region. This model, which combines direct and indirect methods, may be referred to as a Gamma architecture. The Gamma structure may share the organ image identified in step S104 on the one hand, and may share the established plurality of vessel trees when lesion identification and lesion research are performed through the established vessel tree on the other hand.

It should be understood by those skilled in the art that, in practical application, the steps S106 and S108 to S110 may be performed synchronously or asynchronously, which is not a limitation of the present invention.

In step S112, a final lesion region of the organ image is obtained according to the first lesion region and the second lesion region.

For example, the first lesion region and the second lesion region may be combined to determine a final lesion region. In addition, the determined final lesion area can be processed, classified and output correspondingly.

According to the biological organ lesion detection method provided by the embodiment of the invention, an indirect organ lesion detection method is provided, and the method is combined with the existing direct method to carry out omnibearing detection on organ lesions from a wider angle, so that the detection result is greatly improved. Specifically, the biological organ lesion detection method provided by the embodiment of the invention not only directly analyzes the morphology and distribution of organ lesions, but also indirectly reflects the lesions by using various information such as the shape and distribution of blood vessels through analyzing and modeling the morphology and structure of blood vessel regions, thereby achieving better detection success rate by combining with a direct monitoring model and greatly reducing the probability of false negative and false positive.

The Gamma architecture is different from the 'multi-method parallel voting' rule of a parallel structure in pattern recognition, and the two methods of direct detection and indirect detection are organically combined (for example, the calculation morphology is combined with a deep learning model), so that the advantages of the two methods are fully exerted, and the two methods are made to make the best of the two methods by adjusting parameters and algorithm details, so that the detection result can have interpretability and adjustability, and a better detection result is achieved.

Taking the example of lesion recognition on lung CT scan images, lesion detection was performed on 888 low dose CT scan images of the LUNA16 dataset using the method described above. As shown in Table 1, compared with the standard methods of ZNET, masakam, DIAG-CONVNET, 3D Faster-RCNN and the like which are commonly used at present, the method provided by the embodiment of the invention has the experimental effect of least false positive under the condition of keeping high recall rate. Thus, the workload and false alarm rate of the subsequent lesion confirmation of doctors can be reduced.

TABLE 1

It should be clearly understood that the present invention describes how to make and use specific examples, but the principles of the present invention are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.

Fig. 2 is a flow chart illustrating another method of detecting a biological organ lesion according to an exemplary embodiment. The difference from the method 10 for detecting a biological organ lesion shown in fig. 1 is that the method 20 for detecting a biological organ lesion shown in fig. 2 further provides a method for creating a vascular tree from an organ image, and the created vascular tree can be used for lesion recognition of the organ image to obtain the first lesion region, and can be used for studying and judging a pre-detected lesion region image to obtain the second lesion region.

Referring to fig. 2, the biological organ lesion detection method 20 includes:

In step S202, an organ image or a pre-detection lesion area image is acquired.

The organ image Q _l is acquired, or the lesion region is acquired and detected.

In step S204, a plurality of blood vessel regions are segmented from the organ image or the pre-detection lesion region image.

The organ image or the pre-detection lesion area image is subjected to image preprocessing, and voxel values of a non-blood vessel area (for example, a lung image such as an air area of a lung) and other tissue areas except for a blood vessel area and voxels having similar characteristics to the blood vessel can be set to 0 so as to divide a plurality of blood vessel areas from the organ image.

The vascular region includes: blood vessels and voxels with similar characteristics to blood vessels.

One vessel region is R _x,r_x e R, where R is the set of vessel regions segmented from the organ image Q _l. Each vascular region is a separate three-dimensional region and each vascular region is not adjacent to other vascular regions. That is, the adjacent voxel values of each vessel region are all 0. Fig. 3 is a cross-sectional view of a vascular region at a layer, according to an example. Wherein, the black voxel value is 0, and the white voxel value is greater than 0.

In step S206, an association matrix of a plurality of blood vessel regions is established according to the proximity relation between the plurality of blood vessel regions.

The correlation matrix of the plurality of vessel regions may be denoted as M for representing the neighbor relation between the vessel regions, i.e. which vessel regions are closer. Because the vessel regions that are closer together are likely vessels that are connected together in the image, but these vessels are disconnected due to noise from the CT scan or when decision is made by thresholding. The construction of the correlation matrix helps to establish possible connection relations between the vascular regions.

In step S208, a blood vessel segment and a blood vessel bifurcation point for each blood vessel region are acquired.

For example, the vessel segments and vessel bifurcation points contained in each vessel region may be obtained by calculating fractal (also referred to as calculating morphology).

In step S210, a vessel segment, a vessel bifurcation point, and a connection relationship between the vessel segments and the vessel bifurcation points, which are included in each vessel region, are constructed as a vessel map using the vessel segment and the vessel bifurcation point as nodes, based on the correlation matrix.

For example, the vessel segments, the vessel bifurcation points, and the connection relations between the vessel segments and the vessel bifurcation points included in each vessel region may be first constructed as a preliminary vessel map, and then the connection relations between the vessel regions may be supplemented to the preliminary vessel map according to the correlation matrix M to obtain a final vessel map (e.g., may be denoted as G).

In step S212, a corresponding plurality of vessel trees are created based on the plurality of connected subgraphs in the vessel map.

The nodes in each vessel tree are vessel segments contained in the corresponding connected subgraph.

The vessel map may include a plurality of connected sub-maps, each of which has vessel segments connected to each other, and vessels between the connected sub-maps are disconnected from each other. Based on each connected subgraph, corresponding blood vessel trees are established. For example, the blood vessels between the left and right lungs are not connected with each other, so that the blood vessels belong to different connected subgraphs, and different blood vessel trees can be respectively constructed.

According to the method for generating the blood vessel tree, the morphology, structure, characteristics and connection relation among blood vessels of the blood vessel image can be analyzed aiming at the separated blood vessel image, and an accurate blood vessel tree can be constructed. The vessel tree can be used for lesion recognition, and can be used for further researching and judging lesion areas recognized by other modes so as to obtain a more accurate lesion recognition result. Fig. 4 and 5 show a vessel tree generated according to the vessel tree generation method described above based on scanned retinal images and lung images, respectively. Fig. 4 (a) and 5 (a) show the scanned retinal image and lung image, respectively, and fig. 4 (b) and 5 (b) show the vessel tree corresponding to the retinal image and lung image, respectively. As can be seen from fig. 4 and 5, by this method, an accurate vessel tree can be generated based on the acquired medical images.

Fig. 6 is a flowchart illustrating yet another method of detecting a biological organ lesion according to an exemplary embodiment. Unlike the biological organic lesion detection method 20 shown in fig. 2, the biological organic lesion detection method 30 shown in fig. 6 further provides a method of establishing an association matrix of a plurality of blood vessel regions according to a proximity relationship between the plurality of blood vessel regions.

Referring to fig. 6, a medical image-based vessel tree generation method 30 includes:

in step S302, for each layer of image of the medical image, a plurality of blood vessel regions are processed:

1. The spatial index structure I _k is inserted with the region covered by the plurality of blood vessel regions r _x as a node.

The spatial index structure I _k may be, for example, an existing Rtree structure index, or may be an existing other spatial index structure.

2. For each vessel region r _x, the voxels adjacent thereto are expanded, obtaining an expanded region of that vessel region, which can be denoted as r _x'.

3. And searching an overlapped blood vessel region r _y overlapped with the blood vessel region r _x in the spatial index structure I _k by taking the expansion region r _x' of the blood vessel region as a spatial query condition.

4. And assigning values to corresponding elements in the association matrix M according to the expansion times.

For example, the number of extensions z=0 is initialized, and r _x'＝r_x.

If element M _xy in the correlation matrix M is equal to 0 or M _xy is greater than z, then let z be assigned to M _xy and M _yx.

After the expansion is finished, if the expansion times z do not reach a preset threshold value (such as 2-6 voxels), the step 2 is skipped, and the execution is repeated.

In step S304, an association matrix of a plurality of blood vessel regions is obtained.

After the above operations are completed for each layer, a final correlation matrix of a plurality of blood vessel regions is established.

Fig. 7 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment. Unlike the medical image-based vessel tree generation method 20 shown in fig. 2, the biological organ lesion detection method 40 shown in fig. 7 further provides a method of acquiring a vessel segment and a vessel bifurcation point of each vessel region, respectively.

Referring to fig. 7, a medical image-based vessel tree generation method 40 includes: the following steps are performed separately for each vessel region:

In step S402, a corresponding plurality of three-dimensional skeletons are generated for the vascular region by calculating fractal.

The corresponding three-dimensional skeleton can be generated for each vascular region by adopting the existing skeleton extraction (skeletonization) method in computational fractal. Fig. 8 shows a cross-sectional view of the three-dimensional skeleton generated for the vascular region shown in fig. 3 in one layer.

In step S404, a blood vessel bifurcation point is acquired from each three-dimensional skeleton, respectively.

The vessel bifurcation point is a voxel surrounded by 2 or more adjacent voxels, for example, gray voxels as in fig. 8.

In a specific implementation, for example, the voxel value of the bifurcation point of the blood vessel may be set to-1, and marked as a virtual node, and added to the virtual node set S.

In step S406, for each three-dimensional skeleton: removing the bifurcation points of the blood vessels to obtain a plurality of blood vessel segment skeletons in the three-dimensional skeleton; performing region growing on the plurality of vessel segment skeletons respectively to obtain a plurality of vessel segments respectively corresponding to the plurality of vessel segment skeletons; and determining attributes of the plurality of vessel segments, respectively.

After each vessel bifurcation s was removed, each vessel segment skeleton as shown in fig. 9 b was obtained by a labeling (labeling) method of the connected region. The vessel segment skeleton is actually the centerline of the vessel segment without any vessel branches.

Each vessel segment skeleton is subjected to region growing and restored to the vessel segment as shown in fig. 9 (c). Specifically, voxels in adjacent vessel regions are sequentially expanded circumferentially for each vessel segment until no vessel region voxels can be expanded.

The algorithm scheme may include, for example: and (5) carrying out algorithm design based on a Matrix-Vector operation framework.

A voxel correlation matrix a of the vessel region is defined, wherein when two voxels are adjacent, the element in the corresponding matrix a is 1, otherwise 0. The vessel segments of these voxels are assigned a vector b. Since vessel segment skeletons are known and labeled with numbers greater than 0, points therein are assigned on vector b as the number of vessel segment skeletons.

The following operations were repeated:

1. let g _ij＝a_ijb_j denote that voxel i can reach the vessel skeleton b _j by side a _ij;

2. Let b _i'＝max(g_i1,g_i2,…,g_in), which determines the final reachable vessel segment skeleton of voxel i;

3. If b _i is equal to 0, then b _i＝b_i' is enabled.

Until all elements in vector b are not 0, an assignment is obtained, thereby completing the region growing from the vessel segment skeleton to the vessel segment.

The set of vessel segments may be denoted as V, for example, with each vessel segment V e V.

The attributes of each vessel segment are then calculated separately, and the vessel segment attributes may include: the length, radius, direction, voxel set, etc. of the vessel segment may be denoted as { l, r, θ, C }, respectively.

Wherein the set of voxels corresponding to each vessel segment v is denoted C.

The radius of the vessel segment v can be calculated by the formula (1):

Wherein the function dist (P, P _c) represents the distance of the point P belonging to O from the skeleton P _c to which the vessel segment v corresponds.

Calculate the direction θ for each vessel segment v: the main distribution direction of all voxels C of a vessel segment is taken as the direction of the vessel segment v. The voxel set C of the vessel segment is represented as a3 XN voxel point coordinate matrix, and each three-dimensional column vector C epsilon C is one of the voxel coordinates. Thus, the direction of voxel point distribution, i.e. the direction of the voxel point distribution, can be obtained by principal component analysis (PRINCIPAL COMPONENT ANALYSIS, PCA)

Wherein the method comprises the steps ofIs the average coordinates of the voxels in C.

The length l of the vessel segment v can be calculated by the formula (2):

Wherein T represents the transpose.

Fig. 10 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment. Unlike the biological organic lesion detection method 20 shown in fig. 2, the biological organic lesion detection method 50 shown in fig. 10 further provides a method of creating a corresponding plurality of vessel trees based on a plurality of connected subgraphs in the vessel map.

Referring to fig. 10, the biological organ lesion detection method 50 includes:

In step S502, for each connected subgraph, a vessel segment having the closest distance to the heart, the largest radius and only one vessel bifurcation point adjacent thereto is selected as the root node of the vessel tree.

For example, the root node may be denoted as v _c. Fig. 11 shows a process of generating a vessel tree from the vessel segments as shown in fig. 9. Wherein fig. 11 (a) is a schematic view of a plurality of vessel segments and vessel bifurcation points, and fig. 11 (b) is a schematic view of one connected sub-graph in the vessel graph constructed from the vessel segments and vessel bifurcation points in fig. 11 (a). Fig. 11 (c) shows a schematic diagram of a vessel tree generated from the connected subgraph shown in fig. 11 (b).

As shown in fig. 11 (b), the vessel segment v ₀ conforms to the selection rule of the root node of the vessel tree.

In step S504, starting from the root node, all nodes in the connected subgraph are sequentially processed, including: for each adjacent vessel bifurcation of the node:

1. acquiring adjacent nodes of the bifurcation point of the blood vessel.

Taking the example of processing a root node vessel segment v ₀ as in fig. 11 (b), the adjacent nodes of its vessel bifurcation s ₀ are vessel segments v ₁ and v ₄, respectively.

2. One of the adjacent nodes is selected as a trunk node of the vessel tree, and other adjacent nodes are marked as branch nodes.

Selecting one of the adjacent nodes as a trunk node of the vessel tree may include, for example: selecting a trunk node according to a formula (3):

Where v _τ denotes the backbone node, Representing a set of adjacent nodes of the vessel bifurcation. Sigma=2 to 5 voxels representing the upper limit of the radius difference between the vessel nodes

3. The node is connected with the trunk node, and is formed and marked as trunk side tau.

4. The node is connected with the branch node, and is formed and marked as a branch edge b.

The constructed vessel tree is shown in fig. 11 (c).

Fig. 12 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment. Unlike the biological organ lesion detection method 20 shown in fig. 2, the biological organ lesion detection method 60 shown in fig. 12 further provides a method of lesion recognition of an organ image based on the generated vascular tree.

Referring to fig. 12, a lesion recognition method 60 based on a vessel tree includes:

In step S202, an organ image is acquired.

In step S204, a plurality of blood vessel regions are segmented from the organ image.

The steps S202 to S210 are the same as those shown in fig. 2, and are not described here again.

In step S602, lesion recognition is performed according to a predefined lesion recognition rule based on a plurality of vessel trees.

In some embodiments, the lesion recognition rules include some or all of the following rules:

And when the number of nodes in the blood vessel tree is smaller than a preset threshold value of the number of nodes, determining that a lesion exists in a region corresponding to the blood vessel tree.

For example, if the vessel tree has only isolated root nodes, or very few nodes, i.e., the tree root depth is small, it is determined that there is a lesion in the region corresponding to the vessel tree. The threshold value of the number of the nodes can be set according to actual requirements when the node is applied.

And when the radius difference value between the node in the vessel tree and the adjacent node is larger than a preset first radius threshold value, determining that the region corresponding to the node has lesions.

The first radius threshold may be set to 1 to 5 pixel numbers, for example, but the present invention is not limited thereto.

When traversing each node of the blood vessel tree by adopting the breadth-first traversing strategy, determining that lesions exist in the region corresponding to the node of which the radius difference value of the two nodes accessed before and after is out of the preset radius range.

The radius range may be set to [ -1,2], for example, but the invention is not limited thereto.

When the radius of the branch node of the blood vessel tree is larger than a preset second radius threshold value, determining that a lesion exists in the region corresponding to the branch node.

The second radius threshold may be set to 1 to 5, for example, but the present invention is not limited thereto.

For the medical image which is an image of a bilateral symmetry organ, when the KL distance of the radius distribution curves of the nodes in the two vessel trees with the largest left and right sides of the organ is larger than a preset first distance threshold, determining that lesions exist in the areas corresponding to the two vessel trees.

KL distance is an abbreviation for the Kullback-Leibler difference (Kullback-Leibler Divergence), also called relative entropy (Relative Entropy). It measures the difference between two probability distributions in the same event space.

Specifically, assuming that the radius distribution vectors of the nodes in the two vessel trees are p1 and p2, the KL distances thereof are calculated according to formula (4):

Wherein, p1 and p2 are normalized vectors, p1 (x) and p2 (x) are radial distribution functions of nodes in two vessel trees respectively, that is, the number proportion of vessel segments with the radius of x is p1 (x) and p2 (x) respectively.

Fig. 13 shows a schematic representation of a lung lesion identified according to the identification rule. As shown in fig. 13 (a), KL distances between the distribution curves of the radii of the nodes in the corresponding two vessel trees of the left and right lungs are larger than the first distance threshold, the CT scan of each layer is correspondingly taken out, a large lesion as shown in fig. 13 (b) is found, and the lesion can be seen in many layers.

For the medical image which is an image of a left-right asymmetric organ, when the KL distance between the radius distribution curve of the node in the blood vessel tree and the standard radius curve is larger than a preset second distance threshold value, determining that a lesion exists in the region corresponding to the blood vessel tree.

The standard radius curve can be obtained, for example, by counting the distribution curve of the node radius of all normal populations in the normalized vessel tree.

The standardized method is obtained by carrying out the following transformation on (x, y, z) E Q _l:

x’＝(x-x_min)/(x_max-x_min)

y’＝(y-y_min)/(y_max-x_min)

z’＝(z-z_min)/(z_max-z_min)

ql represents the organ segmented region tensor of the CT image three-dimensional tensor Q, and x_min, x_max represent the minimum abscissa and the maximum abscissa of the non-0 voxel coordinates in Ql. The other coordinates are the same.

The first distance threshold and the second distance threshold can be set according to actual requirements when the first distance threshold and the second distance threshold are applied.

Fig. 14 is a flowchart illustrating yet another method for detecting a biological organ lesion according to an exemplary embodiment. Unlike the biological organ lesion detection method 20 shown in fig. 2, the biological organ lesion detection method 70 shown in fig. 14 further provides a method of performing lesion research on the pre-detected lesion area image based on the generated vascular tree.

Referring to fig. 14, a lesion recognition method 70 based on a vessel tree includes:

in step S202, a pre-detection lesion region image is acquired.

In step S204, a plurality of blood vessel regions are segmented from the pre-detection lesion region image.

In step S702, lesion research and judgment are performed on the region to be researched according to a predefined lesion research and judgment rule based on a plurality of blood vessel trees.

In some embodiments, the lesion development rule includes some or all of the following rules:

And when the difference value between the radius of the region to be determined and the radius of the node of the blood vessel tree where the region to be determined is greater than a preset first radius threshold value, determining the region to be determined as a lesion region.

The first radius threshold may be set to 1 to 5, for example, but the present invention is not limited thereto.

And when the node number of the blood vessel tree where the region to be determined is located is smaller than a preset first node number threshold value, determining the region to be determined as a lesion region.

And when the difference between the radius of the region to be determined and the radius of all adjacent nodes of the blood vessel tree where the region to be determined is located is larger than a preset second radius threshold value, determining the region to be determined as a lesion region.

And when the radius of the node of the blood vessel tree where the region to be determined is located is larger than a third radius threshold (for example, 0.5-2), and the node of the subtree of the node is smaller than a second node number threshold, determining the region to be determined as a lesion region.

It should be noted that, when the above thresholds are applied, they may be set according to actual requirements, and the present invention is not limited thereto.

Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments are implemented as a computer program executed by a CPU. When executed by a CPU, performs the functions defined by the above-described method provided by the present invention. The program may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic disk or an optical disk, etc.

Furthermore, it should be noted that the above-described figures are merely illustrative of the processes involved in the method according to the exemplary embodiment of the present invention, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

The following are examples of the apparatus of the present invention that may be used to perform the method embodiments of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.

Referring to fig. 15, the biological organ lesion detection device 100 includes: the image acquisition module 1002, the image recognition module 1004, the lesion recognition module 1006, the lesion pre-detection module 1008, the lesion research module 1010, and the lesion determination module 1012.

The image obtaining module 1002 is configured to obtain a medical image of an organ, where the medical image includes a plurality of voxels.

The image recognition module 1004 is configured to recognize an organ image from the medical image.

The lesion recognition module 1006 is configured to perform lesion recognition on the organ image, and obtain a first lesion region in the organ image.

The lesion pre-detection module 1008 is configured to perform lesion pre-detection on the organ image through a trained deep learning model, and obtain a pre-detected lesion region image in the organ image.

The lesion research module 1010 is configured to research and judge the pre-detected lesion area image, and obtain a second lesion area in the organ image.

The lesion determination module 1012 is configured to obtain a final lesion region of the organ image according to the first lesion region and the second lesion region.

In some embodiments, the biological organ lesion detection device 100 further comprises: and the interference removing module is used for removing the interference image in the organ image before the lesion recognition is carried out on the organ image or the lesion pre-detection is carried out on the organ image through a trained deep learning model, so as to obtain the organ image from which the interference image is removed.

In some embodiments, lesion recognition module 1006 includes: the system comprises an image acquisition unit, a region segmentation unit, a matrix establishment unit, a blood vessel segment acquisition unit, a blood vessel map construction unit, a blood vessel tree establishment unit and a lesion identification unit. The image acquisition unit is used for acquiring the organ image. The region segmentation unit is used for segmenting a plurality of blood vessel regions from the organ image, and the blood vessel regions comprise: blood vessels and voxels with similar characteristics to blood vessels. The matrix establishing unit is used for establishing an association matrix of the plurality of blood vessel areas according to the adjacent relation among the plurality of blood vessel areas. The blood vessel segment acquisition unit is used for respectively acquiring blood vessel segments and blood vessel bifurcation points of each blood vessel region. The blood vessel map construction unit is used for constructing the blood vessel map which takes the blood vessel segments and the blood vessel bifurcation points as nodes according to the incidence matrix and the connection relation among the blood vessel segments, the blood vessel bifurcation points and the connection relation among the blood vessel segments and the blood vessel bifurcation points which are respectively contained in each blood vessel region. The vessel tree establishing unit is used for establishing a plurality of corresponding vessel trees based on a plurality of connected subgraphs in the vessel graph. The lesion recognition unit is used for recognizing lesions of the organ images based on the plurality of vessel trees according to a predefined lesion rule. The nodes in the vessel tree are the vessel segments contained in the connected subgraph corresponding to the nodes.

In some embodiments, the lesion recognition rule includes some or all of the following rules: when the number of nodes in the blood vessel tree is smaller than a preset threshold value of the number of nodes, determining a region corresponding to the blood vessel tree as the first lesion region; when the radius difference value of the node in the blood vessel tree and the adjacent node is larger than a preset first radius threshold value, determining the area corresponding to the node as the first lesion area; when traversing each node of the vessel tree by adopting a breadth-first traversing strategy, determining a region corresponding to a node, of which the radius difference value of two nodes accessed before and after is outside a preset radius range, as the first lesion region; when the radius of a branch node of the blood vessel tree is larger than a preset second radius threshold value, determining a region corresponding to the branch node as the first lesion region; for the medical image which is an image of a bilateral symmetry organ, when the KL distance of the radius distribution curve of the node in the two largest vessel trees on the left and right sides of the organ is larger than a preset first distance threshold, determining the corresponding areas of the two vessel trees as the first lesion area; and when the KL distance between the radius distribution curve of the nodes in the blood vessel tree and the standard radius curve is larger than a preset second distance threshold, determining the region corresponding to the blood vessel tree as the first lesion region.

In some embodiments, the deep learning model is ResNet neural network model, and the biological organ lesion detection device 100 further includes: and the model training unit is used for inputting the training set which is manually marked with the lesion region into the ResNet neural network model for training by adopting an Adam algorithm so as to obtain the trained deep learning model.

In some embodiments, lesion development module 1010 includes: the device comprises an image acquisition unit, a region segmentation unit, a matrix establishment unit, a blood vessel segment acquisition unit, a blood vessel map construction unit, a blood vessel tree establishment unit and a lesion research and judgment unit. The image acquisition unit is used for acquiring the image of the pre-detection lesion area. The region segmentation unit is used for segmenting a plurality of blood vessel regions from the pre-detection lesion region image, and the blood vessel regions comprise: blood vessels and voxels with similar characteristics to blood vessels. The matrix establishing unit is used for establishing an association matrix of the plurality of blood vessel areas according to the adjacent relation among the plurality of blood vessel areas. The blood vessel segment acquisition unit is used for respectively acquiring blood vessel segments and blood vessel bifurcation points of each blood vessel region. The blood vessel map construction unit is used for constructing the blood vessel map which takes the blood vessel segments and the blood vessel bifurcation points as nodes according to the incidence matrix and the connection relation among the blood vessel segments, the blood vessel bifurcation points and the connection relation among the blood vessel segments and the blood vessel bifurcation points which are respectively contained in each blood vessel region. The vessel tree establishing unit is used for establishing a plurality of corresponding vessel trees based on a plurality of connected subgraphs in the vessel graph. The lesion research unit is used for conducting lesion research on the region to be researched according to a predefined lesion research rule based on the plurality of blood vessel trees. The nodes in the vessel tree are the vessel segments contained in the connected subgraph corresponding to the nodes.

In some embodiments, the lesion development rule includes some or all of the following rules: when the difference value between the radius of the region to be ground and the radius of the node of the blood vessel tree where the region to be ground is located is larger than a preset first radius threshold value, determining that the region to be ground is the second lesion region; when the node number of the blood vessel tree where the region to be determined is located is smaller than a preset first node number threshold value, determining the region to be determined as the second lesion region; when the difference between the radius of the region to be determined and the radius of all adjacent nodes of the blood vessel tree where the region to be determined is located is larger than a preset second radius threshold value, determining the region to be determined as the second lesion region; and when the radius of the node of the blood vessel tree where the region to be determined is located is larger than a third radius threshold and the node of the subtree of the node is smaller than a second node number threshold, determining the region to be determined as the second lesion region.

It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

Fig. 16 is a schematic diagram showing a structure of an electronic device according to an exemplary embodiment. It should be noted that the electronic device shown in fig. 16 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present invention.

As shown in fig. 16, the electronic device 800 is embodied in the form of a general-purpose computer device. The components of the electronic device 800 include: at least one Central Processing Unit (CPU) 801 that can perform various appropriate actions and processes according to program code stored in a Read Only Memory (ROM) 802 or program code loaded from at least one storage unit 808 into a Random Access Memory (RAM) 803.

In particular, according to an embodiment of the present invention, the program code may be executed by the central processing unit 801, such that the central processing unit 801 performs the steps according to various exemplary embodiments of the present invention described in the method embodiment section above in this specification. For example, the central processing unit 801 may perform the steps as shown in fig. 1 to 15.

In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

The following components are connected to the I/O interface 805: an input unit 806 including a keyboard, a mouse, and the like; an output unit 807 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage unit 808 including a hard disk or the like; and a communication unit 809 including a network interface card such as a LAN card, modem, or the like. The communication unit 809 performs communication processing via a network such as the internet. The drive 810 is also connected to the I/O interface 805 as needed. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 810, so that a computer program read out therefrom is installed into the storage unit 808 as needed.

Referring to fig. 17, a program product 900 according to an embodiment of the present invention configured to implement the above-described method is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable medium carries one or more programs, which when executed by one of the devices, cause the computer-readable medium to implement the functions as shown in fig. 1-15.

The exemplary embodiments of the present invention have been particularly shown and described above. It is to be understood that this invention is not limited to the precise arrangements, instrumentalities and instrumentalities described herein; on the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for detecting a biological organ lesion, comprising:

Acquiring a medical image of an organ, the medical image comprising a plurality of voxels;

Identifying an organ image from the medical image;

Performing lesion recognition on a blood vessel region acquired from the organ image to acquire a first lesion region in the organ image, wherein the first lesion region comprises: carrying out lesion recognition on the organ image according to the blood vessel tree in the blood vessel area and a predefined lesion rule to obtain a first lesion area;

performing lesion pre-detection on the organ image through a trained deep learning model to obtain a pre-detection lesion area image in the organ image;

Judging the pre-detected lesion area image to obtain a second lesion area in the organ image; and

And obtaining a final lesion region of the organ image according to the first lesion region and the second lesion region.

2. The method of claim 1, wherein prior to lesion recognition of the organ image or pre-lesion detection of the organ image by a trained deep learning model, the method further comprises: and removing the interference image in the organ image to obtain the organ image from which the interference image is removed.

3. The method of claim 2, wherein performing lesion recognition on the organ image to obtain a first lesion region in the organ image comprises:

acquiring the organ image;

Segmenting a plurality of blood vessel regions from the organ image, the blood vessel regions comprising: a blood vessel and voxels having similar characteristics to the blood vessel;

establishing an incidence matrix of the plurality of blood vessel areas according to the adjacent relation among the plurality of blood vessel areas;

Respectively acquiring a blood vessel section and a blood vessel bifurcation point of each blood vessel region;

According to the incidence matrix, constructing the blood vessel segments, the blood vessel bifurcation points and the connection relations among the blood vessel segments, the blood vessel bifurcation points which are respectively contained in each blood vessel region into a blood vessel graph taking the blood vessel segments and the blood vessel bifurcation points as nodes;

establishing a plurality of corresponding vessel trees based on a plurality of connected subgraphs in the vessel graph; and

Based on the plurality of vessel trees, performing lesion recognition on the organ images according to a predefined lesion rule;

the nodes in the vessel tree are the vessel segments contained in the connected subgraph corresponding to the nodes.

4. A method according to claim 3, wherein the lesion recognition rule comprises some or all of the following rules:

when the number of nodes in the blood vessel tree is smaller than a preset threshold value of the number of nodes, determining a region corresponding to the blood vessel tree as the first lesion region;

when the radius difference value of the node in the blood vessel tree and the adjacent node is larger than a preset first radius threshold value, determining the area corresponding to the node as the first lesion area;

when traversing each node of the vessel tree by adopting a breadth-first traversing strategy, determining a region corresponding to a node, of which the radius difference value of two nodes accessed before and after is outside a preset radius range, as the first lesion region;

When the radius of a branch node of the blood vessel tree is larger than a preset second radius threshold value, determining a region corresponding to the branch node as the first lesion region;

For the medical image which is an image of a bilateral symmetry organ, when the KL distance of the radius distribution curve of the node in the two largest vessel trees on the left and right sides of the organ is larger than a preset first distance threshold, determining the corresponding areas of the two vessel trees as the first lesion area;

And when the KL distance between the radius distribution curve of the nodes in the blood vessel tree and the standard radius curve is larger than a preset second distance threshold, determining the region corresponding to the blood vessel tree as the first lesion region.

5. The method of claim 2, wherein the deep learning model is a ResNet neural network model, the method further comprising: and (3) inputting the training set artificially marked with the lesion area into the ResNet neural network model for training by adopting an Adam algorithm to obtain the trained deep learning model.

6. The method of claim 2, wherein the studying the pre-detected lesion image to obtain the second lesion in the organ image comprises:

acquiring the pre-detection lesion area image;

Segmenting a plurality of blood vessel regions from the pre-detection lesion region image, wherein the blood vessel regions comprise: a blood vessel and voxels having similar characteristics to the blood vessel;

Based on the plurality of blood vessel trees, performing lesion research and judgment on a region to be researched and judged according to a predefined lesion research and judgment rule;

7. The method of claim 6, wherein the lesion development rule comprises some or all of the following rules:

When the difference value between the radius of the region to be ground and the radius of the node of the blood vessel tree where the region to be ground is located is larger than a preset first radius threshold value, determining that the region to be ground is the second lesion region;

When the node number of the blood vessel tree where the region to be determined is located is smaller than a preset first node number threshold value, determining the region to be determined as the second lesion region;

When the difference between the radius of the region to be determined and the radius of all adjacent nodes of the blood vessel tree where the region to be determined is located is larger than a preset second radius threshold value, determining the region to be determined as the second lesion region;

and when the radius of the node of the blood vessel tree where the region to be determined is located is larger than a third radius threshold and the node of the subtree of the node is smaller than a second node number threshold, determining the region to be determined as the second lesion region.

8. A biological organ lesion detection device, comprising:

The image acquisition module is used for acquiring a medical image of an organ, wherein the medical image comprises a plurality of voxels;

the image identification module is used for identifying an organ image from the medical image;

The lesion recognition module is used for recognizing lesions of the blood vessel region acquired in the organ image to acquire a first lesion region in the organ image, and the first lesion region comprises: carrying out lesion recognition on the organ image according to the blood vessel tree in the blood vessel area and a predefined lesion rule to obtain a first lesion area;

The lesion pre-detection module is used for pre-detecting the lesions of the organ images through a trained deep learning model to obtain pre-detected lesion area images in the organ images;

The lesion research and judgment module is used for researching and judging the pre-detected lesion area image to obtain a second lesion area in the organ image; and

And the lesion determining module is used for obtaining a final lesion area of the organ image according to the first lesion area and the second lesion area.

9. A computer device, comprising: memory, a processor and executable instructions stored in the memory and executable in the processor, wherein the processor implements the method of any of claims 1-7 when executing the executable instructions.

10. A computer readable storage medium having stored thereon computer executable instructions which when executed by a processor implement the method of any of claims 1-7.