CN115170452A - Condyle CBCT image data enhancement method based on spatial central axis - Google Patents

Abstract

A condyle CBCT image data enhancement method based on a space central axis comprises the steps of firstly dividing an original condyle CBCT image to obtain a sample set for model training; then, carrying out image preprocessing on the training sample set, and eliminating the interference of other soft tissues in the image to obtain a condyle interested region; and then, inputting the condylar interesting region and the manual labeling result into a data enhancement module, and finally obtaining the condylar CBCT image which can be used as an available sample through the processing of a central shaft calculation module and an image rotation processing module, so that the condylar CBCT image training sample set is subjected to data enhancement. The condyle segmentation method based on the CBCT image improves the accuracy of condyle segmentation by using the CBCT image training sample set of the data-enhanced condyle and the generalization capability of a model.

Description

Condyle CBCT image data enhancement method based on spatial central axis

Technical Field

The invention belongs to the field of medical image processing, and particularly relates to a condyle CBCT image data enhancement method based on a space central shaft.

Background

Temporomandibular joint disorder (TMD) is the most common disease of the oral and maxillofacial region and is a generic term for a group of diseases in which the pathogenesis is not fully understood and the temporomandibular joint (TMJ) is unable to function normally. And the incidence rate is high, and the disease is ranked in the fourth place in common oral diseases. Temporomandibular arthritis (TMJ-OA) is a subtype of TMD and can cause severe joint pain, dysfunction, malocclusions and health related decline in quality of life.

The condyloid process, also known as the condyloid process or articular process, is one of the major growth centers of the mandible. The structure of the condyle can be divided into two parts, namely a condylar head and a condylar neck, the upper end of the condyle expands to the condylar head, and the reduced part of the condyle lower part is called the condylar neck. An imaginary line passing through the center of the condyle in the mandibular direction is called the central axis of the condyle; the longest diameter on the condylar head is referred to as the major axis of the condylar head, and the shortest diameter is referred to as the minor axis of the condylar head. Changes in the morphology of the condyles were found in TMJ-OA patients, including thinning of cortical bone resorption, thickening of trabecular bone, cystoid formation between trabecular bones, and the like. Therefore, TMJ-OA can be diagnosed by analyzing the change in the structure of the condyles. In addition, cone Beam Computed Tomography (CBCT) is a well-established imaging method for head and neck regions, and is recommended as one of the most reliable methods for diagnosing TMJ-OA, since it can display normal tissue structures and lesion tissues in three dimensions (axial, coronal, and sagittal), avoiding the disadvantages of overlapping images on two-dimensional images, and the integrity of the condylar structures can be observed by imaging.

In recent years, deep learning techniques have made many breakthroughs in computer vision applications, including segmentation of medical images. This success prompted researchers to plan the use of deep learning models to identify the condyles in CBCT images and to diagnose TMJ-OA. However, compared with other computer vision tasks, the medical image acquisition and labeling cost is high, and since the CBCT image is three-dimensional, a doctor needs to trace the condylar contour piece by piece for labeling during labeling, which is more tedious and time-consuming than the labeling work of a two-dimensional image. Therefore, the original CBCT image of the condyle needs to be processed to expand the number of available samples, so as to reduce the labeling time of the doctor and improve the working efficiency.

However, at present, there are few studies for studying condylar segmentation by deep learning, so that the condylar CBCT image is usually processed by a general data enhancement method. In the existing general data enhancement methods, for example: geometric transformations such as rotation, inversion and cropping are not completely applicable to CBCT image data of the condyles, because unlike two-dimensional images, the spatial position relationship of the condyles in the CBCT image is considered in three-dimensional images such as CBCT. Therefore, the augmented data training model obtained by the data enhancement method of geometric transformation may result in the model being less effective than the model trained before augmentation. In addition, the relative spatial positions of the condyles of different people are different, and if fixed processing parameters are adopted, data such as an interested area without the condyles in the image may be generated, and the data cannot improve the segmentation effect of the condylar segmentation model. Because the area of interest of the condyle in the CBCT image of the condyle is small, the condyle segmentation task is interfered by adopting a data enhancement mode of pixel transformation such as salt and pepper noise, gaussian noise and the like. Therefore, the current data enhancement method applicable to the Condyle CBCT image has defects.

Disclosure of Invention

In order to overcome the defects of the prior art, the invention provides a method for enhancing the condyle CBCT image data based on a spatial central axis, which can perform geometric transformation on the condyle CBCT image by finding the central axis of the condyle under the condition of less original data quantity, thereby generating data which can be used as an available sample, so that the CBCT image training sample set of the condyle is enhanced by the data, and further the accuracy of performing condyle segmentation by using the CBCT image training sample set of the condyle after data enhancement and the generalization capability of a model are improved.

The technical scheme adopted by the invention for solving the technical problem is as follows:

a condyle CBCT image data enhancement method based on a spatial central axis comprises the following steps:

s1, taking the Condyle CBCT image as a training set and an artificial labeling result D _train1 Inputting the data into a central shaft calculation module to obtain D _train1 Central axes of all condyles in the cluster form a central axis set L _center ；

S2, training set D _train1 And a central axis set L _center Inputting the data into a rotating module to obtain a processed data set D _train2 ；

S3, mixing D _train2 Inputting the data into a long shaft calculation module to obtain D _train2 All the long axes form a long axis set L _long ；

S4, training set D _train2 And major axis set L _long Inputting the data into a rotating module to obtain a data set D finally used for training the condylar segmentation model _train3 。

Further, in the step S1, D is added _train1 Inputting the data into a central shaft calculation module to obtain D _train1 The central axes of all condyles in the condyle form a central axis set L _center The process is as follows:

step 1.1, the size is D ₁ ×H ₁ ×W ₁ Manual annotation of results M ¹ (M ¹ ∈D _train1 ) Along the Z-axis direction to obtain D ₁ Sheet size of D ₁ ×H ₁ ×W ₁ The slice of the result is marked manually to obtain a slice set S ¹ ；

Step 1.2, slice set S ¹ Subjecting each slice to image graying and binarization to obtain D1 binarized images, and forming a new slice set S ² ；

Step 1.3, according to the slice set S ² Obtaining a point coordinate set P of the contour of the manually marked area in all the slices by means of a contour detection algorithm in the cv2 library ¹ ；

Step 1.4, slicing s according to each artificial labeling result _1，i (s _1，i ∈S ² ，i∈[0，D ₁ -1]) Set of contour point coordinates p in (1) _1，i (p _1，i ∈P ¹ ，i∈[0，D ₁ -1]) The corresponding geometric center is calculated, and the geometric center coordinates of the contour can be based on the slice s _1，i Calculating the zero order moment and the first order moment;

section s _1，i The formula for calculating the (p + q) order moment of (a) is as follows:

wherein p is _{1i_j} (x, y) denotes slice s _1i Set of contour point coordinates p _1，i One point coordinate (p) of _{1，i_j} ∈p _1，i ，i∈[0，D ₁ -1]J =0,1,2,. Cndot.), X, Y denote the contour point coordinate set p, respectively _1，i Coordinate values of all x-axis and y-axis;

section s _1i Geometric center p of the profile _c，i (c _x ，c _y ) The coordinate calculation formula of (c) is as follows:

c _x ＝m ₁₀ /m ₀₀

c _y ＝m ₀₁ /m ₀₀

wherein m is ₀₀ Is a binarized slice s _1，i Represents the sum of all white areas, i.e. pixel values, in the slice, 1, m ₁₀ Is a binarized slice s _1，i Represents the sum of the x-coordinates of all white area pixels in the slice, m ₀₁ Is a binarized slice s _1，i Represents the sum of the y-coordinates of all white area pixels in the slice;

step 1.5, slice set S ² All the slices are subjected to the operation of the step 1.4 to obtain the geometric center of the outline of the artificial labeling result in each slice, and finally a geometric center set P is obtained ² (p _c，i ∈P ² ，i∈[0，D ₁ -1])；

Step 1.6, according to the obtained geometric center set P ² Fitting a space straight line, wherein the expression of a standard equation of the space straight line is as follows:

converting the formula (1) to obtain a new expression as follows:

wherein m, n, k, x ₀ ，y ₀ ，z ₀ Are all constants, so equation (2) can be simplified to obtain a new expression as follows:

wherein k is ₁ ，k ₂ ，b ₁ ，b ₂ Represents a constant;

the constant k can be obtained by using the least square method fitting principle ₁ ，k ₂ ，b ₁ ，b ₂ Wherein the least squares fitting principle is as follows: fitting the spatial straight line expression of equation (3) and minimizing the sum of the squares of the residuals, wherein the equation for the sum of the squares of the residuals is as follows:

Q ₁ ＝Σ(x _i -(k ₁ ·z _i +b ₁ )) ²

Q ₂ ＝Σ(y _i -(k ₂ ·z _i +b ₂ )) ²

wherein z is _i Representation of slices s _1，i Marking the result M manually ¹ The coordinate value, x, on the z-axis corresponding to (1) _i 、y _i Respectively represent slices s _1，i Geometric center p _c，i The coordinate values corresponding to the x-axis and the y-axis of the optical disk;

when Q is ₁ 、Q ₂ When the minimum, i.e. the sum of squared residuals is minimum, the constant k can be obtained ₁ ，k ₂ ，b ₁ ，b ₂ Because of Q ₁ 、Q ₂ Is about k ₁ ，k ₂ ，b ₁ ，b ₂ Function of (2) when Q ₁ 、Q ₂ Is 0, the corresponding minimum value is reached, so that k is respectively assigned ₁ ，k ₂ ，b ₁ ，b ₂ The derivation can be derived as follows:

thereby obtaining k ₁ ，k ₂ ，b ₁ ，b ₂ The expression of (c) is as follows:

traverse the set of geometric centers P ² All geometric centers in k ₁ ，k ₂ ，b ₁ ，b ₂ In the expression (2), a constant k is obtained ₁ ，k ₂ ，b ₁ ，b ₂ Finally, the space straight line expression of the fitting formula (3) can be obtained, and the space straight line obtained by fitting is M ¹ Central axis of (l) _center，i ；

Step 1.7, training set D _train1 All the manual labeling results in the step (1.1) to the step (1.6) are carried out, and finally a central shaft data set L is obtained _center (l _center，i ∈L _center ，i∈[0，D ₁ -1])。

Still further, in step S2, training set D is used _train1 And a central axis set L _center Inputting the data into a rotating module to obtain processed data D _train2 The process is as follows:

step 2.1, central shaft l obtained according to central shaft calculation module _center，i (l _center，i ∈L _center I =0,1, 2. -) in size D ₁ ×H ₁ ×W ₁ (ii) manually annotating the results M ¹ The central axis l is calculated _center，i The included angle between the condylar CBCT image I and the z axis is determined according to the included angle ¹ And manually labeling the result M ¹ (I ¹ ∈D _train1 ，M ¹ ∈D _train1 ) Rotate so that the central axis l of the condyles _center，i Rotating to z axis, and performing resampling to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of the condyles I ² And manually labeling the result M ² ；

Step 2.2, training set D _train1 Performing the operation of the step 2.1 on all the Condyle CBCT images and the manual labeling result to finally obtain processed data D _train2 (I ² ∈D _train2 ，M ² ∈D _train2 )。

Further, in step S3, D is set _train2 Inputting the data into a long shaft calculation module to obtain D _train2 All the long axes form a long axis set L _long The process is as follows:

step 3.1, give a threshold T ¹ Marking the result M manually ² (M ² ∈D _train2 ) Z-axis coordinate value of less than T ¹ Is discarded, thereby obtaining a region of interest I containing only the condylar-process ^h _ROI ；

Step 3.2, the interested area I of the condylar prominence ^h _ROI Along the Z-axis to obtain D ₁ Sheet size of D ₁ ×H ₁ ×W ₁ To obtain a slice set S ³ ；

Step 3.3, set S of slices ³ Subjecting each slice to image graying and binarization processing to obtain D1 binarized images, and forming a new slice set S ⁴ ；

Step 3.4, according to the slice set S ⁴ Obtaining a point coordinate set P of the contour of the region of interest of the condylar head in all the slices by means of a contour detection algorithm in the cv2 library ³ ；

Step 3.5, slicing s according to the region of interest of each condylar eminence _2，i (s _2，i ∈S ⁴ ，i∈[0，D ₁ -1]) Set of contour point coordinates p in _2，i (p _2，i ∈P ³ ，i∈[0，D ₁ -1]) Recording the coordinates of the two points of the contour having the longest line segment, i.e.Recording the coordinates of the long axis of the condylar process to obtain D ₁ A line segment set L consisting of line segments;

and 3.6, fitting a plane straight line according to the obtained line segment set L, wherein the general expression of the plane straight line equation is as follows:

y＝k ₃ ·x+b ₃ (4)

obtaining a constant k according to the principle of least square method ₃ ，b ₃ Wherein the least squares fitting principle is as follows: fitting the planar straight-line expression of equation (4) and minimizing the sum of squared residuals, wherein the equation for the sum of squared residuals is as follows:

Q ₃ ＝Σ(k ₃ ·x _i +b ₃ -y _i ) ²

wherein x is _i 、y _i Respectively represent slices s _2，i Coordinate values of the x-axis and the y-axis of the long axis of the epicondyle,

when Q is ₃ When the minimum is the sum of the squares of the residual errors, the constant k is obtained ₃ ，b ₃ Because of Q ₃ Is about k ₃ ，b ₃ Function of (2) when Q ₃ Is 0, the corresponding minimum value is taken, so, for k respectively ₃ ，b ₃ The derivation of the partial derivatives can be found as follows:

thereby obtaining k ₃ ，b ₃ The expression of (a) is as follows:

the arithmetic mean value expressed therein is calculated as follows:

traverse all the lengths in the line segment set LShaft, lead-in k ₃ ，b ₃ In the expression (c), a constant k is obtained ₃ ，b ₃ Finally, the plane straight line expression of the fitting formula (4) can be obtained, and the plane straight line obtained by fitting is M ² The long axis of the condylar-process head of (a);

step 3.7, training set D _train2 In (3.1E) all the manual labeling results are processed in step

Operation of step 3.6 to obtain D _train2 Medial condylar-procephalic long-axis dataset L _long 。

In the step S4, a training set D is set _train2 And major axis set L _long Inputting the data into a rotating module to obtain a data set D finally used for training the condylar segmentation model _train3 The process is as follows:

step 4.1, calculating the module according to the long axis of the condylar head to obtain the long axis l of the condylar head _i (l _i ∈L _long I =0,1,2. -) in size D ₁ ×H ₁ ×W ₁ (ii) manually annotating the results M ² Calculating the included angle between the long axis and the x axis, and taking the Condyle CBCT image I as the reference ² And manually labeling the result M ² (I ² ∈D _train1 ，M ² ∈D _train1 ) Rotate so that the long axis l of the condyles _i Rotating to x axis, and performing resampling to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of condyles I ³ And manually labeling the result M ³ ；

Step 4.2, training set D _train2 All the CBCT images of the condyles and the manual labeling result in the step 4.1 are operated, and processed data D are obtained finally _train3 (I ³ ∈D _train3 ，M ³ ∈D _train3 )。

According to the method for enhancing the Condyle CBCT image data based on the spatial central axis, firstly, data used for Condyle segmentation model training are obtained through data division. Then, the interference of other soft tissues is eliminated through data preprocessing. And finally, performing data enhancement through a central shaft calculation module and an image rotation processing module to finally obtain a CBCT image training sample set of the condyles after data enhancement.

The invention has the following beneficial effects: the accuracy of condylar process segmentation performed by using the CBCT image training sample set of the data-enhanced posterior condylar process and the generalization capability of the model are improved.

Drawings

FIG. 1 is a flowchart of a method for enhancing Condyle CBCT image data based on a spatial central axis according to the present application;

FIG. 2 is a schematic diagram of a module related to the method for enhancing CBCT image data of condyles based on a spatial central axis according to the present application;

fig. 3 is a comparison graph of the original condylar data of the present application and the data after data enhancement on the same spatial position, wherein (a) is the original condylar data, and (b) is the image obtained by the method of the present application.

Detailed Description

The invention is further described below with reference to the accompanying drawings.

Referring to fig. 1 to 3, a method for enhancing CBCT image data of condyles based on a spatial central axis includes the following steps:

s1, taking the Condyle CBCT image as a training set and manually marking a result D _train1 Inputting the data into a central shaft calculation module to obtain D _train1 Central axes of all condyles in the cluster form a central axis set L _center 。

This application training set D _train1 The method is obtained by dividing an original Condyle CBCT image and an artificial labeling result according to a certain proportion, including setting the proportion of the data quantity in a training set, a verification set and a test set, and simultaneously carrying out D-pair analysis _train1 Has been pre-processed, including the operations of image cropping and image normalization, as shown in fig. 2, by D _train1 Inputting the data into a central shaft calculation module to obtain D _train1 The central axes of all condyles in the condyle form a central axis set L _center The process is as follows:

step 1.1, the size is D ₁ ×H ₁ ×W ₁ Manual annotation of results M ¹ (M ¹ ∈D _train1 ) Along the Z-axis direction to obtain D ₁ Sheet size of D ₁ ×H ₁ ×W ₁ The slice of the result is marked manually to obtain a slice set S ¹ ；

Step 1.2, slice set S ¹ Subjecting each slice to image graying and binarization processing to obtain D1 binarized images, and forming a new slice set S ² ；

Step 1.3, according to the slice set S ² Obtaining a point coordinate set P of the contour of the manually marked area in all the slices by means of a contour detection algorithm in the cv2 library ¹ ；

Step 1.4, slicing s according to each artificial labeling result _1，i (s _1，i ∈S ² ，i∈[0，D ₁ -1]) Set of contour point coordinates p in _1，i (p _1，i ∈P ¹ ，i∈[0，D ₁ -1]) The corresponding geometric center is calculated, and the geometric center coordinates of the contour can be based on the slice s _1，i The zero order moment and the first order moment are obtained by calculation;

section s _1，i The formula for calculating the (p + q) order moment of (a) is as follows:

wherein p is _{1i_j} (x, y) denotes the slice s _1i Set of contour point coordinates p _1，i One point coordinate (p) of _{1，i_j} ∈p _1，i ，i∈[0，D ₁ -1]J =0,1,2,. Cndot.), X, Y denote the contour point coordinate set p, respectively _1，i Coordinate values of all x-axis and y-axis;

section s _1i Geometric center p of the profile _c，i (c _x ，c _y ) The coordinate calculation formula of (c) is as follows:

c _x ＝m ₁₀ /m ₀₀

c _y ＝m ₀₁ /m ₀₀

wherein m is ₀₀ Is a binarized slice s _1，i OfOrder moment, representing the sum of all white areas in the slice, i.e. pixel values of 1, m ₁₀ Is a binarized slice s _1，i Represents the sum of the x-coordinates of all white area pixels in the slice. m is a unit of ₀₁ Is a binarized slice s _1，i Represents the sum of the y-coordinates of all white area pixels in the slice;

step 1.5, slice set S ² All the slices are subjected to the operation of the step 1.4 to obtain the geometric center of the outline of the artificial labeling result in each slice, and finally a geometric center set P is obtained ² (p _c，i ∈P ² ，i∈[0，D ₁ -1])；

Step 1.6, according to the obtained geometric center set P ² Fitting a space straight line, wherein the expression of a standard equation of the space straight line is as follows:

converting the formula (1) to obtain a new expression as follows:

wherein m, n, k, x ₀ ，y ₀ ，z ₀ All are constants, so equation (2) can be simplified to obtain a new expression as follows:

wherein k is ₁ ，k ₂ ，b ₁ ，b ₂ Represents a constant;

the constant k can be obtained by using the least square method fitting principle ₁ ，k ₂ ，b ₁ ，b ₂ The optimum value of (2). The least square method fitting principle is as follows: fitting the spatial straight line expression of equation (3) and minimizing the sum of the squares of the residuals, where the residuals are flatThe formula for the sum of squares is as follows:

Q ₁ ＝Σ(x _i -(k ₁ ·z _i +b ₁ )) ²

Q ₂ ＝Σ(y _i -(k ₂ ·z _i +b ₂ )) ²

wherein z is _i Representation of the slice s _1，i Marking the result M manually ¹ The coordinate value on the corresponding z-axis. x is the number of _i 、y _i Respectively represent slices s _1，i Geometric center p _c，i The coordinate values corresponding to the x axis and the y axis of (2);

when Q is ₁ 、Q ₂ When the minimum, i.e. the sum of squared residuals is minimum, the constant k can be obtained ₁ ，k ₂ ，b ₁ ，b ₂ Because of Q ₁ 、Q ₂ Is about k ₁ ，k ₂ ，b ₁ ，b ₂ Function of (2) when Q ₁ 、Q ₂ Is 0, the corresponding minimum value is reached, so that k is respectively assigned ₁ ，k ₂ ，b ₁ ，b ₂ The derivation can be derived as follows:

thereby obtaining k ₁ ，k ₂ ，b ₁ ，b ₂ The expression of (c) is as follows:

traverse the geometric center set P ² All geometric centers in k ₁ ，k ₂ ，b ₁ ，b ₂ In the expression (2), a constant k is obtained ₁ ，k ₂ ，b ₁ ，b ₂ The optimum value of (a) is set,finally, a space straight line expression of the fitting formula (3) can be obtained, and the space straight line obtained by fitting is M ¹ Central axis of (l) _center，i ；

Step 1.7, training set D _train1 In (1.1E) all the manual labeling results

Step 1.6, finally obtaining a central axis data set L _center (l _center，i ∈L _center ，i∈[0，D ₁ -1])；

Step S2, training set D _train1 And a central axis set L _center Inputting the data into a rotating module to obtain a processed data set D _train2 。

As shown in FIG. 2, the present application describes a training set D _train1 And a central axis set L _center Input to the rotation module, the process is as follows:

step 2.1, calculating the central axis l obtained by the module according to the central axis _center，i (l _center，i ∈L _center I =0,1,2. -) in size D ₁ ×H ₁ ×W ₁ Manual annotation of results M ¹ The central axis l is calculated _center，i Angle to the z-axis. According to the included angle, the Condyle CBCT image I is obtained ¹ And manually labeling the result M ¹ (I ¹ ∈D _train1 ，M ¹ ∈D _train1 ) Rotate so that the central axis l of the condyles _center，i Rotating to z axis, and performing resampling operation to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of the condyles I ² And manually labeling the result M ² ；

Step 2.2, training set D _train1 Performing the operation of the step 2.1 on all the Condyle CBCT images and the manual labeling result to finally obtain processed data D _train2 (I ² ∈D _train2 ，M ² ∈D _train2 )；

Step S3, adding D _train2 Inputting the data into a long axis calculation module to obtain D _train2 All the long axes in the set form a long axis set L _long 。

As shown in fig. 2, the long axis calculation module described in the present application performs the following process:

step 3.1, a threshold T is given ¹ Marking the result M by manpower ² (M ² ∈D _train2 ) Z-axis coordinate value of less than T ¹ So as to obtain a region of interest I containing only the condylar-head ^h _ROI ；

Step 3.2, the interested area I of the condylar prominence ^h _ROI Along the Z-axis direction to obtain D ₁ Sheet size of D ₁ ×H ₁ ×W ₁ To obtain a slice set S ³ ；

Step 3.3, slice set S ³ Subjecting each slice to image graying and binarization to obtain D1 binarized images, and forming a new slice set S ⁴ ；

Step 3.4, according to slice set S ⁴ Obtaining a point coordinate set P of the contour of the region of interest of the condylar head in all the slices by means of a contour detection algorithm in the cv2 library ³ ；

Step 3.5, slicing s according to the region of interest of each condylar eminence _2，i (s _2，i ∈S ⁴ ，i∈[0，D ₁ -1]) Set of contour point coordinates p in (1) _2，i (p _2，i ∈P ³ ，i∈[0，D ₁ -1]) Recording coordinates of two points with the longest line segment in the contour, namely recording coordinates of the long axis of the condylar process, and finally obtaining D ₁ A line segment set L consisting of line segments;

and 3.6, fitting a plane straight line according to the obtained line segment set L, wherein the general expression of the plane straight line equation is as follows:

y＝k ₃ ·x+b ₃ (4)

obtaining a constant k according to the principle of least square method ₃ ，b ₃ Wherein the least squares fitting principle is as follows: fitting the planar straight-line expression of equation (4) and minimizing the sum of squared residuals, wherein the equation for the sum of squared residuals is as follows:

Q ₃ ＝Σ(k ₃ ·x _i +b ₃ -y _i ) ²

wherein x _i 、y _i Respectively represent slices s _2，i Coordinate values of the x-axis and the y-axis of the epicondylium major axis;

when Q is ₃ When the minimum, i.e. the sum of squared residuals is minimum, the constant k can be obtained ₃ ，b ₃ Because of Q ₃ Is about k ₃ ，b ₃ Function of (2) when Q ₃ Is 0, the corresponding minimum value is reached, so that k is respectively assigned ₃ ，b ₃ The derivation can be derived as follows:

thereby obtaining k ₃ ，b ₃ The expression of (a) is as follows:

the arithmetic mean value expressed therein is calculated as follows:

traversing all long axes in the line segment set L, and substituting k ₃ ，b ₃ In the expression (2), a constant k is obtained ₃ ，b ₃ Finally, the plane straight line expression of the fitting formula (4) can be obtained, and the plane straight line obtained by fitting is M ² The long axis of the condylar-process head of (a);

step 3.7, training set D _train2 In (3.1E) all the manual labeling results are processed in step

Step 3.6 operation, D is finally obtained _train2 Long axis dataset L of the medial condylar process _long ；

Step S4, training set D _train2 And major axis set L _long Inputting into a rotating module to obtain final productData set D for training condylar segmentation model _train3 。

As shown in FIG. 2, the training set D is described in the present application _train2 And major axis set L _long Input to the rotation module, the process is as follows:

step 4.1, according to the long shaft l of the condylar head obtained by the long shaft calculation module _i (l _i ∈L _long I =0,1, 2. -) in size D ₁ ×H ₁ ×W ₁ Manual annotation of results M ² Calculating the included angle between the long axis and the x axis, and obtaining the Condyle CBCT image I according to the included angle ² And manually labeling the result M ² (I ² ∈D _train1 ，M ² ∈D _train1 ) Rotate so that the long axis l of the condyles _i Rotating to x axis, and performing resampling to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of the condyles I ³ And manually labeling the result M ³ ；

Step 4.2, training set D _train2 Performing the operation of the step 4.1 on all the Condyle CBCT images and the manual labeling result to finally obtain processed data D _train3 (I ³ ∈D _train3 ，M ³ ∈D _train3 )。

It should be noted that a comparative image of a sample of the data after data enhancement and the original data of the condyle at the same spatial position, i.e., the same x, y, and z-axis coordinates, is shown in fig. 3, where (a) is the original data of the condyle, and (b) is the image obtained by the method provided by the present application. As shown in (b), under the condition that the spatial position is the same, the condyle information of each axial surface in (b) is more abundant, so that the accuracy of the condyle segmentation model is improved.

In the present application, D is the depth of the picture, H is the height of the picture, W is the width of the picture, and the subscripts of the letters indicate numbers for distinguishing the dimensions of different feature maps.

The central shaft of the condylar process is calculated through the central shaft calculating module. And finding the optimal axial surface of the condyle by combining the central axis of the condyle through the image rotation processing module. The processed data is expanded to a training sample set, so that the accuracy of condyle segmentation performed by using the CBCT image training sample set of the data-enhanced posterior condyles and the generalization capability of the model are improved.

The embodiments described in this specification are merely exemplary of implementations of the inventive concepts and are provided for illustrative purposes only. The scope of the present invention should not be construed as being limited to the particular forms set forth in the examples, but rather as being defined by the claims and the equivalents thereof which can occur to those skilled in the art upon consideration of the present inventive concept.

Claims (5)

1. A method for enhancing Condyle CBCT image data based on a spatial central axis is characterized by comprising the following steps:

s1, taking the Condyle CBCT image as a training set and manually marking a result D _train1 Inputting the data into a central shaft calculation module to obtain D _train1 Central axes of all condyles in the cluster form a central axis set L _center ；

S2, training set D _train1 And a central axis set L _center Inputting the data into a rotating module to obtain a processed data set D _train2 ；

S3, mixing D _train2 Inputting the data into a long shaft calculation module to obtain D _train2 All the long axes in the set form a long axis set L _long ；

S4, training set D _train2 And major axis set L _long Inputting the data into a rotating module to obtain a data set D finally used for training the condylar segmentation model _train3 。

2. The method for enhancing spatial centric axis-based CBCT image data of condyles as claimed in claim 1, wherein in said step S1, D is determined _train1 Inputting the data into a central shaft calculation module to obtain D _train1 Central axes of all condyles in the cluster form a central axis set L _center The process is as follows:

step 1.1, the size is D ₁ ×H ₁ ×W ₁ Manual annotation of results M ¹ (M ¹ ∈D _train1 ) Along the Z-axis direction to obtain D ₁ The size of the sheet is D ₁ ×H ₁ ×W ₁ The slice of the result is marked manually to obtain a slice set S ¹ ；

Step 1.2, slice set S ¹ Subjecting each slice to image graying and binarization processing to obtain D1 binarized images, and forming a new slice set S ² ；

Step 1.3, according to slice set S ² Obtaining a point coordinate set P of the contour of the manually marked area in all the slices by means of a contour detection algorithm in the cv2 library ¹ ；

Step 1.4, slicing s according to each artificial labeling result _1，i (s _1，i ∈S ² ，i∈[0，D ₁ -1]) Set of contour point coordinates p in _1，i (p _1，i ∈P ¹ ，i∈[0，D ₁ -1]) The corresponding geometric center is calculated, and the geometric center coordinates of the contour can be based on the slice s _1，i The zero order moment and the first order moment are obtained by calculation;

section s _1，i The (p + q) order moment of (c) is calculated as follows:

wherein p is _{1i_j} (x, y) denotes slice s _1i Set of contour point coordinates p _1，i One point coordinate (p) of _{1，i_j} ∈p _1，i ，i∈[0，D ₁ -1]J =0,1, 2.)), X, Y represent the contour point coordinate set p, respectively _1，i Coordinate values of all x-axis and y-axis;

section s _1i Geometric center p of the profile _c，i (c _x ，c _y ) The coordinate calculation formula of (c) is as follows:

c _x ＝m ₁₀ /m ₀₀

c _y ＝m ₀₁ /m ₀₀

wherein m is ₀₀ Is after binarizationSection(s) of _1，i Zero order moment of (b), representing the sum of all white areas in the slice, i.e. pixel values of 1, m ₁₀ Is the binarized slice s _1，i Represents the sum of the x-coordinates of all white area pixels in the slice, m ₀₁ Is a binarized slice s _1，i Represents the sum of the y-coordinates of all white area pixels in the slice;

step 1.5, slice set S ² All the slices are subjected to the operation of the step 1.4 to obtain the geometric center of the outline of the artificial labeling result in each slice, and finally a geometric center set P is obtained ² (p _c，i ∈P ² ，i∈[0，D ₁ -1])；

Step 1.6, according to the obtained geometric center set P ² Fitting a space straight line, wherein the expression of a standard equation of the space straight line is as follows:

converting the formula (1) to obtain a new expression as follows:

wherein m, n, k, x ₀ ，y ₀ ，z ₀ Are all constants, so equation (2) can be simplified to obtain a new expression as follows:

wherein k is ₁ ，k ₂ ，b ₁ ，b ₂ Represents a constant;

the constant k can be obtained by using the least square fitting principle ₁ ，k ₂ ，b ₁ ，b ₂ Wherein the least squares fitting principle is as follows: fitting equation (3)A spatial straight line expression, and minimizes the sum of the squares of the residuals, wherein the formula of the sum of the squares of the residuals is as follows:

Q ₁ ＝Σ(x _i -(k ₁ ·z _i +b ₁ )) ²

Q ₂ ＝Σ(y _i -(k ₂ ·z _i +b ₂ )) ²

wherein z is _i Representation of the slice s _1，i Marking the result M manually ¹ The coordinate value, x, on the z-axis corresponding to (1) _i 、y _i Respectively represent slices s _1，i Geometric center p _c，i The coordinate values corresponding to the x axis and the y axis of (2);

when Q is ₁ 、Q ₂ When the minimum, i.e. the sum of squared residuals is minimum, the constant k can be obtained ₁ ，k ₂ ，b ₁ ，b ₂ Because of Q ₁ 、Q ₂ Is about k ₁ ，k ₂ ，b ₁ ，b ₂ Function of (2) when Q ₁ 、Q ₂ Is 0, the corresponding minimum value is reached, so that k is respectively assigned ₁ ，k ₂ ，b ₁ ，b ₂ The derivation can be derived as follows:

thereby obtaining k ₁ ，k ₂ ，b ₁ ，b ₂ The expression of (c) is as follows:

traverse the geometric center set P ² All geometric centers in, into k ₁ ，k ₂ ，b ₁ ，b ₂ In the expression (2), a constant k is obtained ₁ ，k ₂ ，b ₁ ，b ₂ Finally, the space straight line expression of the fitting formula (3) can be obtained, and the space straight line obtained by fitting is M ¹ Central axis of (l) _center，i ；

Step 1.7, training set D _train1 All the manual labeling results in the step (1.1) to the step (1.6) are operated, and finally a central shaft data set L is obtained _center (l _center，i ∈L _center ，i∈[0，D ₁ -1])。

3. The method for spatial central axis-based CBCT image data enhancement of condyles as claimed in claim 1 or 2, wherein in said step S2, a training set D is set _train1 And a central axis set L _center Inputting the data into a rotating module to obtain processed data D _train2 The process is as follows:

step 2.1, central shaft l obtained according to central shaft calculation module _center，i (l _center，i ∈L _center I =0,1, 2. -) in size D ₁ ×H ₁ ×W ₁ Manual annotation of results M ¹ The central axis l is calculated _center，i The included angle between the Condyle CBCT image I and the z axis is determined according to the included angle ¹ And manually labeling the result M ¹ (I ¹ ∈D _train1 ，M ¹ ∈D _train1 ) Rotating so that the central axis l of the condyle _center，i Rotating to z axis, and performing resampling to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of the condyles I ² And manually labeling the result M ² ；

Step 2.2, training set D _train1 Performing the operation of the step 2.1 on all the Condyle CBCT images and the manual labeling result to finally obtain processed data D _train2 (I ² ∈D _train2 ，M ² ∈D _train2 )。

4. The method according to claim 1 or 2The CBCT image data enhancement method for condyle based on spatial central axis is characterized in that in step S3, D is calculated _train2 Inputting the data into a long shaft calculation module to obtain D _train2 All the long axes in the set form a long axis set L _long The process is as follows:

step 3.1, a threshold T is given ¹ Marking the result M manually ² (M ² ∈D _train2 ) Z-axis coordinate value of less than T ¹ So as to obtain a region of interest I containing only the condylar-head ^h _ROI ；

Step 3.2, the interested area I of the condylar prominence ^h _ROI Along the Z-axis to obtain D ₁ The size of the sheet is D ₁ ×H ₁ ×W ₁ To obtain a slice set S ³ ；

Step 3.3, slice set S ³ Carrying out image graying and binarization processing on each slice to obtain D ₁ The image after binarization is opened to form a new slice set S ⁴ ；

Step 3.4, according to slice set S ⁴ Obtaining a point coordinate set P of the contour of the region of interest of the condylar head in all the slices by means of a contour detection algorithm in the cv2 library ³ ；

Step 3.5 slicing s according to the region of interest of each condylar eminence _2，i (s _2，i ∈S ⁴ ，i∈[0，D ₁ -1]) Set of contour point coordinates p in _2，i (p _2，i ∈P ³ ，i∈[0，D ₁ -1]) Recording coordinates of two points with the longest line segment in the contour, namely recording coordinates of the long axis of the condylar process, and finally obtaining D ₁ A line segment set L consisting of line segments;

and 3.6, fitting a plane straight line according to the obtained line segment set L, wherein the general expression of the plane straight line equation is as follows:

y＝k ₃ ·x+b ₃ (4)

obtaining a constant k according to the principle of least square method ₃ ，b ₃ Wherein the least squares fitting principle is as follows: plane straight line table of fitting formula (4)And minimizing the sum of squared residuals, wherein the sum of squared residuals is given by:

Q ₃ ＝Σ(k ₃ ·x _i +b ₃ -y _i ) ²

wherein x is _i 、y _i Respectively represent slices s _2，i Coordinate values of the x-axis and the y-axis of the long axis of the epicondyle,

when Q is ₃ When the minimum is the sum of the squares of the residual errors, the constant k is obtained ₃ ，b ₃ Because of Q ₃ Is about k ₃ ，b ₃ Function of (2) when Q ₃ Is 0, the corresponding minimum value is taken, so, for k respectively ₃ ，b ₃ The derivation can be derived as follows:

thereby obtaining k ₃ ，b ₃ The expression of (a) is as follows:

the arithmetic mean value expressed therein is calculated as follows:

traversing all long axes in the line segment set L and substituting k into ₃ ，b ₃ In the expression (2), a constant k is obtained ₃ ，b ₃ Finally, the plane straight line expression of the fitting formula (4) can be obtained, and the plane straight line obtained by fitting is M ² The long axis of the condylar-process head of (a);

step 3.7, training set D _train2 All the manual labeling results in (1) are subjected to the operations of the step (3.1) to the step (3.6), and finally D is obtained _train2 Medial condylar-procephalic long-axis dataset L _long 。

5. The spatial central axis-based Condyle CBCT image data enhancement method as claimed in claim 1 or 2, wherein in the step S4, a training set D is set _train2 And major axis set L _long Inputting the data into a rotating module to obtain a data set D finally used for training the condylar segmentation model _train3 The process is as follows:

step 4.1, calculating the module according to the long axis of the condylar head to obtain the long axis l of the condylar head _i (l _i ∈L _long I =0,1, 2. -) in size D ₁ ×H ₁ ×W ₁ Manual annotation of results M ² Calculating the included angle between the long axis and the x axis, and taking the Condyle CBCT image I as the reference ² And manually labeling the result M ² (I ² ∈D _train1 ，M ² ∈D _train1 ) Rotate so that the long axis l of the condyles _i Rotating to x axis, and performing resampling to obtain D ₁ ×H ₁ ×W ₁ New CBCT image of condyles I ³ And manually labeling the result M ³ ；

Step 4.2, training set D _train2 All the CBCT images of the condyles and the manual labeling result in the step 4.1 are operated, and processed data D are obtained finally _train3 (I ³ ∈D _train3 ，M ³ ∈D _train3 )。

Images