CN109410185A

CN109410185A - A kind of image partition method, device and storage medium

Info

Publication number: CN109410185A
Application number: CN201811176821.5A
Authority: CN
Inventors: 边成
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd; Tencent Cloud Computing Beijing Co Ltd
Priority date: 2018-10-10
Filing date: 2018-10-10
Publication date: 2019-03-01
Anticipated expiration: 2038-10-10
Also published as: CN109410185B

Abstract

The embodiment of the invention discloses a kind of image partition method, device and storage mediums；The embodiment of the present invention is after getting 3 d medical images to be split, first the target area (target area includes target object) in the 3 d medical images can be intercepted using two-dimentional parted pattern, obtain candidate region, then again using the type of each voxel in three-dimensional pyramid analysis neural network forecast candidate region, and the target object in the candidate region is split based on the type of prediction, to obtain final segmentation result；The program can prevent three-dimensional spatial information from losing, while to ensure that the marginal layer structure of target object can be come out by Accurate Prediction, training is not easy and the generation of the situation of prediction result inaccuracy caused by avoiding the ratio for accounting for entire original medical image due to target object too small, can greatly improve the accuracy of medical image segmentation.

Description

A kind of image partition method, device and storage medium

Technical field

The present invention relates to fields of communication technology, and in particular to a kind of image partition method, device and storage medium.

Background technique

Auricular fibrillation (abbreviation atrial fibrillation) is the most common perpetual arrhythmia symptom.Accurately from it is three-dimensional (3D, It 3Dimensions) is partitioned into left atrial region in nuclear magnetic resonance image (MRI, Magnetic Resonance Imaging), with The variation for observing atrium sinistrum is helpful to auricular fibrillation is eliminated.

In the conventional technology, the segmentation of left atrial region generally requires artificial from nuclear magnetic resonance image by medical staff It is operable to realize, not only efficiency is lower, but also very dependent on the professional standing and experience of executor.For this purpose, it is existing again Propose the technology divided automatically, such as U-Net (Convolutional Networks for Biomedical Image Segmentation, a kind of for dividing the network of cell image) network.Wherein, U-Net be two dimension (2D, Network structure 2Dimensions) designed on image, it can by predict the classification of each pixel come to medical image into Row segmentation.

But in the research and practice process to the prior art, it was found by the inventors of the present invention that using U-Net into When row image segmentation, three-dimensional spatial information can be lacked, can not be gone out by Accurate Prediction so as to cause the atrium structure of some marginal layers Come, that is to say, that the accuracy of existing splitting scheme is not good enough.

Summary of the invention

The embodiment of the present invention provides a kind of image partition method, device and storage medium, and medical image segmentation can be improved Accuracy.

The embodiment of the present invention provides a kind of image partition method, comprising:

Obtain 3 d medical images to be split；

The target area in the 3 d medical images is intercepted using preset two-dimentional parted pattern, obtains candidate Region, the target area include target object；

Using the type of each voxel in candidate region described in preset three-dimensional pyramid analysis neural network forecast；

The target object in the candidate region is split based on the type of prediction, obtains segmentation result.

Correspondingly, the embodiment of the present invention also provides a kind of image segmentation device, including acquiring unit, interception unit, prediction Unit and cutting unit, as follows:

Acquiring unit, for obtaining 3 d medical images to be split；

Interception unit, for being carried out using preset two-dimentional parted pattern to the target area in the 3 d medical images Interception, obtains candidate region, and the target area includes target object；

Predicting unit, for using each voxel in candidate region described in preset three-dimensional pyramid analysis neural network forecast Type；

Cutting unit is split the target object in the candidate region for the type based on prediction, is divided Cut result.

Optionally, in some embodiments, the interception unit may include conversion subunit, interception subelement and fusion Subelement, as follows:

Conversion subunit, for the 3 d medical images to be converted to multiple two-dimensional medical images；

Intercept subelement, for using preset two-dimentional parted pattern to the target area in every two-dimensional medical images into Row interception, obtains multiple two dimensional images to be fused；

It merges subelement and obtains candidate region for multiple described two dimensional images to be fused to be fused to 3-D image.

Optionally, in some embodiments, the interception subelement specifically can be used for dividing mould using preset two dimension Type predicts the type of pixel in every two-dimensional medical images, determines target area in two-dimensional medical figure according to prediction result Boundary point as in, is intercepted based on determining boundary point pair target area, obtains multiple two dimensional images to be fused.

Optionally, in some embodiments, the three-dimensional pyramid analysis network includes residual error network and three-dimensional pyramid Parted pattern, the predicting unit includes extracting subelement and prediction subelement, as follows:

The extraction subelement obtains candidate regions for carrying out feature extraction to the candidate region by residual error network The characteristic information in domain；

The prediction subelement, for predicting the time according to the characteristic information using three-dimensional pyramid parted pattern The type of each voxel in favored area.

Optionally, in some embodiments, the extraction subelement specifically can be used for through the first residual of residual error network Difference module, the second residual error module and third residual error module carry out process of convolution to the candidate region；Using the residual error network Four-infirm difference module and the 5th residual error module empty process of convolution is carried out to the candidate region after process of convolution, obtain candidate regions The characteristic information in domain.

Optionally, in some embodiments, the predicting unit can also include optimization subelement, as follows:

The optimization subelement, for expanding default size outward, being waited after being expanded centered on the candidate region Favored area, from the region of pre-set dimension is cut after the expansion in candidate region at random as candidate region after optimization；

Then at this point, extracting subelement, specifically can be used for carrying out candidate region after the optimization by residual error network special Sign is extracted, the characteristic information of candidate region after being optimized.

Optionally, in some embodiments, the three-dimensional pyramid parted pattern include three-dimensional pyramid pond layer, on adopt Sample layer and regularization layer, the prediction subelement specifically can be used for being believed using three-dimensional pyramid pond layer according to the feature Breath carries out pond to the candidate region, by up-sampling layer by the size of the output up-sampling of Chi Huahou to Chi Huaqian, and will Output after up-sampling is attached, and is carried out Regularization to the output after connection using regularization layer, is obtained the candidate The type of each voxel in region.

Optionally, in some embodiments, the cutting unit specifically can be used for the type according to prediction, from described Screening meets the voxel of target object voxel type in candidate region, obtains candidate voxels collection, true according to the candidate voxels collection Fixed boundary point of the target object in candidate region, based on target object in candidate region described in determining boundary point pair into Row segmentation, obtains segmentation result.

Optionally, in some embodiments, described image segmenting device can also include the first acquisition unit and the first instruction Practice unit, as follows:

First acquisition unit, for acquiring multiple 3 d medical images samples for being labelled with target area；

First training unit, for being carried out according to the 3 d medical images sample to preset semantic segmentation model Training, obtains two-dimentional parted pattern.

Optionally, in some embodiments, first training unit may include the first division subelement, the first calculating Subelement and the first convergence subelement, as follows:

First divides subelement, for the 3 d medical images sample to be divided into positive sample and negative sample, obtains the One training sample set, the positive sample are the target area in the 3 d medical images sample, and the negative sample is described three Tie up the region in medical image sample in addition to target area；

First computation subunit, the positive sample for being concentrated using preset semantic segmentation model to first training sample This and negative sample are calculated, and are obtained the first training sample and are concentrated the predicted value of each positive sample and the predicted value of negative sample；

First convergence subelement, for obtain the first training sample concentrate each positive sample true value and negative sample it is true Real value, and according to the true value of each positive sample and the true value and predicted value of predicted value and each negative sample to institute Predicate justice parted pattern is restrained, and two-dimentional parted pattern is obtained.

Optionally, in some embodiments, the first convergence subelement, specifically can be used for calculating each positive sample Error between predicted value and true value obtains the corresponding positive sample error of each positive sample；Calculate the prediction of each negative sample Error between value and true value, obtains the corresponding negative sample error of each negative sample；Screening negative sample error is greater than the set value Negative sample as difficult negative sample；The negative sample error of difficult negative sample and all positive sample errors are subjected to the reversed of convolution It propagates.

Optionally, in some embodiments, first training unit, specifically can be used in multiple graphics processors Load the same preset semantic segmentation model；According to the 3 d medical images sample respectively to the multiple graphics processor In semantic segmentation model to being trained；Semantic point after the training for selecting accuracy rate optimal in semantic segmentation model after training Cut model；The model parameter of semantic segmentation model after the training of selection is loaded onto all semantic segmentation models, and returns and holds Row is according to the 3 d medical images sample respectively to the semantic segmentation model in the multiple graphics processor to being trained The step of, until the convergence of all semantic segmentation models finishes；It is accurate to select from multiple semantic segmentation models after convergence The optimal semantic segmentation model of rate is as two-dimentional parted pattern.

Optionally, in some embodiments, described image segmenting device can also include the second acquisition unit and the second instruction Practice unit, as follows:

Second acquisition unit is labelled with voxel type for acquiring multiple, and includes the 3-D image sample of target object；

Second training unit obtains three for being trained according to the 3-D image sample to preset analysis network It ties up pyramid and analyzes network.

Optionally, in some embodiments, second training unit may include the second division subelement, the second calculating Subelement and the second convergence subelement, as follows:

Second divides subelement, for 3-D image sample to be divided into positive sample and negative sample, obtains the second training sample This collection, the positive sample are the target object in the 3-D image sample, and the negative sample is in the 3-D image sample Region in addition to target object；

Second computation subunit, positive sample for being concentrated using preset analysis network to second training sample with Negative sample is calculated, and is obtained the second training sample and is concentrated the predicted value of each positive sample and the predicted value of negative sample；

Second convergence subelement, for obtain the second training sample concentrate each positive sample true value and negative sample it is true Real value, and according to the true value of each positive sample and the true value and predicted value of predicted value and each negative sample to institute It states analysis network to be restrained, obtains three-dimensional pyramid analysis network.

Second training unit specifically can be used in multiple graphics processors loading the same preset analysis net Network；According to the 3-D image sample respectively to the analysis network in the multiple graphics processor to being trained；From training The post exercise analysis network for selecting accuracy rate optimal in post analysis network；The model parameter of the post exercise analysis network of selection is added It is loaded onto all analysis networks, and returns to execution according to the 3-D image sample respectively in the multiple graphics processor The step of network is to being trained is analyzed, until all analysis network convergences finish；From multiple analysis networks after convergence The middle analysis network for selecting accuracy rate optimal analyzes network as three-dimensional pyramid.

In addition, the embodiment of the present invention also provides a kind of storage medium, the storage medium is stored with a plurality of instruction, the finger It enables and being loaded suitable for processor, to execute the step in any image partition method provided in an embodiment of the present invention.

The embodiment of the present invention, can be first using two-dimentional parted pattern to this after getting 3 d medical images to be split Target area (target area includes target object) in 3 d medical images is intercepted, and obtains candidate region, then again Using the type of each voxel in three-dimensional pyramid analysis neural network forecast candidate region, and based on the type of prediction to the time Target object in favored area is split, to obtain final segmentation result；Since the program can be first using two dimension segmentation Model carries out region detection to the 3 d medical images, then is finely divided using three-dimensional pyramid analysis network candidate region It cuts, therefore, can prevent three-dimensional spatial information from losing, to ensure that the marginal layer structure of target object can be gone out by Accurate Prediction While coming, training is not easy (positive and negative sample caused by avoiding the ratio for accounting for entire original medical image due to target object too small This imbalance) and prediction result inaccuracy situation generation, the accuracy of medical image segmentation can be greatly improved.

Detailed description of the invention

To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached Figure.

Fig. 1 a is the schematic diagram of a scenario of image partition method provided in an embodiment of the present invention；

Fig. 1 b is the flow chart of image partition method provided in an embodiment of the present invention；

Fig. 1 c is the partial structure diagram of two dimension segmentation network provided in an embodiment of the present invention；

Fig. 1 d is the schematic diagram of competition training provided in an embodiment of the present invention；

Fig. 1 e is the partial structure diagram of three-dimensional pyramid analysis network provided in an embodiment of the present invention；

Fig. 2 a is another flow chart of image partition method provided in an embodiment of the present invention；

Fig. 2 b is the processing schematic of two dimension segmentation network provided in an embodiment of the present invention；

Fig. 2 c is the processing schematic of three-dimensional pyramid analysis network provided in an embodiment of the present invention；

Fig. 3 a is the structural schematic diagram of image segmentation device provided in an embodiment of the present invention；

Fig. 3 b is another structural schematic diagram of image segmentation device provided in an embodiment of the present invention；

Fig. 4 is the structural schematic diagram of the network equipment provided in an embodiment of the present invention.

Specific embodiment

Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts Example, shall fall within the protection scope of the present invention.

The embodiment of the present invention provides a kind of image partition method, device and storage medium.Wherein, which can collect At in the network device, which can be server, be also possible to the equipment such as terminal.

So-called image segmentation refers to dividing the image into several regions specific, with unique properties, and proposes to feel The technology and process of targets of interest.And in embodiments of the present invention, it generally refers to be split medical image, and find out institute The target object needed, for example the region analyzed for uterine cancer is partitioned into from gynecatoptron image, for another example, from nuclear magnetic resonance figures As in be partitioned into left atrial region, etc., this target object split (i.e. area image) is subsequent can be for medical staff Or other medical experts analyze, to make further operation.

In embodiments of the present invention, two kinds of network is mainly used, one is by being instructed to semantic segmentation model Two dimension parted pattern obtained from white silk such as can specifically be improved by the network structure to U-Net and be obtained, another is then It is three-dimensional pyramid analysis network, is obtained for example, can be improved by the network structure to PSPNet.Due to U-Net Belong to the network structure designed on 2d, it, can be because lacking if be directly applied in the segmentation of 3 d medical images Three-dimensional spatial information and cause some marginal layer structures of target object that can not be come out by Accurate Prediction (two dimension segmentation is artificial right Data carry out slice and throw into network training, and two-dimensional network once can only see the image after a slice, so can not observe To the information between slice, lead to the deficiency of three-dimensional semantic information).And if directlying adopt PSPNet, meeting again Because it is too small that target object accounts for entire original image ratio, and causes positive negative sample extremely unbalanced, not only it is not easy to train, but also pre- The accuracy rate for surveying result is not also high.So the embodiment of the present invention is improved and has been melted to the structure of the network of both types It closes, as follows:

Referring to Fig. 1 a, firstly, the network equipment for being integrated with image segmentation device is getting 3 D medical to be split After image, the target area in the 3 d medical images can be intercepted using preset two-dimentional parted pattern, with rough Ground Split goes out the suspicious region where target object, wherein the suspicious region is known as candidate region in embodiments of the present invention, so Afterwards, then using preset three-dimensional pyramid the type of each voxel in the neural network forecast candidate region is analyzed, and based on prediction Type is split the target object in the candidate region, to obtain final segmentation result, i.e., divides the target object It comes out.

Due to before being split using three-dimensional pyramid analysis network, by include target object region from Divide in 3 d medical images to be split and has come out, input --- the candidate region to analyze network as three-dimensional pyramid, And the ratio that target object accounts for entire candidate region is larger, is not in that training is not caused by positive and negative sample imbalance therefore Easy situation.Moreover, the use of three-dimensional pyramid analysis network is also possible to prevent three-dimensional spatial information loss, so, it can be true The marginal layer structure for protecting target object can be come out by Accurate Prediction, improve the accuracy of segmentation.

It is described in detail separately below.It should be noted that the following description sequence is not as excellent to embodiment The restriction of choosing sequence.

The present embodiment will be described from the angle of image segmentation device, which specifically can integrate in net In network equipment, which can be server, be also possible to the equipment such as terminal；Wherein, which may include mobile phone, puts down The equipment such as plate computer, laptop and individual calculus (PC, Personal Computer).

A kind of image partition method, comprising: 3 d medical images to be split are obtained, using preset two-dimentional parted pattern Target area (target area includes target object) in the 3 d medical images is intercepted, obtains candidate region, so Afterwards, the type of each voxel in the neural network forecast candidate region, and the class based on prediction are analyzed using preset three-dimensional pyramid Type is split the target object in the candidate region, obtains segmentation result.

As shown in Figure 1 b, the detailed process of the image partition method can be such that

101,3 d medical images to be split are obtained.

For example, specifically being come by each medical image acquisition device, such as NMR imaging instrument, such as gynecatoptron or endoscope Image Acquisition is carried out to life body tissue, and then is supplied to the image segmentation device, that is, image segmentation device specifically can receive The 3 d medical images to be split that medical image acquisition device is sent.

Wherein, 3 d medical images to be split refer to the 3 d medical images for needing to carry out image segmentation, so-called three Medical image is tieed up, refers to certain component part by medical image acquisition device to life entity, such as stomach, the heart of human body Dirty, throat and vagina etc. carry out the obtained image of image collection.And life entity refers to form of life, and can be to the external world The independent individual accordingly reflected, such as people, cat or dog etc. are made in stimulation.

102, the target area in the 3 d medical images is intercepted using preset two-dimentional parted pattern, is waited Favored area；For example, specifically can be such that

(1) 3 d medical images are converted into multiple two-dimensional medical images.

For example, with the coordinate of the 3 d medical images including for (x, y, z), then can specifically be constituted with x-axis and y-axis Face be plane, the 3 d medical images are cut along z-axis, obtain multiple two-dimensional medical images, wherein the grain of cutting Degree can be depending on the demand of practical application, and therefore not to repeat here.

(2) target area in every two-dimensional medical images is intercepted using preset two-dimentional parted pattern, is obtained Multiple two dimensional images to be fused.

For example, can specifically be carried out using preset two-dimentional parted pattern to the type of pixel in every two-dimensional medical images Prediction, determines boundary point of the target area in two-dimensional medical images according to prediction result, and based on determining boundary point pair mesh Mark region is intercepted, multiple two dimensional images to be fused are obtained.

Wherein, which includes target object, and the target area and target object can be according to practical applications Demand is configured, for example, the target area can be set to heart, for another example, if target if target object is atrium sinistrum Object is 12 duodenum 12, then the target area can be set to stomach, etc..

It, then at this point, specifically can be using default for target area is heart for example, using the target object as atrium sinistrum Two-dimentional parted pattern the type of pixel in every two-dimensional medical images is predicted, then, filtered out according to prediction result Belong to the pixel of heart, and then determine boundary point of the heart in the two-dimensional medical images, subsequently, based on determining boundary point Region where heart is intercepted, multiple two dimensional images to be fused about heart can be obtained.

(3) by this, multiple two dimensional images to be fused are fused to 3-D image, obtain candidate region.

3-D image can be rebuild by multiple two dimensional images to be fused according to this, obtain candidate region.

For example, still using target object as atrium sinistrum, for target area is heart, due to having obtained multiple in (2) About the two dimensional image to be fused of heart, therefore, multiple can be rebuild at this time about the two dimensional image to be fused of heart according to this Three-dimensional heart area, and using the heart area as candidate region.

Optionally, which can be labelled with the 3 d medical images sample of target area by multiple Training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, can also be by the figure As segmenting device is voluntarily trained；I.e. in step " using preset two-dimentional parted pattern to the mesh in the 3 d medical images Mark region is intercepted " before, which can also include:

Multiple 3 d medical images samples for being labelled with target area are acquired, according to the 3 d medical images sample to default Semantic segmentation model be trained, obtain two-dimentional parted pattern.For example, can be such that

A, multiple 3 d medical images samples for being labelled with target area are acquired.

For example, can specifically acquire multiple 3 d medical images as raw data set, such as from database or network etc. The raw data set is obtained, then the image in the raw data set is pre-processed, to obtain meeting default semantic segmentation Then the image of the input standard of model carries out the mark of target area to these pretreated images, multiple can be obtained It is labelled with the 3 d medical images sample of target area.

Wherein, pretreatment may include the operation such as duplicate removal, cutting, rotation and/or overturning.For example, to preset semantic segmentation The input size of model be " 512*512 (wide * high) " for, then at this point it is possible to be by the image cropping in raw data set " 512*512 " size, it is, of course, also possible to further other pretreatment operations be carried out to these images, such as (- 15 -+15) Random-Rotation operation and Random Level turning operation, etc..

It should be noted that after obtaining multiple and being labelled with the 3 d medical images sample of target area, in addition to there is part Data can by execute step B can also be using partial data as verify data to obtain except the first training sample set Collection verifies the two-dimentional parted pattern obtained in training process or after training so as to subsequent.

B, the 3 d medical images sample is divided into positive sample and negative sample, obtains the first training sample set.

Wherein, which is the target area in the 3 d medical images sample, which is the 3 D medical figure Region in decent in addition to target area.

For example, the positive sample is the heart area in the 3 d medical images sample if the target area is heart, and Negative sample is the region in the 3 d medical images sample than the heart, and so on, etc..

C, the training sample that first training sample is concentrated using preset semantic segmentation model, such as positive sample and negative Sample is calculated, and the predicted value that the first training sample concentrates each training sample, such as the predicted value of each positive sample are obtained With the predicted value of each negative sample.

Wherein, which can be depending on the demand of practical application, for example, can be by U- The structure of Net network, which improves, to be obtained, specifically, can be by encoder section (the i.e. down-sampling portion in U-Net network Point) residual error network is replaced with, such as ResNet101 network.

For example, with reference to Fig. 1 c, in the semantic segmentation model, encoder section is the ResNet101 network of a standard, and Decoder section is then made of four up-sampling layers.Wherein, the encoder section of the semantic segmentation model includes residual error module (ResBlock) 1, residual error module 2, residual error module 3, residual error module 4 and residual error module 5.Decoder section then may include decoding Device 1, decoder 2, decoder 3 and decoder 4.

After the training sample that first training sample is concentrated is input to the semantic segmentation model, successively by residual error module 1, residual error Module 2, residual error module 3, residual error module 4 and residual error module 5 are encoded (i.e. down-sampling operation) to the training sample.Residual error mould The output of block 1~4 all has Liang Ge branch, and one sends next residual error module to, another sends corresponding decoder to. For example, residual error module 1 will export send residual error module 2 to while, also send output to decoder 1；Residual error module 2 exists While sending output to residual error module 3, also output is sent to decoder 2；Residual error module 3 sends residual error to will export While module 3, also output is sent to decoder 3；Residual error module 4 will export send residual error module 5 to while, also will Output sends decoder 4 to, and so on, etc..Residual error module 5 is then slightly different, although output there are two also, it An output send decoder 4 to, another output then needs to be attached (concat) with the output of decoder 1~4, with Finally obtain the predicted value of training sample, such as the predicted value or the predicted value of negative sample, etc. of positive sample.

It should be noted that, wherein decoder 1~4 all has two inputs, for example, decoder 4 is in addition to can be by residual error mould Except the output that block 4 transmits is as one of input, it is also necessary to up-sampled by the output to residual error module 5 to obtain Another input；And on the one hand output that decoder 3 then transmits residual error module 3 is as one of input, other side It is up-sampled by the output to decoder 4 to obtain another input；Decoder 2 and decoder 1 are similar with decoder 3, It is detailed in Fig. 1 c.Each decoder can by way of fusion (such as " adding ", the "+" number being detailed in Fig. 1 c) will be respectively received Two inputs be attached, and then obtain the output of each decoder.

Optionally, the quantity and parameter and the parameter of decoder of convolutional layer can be according to realities in each residual error module Depending on the demand of border application.

In addition, it should be noted that, the parameter of each residual error module can use residual error network, such as ResNet101 network The ginseng of pre-training on ImageNet (one is used for the large-scale visible database of visual object identification software research) data set Number, and respectively up-sample layer and variance 0.01, mean value is then used to be initialized for 0 Gaussian Profile.

D, it obtains the first training sample and concentrates the true value of each positive sample and the true value of negative sample.

For example, can specifically determine that the first training sample concentrates each instruction according to the mark of each 3 d medical images sample Practice sample true value (i.e. mark value), such as each positive sample true value (i.e. mark value) and each negative sample true value (i.e. mark value).

E, according to the true value and predicted value of the true value of each positive sample and predicted value and each negative sample to the language Adopted parted pattern is restrained, and two-dimentional parted pattern is obtained.

For example, the error between the predicted value and true value of each positive sample can be specifically calculated, each positive sample is obtained Error between corresponding positive sample error, and the predicted value and true value of each negative sample of calculating, obtains each negative sample Then all negative sample errors and all positive sample errors are carried out the backpropagation of convolution, to reach by corresponding negative sample error To convergent purpose, two-dimentional parted pattern is finally obtained.

Optionally, in order to avoid problem unbalanced between positive negative sample, if being much larger than positive sample than negative sample quantity Quantity (difference of negative sample quantity and positive sample quantity is greater than predetermined threshold) can be with then when carrying out the backpropagation of convolution Biggish negative sample error will be only worth and all positive sample errors carry out backpropagation, i.e., step is " according to each positive sample The true value and predicted value of true value and predicted value and each negative sample restrain the semantic segmentation model " it can also be with It is as follows:

The error (calculating the loss of each positive sample) between the predicted value and true value of each positive sample is calculated, is obtained The corresponding positive sample error of each positive sample, and the error calculated between the predicted value and true value of each negative sample (are counted Calculate the loss of each negative sample), the corresponding negative sample error of each negative sample is obtained, then, screening negative sample error, which is greater than, to be set The negative sample of definite value carries out convolution as difficult negative sample, by the negative sample error of difficult negative sample and all positive sample errors Backpropagation, for example, the negative sample error of difficult negative sample and all positive sample errors can specifically add up.

Similarly, if positive sample quantity is much larger than negative sample quantity, (difference of positive sample quantity and negative sample quantity is greater than pre- Determine threshold value), then when carrying out the backpropagation of convolution, it will can only be worth biggish positive sample error and all negative samples miss Difference carries out backpropagation, i.e., step " according to the true value of the true value of each positive sample and predicted value and each negative sample and Predicted value restrains the semantic segmentation model " also it can be such that

The error (calculating the loss of each positive sample) between the predicted value and true value of each positive sample is calculated, is obtained The corresponding positive sample error of each positive sample, and the error calculated between the predicted value and true value of each negative sample (are counted Calculate the loss of each negative sample), the corresponding negative sample error of each negative sample is obtained, then, screening positive sample error, which is greater than, to be set The positive sample of definite value carries out convolution as difficult positive sample, by the positive sample error of difficult positive sample and all negative sample errors Backpropagation, for example, the positive sample error of difficult positive sample and all negative sample errors can specifically add up.

Optionally, it only will can also be worth biggish positive sample error and the biggish negative sample error of value is reversely passed It broadcasts, i.e. the positive sample that is greater than the set value of screening positive sample error is as difficult positive sample, and screening negative sample error is greater than and sets The negative sample of definite value is as difficult negative sample, then, by the negative sample of the positive sample error of difficult positive sample and difficult negative sample Error carries out the backpropagation, etc. of convolution, and therefore not to repeat here.But, for convenience, in embodiments of the present invention, By being illustrated so that if negative sample quantity is much larger than positive sample quantity as an example.

Wherein, calculate the error between predicted value and true value mode can there are many, for example, can be by preset Loss function calculates the error between predicted value and true value, and predicted value and true is such as calculated by cross entropy loss function Intersection entropy loss between value, etc..

In addition, it should be noted that, setting value can be depending on the demand of practical application, which can be a specific Numerical value, be also possible to function, for example, if the number of positive sample is that k can be to negative sample after obtaining negative sample error This error 2*K before selecting, only carries out negative sample corresponding to the 2*K negative sample error according to being ranked up from big to small Backpropagation (certainly, it is also desirable to which positive sample is carried out to the backpropagation of convolution) of convolution, etc..

Optionally, in order to improve training effectiveness, and the problem that alleviation neural metwork training is unstable, to the two dimension point , can also be by the way of " competition training ", for example, referring to Fig. 1 d when cutting model and being trained, it can be in multiple graphics process The same semantic segmentation model is loaded on device (GPU, Graphics Processing Unit), is then trained respectively, is passed through It crosses certain frequency of training and then model after each training is verified using verifying collection, the best model that will be obtained As " victor ", (black small circle in such as Fig. 1 d, the i.e. model at the t times in GPU3 are victor, at the t+1 times Model in GPU2 is victor, etc.), then the model parameter of " victor " is loaded on all models, is repeated This iterative process, until continueing to that model is finally restrained；I.e. step is " according to the 3 d medical images sample to preset semanteme Parted pattern is trained, and obtains two-dimentional parted pattern " may include:

The same preset semantic segmentation model is loaded in multiple graphics processors, according to the 3 d medical images sample Respectively to the semantic segmentation model in multiple graphics processor to being trained, from selection is quasi- in semantic segmentation model after training Semantic segmentation model after the optimal training of true rate, is loaded onto all languages for the model parameter of semantic segmentation model after the training of selection In adopted parted pattern, and execution is returned according to the 3 d medical images sample respectively to the semanteme in multiple graphics processor point The step of model is to being trained is cut, until the convergence of all semantic segmentation models finishes.

Wherein it is possible to using validation data set, (validation data set may include multiple three-dimensionals for being labelled with target area Medical image sample) semantic segmentation model after training is verified, come select accuracy rate optimal training after semantic segmentation mould Type, for example, the image that verify data is concentrated can be specifically input to after each training in semantic segmentation model, to be predicted Value, then, obtained predicted value is compared with mark value, and the accuracy rate of semantic segmentation model after training can be obtained.

It should be noted that in each graphics processor, semantic segmentation model (the two-dimentional parted pattern before training) Training is identical as the training process of semantic segmentation model described in step C~E, and therefore not to repeat here.

103, using the type of each voxel in preset three-dimensional pyramid analysis neural network forecast candidate region.

Wherein, three-dimensional pyramid analysis network may include two parts, and a part is residual error network, for encoding, with Feature extraction is carried out to candidate region, another part uses the structure in pyramid pond, is mainly used for decoding, to predict voxel class Therefore type is known as three-dimensional pyramid parted pattern in embodiments of the present invention.

If three-dimensional pyramid analysis network includes residual error network and three-dimensional pyramid parted pattern, step is " using pre- If three-dimensional pyramid analysis the neural network forecast candidate region in each voxel type " may include:

(1) feature extraction is carried out to the candidate region by residual error network, obtains the characteristic information of candidate region.

For example, can specifically pass through the first residual error module (ResBlock1, abbreviation RB1) of preset residual error network, second Residual error module (ResBlock2, abbreviation RB2), third residual error module (ResBlock3, abbreviation RB3), four-infirm difference module (ResBlock4, abbreviation RB4) and the 5th residual error module (ResBlock5, abbreviation RB5) carry out process of convolution to the candidate region (Convolution), the characteristic information of candidate region is obtained.

In another example specifically can be residual by the first residual error module, the second residual error module and third of preset residual error network Difference module carries out process of convolution to the candidate region, then, using the four-infirm difference module and the 5th residual error mould of the residual error network Block carries out empty convolution (also known as expansion convolution, Dilated Convolution) processing to the candidate region after process of convolution, Obtain the characteristic information of candidate region.

Wherein, which can be ResNext101 or other residual error networks.Optionally, in order to improve Treatment effeciency reduces the consumption of resource, the residual error network other than it can be obtained by the image segmentation device self training, The preset residual error network can also be obtained by transfer learning, for example, being supplied to after being trained by other equipment The image segmentation device, etc..

Wherein, (also referred to as spreading rate, i.e. dilation rate, the parameter define at convolution kernel for the ratio of empty convolution The spacing being respectively worth when managing data) it can be configured according to the demand of practical application, for example be set as 2 and 5, i.e., by the 4th residual error The ratio of the empty convolution of module is set as 2, sets 5 for the ratio of the empty convolution of the 5th residual error module.In addition, first is residual The ratio of difference module, the second residual error module and third residual error module convolution can also depending on the demand of practical application, such as It can be set to 1, etc., for details, reference can be made to the ResNext101 network portions in Fig. 1 e.

Optionally, since candidate region is to divide network by two dimension to divide, two dimension segmentation in order to prevent Influence of the network segmentation inaccuracy to final segmentation result can carry out data augmentation (due to that can obtain to obtained candidate region The data volume taken is limited, and trains neural network that a large amount of data is needed therefore can to increase data by data augmentation Amount), for example obtained candidate region is expanded, for example, the rectangle frame of the candidate region can be expanded outward default big It is small, then cut the region of pre-set dimension at random in the frame after the expansion again；" pass through preset residual error network pair in step The candidate region carries out feature extraction " before, which can also include:

Centered on the candidate region, expand default size outward, candidate region after being expanded is candidate after the expansion The region of pre-set dimension is cut in region at random as candidate region after optimization.

Then at this point, step " carries out feature extraction to the candidate region by residual error network, obtains the feature letter of candidate region Breath " is specifically as follows: feature extraction is carried out to candidate region after the optimization by preset residual error network, it is candidate after being optimized The characteristic information in region.

Optionally, when optimizing to candidate region, the processing of other data augmentation can also be carried out, for example, can be with By the region of the pre-set dimension cut at random, prolong z-axis Random-Rotation (0 °, 90 °, 180 °, 270 °), etc..

(2) it using three-dimensional pyramid parted pattern, is predicted according to the characteristic information of the candidate region every in the candidate region The type of a voxel.

Wherein, the three-dimensional pyramid parted pattern may include three-dimensional pyramid pond layer (Pyramid Pool), on adopt Sample layer (Up-Sample) and regularization layer then predict that the mode of the type of each voxel in the candidate region specifically can be such that

Using three-dimensional pyramid pond layer, pond is carried out to the candidate region according to the characteristic information of the candidate region, is led to Up-sampling layer is crossed by the size of the output up-sampling of Chi Huahou to Chi Huaqian, and the output after up-sampling is attached (concat), Regularization is carried out to the output after connection using regularization layer, obtains each voxel in the candidate region Type.

Wherein, three-dimensional pyramid pond layer may include multiple convolution (Conv) layer, crowd standardization (BN, Batch Normalization, also referred to as batch are normalized) layer and activation primitive (Relu) layer, it can by convolution, batch standardization and Activation primitive etc. carries out pond processing to the candidate region, for example, reference can be made to Fig. 1 e.

Wherein, regularization layer can be used for preventing over-fitting, specifically can be according to the demand flexible choice of practical application just Then change mode, for example, can refer to using random inactivation regularization (Dropout), so-called Dropout in deep learning network Training process in, for neural network unit, it is temporarily abandoned from network according to certain probability (such as 10%).For Description is convenient, in embodiments of the present invention, is illustrated by taking Dropout as an example.

Wherein, three-dimensional pyramid analysis network can be labelled with voxel type by multiple, and include the three of target object Dimension image pattern training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, It can be voluntarily trained by the image segmentation device；" it is somebody's turn to do using preset three-dimensional pyramid analysis neural network forecast in step Before the type of each voxel in candidate region ", which can also include:

It acquires multiple and is labelled with voxel type, and include the 3-D image sample of target object, it is decent according to the three-dimensional figure This is trained preset analysis network, obtains three-dimensional pyramid analysis network.For example, can be such that

A, multiple are acquired and is labelled with voxel type, and includes the 3-D image sample of target object.

For example, multiple 3 d medical images comprising target object can be specifically acquired as raw data set, such as from Database or network etc. obtain the raw data set, then pre-process to the image in the raw data set, to be expired Then the image of the input standard of sufficient presupposition analysis network carries out the mark of voxel type to these pretreated images, i.e., Multiple can be obtained and is labelled with voxel type, and includes the 3-D image sample of target object.

Wherein, pretreatment may include the operation such as duplicate removal, cutting, rotation and/or overturning.For example, with presupposition analysis network Input size be " 128*128*32 (wide * high * deep) " for, then at this point it is possible to be by the image cropping in raw data set " 28*128*32 " size, it is, of course, also possible to further carry out other pretreatment operations to these images.

For another example, it can also directly acquire and network is divided to the obtained time of 3 d medical images sample decomposition by two dimension Favored area (i.e. the corresponding candidate region of 3 d medical images sample), then, to these corresponding times of 3 d medical images sample Favored area carries out the mark of voxel type, multiple can be obtained and is labelled with voxel type, and includes the 3-D image of target object Sample.

Optionally, influence of the two dimension segmentation network segmentation inaccuracy to final segmentation result in order to prevent, can be to obtaining The corresponding candidate region of 3 d medical images sample carry out data augmentation, for example expanded, for example, specifically can will be candidate The external frame (such as external rectangle frame) in region expands default size outward, for example expands the big of 40*40*10 (wide * high * is deep) It is small, then cut the region of pre-set dimension at random in the frame after expansion again, for example cut the area of 128*128*32 size at random Domain etc.；Optionally, other data augmentation processing can also be carried out, for example, can be by the pre-set dimension cut at random Z-axis Random-Rotation (0 °, 90 °, 180 °, 270 °), etc. is prolonged in region, subsequently, just these regions is carried out with the mark of voxel type Note, is labelled with voxel type to obtain multiple, and include the 3-D image sample of target object.

It should be noted that be labelled with voxel type obtaining multiple, and after including the 3-D image sample of target object, In addition to have partial data can by execute step B, to obtain except the second training sample set, can also using partial data as Validation data set verifies the three-dimensional pyramid analysis network obtained in training process or after training so as to subsequent.

B, 3-D image sample is divided into positive sample and negative sample, obtains the second training sample set.

Wherein, which is the target object in the 3-D image sample, which is in the 3-D image sample Region in addition to target object.

For example, the positive sample is the left atrial region in the 3-D image sample if the target object is atrium sinistrum, and Negative sample is the region in the 3-D image sample in addition to atrium sinistrum, and so on, etc..

C, the training sample that second training sample is concentrated using preset analysis network, such as positive sample and negative sample It is calculated, obtains the predicted value that the second training sample concentrates each training sample, such as the predicted value of each positive sample and each The predicted value of a negative sample.

Wherein, which can be configured according to the demand of practical application, for example, can be by right The structure of PSPNet network, which improves, to be obtained, specifically, the encoder section in PSPNet network can be replaced with residual Poor network, such as ResNet101 network, and the rest part of PSPNet network is then used as three-dimensional pyramid parted pattern part.

Wherein, residual error network may include the first residual error module (RB1), the second residual error module (RB2), third residual error module (RB3), four-infirm difference module (RB4) and the 5th residual error module (RB5).

Optionally, in order to avoid down-sampling multiple is excessive, cause some small targets that can not restore, it can be by residual error network The convolutional layers of several subsequent residual error modules replace with empty convolutional layer, for example, by the second residual error module, third residual error mould The convolutional layer of block, four-infirm difference module and the 5th residual error module replaces with empty convolutional layer, alternatively, by third residual error module, Four-infirm difference module and the convolutional layer of the 5th residual error module replace with empty convolutional layer, etc..But excessive empty convolutional layer is again " grid effect " (i.e. the appearance " grid ") that may result in prediction result, therefore, by multiple practical studies and experiment, at this In inventive embodiments, the convolutional layer of four-infirm difference module and the 5th residual error module is only replaced with into empty convolutional layer, and such as Fig. 1 e Shown, the ratio (i.e. spreading rate) of the empty convolution of the four-infirm difference module and the 5th residual error module can be respectively set to 2 Hes 5.That is:

It, can be with after the training sample (positive sample or negative sample) that second training sample is concentrated inputs the preset analysis network After successively carrying out process of convolution to the training sample by the first residual error module, the second residual error module and third residual error module, then by Four-infirm difference module and the 5th residual error module carry out empty process of convolution to the data after process of convolution, obtain the spy of training sample Then reference breath sends the characteristic information of training sample to three-dimensional pyramid pond layer and carries out pond processing, then by up-sampling Output after up-sampling is attached by layer (Up-Sample) by the size of the output up-sampling of Chi Huahou to Chi Huaqian (concat), it hereafter, can use regularization layer and Regularization carried out to the output after connection, the training sample can be obtained In each voxel type.

Wherein, regularization layer can be used for preventing over-fitting, specifically can be according to the demand flexible choice of practical application just Then change mode, for example, Dropout can be used.

Optionally, the up-sampling layer in three-dimensional pyramid parted pattern part is defeated according to the demand of practical application adjustment pondization Size out, in addition, the up-sampling layer in the three-dimensional pyramid parted pattern part can also be using different empty convolution ratios Empty convolutional coding structure be replaced, therefore not to repeat here.

D, it obtains the second training sample and concentrates the true value of each positive sample and the true value of negative sample.

For example, can specifically determine that the second training sample concentrates each trained sample according to the mark of each 3-D image sample This true value (i.e. mark value), for example, each positive sample true value (i.e. mark value) and each negative sample true value (i.e. Mark value).

E, according to the true value and predicted value of the true value of each positive sample and predicted value and each negative sample to this point Analysis network is restrained, and three-dimensional pyramid analysis network is obtained.

For example, the error between the predicted value and true value of each positive sample can be specifically calculated, each positive sample is obtained Error between corresponding positive sample error, and the predicted value and true value of each negative sample of calculating, obtains each negative sample Then all negative sample errors and all positive sample errors are carried out the backpropagation of convolution by corresponding negative sample error, than Such as, all negative sample errors and all positive sample errors can specifically be added up, to reach convergent purpose, is finally obtained Three-dimensional pyramid analyzes network.

Optionally, in order to improve training effectiveness, and the problem that alleviation neural metwork training is unstable, with training two dimension point Cut model similarly, when being trained to three-dimensional pyramid analysis network, equally can also using by the way of " competition train ", For example, as shown in Figure 1 d, the same analysis network being loaded in multiple graphics processors, is then trained respectively, pass through It crosses certain frequency of training and then each post exercise analysis network is verified using verifying collection, it is best by what is obtained It analyzes network and is used as " victor " (black small circle in such as Fig. 1 d), the analysis network parameter of " victor " is then loaded into institute Have on analysis network, repeat this iterative process, until continueing to that analysis network is finally restrained, required three can be obtained It ties up pyramid and analyzes network；That is, step " is trained preset analysis network according to the 3-D image sample, three Vygen words Tower analyzes network " may include:

The same preset analysis network is loaded in multiple graphics processors, according to the 3-D image sample respectively to this Analysis network in multiple graphics processors is to being trained, after the training for selecting accuracy rate optimal in post exercise analysis network Analyze network, the model parameter of the post exercise analysis network of selection is loaded onto all analysis networks, and return execution according to The step of 3-D image sample is respectively to the analysis network in multiple graphics processor to being trained, until all analyses Network convergence finishes, and the analysis network for selecting accuracy rate optimal from multiple analysis networks after convergence is as three Vygen words Tower analyzes network.

Wherein it is possible to using validation data set (validation data set may include it is multiple be labelled with voxel type, and include The 3-D image sample of target object) post exercise analysis network is verified, come the post exercise analysis for selecting accuracy rate optimal Network, for example, the image that verify data is concentrated can be specifically input in each post exercise analysis network, to be predicted Value, then, obtained predicted value is compared with mark value, the accuracy rate of post exercise analysis network can be obtained.

It should be noted that in each graphics processor, analysis network (the three-dimensional pyramid before training analyzes network) The described analysis training process of network of training and step C~E it is identical, therefore not to repeat here.

104, the target object in the candidate region is split based on the type of prediction, obtains segmentation result；For example, Specifically it can be such that

According to the type of prediction, screening meets the voxel of target object voxel type from the candidate region, obtains candidate Voxel collection determines boundary point of the target object in candidate region according to the candidate voxels collection, based on determining boundary point pair Target object is split in the candidate region, obtains segmentation result.

Optionally, in order to improve the accuracy of segmentation, the 3 d medical images to be split can also be carried out reducing or Amplification, to obtain the image of certain ratio (such as 0.8,0.9,1.0,1.1 and 1.2), then using these images as newly to The 3 d medical images of segmentation are split respectively, to obtain the corresponding segmentation result of these images, and by each segmentation result Original size is adjusted to (for example, if the new 3 d medical images to be split and original according to the ratio zoomed in or out before There is image to compare, reduce 0.8 times, then needs to reduce this 0.8 times of Image Adjusting at this time and return original size, that is, amplify 1.25 times), finally, segmentation result adjusted is averaged again, final point of the 3 d medical images to be split as this Cut result.

From the foregoing, it will be observed that the present embodiment after getting 3 d medical images to be split, first can divide mould using two dimension Type intercepts the target area (target area includes target object) in the 3 d medical images, obtains candidate region, Then the type of each voxel in the neural network forecast candidate region, and the type pair based on prediction are analyzed using three-dimensional pyramid again Target object in the candidate region is split, to obtain final segmentation result；Since the program can be first using two dimension Parted pattern carries out regional prediction to the 3 d medical images, then carries out essence to candidate region using three-dimensional pyramid analysis network Subdivision is cut, and therefore, can prevent three-dimensional spatial information from losing, to ensure that the marginal layer structure of target object can be by accurate pre- While measuring next, training is not easy (just caused by avoiding the ratio for accounting for entire original medical image due to target object too small Negative sample is uneven) and prediction result inaccuracy situation generation, the accurate of medical image segmentation can be greatly improved Property.

According to method described in upper one embodiment, citing is described in further detail below.

In the present embodiment, it will be specifically integrated in the network equipment with the image segmentation device, and will be specially with target area Heart, target object are specially to be illustrated for atrium sinistrum.

(1) two-dimentional parted pattern and three-dimensional pyramid analysis network are trained respectively, are specifically can be such that

1, the training of two-dimentional parted pattern.

Firstly, the network equipment acquires multiple 3 d medical images sample (such as three-dimensional NMRs for being labelled with heart area Image), wherein there is part to can be used as training dataset, there is part to can be used as validation data set, for example, can be by these 3 d medical images sample is divided into training dataset and validation data set, etc. according to the ratio of 4:1 at random.

Secondly, the 3 d medical images sample that training data is concentrated is divided into positive sample and negative sample by the network equipment, than Such as, in the present embodiment, the heart area in the 3 d medical images sample can be divided into positive sample, by the 3 D medical Region division in image pattern than the heart is negative sample, to obtain the first training sample set.

Furthermore positive sample and negative sample that the network equipment concentrates first training sample using preset semantic segmentation model This is calculated, and is obtained the first training sample and is concentrated the predicted value of each positive sample and the predicted value of each negative sample.Wherein, should Preset semantic segmentation model depending on the demand of practical application, can specifically may refer to the embodiment and Fig. 1 c of front, Therefore not to repeat here.

Finally, the network equipment, which obtains the first training sample according to the mark on 3 d medical images sample, concentrates each positive sample The true value of this true value and negative sample, and according to the true value of each positive sample and predicted value and each negative sample True value and predicted value restrain the semantic segmentation model, and two-dimentional parted pattern can be obtained.

Wherein, it can specifically be restrained by loss function, for example, can be calculated according to default loss function each Error between the predicted value and true value of positive sample obtains the corresponding positive sample error of each positive sample, and calculates each Error between the predicted value and true value of negative sample obtains the corresponding negative sample error of each negative sample, then, will be all negative The backpropagation that sample error and all positive sample errors carry out convolution finally obtains two-dimentional segmentation to reach convergent purpose Model.

Optionally, it can also be verified and be adjusted using the two-dimentional parted pattern that validation data set obtains training, with Improve the accuracy of the two dimension parted pattern.

Optionally, in order to avoid problem unbalanced between positive negative sample, for example, if negative sample quantity is much larger than positive sample Quantity will can also only be worth biggish negative sample error and all positive sample errors then when carrying out the backpropagation of convolution Carry out backpropagation；For another example, if positive sample quantity is much larger than negative sample quantity, when carrying out the backpropagation of convolution, It will can only be worth biggish positive sample error and all negative sample errors carry out backpropagation, and so on, etc..In addition, Optionally, in order to improve training effectiveness, and alleviate the unstable problem of neural metwork training, to the two dimension parted pattern into When row training, it can also be detailed in the embodiment of front, therefore not to repeat here by the way of " competition training ".

2, the training of three-dimensional pyramid analysis network.

The network equipment acquires multiple and is labelled with voxel type, and includes the 3-D image sample of atrium sinistrum, with training two dimension Divide network type, may be otherwise using collected partial 3-D image pattern as training dataset, have part that can make For validation data set, specific division proportion can be depending on the demand of practical application.

Secondly, 3-D image sample is divided into positive sample and negative sample by the network equipment, the second training sample set is obtained, than Such as, in the present embodiment, the left atrial region of 3-D image sample can be divided into positive sample, it will be in the 3-D image sample Region division in addition to atrium sinistrum is negative sample, to obtain the second training sample set.

Furthermore positive sample that the network equipment is concentrated second training sample using preset analysis network and negative sample into Row calculates, and obtains the second training sample and concentrates the predicted value of each positive sample and the predicted value of each negative sample.Wherein, this is default Analysis network can be configured according to the demand of practical application, for details, reference can be made to the embodiments of front and Fig. 1 e, herein not It repeats.

Finally, the network equipment, which obtains the second training sample according to the mark on 3-D image sample, concentrates each positive sample The true value of true value and negative sample, and according to the true of the true value of each positive sample and predicted value and each negative sample Value and predicted value restrain the analysis network, obtain three-dimensional pyramid analysis network.

Wherein, can specifically be restrained by loss function, for example, can calculate each positive sample predicted value and Error between true value, obtains the corresponding positive sample error of each positive sample, and calculate each negative sample predicted value and Error between true value obtains the corresponding negative sample error of each negative sample, then, by all negative sample errors and it is all just Sample error carries out the backpropagation of convolution, for example, all negative sample errors and all positive sample errors can specifically be carried out It is cumulative, to reach convergent purpose, finally obtain three-dimensional pyramid analysis network.

Optionally, the three-dimensional pyramid analysis network that training obtains can also be verified and is adjusted using validation data set It is whole, to improve the accuracy of three-dimensional pyramid analysis network.

Optionally, in order to improve training effectiveness, and the problem that alleviation neural metwork training is unstable, with training two dimension point Cut model similarly, when being trained to three-dimensional pyramid analysis network, equally can also using by the way of " competition train ", It is detailed in the embodiment of front, therefore not to repeat here.

(2) network is analyzed by the trained two-dimentional parted pattern and three-dimensional pyramid, it can be to be split three It ties up medical image and carries out image segmentation, for details, reference can be made to Fig. 2 a, Fig. 2 b and Fig. 2 c.

As shown in Figure 2 a, a kind of dividing method of image, detailed process can be such that

201, the network equipment acquires nuclear magnetic resonance image.

For example, the network equipment can receive the nuclear magnetic resonance image of user's input, alternatively, receiving the core that other equipment are sent Magnetic resonance image, wherein the nuclear magnetic resonance image can by NMR imaging instrument to certain component part of life entity, such as The internal organ of human body such as heart etc. carries out image collection to obtain.

202, the network equipment pre-processes the nuclear magnetic resonance image, obtains 3 d medical images to be split.

Wherein, pretreatment may include the operation such as duplicate removal, cutting, rotation and/or overturning.

For example, by taking the input size (wide * high) for presetting semantic segmentation model is " 512*512 " as an example, then at this point, network is set Standby that the nuclear magnetic resonance image can be cut to " 512*512 " size, certainly, the network equipment can also be further total to the nuclear-magnetism The image that shakes carries out other pretreatment operations, the Random-Rotation operation of such as (- 15 -+15) and Random Level turning operation, etc..

203, the 3 d medical images are converted to multiple two-dimensional medical images by the network equipment.

For example, with the coordinate of the 3 d medical images including for (x, y, z), then the network equipment can be with x-axis and y-axis institute The face of composition is plane, cuts along z-axis to the 3 d medical images, obtains multiple two-dimensional medical images, wherein cutting Granularity can be depending on the demand of practical application, therefore not to repeat here.

204, the network equipment using trained two-dimentional parted pattern to the target area in every two-dimensional medical images into Row interception, obtains multiple two dimensional images to be fused.

For example, the network equipment specifically can be using trained two-dimentional parted pattern to the picture in every two-dimensional medical images Plain type is predicted, determines boundary point of the heart (i.e. target area) in two-dimensional medical images according to prediction result, such as The pixel for belonging to heart can be specifically filtered out according to prediction result, and then determines boundary of the heart in the two-dimensional medical images Point, etc., then, is intercepted based on the region where determining boundary point pair heart, obtains multiple two dimensional images to be fused.

Wherein, as shown in Figure 2 b, the encoder section of the two dimension parted pattern is a residual error network ResNet101, tool Body includes residual error module 1, residual error module 2, residual error module 3, residual error module 4 and residual error module 5；And decoder section then can wrap Include four up-sampling layers, i.e. decoder 1, decoder 2, decoder 3 and decoder 4.If the two dimension parted pattern includes institute as above The structure stated, then step 204 specifically can be such that

B referring to fig. 2 can be successively by residual error mould after two-dimensional medical images are imported the two dimension parted pattern by the network equipment Block 1 is encoded (i.e. down-sampling operation) to the two-dimensional medical images, and then, residual error module 1 sends output to residual error module 2 With decoder 1；Residual error module 2 encodes the output after the output for receiving residual error module 1, then, residual error module 2 Send the output of itself to residual error module 3 and decoder 2；Similar, residual error module 3 is in the output for receiving residual error module 2 Afterwards, which encode and send the output of itself to residual error module 4 and decoder 3；The rest may be inferred, residual error module 4 After the output to residual error module 3 encodes, also the output of itself is sent to residual error module 5 and decoder 4, then, by Residual error module sends decoder 4 and articulamentum (i.e. concat in Fig. 2 b) to after encoding to the output of residual error module 4.Solution Code device 4 after the output to residual error module 5 up-samples, will up-sample result and the output of residual error module 4 that receives into Row fusion (such as " adding ", the "+" number being detailed in Fig. 2 b), and send fused output to decoder 3 and articulamentum；It is similar , decoder 3 will up-sample the output of result with the residual error module 3 received after the output to decoder 4 up-samples It is merged, and sends fused output to decoder 2 and articulamentum；Similarly, decoder 2 is in the output to decoder 3 After being up-sampled, up-sampling result is merged with the output of the residual error module 2 received, and fused output is passed Give decoder 1 and articulamentum；And decoder 1 by up-sampling result and receives after the output to decoder 2 up-samples To the output of residual error module 1 merged, and send fused output to articulamentum, it is residual by what is received by articulamentum The output of difference module 5, the output of decoder 4, the output of decoder 3, the output of decoder 2 and the output of decoder 1 are connected It connects, obtains the prediction result of the type of pixel in two-dimensional medical images, the pixel for belonging to heart is filtered out according to prediction result, into And determine boundary point of the heart in the two-dimensional medical images, and cut based on the region where determining boundary point pair heart It takes, multiple two dimensional images to be fused can be obtained.

205, by this, multiple two dimensional images to be fused are fused to 3-D image to the network equipment, obtain candidate region.

For example, still using target object as atrium sinistrum, for target area is heart, as shown in Figure 2 b, due in step In 204, obtained multiple two dimensional images to be fused about heart, therefore, at this time the network equipment can according to this multiple about The two dimensional image to be fused of heart is redeveloped into 3-D image, and obtains the external frame (such as external rectangle frame) of the 3-D image, It can obtain the heart area (i.e. candidate region).

206, the network equipment is expanded preset size outward centered on the candidate region, candidate region after being expanded, from The region of pre-set dimension is cut after the expansion in candidate region at random as candidate region after optimization.

For example, the network equipment can carry out data to the heart area after obtaining heart area (i.e. candidate region) Augmentation, for example expand the size of 40*40*10 (wide * high * deep) outward to the corresponding external frame of the heart area, then, then from expansion The region of 128*128*32 size is cut in frame after big as candidate region after optimization, for details, reference can be made to Fig. 2 b.

Optionally, other data augmentation processing can also be carried out, for example, the pre-set dimension that will can be cut at random Region, prolong z-axis Random-Rotation (0 °, 90 °, 180 °, 270 °), etc..

207, the residual error network that the network equipment passes through that three-dimensional pyramid is analyzed in network carries out candidate region after the optimization Feature extraction, the characteristic information of candidate region after being optimized.For example, then having so that the residual error network is ResNext101 as an example Body can be such that

By the first residual error module of ResNext101, the second residual error module, third residual error module, four-infirm difference module and 5th residual error module carries out process of convolution, the characteristic information of candidate region after being optimized to candidate region after the optimization.

Optionally, in order to avoid down-sampling multiple is excessive, cause some small targets that can not restore, it can be by residual error network The convolutional layers of several subsequent residual error modules replace with empty convolutional layer.It at the same time, in order to prevent " grid effect ", can The convolutional layer of four-infirm difference module and the 5th residual error module is only replaced with empty convolutional layer, and by four-infirm difference module and The ratio of the empty convolution of five residual error modules is respectively set to 2 and 5.I.e. as shown in Figure 2 c, step 207 also may include:

By the first residual error module, the second residual error module and third residual error module of ResNext101 to candidate after the optimization Region carry out process of convolution, then, using the four-infirm difference module and the 5th residual error module of the residual error network to process of convolution after Candidate region carry out empty process of convolution, the characteristic information of candidate region after being optimized.

It should be noted that the ratio of above-mentioned cavity convolution is only example, it should be appreciated that the ratio of empty convolution can To be configured according to the demand of practical application.In addition, the first residual error module, the second residual error module and third residual error module convolution Ratio can also be depending on the demand of practical application, for example can be set to 1, etc., referring to fig. 2 c.

208, the network equipment is predicted using three-dimensional pyramid parted pattern according to the characteristic information of candidate region after the optimization After the optimization in candidate region each voxel type.

Wherein, which may include three-dimensional pyramid pond layer, up-sampling layer (Up-Sample) Then as shown in Figure 2 c predict that the mode of the type of each voxel in candidate region after the optimization specifically can be as with regularization layer Under:

The characteristic information of candidate region after optimization is inputted three-dimensional pyramid pond layer by the network equipment, with to waiting after the optimization Favored area (such as heart area after optimization) carries out pond, then, is up-sampled the output of Chi Huahou to pond by up-sampling layer Size before change, and the output after up-sampling is attached (concat), recycle regularization layer to the output after connection into Row Regularization obtains the type of each voxel in candidate region after the optimization.

Wherein, regularization layer can be used for preventing over-fitting, specifically can be according to the demand flexible choice of practical application just Then change mode, for example, can be using random inactivation regularization (Dropout).

209, the network equipment is split the atrium sinistrum in candidate region after the optimization based on the type of prediction, is divided Cut result.

For example, the network equipment can be according to the type of prediction, from (such as the heart area after optimization of candidate region after the optimization Domain) in screening meet the voxel of atrium sinistrum voxel type, obtain candidate voxels collection, then, determining according to the candidate voxels collection should The atrium sinistrum boundary point in candidate region after optimization, based on atrium sinistrum in candidate region after the optimization of determining boundary point pair into Row segmentation, obtains segmentation result, referring to fig. 2 c.

Optionally, in order to improve the accuracy of segmentation, the 3 d medical images to be split can also be carried out reducing or Amplification, to obtain the image of certain ratio (such as 0.8,0.9,1.0,1.1 and 1.2), then using these images as newly to The 3 d medical images of segmentation are split respectively, to obtain the corresponding segmentation result of these images, and by each segmentation result It is adjusted to original size according to the ratio zoomed in or out before, for example, if new 3 d medical images to be split are original 0.8 times of image then needs to reduce this at this time 0.8 times of Image Adjusting and returns original size, that is, amplifies 1.25 times, with such Push away, etc., finally, segmentation result adjusted is averaged again, final point of the 3 d medical images to be split as this Cut result.

In addition, it should be noted that, the execution hardware environment of the program can according to actual needs depending on, for example, can use Pytorch is realized, and operation, etc. on the Nvidia Tesla P40 video card.

From the foregoing, it will be observed that the present embodiment after getting 3 d medical images to be split, first can divide mould using two dimension Type intercepts the heart area in the 3 d medical images, obtains candidate region, then after expanding to candidate region with Machine cuts the region of pre-set dimension as candidate region after optimization, then just analyzes the neural network forecast optimization using three-dimensional pyramid Afterwards in candidate region each voxel type, and the atrium sinistrum in the candidate region is split based on the type of prediction, with Obtain final segmentation result；Since the program can be first pre- to 3 d medical images progress region using two-dimentional parted pattern It surveys, and optimizes to prevent influence of the two dimension segmentation network segmentation inaccuracy to final segmentation result, then use three Vygen words Therefore tower, which analyzes network, can prevent three-dimensional spatial information from losing, to ensure candidate region progress fine segmentation after optimization While atrium sinistrum marginal layer structure can be come out by Accurate Prediction, avoid accounting for entire original nuclear magnetic resonance image due to atrium sinistrum Ratio it is too small caused by training be not easy (positive and negative sample imbalance) and prediction result inaccuracy situation generation, can To greatly improve the accuracy of image segmentation；Moreover, because the residual error network in the three-dimensional pyramid analysis network of the present embodiment Four-infirm difference module and the convolutional layer of the 5th residual error module replace with empty convolutional layer, and it is four-infirm difference module and the 5th is residual The ratio of the empty convolution of difference module is respectively set to 2 and 5, so, both can to avoid due to down-sampling multiple it is excessive, it is caused The case where some small targets can not be restored, and " grid effect " can be prevented.

Following table one be the program subsequent sections segmentation result (stage1), atrium sinistrum divide stage (stage2), with And the experimental result of total effect (Ensemble) and each neural network of tradition in atrium sinistrum segmentation task, wherein " dice What table " characterized is the similarity of predicted value and true value (mark value), better closer to 1 between 0 to 1.

Table one:

Method	PSPNet	PSPNetD	U-net	DeeLabv3	SegNet	GCN	U-net3d	Stage1	Stage2	Ensemble
											Dice (%)	87.904	92.053	89.638	88.456	90.762	91.812	92.860	93.004	93.540	93.830

As it can be seen that for the accuracy of this programme segmentation outclass existing scheme.In addition, the present embodiment is in training two dimension segmentation It, can also be using only to " tired for being easy to produce the scene of positive and negative imbalanced training sets when model and three-dimensional pyramid analysis network Difficult negative sample " and all positive samples carry out the mode of backpropagation to be improved, and using modes such as " competition training " come Improve the stability of training effectiveness and neural metwork training.

In order to better implement above method, correspondingly, the embodiment of the present invention also provides a kind of image segmentation device, the figure As segmenting device specifically can integrate in the network device, which can be server, be also possible to the equipment such as terminal.

For example, as shown in Figure 3a, which may include acquiring unit 301, interception unit 302, prediction list Member 303 and cutting unit 304, as follows:

(1) acquiring unit 301；

Acquiring unit 301, for obtaining 3 d medical images to be split.

For example, specifically being come by each medical image acquisition device, such as NMR imaging instrument, such as gynecatoptron or endoscope Image Acquisition is carried out to life body tissue, and then is supplied to the acquiring unit 301, that is, acquiring unit 301 specifically can be used for connecing The original image that medical image acquisition device is sent is received, using the original image as 3 d medical images to be split.

Optionally, if the original image that acquiring unit 301 receives fails to comply with the input standard of two dimension segmentation network, Acquiring unit 301 can be also used for pre-processing the original image received, obtain 3 d medical images to be split.

(2) interception unit 302；

Interception unit 302, for using preset two-dimentional parted pattern to the target area in the 3 d medical images into Row interception, obtains candidate region；Wherein, which includes target object.

Optionally, in some embodiments, which may include conversion subunit, interception subelement and melts Zygote unit, as follows:

Conversion subunit 3021, for the 3 d medical images to be converted to multiple two-dimensional medical images.

For example, with the coordinate of the 3 d medical images including for (x, y, z), then the conversion subunit, can specifically use In the face constituted using x-axis and y-axis as plane, the 3 d medical images are cut along z-axis, obtain multiple two-dimensional medicals Image.Wherein, the granularity of cutting can be depending on the demand of practical application, and therefore not to repeat here.

Intercept subelement, for using preset two-dimentional parted pattern to the target area in every two-dimensional medical images into Row interception, obtains multiple two dimensional images to be fused.

Optionally, the interception subelement specifically can be used for using preset two-dimentional parted pattern to every two-dimensional medical Type of pixel in image is predicted, determines boundary point of the target area in two-dimensional medical images, base according to prediction result It is intercepted in determining boundary point pair target area, obtains multiple two dimensional images to be fused.

Subelement is merged, 3-D image is fused to for multiple two dimensional images to be fused by this, obtains candidate region.

That is the fusion subelement can be used for multiple two dimensional images to be fused according to this and rebuild 3-D image, obtains candidate Region.

(3) predicting unit 303；

Predicting unit 303, for using each voxel in preset three-dimensional pyramid analysis neural network forecast candidate region Type.

Wherein, three-dimensional pyramid analysis network may include residual error network and three-dimensional pyramid parted pattern, the residual error The structure of network and three-dimensional pyramid parted pattern is for details, reference can be made to the embodiment of front, and therefore not to repeat here.

If three-dimensional pyramid analysis network includes residual error network and three-dimensional pyramid parted pattern, the predicting unit 302 may include extracting subelement and prediction subelement, as follows:

The extraction subelement obtains candidate region for carrying out feature extraction to the candidate region by residual error network Characteristic information.

For example, extracting subelement, it specifically can be used for the first residual error module, the second residual error by preset residual error network Module, third residual error module, four-infirm difference module and the 5th residual error module carry out process of convolution to the candidate region, obtain candidate The characteristic information in region.

For another example, subelement is extracted, specifically can be used for the first residual error module by preset residual error network, second residual Difference module and third residual error module carry out process of convolution to the candidate region, then, using the four-infirm differential mode of the residual error network Block and the 5th residual error module carry out empty process of convolution to the candidate region after process of convolution, obtain the feature letter of candidate region Breath.

The prediction subelement, for being somebody's turn to do according to the prediction of the characteristic information of candidate region using three-dimensional pyramid parted pattern The type of each voxel in candidate region.

Optionally, since candidate region is to divide network by two dimension to divide, two dimension segmentation in order to prevent Influence of the network segmentation inaccuracy to final segmentation result, can carry out data augmentation to obtained candidate region, for example carry out Expand, for example, the rectangle frame of the candidate region can be expanded to default size outward, it is then random in the frame after the expansion again Cut the region of pre-set dimension；I.e. in some embodiments, which can also include optimization subelement, as follows:

The optimization subelement, for expanding default size, candidate regions after being expanded outward centered on the candidate region Domain, from the region of pre-set dimension is cut after the expansion in candidate region at random as candidate region after optimization.

Then at this point, extracting subelement, specifically can be used for carrying out feature to candidate region after the optimization by residual error network It extracts, the characteristic information of candidate region after being optimized.

For example, the optimization subelement can expand 40*40*10's (wide * high * is deep) centered on the candidate region outward Then size cuts the region of 128*128*32 size as candidate region after optimization at random in the frame after expansion again, then by It extracts subelement and feature extraction, the feature of candidate region after being optimized is carried out to candidate region after the optimization by residual error network Information.

Optionally, optimization subelement can also carry out the processing of other data augmentation when optimizing to candidate region, than Such as, z-axis Random-Rotation (0 °, 90 °, 180 °, 270 °), etc. can be prolonged into the region of the pre-set dimension cut at random.

Optionally, in some embodiments, the three-dimensional pyramid parted pattern may include three-dimensional pyramid pond layer, on Sample level and regularization layer, then:

The prediction subelement specifically can be used for being believed using three-dimensional pyramid pond layer according to the feature of the candidate region Breath carries out pond to the candidate region, by up-sampling layer by the size of the output up-sampling of Chi Huahou to Chi Huaqian, and will be upper Output after sampling is attached, and is carried out Regularization to the output after connection using regularization layer, is obtained the candidate region In each voxel type.

(4) cutting unit 304；

Cutting unit 304 is split the target object in the candidate region for the type based on prediction, is divided Cut result.

For example, the cutting unit 304, specifically can be used for the type according to prediction, screens and meet from the candidate region The voxel of target object voxel type, obtains candidate voxels collection, determines the target object in candidate regions according to the candidate voxels collection Boundary point in domain is split based on target object in the determining boundary point pair candidate region, obtains segmentation result.

Optionally, in order to improve the accuracy of segmentation, acquiring unit 301 can also be to the 3 d medical images to be split It is zoomed in or out, it is then single by interception again to obtain the image of certain ratio (such as 0.8,0.9,1.0,1.1 and 1.2) Member 302, predicting unit 303 and cutting unit 304 carry out respectively using these images as new 3 d medical images to be split Segmentation, to obtain the corresponding segmentation result of these images, and by each segmentation result according to the ratio tune zoomed in or out before Whole is original size, finally, segmentation result adjusted is averaged by cutting unit 304 again, as this to be split three Tie up the final segmentation result of medical image.

Optionally, which can be labelled with the 3 d medical images sample of target area by multiple Training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, can also be by the figure As segmenting device is voluntarily trained；I.e. as shown in Figure 3b, which can also include 305 He of the first acquisition unit First training unit 306, as follows:

First acquisition unit 305, for acquiring multiple 3 d medical images samples for being labelled with target area.

First training unit 306, for being instructed according to the 3 d medical images sample to preset semantic segmentation model Practice, obtains two-dimentional parted pattern.

Optionally, which may include the first division subelement, the first computation subunit and the first convergence Subelement, as follows:

First division subelement obtains first for the 3 d medical images sample to be divided into positive sample and negative sample Training sample set.

Wherein, which is the target area in the 3 d medical images sample, which is the 3 D medical figure Region in decent in addition to target area.For example, the positive sample is the 3 D medical figure if the target area is heart Heart area in decent, and negative sample is the region in the 3 d medical images sample than the heart, and so on, etc. Deng.

First computation subunit, the positive sample for being concentrated using preset semantic segmentation model to first training sample It is calculated with negative sample, obtains the first training sample and concentrate the predicted value of each positive sample and the predicted value of negative sample.

Wherein, which can be depending on the demand of practical application, for example, can be by U- The structure of Net network, which improves, to be obtained, specifically, can be by encoder section (the i.e. down-sampling portion in U-Net network Point) residual error network is replaced with, such as ResNet101 network, for details, reference can be made to the embodiments of front, and therefore not to repeat here.

First convergence subelement, for obtain the first training sample concentrate each positive sample true value and negative sample it is true Real value, and according to the true value and predicted value of the true value of each positive sample and predicted value and each negative sample to the semanteme Parted pattern is restrained, and two-dimentional parted pattern is obtained.

Optionally, in order to avoid problem unbalanced between positive negative sample, positive sample quantity is much larger than in negative sample quantity When, when carrying out the backpropagation of convolution, it will can only be worth biggish negative sample error and all positive sample errors carry out instead To propagation, it may be assumed that

The first convergence subelement, specifically can be used for calculating the mistake between the predicted value and true value of each positive sample Difference obtains the corresponding positive sample error of each positive sample；The error between the predicted value and true value of each negative sample is calculated, is obtained To the corresponding negative sample error of each negative sample；The negative sample that screening negative sample error is greater than the set value is as difficult negative sample； The negative sample error of difficult negative sample and all positive sample errors are carried out to the backpropagation of convolution.

Similarly, if positive sample quantity is much larger than negative sample quantity, when carrying out backpropagation to positive sample, can also only will It is worth biggish positive sample error and carries out backpropagation, and so on, therefore not to repeat here.

In addition, it should be noted that, setting value can be depending on the demand of practical application, and therefore not to repeat here.

Optionally, in order to improve training effectiveness, and the problem that alleviation neural metwork training is unstable, to the two dimension point It, can also be by the way of " competition training " when cutting model and being trained, it may be assumed that

First training unit specifically can be used in multiple graphics processors loading the same preset semantic segmentation Model；According to the 3 d medical images sample respectively to the semantic segmentation model in multiple graphics processor to being trained； From semantic segmentation model after the training for selecting accuracy rate optimal in semantic segmentation model after training；By semantic point after the training of selection The model parameter for cutting model is loaded onto all semantic segmentation models, and is returned to execution and distinguished according to the 3 d medical images sample The step of to the semantic segmentation model in multiple graphics processor to being trained, until all semantic segmentation models have been restrained Finish；The semantic segmentation model for selecting accuracy rate optimal from multiple semantic segmentation models after convergence divides mould as two dimension Type.

Wherein, three-dimensional pyramid analysis network can be labelled with voxel type by multiple, and include the three of target object Dimension image pattern training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, It can be voluntarily trained by the image segmentation device；I.e. as shown in Figure 3b, which can also adopt including second Collect unit 307 and the second training unit 308, as follows:

Second acquisition unit 307 is labelled with voxel type for acquiring multiple, and include target object three-dimensional figure it is decent This.

Second training unit 308 obtains three for being trained according to the 3-D image sample to preset analysis network It ties up pyramid and analyzes network.

For example, second training unit 308 may include the second division subelement, the second computation subunit and the second convergence Subelement, as follows:

Second divides subelement, for 3-D image sample to be divided into positive sample and negative sample, obtains the second training sample This collection.

Wherein, which is the target object in the 3-D image sample, which is in the 3-D image sample Region in addition to target object.For example, the positive sample is in the 3-D image sample if the target object is atrium sinistrum Left atrial region, and negative sample is the region in the 3-D image sample in addition to atrium sinistrum, and so on, etc..

Second computation subunit, positive sample for being concentrated using preset analysis network to second training sample and is born Sample is calculated, and is obtained the second training sample and is concentrated the predicted value of each positive sample and the predicted value of negative sample.

Wherein, which can be configured according to the demand of practical application, for example, can be by right The structure of PSPNet network, which improves, to be obtained, specifically, the encoder section in PSPNet network can be replaced with residual Poor network, such as ResNet101 network, and the rest part of PSPNet network is then used as three-dimensional pyramid parted pattern part, tool Body can be found in the embodiment of front, and therefore not to repeat here.

Second convergence subelement, for obtain the second training sample concentrate each positive sample true value and negative sample it is true Real value, and according to the true value and predicted value of the true value of each positive sample and predicted value and each negative sample to the analysis Network is restrained, and three-dimensional pyramid analysis network is obtained.

For example, the second convergence subelement, specifically can be used for calculating between the predicted value and true value of each positive sample Error, obtain the corresponding positive sample error of each positive sample, and calculate between the predicted value and true value of each negative sample Error, obtain the corresponding negative sample error of each negative sample, then, by all negative sample errors and all positive sample errors into The backpropagation of row convolution, for example, all negative sample errors and all positive sample errors can specifically add up, to reach Convergent purpose finally obtains three-dimensional pyramid analysis network.

Optionally, in order to improve training effectiveness, and the problem that alleviation neural metwork training is unstable, with training two dimension point Cut model similarly, when being trained to three-dimensional pyramid analysis network, equally can also using by the way of " competition train ", That is:

Second training unit 308 specifically can be used in multiple graphics processors loading the same preset analysis Network；According to the 3-D image sample respectively to the analysis network in multiple graphics processor to being trained；After training Analyze the post exercise analysis network that selection accuracy rate is optimal in network；The model parameter of the post exercise analysis network of selection is loaded Extremely in all analysis networks, and execution is returned according to the 3-D image sample respectively to the analysis net in multiple graphics processor The step of network is to being trained, until all analysis network convergences finish；It is selected from multiple analysis networks after convergence The optimal analysis network of accuracy rate analyzes network as three-dimensional pyramid.

When it is implemented, above each unit can be used as independent entity to realize, any combination can also be carried out, is made It is realized for same or several entities, the specific implementation of above each unit can be found in the embodiment of the method for front, herein not It repeats again.

From the foregoing, it will be observed that the image segmentation device of the present embodiment gets 3 D medical figure to be split in acquiring unit 301 As after, first the target area in the 3 d medical images can be intercepted using two-dimentional parted pattern by interception unit 302, Candidate region is obtained, the neural network forecast candidate is then analyzed using three-dimensional pyramid by predicting unit 303 and cutting unit 304 again The type of each voxel in region, and the target object in the candidate region is split based on the type of prediction, to obtain Final segmentation result；Since the program first can carry out regional prediction to the 3 d medical images using two-dimentional parted pattern, Therefore carrying out fine segmentation to candidate region using three-dimensional pyramid analysis network again can prevent three-dimensional spatial information from losing It loses, while to ensure that the marginal layer structure of target object can be come out by Accurate Prediction, avoids accounting for due to target object entirely Training is not easy and the generation of the situation of prediction result inaccuracy caused by the ratio of original medical image is too small, Ke Yi great The big accuracy for improving medical image segmentation.

In addition, in the two-dimentional parted pattern of training and three-dimensional pyramid analysis network, for being easy to produce positive negative sample not Balanced scene, the image segmentation device can also use and only carry out backpropagation to " difficult negative sample " and all positive samples Mode is improved, and the stability of training effectiveness and neural metwork training is improved using modes such as " competition are trained ".

In addition, the embodiment of the present invention also provides a kind of network equipment, as shown in figure 4, it illustrates institutes of the embodiment of the present invention The structural schematic diagram for the network equipment being related to, specifically:

The network equipment may include one or more than one processing core processor 401, one or more The components such as memory 402, power supply 403 and the input unit 404 of computer readable storage medium.Those skilled in the art can manage It solves, network equipment infrastructure shown in Fig. 4 does not constitute the restriction to the network equipment, may include more more or fewer than illustrating Component perhaps combines certain components or different component layouts.Wherein:

Processor 401 is the control centre of the network equipment, utilizes various interfaces and connection whole network equipment Various pieces by running or execute the software program and/or module that are stored in memory 402, and are called and are stored in Data in reservoir 402 execute the various functions and processing data of the network equipment, to carry out integral monitoring to the network equipment. Optionally, processor 401 may include one or more processing cores；Preferably, processor 401 can integrate application processor and tune Demodulation processor processed, wherein the main processing operation system of application processor, user interface and application program etc., modulatedemodulate is mediated Reason device mainly handles wireless communication.It is understood that above-mentioned modem processor can not also be integrated into processor 401 In.

Memory 402 can be used for storing software program and module, and processor 401 is stored in memory 402 by operation Software program and module, thereby executing various function application and data processing.Memory 402 can mainly include storage journey Sequence area and storage data area, wherein storing program area can the (ratio of application program needed for storage program area, at least one function Such as sound-playing function, image player function) etc.；Storage data area, which can be stored, uses created number according to the network equipment According to etc..In addition, memory 402 may include high-speed random access memory, it can also include nonvolatile memory, such as extremely A few disk memory, flush memory device or other volatile solid-state parts.Correspondingly, memory 402 can also wrap Memory Controller is included, to provide access of the processor 401 to memory 402.

The network equipment further includes the power supply 403 powered to all parts, it is preferred that power supply 403 can pass through power management System and processor 401 are logically contiguous, to realize management charging, electric discharge and power managed etc. by power-supply management system Function.Power supply 403 can also include one or more direct current or AC power source, recharging system, power failure monitor The random components such as circuit, power adapter or inverter, power supply status indicator.

The network equipment may also include input unit 404, which can be used for receiving the number or character of input Information, and generate keyboard related with user setting and function control, mouse, operating stick, optics or trackball signal Input.

Although being not shown, the network equipment can also be including display unit etc., and details are not described herein.Specifically in the present embodiment In, the processor 401 in the network equipment can be corresponding by the process of one or more application program according to following instruction Executable file be loaded into memory 402, and the application program being stored in memory 402 is run by processor 401, It is as follows to realize various functions:

3 d medical images to be split are obtained, using preset two-dimentional parted pattern to the mesh in the 3 d medical images Mark region (target area includes target object) is intercepted, and candidate region is obtained, then, using preset three-dimensional pyramid The type of each voxel in the neural network forecast candidate region is analyzed, and based on the type of prediction to the target pair in the candidate region As being split, segmentation result is obtained.

Optionally, which can be labelled with the 3 d medical images sample of target area by multiple Training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, can also be by the figure As segmenting device is voluntarily trained；I.e. processor 401 can also run the application program being stored in memory 402, thus Realize following functions:

Multiple 3 d medical images samples for being labelled with target area are acquired, according to the 3 d medical images sample to default Semantic segmentation model be trained, obtain two-dimentional parted pattern.

Similar, three-dimensional pyramid analysis network can be labelled with voxel type by multiple, and include target object 3-D image sample training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, It can also be voluntarily trained by the image segmentation device；I.e. processor 401, which can also be run, is stored in answering in memory 402 With program, to realize following functions:

It acquires multiple and is labelled with voxel type, and include the 3-D image sample of target object, it is decent according to the three-dimensional figure This is trained preset analysis network, obtains three-dimensional pyramid analysis network.

The specific implementation of above each operation can be found in the embodiment of front, and details are not described herein.

From the foregoing, it will be observed that the network equipment of the present embodiment after getting 3 d medical images to be split, can be used first Two-dimentional parted pattern intercepts the target area in the 3 d medical images, obtains candidate region, then again using three-dimensional Pyramid analyzes the type of each voxel in the neural network forecast candidate region, and based on the type of prediction in the candidate region Target object be split, to obtain final segmentation result；Since the program can be first using two-dimentional parted pattern to this 3 d medical images carry out regional prediction, then carry out fine segmentation to candidate region using three-dimensional pyramid analysis network, therefore, It can prevent three-dimensional spatial information from losing, to ensure that it is same that the marginal layer structure of target object can be come out by Accurate Prediction When, training is not easy and prediction result caused by avoiding the ratio for accounting for entire original medical image due to target object too small The generation of the situation of inaccuracy, can greatly improve the accuracy of medical image segmentation.

It will appreciated by the skilled person that all or part of the steps in the various methods of above-described embodiment can be with It is completed by instructing, or relevant hardware is controlled by instruction to complete, which can store computer-readable deposits in one In storage media, and is loaded and executed by processor.

For this purpose, the embodiment of the present invention also provides a kind of storage medium, wherein being stored with a plurality of instruction, which can be located Reason device is loaded, to execute the step in any image partition method provided by the embodiment of the present invention.For example, the instruction Following steps can be executed:

Optionally, which can be labelled with the 3 d medical images sample of target area by multiple Training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, can also be by the figure As segmenting device is voluntarily trained；I.e. following steps can also be performed in the instruction:

Similar, three-dimensional pyramid analysis network can be labelled with voxel type by multiple, and include target object 3-D image sample training forms.After being specifically trained by other equipment, it is supplied to the image segmentation device, alternatively, It can also be voluntarily trained by the image segmentation device；I.e. following steps can also be performed in the instruction:

Wherein, which may include: read-only memory (ROM, Read Only Memory), random access memory Body (RAM, Random Access Memory), disk or CD etc..

By the instruction stored in the storage medium, any image provided by the embodiment of the present invention point can be executed Step in segmentation method, it is thereby achieved that achieved by any image partition method provided by the embodiment of the present invention Beneficial effect is detailed in the embodiment of front, and details are not described herein.

It is provided for the embodiments of the invention a kind of image partition method, device and storage medium above and has carried out detailed Jie It continues, used herein a specific example illustrates the principle and implementation of the invention, and the explanation of above embodiments is only It is to be used to help understand method and its core concept of the invention；Meanwhile for those skilled in the art, according to the present invention Thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be construed as Limitation of the present invention.

Claims

1. a kind of image partition method characterized by comprising

Obtain 3 d medical images to be split；

The target area in the 3 d medical images is intercepted using preset two-dimentional parted pattern, obtains candidate regions Domain, the target area include target object；

2. the method according to claim 1, wherein described use preset two-dimentional parted pattern to the three-dimensional Target area in medical image is intercepted, and candidate region is obtained, comprising:

The 3 d medical images are converted into multiple two-dimensional medical images；

The target area in every two-dimensional medical images is intercepted using preset two-dimentional parted pattern, multiple is obtained and waits melting Close two dimensional image；

Multiple described two dimensional images to be fused are fused to 3-D image, obtain candidate region.

3. according to the method described in claim 2, it is characterized in that, described use preset two-dimentional parted pattern to every two dimension Target area in medical image is intercepted, multiple two dimensional images to be fused are obtained, comprising:

The type of pixel in every two-dimensional medical images is predicted using preset two-dimentional parted pattern；

Boundary point of the target area in two-dimensional medical images is determined according to prediction result；

It is intercepted based on determining boundary point pair target area, obtains multiple two dimensional images to be fused.

4. the method according to claim 1, wherein it is described three-dimensional pyramid analysis network include residual error network and Three-dimensional pyramid parted pattern, using the class of each voxel in candidate region described in preset three-dimensional pyramid analysis neural network forecast Type, comprising:

Feature extraction is carried out to the candidate region by residual error network, obtains the characteristic information of candidate region；

Using three-dimensional pyramid parted pattern, the type of each voxel in the candidate region is predicted according to the characteristic information.

5. according to the method described in claim 4, it is characterized in that, described carry out spy to the candidate region by residual error network Sign is extracted, and the characteristic information of candidate region is obtained, comprising:

The candidate region is rolled up by the first residual error module, the second residual error module and third residual error module of residual error network Product processing；

The candidate region after process of convolution is carried out using the four-infirm difference module and the 5th residual error module of the residual error network empty Hole process of convolution obtains the characteristic information of candidate region.

6. according to the method described in claim 4, it is characterized in that, described carry out spy to the candidate region by residual error network Before sign is extracted, further includes:

Centered on the candidate region, expand default size, candidate region after being expanded outward；

From the region of pre-set dimension is cut after the expansion in candidate region at random as candidate region after optimization；

It is described that feature extraction is carried out to the candidate region by residual error network, obtain the characteristic information of candidate region specifically: Feature extraction, the characteristic information of candidate region after being optimized are carried out to candidate region after the optimization by residual error network.

7. according to the method described in claim 4, it is characterized in that, the three-dimensional pyramid parted pattern includes three-dimensional pyramid Pond layer, up-sampling layer and regularization layer, it is described using three-dimensional pyramid parted pattern, according to characteristic information prediction The type of each voxel in candidate region, comprising:

Using three-dimensional pyramid pond layer, pond is carried out to the candidate region according to the characteristic information；

By up-sampling layer by the size of the output up-sampling of Chi Huahou to Chi Huaqian, and the output after up-sampling is connected It connects；

Regularization is carried out to the output after connection using regularization layer, obtains the class of each voxel in the candidate region Type.

8. the method according to claim 1, wherein it is described based on the type of prediction in the candidate region Target object is split, and obtains segmentation result, comprising:

According to the type of prediction, screening meets the voxel of target object voxel type from the candidate region, obtains candidate body Element collection；

Boundary point of the target object in candidate region is determined according to the candidate voxels collection；

It is split based on target object in candidate region described in determining boundary point pair, obtains segmentation result.

9. method according to any one of claims 1 to 8, which is characterized in that described using preset two-dimentional parted pattern Before being intercepted to the target area in the 3 d medical images, further includes:

Acquire multiple 3 d medical images samples for being labelled with target area；

Preset semantic segmentation model is trained according to the 3 d medical images sample, obtains two-dimentional parted pattern.

10. according to the method described in claim 9, it is characterized in that, it is described according to the 3 d medical images sample to default Semantic segmentation model be trained, obtain two-dimentional parted pattern, comprising:

The 3 d medical images sample is divided into positive sample and negative sample, obtains the first training sample set, the positive sample For the target area in the 3 d medical images sample, the negative sample is that target area is removed in the 3 d medical images sample Region except domain；

The positive sample and negative sample concentrated using preset semantic segmentation model to first training sample are calculated, and are obtained First training sample concentrates the predicted value of each positive sample and the predicted value of negative sample；

It obtains the first training sample and concentrates the true value of each positive sample and the true value of negative sample, and according to each positive sample This true value and the true value and predicted value of predicted value and each negative sample restrain the semantic segmentation model, Obtain two-dimentional parted pattern.

11. according to the method described in claim 10, it is characterized in that, the true value according to each positive sample and pre- The true value and predicted value of measured value and each negative sample restrain the semantic segmentation model, comprising:

The error between the predicted value and true value of each positive sample is calculated, the corresponding positive sample error of each positive sample is obtained；

The error between the predicted value and true value of each negative sample is calculated, the corresponding negative sample error of each negative sample is obtained；

The negative sample that screening negative sample error is greater than the set value is as difficult negative sample；

The negative sample error of difficult negative sample and all positive sample errors are carried out to the backpropagation of convolution.

12. according to the method described in claim 9, it is characterized in that, it is described according to the 3 d medical images sample to default Semantic segmentation model be trained, obtain two-dimentional parted pattern, comprising:

The same preset semantic segmentation model is loaded in multiple graphics processors；

According to the 3 d medical images sample respectively to the semantic segmentation model in the multiple graphics processor to instructing Practice；

From semantic segmentation model after the training for selecting accuracy rate optimal in semantic segmentation model after training；

The model parameter of semantic segmentation model after the training of selection is loaded onto all semantic segmentation models, and returns to execution root According to the 3 d medical images sample respectively to the semantic segmentation model in the multiple graphics processor to the step being trained Suddenly, until the convergence of all semantic segmentation models finishes；

The semantic segmentation model for selecting accuracy rate optimal from multiple semantic segmentation models after convergence is divided as two dimension Model.

13. method according to any one of claims 1 to 8, which is characterized in that described using preset three-dimensional pyramid point It analyses in candidate region described in neural network forecast before the type of each voxel, further includes:

It acquires multiple and is labelled with voxel type, and include the 3-D image sample of target object；

Preset analysis network is trained according to the 3-D image sample, obtains three-dimensional pyramid analysis network.

14. according to the method for claim 13, which is characterized in that it is described according to the 3-D image sample to preset point Analysis network is trained, and obtains three-dimensional pyramid analysis network, comprising:

3-D image sample is divided into positive sample and negative sample, obtains the second training sample set, the positive sample is described three The target object in image pattern is tieed up, the negative sample is the region in the 3-D image sample in addition to target object；

The positive sample and negative sample concentrated using preset analysis network to second training sample are calculated, and obtain second Training sample concentrates the predicted value of each positive sample and the predicted value of negative sample；

It obtains the second training sample and concentrates the true value of each positive sample and the true value of negative sample, and according to each positive sample This true value and the true value and predicted value of predicted value and each negative sample restrain the analysis network, obtain Three-dimensional pyramid analyzes network.

15. according to the method for claim 13, which is characterized in that it is described according to the 3-D image sample to preset point Analysis network is trained, and three-dimensional pyramid analyzes network, comprising:

The same preset analysis network is loaded in multiple graphics processors；

According to the 3-D image sample respectively to the analysis network in the multiple graphics processor to being trained；

The post exercise analysis network for selecting accuracy rate optimal from post exercise analysis network；

The model parameter of the post exercise analysis network of selection is loaded onto all analysis networks, and returns to execution according to described three The step of image pattern is respectively to the analysis network in the multiple graphics processor to being trained is tieed up, until all analysis nets Network convergence finishes；

The analysis network for selecting accuracy rate optimal from multiple analysis networks after convergence analyzes net as three-dimensional pyramid Network.

16. a kind of image segmentation device characterized by comprising

Acquiring unit, for obtaining 3 d medical images to be split；

Interception unit, for being cut using preset two-dimentional parted pattern to the target area in the 3 d medical images It takes, obtains candidate region, the target area includes target object；

Predicting unit, for the class using each voxel in candidate region described in preset three-dimensional pyramid analysis neural network forecast Type；

Cutting unit is split the target object in the candidate region for the type based on prediction, obtains segmentation knot Fruit.

17. a kind of storage medium, which is characterized in that the storage medium is stored with a plurality of instruction, and described instruction is suitable for processor It is loaded, the step in 1 to 15 described in any item image partition methods is required with perform claim.