CN109919212A - The multi-dimension testing method and device of tumour in digestive endoscope image - Google Patents
The multi-dimension testing method and device of tumour in digestive endoscope image Download PDFInfo
- Publication number
- CN109919212A CN109919212A CN201910142912.5A CN201910142912A CN109919212A CN 109919212 A CN109919212 A CN 109919212A CN 201910142912 A CN201910142912 A CN 201910142912A CN 109919212 A CN109919212 A CN 109919212A
- Authority
- CN
- China
- Prior art keywords
- digestive endoscope
- detected
- endoscope image
- image
- classification results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
- Endoscopes (AREA)
Abstract
The embodiment of the invention discloses the multi-dimension testing methods and device of tumour in a kind of digestive endoscope image.This method comprises: carrying out tumor imaging label to each digestive endoscope image sample, and according to preset tumour property grade, tumour property grade mark is carried out to the digestive endoscope image sample for being marked as tumor imaging;To carry out the digestive endoscope image sample after tumour property grade mark as foundation, training obtains digestive endoscope Image detection model;Obtain scope image to be detected;According to digestive endoscope Image detection model, the testing result of scope image to be detected is obtained, testing result includes one of the first classification results, the second classification results and multiple dimensioned tumor region testing result or much information.Implement the embodiment of the present invention, can reduce label cost, while realizing and being accurately positioned to the tumor region in digestive endoscope image.
Description
Technical field
The present invention relates to medical imaging processing technology fields, and in particular to more rulers of tumour in a kind of digestive endoscope image
Spend detection method and device.
Background technique
According to the statistics of the World Health Organization, three kinds of gastric cancer, colorectal cancer and cancer of the esophagus most important alimentary tract cancers arrange root
In high-incidence the first six name of cancer in the whole world, divide column second, the 4th and the 5th in China, the far super lung cancer of summation.Moreover, these three are digested
Road cancer is in rising trend in the disease incidence of China.Since alimentary tract cancer is disease of digestive tract (such as tumor in digestive tract)
On the basis of lesion and come, if tumor in digestive tract early stage be found, cure rate is high, so tumor in digestive tract early screening
It is significant.
And the necessary means that tumor in digestive tract early sieves are digestive endoscopies, although digestive endoscopy China
Through universal, but due to the case where diagnostic level of situation of all-level hospitals doctor is irregular, and the early diagnosis of tumor in digestive tract is early controlled and few,
And cause tumor in digestive tract early diagnostic rate low, many tumor in digestive tract are in middle evening without unified effective screening mode in industry
Phase is just found, and for lesion at alimentary tract cancer, the rate that causes death is high when discovery, makes people very distressed.
But, it allows people to feel more comfort, is that support has both by force with big data with the quick emergence of artificial intelligence technology
The artificial intelligence technology of big computing capability and learning ability brings new development to medical domain, and artificial intelligence technology is constantly real
Application innovation on the directions such as present medical image analysis, auxiliary diagnosis, intelligent sound, cancer morning sieve and body-building biotechnology.
It using artificial intelligence technology assist digestion road endoscopy is logical to carry out the technological means that tumor in digestive tract early sieves currently, existing
It crosses and model is trained using the digestive endoscope image sample of a large amount of handmarkings, and using in model identification alimentary canal
The approximate location and property of tumour in mirror image solve the problems, such as that early screening is difficult, early diagnostic rate is low.
But the tumor region in digestive endoscope image is accurately positioned in order to realize, it needs manually to a large amount of
Tumor region in digestive endoscope image sample carries out precise marking, that is, needs manually accurately to draw a large amount of digestive endoscope shadows
The profile of decent middle tumour could complete the training of model.Therefore, it manually accurately draws in a large amount of alimentary canals in the prior art
The profile of tumour is very time-consuming and laborious in mirror image sample, and label cost is very big.
Summary of the invention
In view of the foregoing drawbacks, the embodiment of the invention discloses a kind of multiple scale detecting sides of tumour in digestive endoscope image
Method and device can reduce label cost, while realizing and being accurately positioned to the tumor region in digestive endoscope image.
First aspect of the embodiment of the present invention discloses a kind of multi-dimension testing method of tumour in digestive endoscope image, packet
It includes:
Tumor imaging label is carried out to each digestive endoscope image sample, and according to preset tumour property grade, it is right
The digestive endoscope image sample for being marked as tumor imaging carries out tumour property grade mark;
To carry out the digestive endoscope image sample after the tumour property grade mark as foundation, training obtains alimentary canal
Scope Image detection model;
Obtain scope image to be detected;
According to the digestive endoscope Image detection model, the testing result of the scope image to be detected is obtained, it is described
Testing result includes one of the first classification results, the second classification results, multiple dimensioned tumor region testing result or a variety of letters
Breath;Wherein, first classification results are for describing whether the scope image to be detected is digestive endoscope image, and described the
Two classification results are for describing normal digestive endoscope image, benign lesion or malignant change.
As an alternative embodiment, in first aspect of the embodiment of the present invention, it is described to obtain scope shadow to be detected
Picture, comprising:
The detection request and scope image to be detected that real-time reception endoscopy equipment is sent;
And it is described according to the digestive endoscope Image detection model, obtain the detection of the scope image to be detected
As a result after, the method also includes:
The testing result is sent to the endoscopy equipment.
As an alternative embodiment, in first aspect of the embodiment of the present invention, the digestive endoscope image inspection
Surveying model includes depth convolution baseline neural network, more classification branching networks and multiple scale detecting branching networks;It is described according to institute
Digestive endoscope Image detection model is stated, the testing result of the scope image to be detected is obtained, comprising:
The scope image to be detected is inputted into the depth convolution baseline neural network, in the depth convolution baseline mind
Multilayer process of convolution is carried out to the scope image to be detected in network, obtains the corresponding feature of the scope image to be detected
Figure;
By the characteristic pattern input branching networks of classifying, to the characteristic pattern in more classification branching networks more
More classification processings are carried out, first classification results and second classification results are obtained;
If second classification results are for describing digestive endoscope image benign lesion or malignant change, by the feature
Figure inputs the multiple scale detecting branching networks, carries out tumor area to the characteristic pattern in the multiple scale detecting branching networks
Domain detection processing, obtains the multiple dimensioned tumor region testing result, and according to first classification results, second point described
Class result and the multiple dimensioned tumor region testing result obtain the testing result of the scope image to be detected;
If second classification results are normal for describing digestive endoscope image, according to first classification results and institute
State the testing result that the second classification results obtain the scope image to be detected.
As an alternative embodiment, in first aspect of the embodiment of the present invention, it is described to input the characteristic pattern
The multiple scale detecting branching networks carry out tumor region detection to the characteristic pattern in the multiple scale detecting branching networks
Processing obtains the multiple dimensioned tumor region testing result, comprising:
The characteristic pattern is inputted into the multiple scale detecting branching networks, by institute in the multiple scale detecting branching networks
It states characteristic pattern and is divided into several super-pixel;
According to second classification results, several target super-pixel are determined, from several described super-pixel to obtain
Obtain the multiple dimensioned tumor region testing result.
As an alternative embodiment, in first aspect of the embodiment of the present invention, the depth convolution baseline nerve
Network includes that depth separates convolutional layer, expansion convolutional layer and dimensionality reduction layer;It is described will be described in the scope image to be detected input
Depth convolution baseline neural network carries out multilayer to the scope image to be detected in the depth convolution baseline neural network
Process of convolution obtains the corresponding characteristic pattern of the scope image to be detected, comprising:
The scope image to be detected is inputted into the depth and separates convolutional layer, is separated in convolutional layer in the depth
Depth is carried out to the scope image to be detected and separates process of convolution, obtains initial characteristics figure;
The initial characteristics figure is inputted into the expansion convolutional layer, the volume of different scale is used in the expansion convolutional layer
Product checks the initial characteristics figure and carries out concurrent operation processing, obtains multidimensional characteristic figure;
The initial characteristics figure is inputted into the dimensionality reduction layer, the multidimensional characteristic figure is carried out in the dimensionality reduction layer one-dimensional
Change processing obtains the corresponding characteristic pattern of the scope image to be detected.
As an alternative embodiment, in first aspect of the embodiment of the present invention, it is described to carry out the tumprigenicity
Digestive endoscope image sample after matter grade mark is foundation, and training obtains digestive endoscope Image detection model, comprising:
The data of digestive endoscope image sample after the progress tumour property grade mark are pre-processed, are obtained
Training data;
Depth convolutional neural networks are trained using the training data, obtain digestive endoscope Image detection mould
Type.
Second aspect of the embodiment of the present invention discloses a kind of multiple scale detecting device of tumour in digestive endoscope image, packet
It includes:
Marking unit, for carrying out tumor imaging label to each digestive endoscope image sample, and according to preset swollen
Tumor property grade carries out tumour property grade mark to the digestive endoscope image sample for being marked as tumor imaging;
Training unit, for carry out the digestive endoscope image sample after the tumour property grade mark as foundation,
Training obtains digestive endoscope Image detection model;
First acquisition unit, for obtaining scope image to be detected;
Second acquisition unit, for obtaining the scope shadow to be detected according to the digestive endoscope Image detection model
The testing result of picture, the testing result include the first classification results, the second classification results and multiple dimensioned tumor region detection knot
One of fruit or much information;Wherein, first classification results are for describing whether the scope image to be detected is to disappear
Change road scope image, second classification results are for describing normal digestive endoscope image, benign lesion or malignant change.
As an alternative embodiment, in second aspect of the embodiment of the present invention, the first acquisition unit, specifically
The detection request sent for real-time reception endoscopy equipment and scope image to be detected;
And described device further include:
Transmission unit, for, according to the digestive endoscope Image detection model, obtaining institute in the second acquisition unit
After the testing result for stating scope image to be detected, the testing result is sent to the endoscopy equipment.
As an alternative embodiment, in second aspect of the embodiment of the present invention, the digestive endoscope image inspection
Surveying model includes depth convolution baseline neural network, more classification branching networks and multiple scale detecting branching networks;Described second obtains
The unit is taken to include:
Convolution subelement, for the scope image to be detected to be inputted the depth convolution baseline neural network, in institute
It states in depth convolution baseline neural network and multilayer process of convolution is carried out to the scope image to be detected, obtain described to be detected interior
The corresponding characteristic pattern of mirror image;
More classification subelements, for the characteristic pattern to be inputted the branching networks of classifying more, in the branches of classifying more
More classification processings are carried out to the characteristic pattern in network, obtain first classification results and second classification results;
Multiple scale detecting subelement, for being used to describe digestive endoscope image benign lesion in second classification results
Or when malignant change, the characteristic pattern is inputted into the multiple scale detecting branching networks, in the multiple scale detecting branching networks
In tumor region detection processing is carried out to the characteristic pattern, obtain the multiple dimensioned tumor region testing result;
First obtain subelement, for second classification results for describe digestive endoscope image benign lesion or
When malignant change, according to first classification results, second classification results and the multiple dimensioned tumor region testing result
Obtain the testing result of the scope image to be detected;
Second obtain subelement, for second classification results for describe digestive endoscope image it is normal when, root
The testing result of the scope image to be detected is obtained according to first classification results and second classification results.
As an alternative embodiment, in second aspect of the embodiment of the present invention, the multiple scale detecting subelement
Include:
Division module, for being used to describe digestive endoscope image benign lesion or malignant diseases in second classification results
When change, the characteristic pattern is inputted into the multiple scale detecting branching networks, it will be described in the multiple scale detecting branching networks
Characteristic pattern is divided into several super-pixel;
Determining module, for determining several mesh from several described super-pixel according to second classification results
Super-pixel is marked, to obtain the multiple dimensioned tumor region testing result.
As an alternative embodiment, in second aspect of the embodiment of the present invention, the depth convolution baseline nerve
Network includes that depth separates convolutional layer, expansion convolutional layer and dimensionality reduction layer;The convolution subelement includes:
Depth separates convolution module, separates convolutional layer for the scope image to be detected to be inputted the depth,
It is separated in convolutional layer in the depth and the separable process of convolution of depth is carried out to the scope image to be detected, obtained initial special
Sign figure;
Convolution module is expanded, for the initial characteristics figure to be inputted the expansion convolutional layer, in the expansion convolutional layer
The middle convolution kernel using different scale carries out concurrent operation processing to the initial characteristics figure, obtains multidimensional characteristic figure;
Dimensionality reduction module, for the initial characteristics figure to be inputted the dimensionality reduction layer, to the multidimensional in the dimensionality reduction layer
Characteristic pattern carries out one-dimensional processing, obtains the corresponding characteristic pattern of the scope image to be detected.
As an alternative embodiment, in second aspect of the embodiment of the present invention, the training unit includes:
Subelement is handled, for the data to the digestive endoscope image sample after the progress tumour property grade mark
It is pre-processed, obtains training data;
Training subelement obtains alimentary canal for being trained using the training data to depth convolutional neural networks
Scope Image detection model.
The third aspect of the embodiment of the present invention discloses a kind of multiple scale detecting device of tumour in digestive endoscope image, packet
It includes:
It is stored with the memory of executable program code;
The processor coupled with the memory;
The processor calls the executable program code stored in the memory, executes the embodiment of the present invention the
The multi-dimension testing method of tumour in a kind of digestive endoscope image disclosed in one side.
Fourth aspect of the embodiment of the present invention discloses a kind of computer readable storage medium, stores computer program, wherein
The computer program executes computer in a kind of digestive endoscope image disclosed in first aspect of the embodiment of the present invention to swell
The multi-dimension testing method of tumor.
The 5th aspect of the embodiment of the present invention discloses a kind of computer program product, when the computer program product is calculating
When being run on machine, so that the computer executes some or all of any one method of first aspect step.
The aspect of the embodiment of the present invention the 6th disclose a kind of using distribution platform, and the application distribution platform is for publication calculating
Machine program product, wherein when the computer program product is run on computers, so that the computer executes first party
Some or all of any one method in face step.
Compared with prior art, the embodiment of the present invention has the advantages that
In the embodiment of the present invention, by carrying out tumor imaging label to each digestive endoscope image sample, and according to pre-
If tumour property grade, to be marked as tumor imaging digestive endoscope image sample carry out tumour property grade mark,
To carry out the digestive endoscope image sample after tumour property grade mark as foundation, training obtains digestive endoscope Image detection
Model is based on this, after obtaining scope image to be detected, can be obtained to be checked according to the digestive endoscope Image detection model
Survey the testing result of scope image, wherein testing result includes the first classification results, the second classification results and multiple dimensioned tumor area
One of domain testing result or much information;Wherein, the first classification results are for describing whether scope image to be detected is to disappear
Change road scope image, the second classification results are for describing normal digestive endoscope image, benign lesion or malignant change, it is seen then that
By training digestive endoscope Image detection model, tumor region is completed using digestive endoscope Image detection model and is positioned, no
The profile for needing manually accurately to draw tumour in digestive endoscope image sample can be achieved with to swollen in digestive endoscope image
Tumor region is accurately positioned, and therefore, implements the embodiment of the present invention, can reduce label cost, while realizing in alimentary canal
Tumor region in mirror image is accurately positioned.
Detailed description of the invention
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to needed in the embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for ability
For the those of ordinary skill of domain, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is the process of the multi-dimension testing method of tumour in a kind of digestive endoscope image disclosed by the embodiments of the present invention
Schematic diagram;
Fig. 2 is the stream of the multi-dimension testing method of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
Journey schematic diagram;
Fig. 3 is the structure of the multiple scale detecting device of tumour in a kind of digestive endoscope image disclosed by the embodiments of the present invention
Schematic diagram;
Fig. 4 is the knot of the multiple scale detecting device of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
Structure schematic diagram;
Fig. 5 is the knot of the multiple scale detecting device of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
Structure schematic diagram;
Fig. 6 is the experiment number that digestive endoscope Image detection model disclosed by the embodiments of the present invention exports the first classification results
According to figure;
Fig. 7 is the experiment number that digestive endoscope Image detection model disclosed by the embodiments of the present invention exports the second classification results
According to figure;
Fig. 8 is that digestive endoscope Image detection model disclosed by the embodiments of the present invention exports multiple dimensioned tumor region detection knot
The experimental data figure of fruit.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that the described embodiment is only a part of the embodiment of the present invention, instead of all the embodiments.Based on this
Embodiment in invention, every other reality obtained by those of ordinary skill in the art without making creative efforts
Example is applied, shall fall within the protection scope of the present invention.
It should be noted that term " first ", " second ", " third " etc. in description and claims of this specification
It is to be not use to describe a particular order for distinguishing different objects.The term " includes " of the embodiment of the present invention and " having "
And their any deformation, it is intended that cover it is non-exclusive include, for example, containing the mistake of a series of steps or units
Journey, method, system, product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include unclear
Other step or units that ground is listed or intrinsic for these process, methods, product or equipment.
The embodiment of the invention discloses the multi-dimension testing methods and device of tumour in a kind of digestive endoscope image, can
Reduction flag cost, at the same realize the tumor region in digestive endoscope image is accurately positioned, below in conjunction with attached drawing into
Row detailed description.
Embodiment one
Referring to Fig. 1, Fig. 1 is the multiple scale detecting of tumour in a kind of digestive endoscope image disclosed by the embodiments of the present invention
The flow diagram of method.Wherein, method described in the embodiment of the present invention is suitable for medical inspection device, medical treatment detection device
Or medical imaging processing equipment etc., the present invention is not especially limited.As shown in Figure 1, in the digestive endoscope image tumour it is more
Size measurement method may comprise steps of:
101, tumor imaging label is carried out to each digestive endoscope image sample, and according to preset tumour property etc.
Grade carries out tumour property grade mark to the digestive endoscope image sample for being marked as tumor imaging.
Wherein, digestive endoscope image sample can be acquired according to true tumor in digestive tract case.
In the embodiment of the present invention, preset tumour property grade can be benign or malignant;Tumor imaging label includes mark
Remember whether each digestive endoscope image sample is tumor imaging, i.e., whether contains tumor information.
The embodiment of the present invention is based on Weakly supervised learning art, by the way that image level is marked, training digestive endoscope shadow
As detection model learns automatically, realization is accurately positioned the tumor region in digestive endoscope image, i.e., by using thick
The label of granularity can allow model to learn fine-grained feature automatically, avoid carrying out artificial precise marking.
102, to carry out the digestive endoscope image sample after tumour property grade mark as foundation, training obtains alimentary canal
Scope Image detection model.
As an alternative embodiment, step 102 may include: to the digestion after progress tumour property grade mark
The data of road scope image sample are pre-processed, and training data is obtained;Using training data to depth convolutional neural networks into
Row training, obtains digestive endoscope Image detection model.
Wherein, pretreatment may include to after label digestive endoscope image sample carry out data enhancing, scaling, at random
The sequence of operations such as rotation and filling.
Implement the embodiment, be capable of increasing data volume, prevent over-fitting, at the same can be improved model generalization ability and
Recognition accuracy.
As another optional embodiment, after executing step 102, backpropagation can be used
(Backpropagation algorithm, BP) algorithm is adjusted digestive endoscope Image detection model.Specifically, from
Several digestive endoscope image samples are obtained in training data, calculate the difference of reality output and ideal output;According to the difference
Value, by the weight of the method backpropagation adjustment weight matrix of Minimum Polarization error, to adjust digestive endoscope Image detection model.
As an example it is assumed that getting a digestive endoscope image sample (X, Yp);Wherein, X is input, and Yp is ideal
Output;X is inputted into digestive endoscope Image detection model, obtains corresponding reality output Op;Calculate reality output Op and ideal
Export the difference of Yp;Disappeared according to the difference by the weight of the method backpropagation adjustment weight matrix of Minimum Polarization error with adjustment
Change road scope Image detection model.
Implement the embodiment, can be improved the recognition accuracy of model.
Optionally, after training obtains digestive endoscope Image detection model, available validation data set, the verifying
Data set is used to verify the recognition accuracy of above-mentioned digestive endoscope Image detection model.Optionally, it can also use at image
Reason device (Graphics Processing Unit, GPU) carries out acceleration operation, improves treatment effeciency and real-time.In practice
It was found that above-mentioned digestive endoscope Image detection model is trained on GPU, and it is 12 that batch processing size parameter, which is arranged, iteration
When number is 100, recognition accuracy is higher, and up to 98%, details experimental data please refers to Fig. 6~Fig. 8.
Referring to Fig. 6, Fig. 6 is the first classification of digestive endoscope Image detection model output knot disclosed by the embodiments of the present invention
The experimental data figure of fruit.Wherein, abscissa is the number of iterations of model, and ordinate is the accuracy rate of the first classification results.As it can be seen that
Model is relatively stable, and accuracy rate is up to 98%.
Referring to Fig. 7, Fig. 7 is the second classification of digestive endoscope Image detection model output knot disclosed by the embodiments of the present invention
The experimental data figure of fruit.Wherein, abscissa is the number of iterations of model, and ordinate is the accuracy rate of the second classification results.As it can be seen that
Model is relatively stable, and accuracy rate is up to 95%.
Referring to Fig. 8, Fig. 8 is that digestive endoscope Image detection model disclosed by the embodiments of the present invention exports multiple dimensioned tumour
The experimental data figure of area detection result.Wherein, abscissa is the number of iterations of model, and ordinate is average hands over and than (Mean
Intersection over Union, MIoU), for evaluating the accuracy rate of multiple dimensioned tumor region testing result.As it can be seen that mould
Type is relatively stable, and MIoU reaches 0.64.
103, scope image to be detected is obtained.
104, according to digestive endoscope Image detection model, the testing result of scope image to be detected, testing result are obtained
Including one of the first classification results, the second classification results and multiple dimensioned tumor region testing result or much information;Wherein,
For describing whether scope image to be detected is digestive endoscope image, the second classification results disappear first classification results for describing
Change normal scope image in road, benign lesion or malignant change.
As an alternative embodiment, digestive endoscope Image detection model may include three network branches, the
One network branches are for exporting the first classification results, and the second network branches are for exporting the second classification results, third network branches
For exporting multiple dimensioned tumor region testing result;It also may include two network branches, first network branch is for exporting the
One classification results and/or the second classification results, the second network branches are for exporting multiple dimensioned tumor region testing result.In this reality
It applies in example, digestive endoscope Image detection model may include three network branches, but in some other possible embodiment,
Digestive endoscope Image detection model may include two network branches, and the present invention is not especially limited.Wherein, in the instruction of model
Practice the stage, binary_crossentropy cross entropy loss function has been used to each network branches in model respectively, it is in parallel
The loss function of all-network branch has been closed, multitask joint training is carried out.
In the embodiment of the present invention, it will be understood that be detected interior if scope image to be detected is not digestive endoscope image
The testing result of mirror image only includes the first classification results, and the first classification results do not disappear for describing scope image to be detected
Change road scope image.
Method described in Fig. 1, by carrying out tumor imaging label to each digestive endoscope image sample, and according to pre-
If tumour property grade, to be marked as tumor imaging digestive endoscope image sample carry out tumour property grade mark,
To carry out the digestive endoscope image sample after tumour property grade mark as foundation, training obtains digestive endoscope Image detection
Model is based on this, after obtaining scope image to be detected, can be obtained to be checked according to the digestive endoscope Image detection model
Survey the testing result of scope image.As it can be seen that being examined by training digestive endoscope Image detection model using digestive endoscope image
It surveys model and completes tumor region positioning, do not need the profile for manually accurately drawing tumour in digestive endoscope image sample, energy
Realization is accurately positioned the tumor region in digestive endoscope image, therefore, implements the embodiment of the present invention, can reduce mark
Remember cost, while realizing and the tumor region in digestive endoscope image is accurately positioned.
In addition to this, implement the embodiment of the present invention, by carrying out multitask joint to digestive endoscope Image detection model
Study, makes digestive endoscope Image detection model multi output model, can be improved the robustness and robustness of model, can not only
Enough realize is accurately positioned the tumor region in digestive endoscope image, moreover it is possible to carry out more points to scope image to be detected
Class.
Embodiment two
Referring to Fig. 2, Fig. 2 is the multiple dimensioned inspection of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
The flow diagram of survey method.Wherein, digestive endoscope Image detection model includes depth convolution baseline neural network, more classification
Branching networks and multiple scale detecting branching networks.As shown in Fig. 2, in the digestive endoscope image tumour multi-dimension testing method
It may comprise steps of:
201~202.Wherein, step 201~202 are identical as step 101~102 described in embodiment one, the present invention
Embodiment repeats no more.
203, the detection request and scope image to be detected that real-time reception endoscopy equipment is sent.
In the embodiment of the present invention, server-side-client platform can be built, server-side is medical imaging processing equipment, visitor
Family end is endoscopy equipment used in doctor.Based on this, endoscopy equipment sends to medical imaging processing equipment and detects
Request, and scope image to be detected to be treated is real-time transmitted to medical imaging processing equipment in the form of video, medical treatment
Image processing arrangement calls alimentary canal after the detection request and scope image to be detected for receiving the transmission of endoscopy equipment
Scope Image detection model handles scope image to be detected, and processing result is returned to endoscopy equipment, scope
Check equipment can real-time display processing result, can help doctor improve diagnosis efficiency, realize real-time diagnosis.
204, scope image to be detected is inputted into depth convolution baseline neural network, in depth convolution baseline neural network
Multilayer process of convolution is carried out to scope image to be detected, obtains the corresponding characteristic pattern of scope image to be detected.
As an alternative embodiment, depth convolution baseline neural network may include depth separate convolutional layer,
Convolutional layer and dimensionality reduction layer are expanded, this is based on, step 204 may include: that scope image to be detected input depth is separated convolution
Layer separates in convolutional layer in depth and carries out the separable process of convolution of depth to scope image to be detected, obtains initial characteristics figure;
Initial characteristics figure is inputted into expansion convolutional layer, initial characteristics figure is carried out using the convolution kernel of different scale in expansion convolutional layer
Concurrent operation processing, obtains multidimensional characteristic figure;Initial characteristics figure is inputted into dimensionality reduction layer, multidimensional characteristic figure is carried out in dimensionality reduction layer
One-dimensional processing, obtains the corresponding characteristic pattern of scope image to be detected.
Wherein, it may include the layer-by-layer convolution of point-by-point convolution sum that depth, which separates convolutional layer,.Specifically, 1 × 1 convolution kernel is first used
Point-by-point convolution (also known as channel fusion) is carried out to multiple channels, 3 × 3 convolution kernels is reused and each channel is successively rolled up respectively
Product.
Implement the embodiment, can reduce parameter amount, improve arithmetic speed, while allowing the correlation in model learning space
With the correlation of interchannel, the detection performance of model is improved.
It should be noted that spy can be expanded under the premise of not sacrificial features spatial resolution using expansion convolutional layer
Sign receives open country, merges the convolution feature of the tumor region of different scale, is precisely identified with the tumor region to different scale.
Wherein, expansion convolutional layer can be expansion convolution (Atrous SpatialPyramid Pooling, ASPP) module.
It is appreciated that the output of ASPP module is multidimensional characteristic figure, in order to which multidimensional characteristic figure is inputted branched networks of classifying more
Network carries out more classification processings, needs to carry out dimension-reduction treatment, multidimensional characteristic figure is stretched as one-dimensional characteristic figure.Wherein, dimensionality reduction layer
It can use Flatten function and carry out dimensionality reduction operation.
205, characteristic pattern is inputted into branching networks of classifying more, characteristic pattern is carried out at more classification in mostly classification branching networks
Reason obtains the first classification results and the second classification results.
In the embodiment of the present invention, branching networks of classifying can be the full articulamentum in neural network more, in the full articulamentum
It is middle to use sigmoid function as activation primitive, Nonlinear Mapping is done to characteristic pattern, output is mapped to the probability point of [0,1]
In cloth, whether to be respectively that digestive endoscope image, digestive endoscope image be normal or benign lesion to scope image to be detected
Or malignant change is classified.
Optionally, sigmoid function is being used as activation primitive, before doing Nonlinear Mapping to characteristic pattern, can adopt
With specification layer (BatchNormalization, BN), characteristic pattern is normalized, model is allowed to restrain faster, is improved
The stability of model, while reducing the training time of model.
As an alternative embodiment, step 205 may include: that characteristic pattern is inputted branching networks of classifying more,
The first probability value that scope image to be detected is digestive endoscope image is obtained in more classification branching networks;Judge first probability
Whether value reaches predetermined probabilities threshold value;If first probability value is not up to predetermined probabilities threshold value, obtain for describing in be detected
Mirror image is not the first classification results of digestive endoscope image;If first probability value reaches predetermined probabilities threshold value, obtains and use
In the first classification results that description scope image to be detected is digestive endoscope image, and, judge the digestive endoscope image
It is whether normal;If normal, it obtains for describing normal second classification results of digestive endoscope image;Obtaining if abnormal, should
Second probability value of digestive endoscope image benign lesion;When second probability value reaches predetermined probabilities threshold value, acquisition is used for
Second classification results of digestive endoscope image malignant change are described;When second probability value is not up to predetermined probabilities threshold value,
Obtain the second classification results for describing digestive endoscope image benign lesion.
As an example it is assumed that predetermined probabilities threshold value is 50%, in mostly classification branching networks it is accessed it is to be detected in
Mirror image is that the first probability value of digestive endoscope image is 60%, it should be apparent that, the first probability value, which reaches, (to be greater than or waits
In) predetermined probabilities threshold value, then determine scope image to be detected for digestive endoscope image.If the digestive endoscope image is not just
Often, and the second probability value of malignant change is 40%, and the second probability value is less than predetermined probabilities threshold value at this time, then determines in alimentary canal
Mirror image benign lesion.
Implement the embodiment, by can be improved multiple scale detecting effect to the more classification of scope image to be detected progress,
The rate of precision of tumor region positioning can be also improved simultaneously.
It is appreciated that the scope image to be detected that endoscopy equipment is taken before not entering alimentary canal after unlatching
It is not digestive endoscope image, therefore the identification of model can be interfered.So the embodiment of the present invention is to scope to be detected
Image is classified, if the first classification results for describe scope image to be detected not to be digestive endoscope image, without pair
Scope image to be detected is further processed, and the testing result of scope image to be detected is directly obtained according to the first classification results.
206, it is normal to judge whether the second classification results are used to describe digestive endoscope image.If so, execute step 207 and
Step 210;Conversely, executing step 208~210.
207, the testing result of scope image to be detected is obtained according to the first classification results and the second classification results.
208, characteristic pattern is inputted into multiple scale detecting branching networks, characteristic pattern is carried out in multiple scale detecting branching networks
Multiple scale detecting processing, obtains multiple dimensioned tumor region testing result.
As an alternative embodiment, step 208 may include: that characteristic pattern is inputted multiple scale detecting branched network
Characteristic pattern is divided into several super-pixel in multiple scale detecting branching networks by network;According to the second classification results, from several
Several target super-pixel are determined in super-pixel, to obtain multiple dimensioned tumor region testing result.
In the embodiment of the present invention, target super-pixel is the region containing tumor information in scope image to be detected, and more rulers
Degree tumor region testing result includes the tag along sort of several target super-pixel, which is for describing the super-pixel
No is the region containing tumor information.
If the second classification results are for describing digestive endoscope image benign lesion or malignant change, then scope to be detected
Include at least one target super-pixel in image, therefore can detect the mesh containing tumor information according to the second classification results
Super-pixel is marked, to obtain multiple dimensioned tumor region testing result.
Specifically, multi-instance learning (Multiple Instance is used in multiple scale detecting branching networks
Learning, MIL) algorithm, using super-pixel as example packet, by the pixel in super-pixel as an example, utilizing the maximum pond in the whole world
Change the maximum value that layer obtains all pixels in each super-pixel, to obtain the tag along sort of each super-pixel.According to this
A little tag along sorts can recognize the tumor region in scope image to be detected, obtain multiple dimensioned tumor region testing result, thus
Realize the accurate positioning to tumor region in scope image to be detected.
It is appreciated that super-pixel is the set of some vicinity points, position of these pixels in characteristic pattern is compared
It is close, and meet certain characteristic, the process for constructing super-pixel belongs to a kind of simple image segmentation, and each cut zone is just
As super-pixel, i.e., each super-pixel corresponds to some region in characteristic pattern.By the way that super-pixel is replaced these pixels
Point carries out image procossing, can reduce data complexity, and can generate preferable preliminary clusters result.
209, it is obtained in be detected according to the first classification results, the second classification results and multiple dimensioned tumor region testing result
The testing result of mirror image.
210, it will test result and be sent to endoscopy equipment.
As an alternative embodiment, more new demand servicing can be built based on Tensorflow Serving open source library
Device, for being called to trained digestive endoscope Image detection model and Version Control is at any time to digestive endoscope shadow
As detection model improves and updates.
Implement method described in Fig. 2, by training digestive endoscope Image detection model, utilizes digestive endoscope image
Detection model completes tumor region positioning, does not need the profile for manually accurately drawing tumour in digestive endoscope image sample, just
It is able to achieve and the tumor region in digestive endoscope image is accurately positioned, therefore, implement the embodiment of the present invention, can reduce
Cost is marked, while realizing and the tumor region in digestive endoscope image is accurately positioned.
In addition to this, it is requested and scope image to be detected, calling by the detection that real-time reception endoscopy equipment is sent
Digestive endoscope Image detection model handles scope image to be detected, and processing result is returned to endoscopy and is set
It is standby, so that endoscopy equipment real-time display processing result, can help doctor to improve diagnosis efficiency, realize real-time diagnosis.
In addition, can expand feature under the premise of not sacrificial features spatial resolution using expansion convolutional layer and receive open country,
The convolution feature of the tumor region of different scale is merged, so that the tumor region to different scale in scope image to be detected carries out
It divides;Convolution algorithm method is separated using the depth of first point-by-point convolution layer-by-layer convolution again, parameter amount is can reduce, improves operation
Speed.
Embodiment three
Referring to Fig. 3, Fig. 3 is the multiple scale detecting of tumour in a kind of digestive endoscope image disclosed by the embodiments of the present invention
The structural schematic diagram of device.As shown in figure 3, the multiple scale detecting device of tumour may include: in the digestive endoscope image
Marking unit 301, for carrying out tumor imaging label to each digestive endoscope image sample, and according to preset
Tumour property grade carries out tumour property grade mark to the digestive endoscope image sample for being marked as tumor imaging.
Training unit 302, for carry out the digestive endoscope image sample after tumour property grade mark as foundation, instruction
Get digestive endoscope Image detection model.
First acquisition unit 303, for obtaining scope image to be detected.
Second acquisition unit 304, for obtaining the inspection of scope image to be detected according to digestive endoscope Image detection model
It surveys as a result, testing result includes one of the first classification results, the second classification results and multiple dimensioned tumor region testing result
Or much information;Wherein, the first classification results are for describing whether scope image to be detected is digestive endoscope image, and second point
Class result is for describing normal digestive endoscope image, benign lesion or malignant change.
As an alternative embodiment, above-mentioned training unit 302, is also used to using BP algorithm, in alimentary canal
Mirror Image detection model is adjusted.
Above-mentioned training unit 302 is used to use BP algorithm, the side for being adjusted digestive endoscope Image detection model
Formula is specifically as follows: above-mentioned training unit 302, for obtaining several digestive endoscope image samples from training data,
Calculate the difference of reality output and ideal output;According to the difference, by the method backpropagation adjustment power square of Minimum Polarization error
The weight of battle array, to adjust digestive endoscope Image detection model.Implement the embodiment, the identification that can be improved model is accurate
Rate.
Device shown in implementing Fig. 3 is examined by training digestive endoscope Image detection model using digestive endoscope image
It surveys model and completes tumor region positioning, do not need the profile for manually accurately drawing tumour in digestive endoscope image sample, energy
Realization is accurately positioned the tumor region in digestive endoscope image, therefore, implements the embodiment of the present invention, can reduce mark
Remember cost, while realizing and the tumor region in digestive endoscope image is accurately positioned.
In addition to this, by carrying out multitask combination learning to digestive endoscope Image detection model, make digestive endoscope
Image detection model is multi output model, can be improved the robustness and robustness of model, can not only realize in alimentary canal
Tumor region in mirror image is accurately positioned, moreover it is possible to carry out more classification to scope image to be detected.
Example IV
Referring to Fig. 4, Fig. 4 is the multiple dimensioned inspection of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
Survey the structural schematic diagram of device.Wherein, the multiple scale detecting device of tumour is by Fig. 3 in digestive endoscope image shown in Fig. 4
Shown in digestive endoscope image the multiple scale detecting device of tumour optimize, compared with Fig. 3, shown in Fig. 4
Digestive endoscope image in tumour multiple scale detecting device in, above-mentioned first acquisition unit 303, specifically for connecing in real time
Receive the detection request and scope image to be detected that endoscopy equipment is sent.
The multiple scale detecting device of tumour can also include: in digestive endoscope image shown in Fig. 4
Transmission unit 305, for, according to digestive endoscope Image detection model, being obtained to be checked in second acquisition unit 304
It surveys after the testing result of scope image, will test result and be sent to endoscopy equipment.
As an alternative embodiment, in digestive endoscope image shown in Fig. 4 tumour multiple scale detecting device
In, digestive endoscope Image detection model may include depth convolution baseline neural network, more classification branching networks and multiple dimensioned
Detection branches network;Above-mentioned second acquisition unit 304 may include:
Convolution subelement 3041 is rolled up for scope image to be detected to be inputted depth convolution baseline neural network in depth
Multilayer process of convolution is carried out to scope image to be detected in product baseline neural network, obtains the corresponding feature of scope image to be detected
Figure.
More classification subelements 3042, it is right in mostly classification branching networks for characteristic pattern to be inputted branching networks of classifying more
Characteristic pattern carries out more classification processings, obtains the first classification results and the second classification results.
Multiple scale detecting subelement 3043, for being used to describe digestive endoscope image benign lesion in the second classification results
Or when malignant change, by characteristic pattern input multiple scale detecting branching networks, in multiple scale detecting branching networks to characteristic pattern into
The processing of row multiple scale detecting, obtains multiple dimensioned tumor region testing result.
First obtain subelement 3044, for the second classification results for describe digestive endoscope image benign lesion or
When malignant change, obtained in be detected according to the first classification results, the second classification results and multiple dimensioned tumor region testing result
The testing result of mirror image.
Second obtain subelement 3045, for the second classification results for describe digestive endoscope image it is normal when, root
The testing result of scope image to be detected is obtained according to the first classification results and the second classification results.
As an alternative embodiment, above-mentioned multiple scale detecting subelement 3043 may include following (not shown)
Module:
Division module, for being used to describe digestive endoscope image benign lesion or malignant change in the second classification results
When, characteristic pattern is inputted into multiple scale detecting branching networks, characteristic pattern is divided into several in multiple scale detecting branching networks
Super-pixel.
Determining module, for determining several target super-pixel from several super-pixel according to the second classification results,
To obtain multiple dimensioned tumor region testing result.
As an alternative embodiment, in digestive endoscope image shown in Fig. 4 tumour multiple scale detecting device
In, depth convolution baseline neural network may include that depth separates convolutional layer, expansion convolutional layer and dimensionality reduction layer;Above-mentioned convolution
Subelement 3041 may include following module (not shown):
Depth separates convolution module, for by scope image to be detected input depth separate convolutional layer, depth can
It separates in convolutional layer and the separable process of convolution of depth is carried out to scope image to be detected, obtain initial characteristics figure.
Convolution module is expanded, for initial characteristics figure to be inputted expansion convolutional layer, uses different rulers in expansion convolutional layer
The convolution kernel of degree carries out concurrent operation processing to initial characteristics figure, obtains multidimensional characteristic figure.
Dimensionality reduction module carries out one-dimensional to multidimensional characteristic figure in dimensionality reduction layer for initial characteristics figure to be inputted dimensionality reduction layer
Processing, obtains the corresponding characteristic pattern of scope image to be detected.
As an alternative embodiment, in digestive endoscope image shown in Fig. 4 tumour multiple scale detecting device
In, above-mentioned training unit 302 may include following subelement (not shown):
Subelement is handled, is carried out for the data to the digestive endoscope image sample after progress tumour property grade mark
Pretreatment obtains training data.
Training subelement obtains digestive endoscope for being trained using training data to depth convolutional neural networks
Image detection model.
As an alternative embodiment, above-mentioned more classification subelements 3042 may include following mould (not shown)
Block:
First obtains module, for characteristic pattern to be inputted branching networks of classifying more, obtained in mostly classification branching networks to
Detect the first probability value that scope image is digestive endoscope image;
First judgment module, for judging whether first probability value reaches predetermined probabilities threshold value;
Second obtains module, for judging that first probability value is not up to predetermined probabilities threshold value in first judgment module
When, it obtains for describing the first classification results that scope image to be detected is not digestive endoscope image;
Above-mentioned second obtains module, is also used to judge that first probability value is not up to predetermined probabilities in first judgment module
When threshold value, obtain for describing the first classification results that scope image to be detected is not digestive endoscope image;
Second judgment module, for judging that first probability value is not up to predetermined probabilities threshold value in first judgment module
When, judge whether the digestive endoscope image is normal;
Third obtains module, for when the second judgment module judges that the digestive endoscope image is normal, acquisition to be used for
Normal second classification results of digestive endoscope image are described;
Above-mentioned third obtains module, is also used to judge that the digestive endoscope image is abnormal in the second judgment module
When, obtain the second probability value of the digestive endoscope image benign lesion;And reach predetermined probabilities threshold in second probability value
When value, the second classification results for describing digestive endoscope image malignant change are obtained;And it is not reached in second probability value
When to predetermined probabilities threshold value, the second classification results for describing digestive endoscope image benign lesion are obtained.
Implement the embodiment, by can be improved multiple scale detecting effect to the more classification of scope image to be detected progress,
The rate of precision of tumor region positioning can be also improved simultaneously.
Implement device shown in Fig. 4, by training digestive endoscope Image detection model, is examined using digestive endoscope image
It surveys model and completes tumor region positioning, do not need the profile for manually accurately drawing tumour in digestive endoscope image sample, energy
Realization is accurately positioned the tumor region in digestive endoscope image, therefore, implements the embodiment of the present invention, can reduce mark
Remember cost, while realizing and the tumor region in digestive endoscope image is accurately positioned.
In addition to this, it is requested and scope image to be detected, calling by the detection that real-time reception endoscopy equipment is sent
Digestive endoscope Image detection model handles scope image to be detected, and processing result is returned to endoscopy and is set
It is standby, so that endoscopy equipment real-time display processing result, can help doctor to improve diagnosis efficiency, realize real-time diagnosis.
In addition, can expand feature under the premise of not sacrificial features spatial resolution using expansion convolutional layer and receive open country,
The convolution feature for merging the tumor region of different scale, is identified with the tumor region to different scale;Using first point-by-point volume
The depth of product convolution layer-by-layer again separates convolution algorithm method, can reduce parameter amount, improves arithmetic speed.
Embodiment five
Referring to Fig. 5, Fig. 5 is the multiple dimensioned inspection of tumour in another digestive endoscope image disclosed by the embodiments of the present invention
Survey the structural schematic diagram of device.As shown in figure 5, the multiple scale detecting device of tumour may include: in the digestive endoscope image
It is stored with the memory 501 of executable program code;
The processor 502 coupled with memory 501;
Wherein, processor 502 calls the executable program code stored in memory 501, and it is any one to execute FIG. 1 to FIG. 2
The multi-dimension testing method of tumour in kind digestive endoscope image.
The embodiment of the present invention discloses a kind of computer readable storage medium, stores computer program, wherein the computer
Program makes the multi-dimension testing method of tumour in computer execution any one digestive endoscope image of FIG. 1 to FIG. 2.
A kind of computer program product is also disclosed in the embodiment of the present invention, wherein when computer program product on computers
When operation, so that computer executes some or all of the method in such as above each method embodiment step.
The embodiment of the present invention is also disclosed a kind of using distribution platform, wherein using distribution platform for issuing computer journey
Sequence product, wherein when computer program product is run on computers, so that computer executes such as the above each method embodiment
In some or all of method step.
In various embodiments of the present invention, it should be appreciated that magnitude of the sequence numbers of the above procedures are not meant to execute suitable
Successively, the execution sequence of each process should be determined by its function and internal logic the certainty of sequence, without coping with the embodiment of the present invention
Implementation process constitutes any restriction.
Above-mentioned unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be object unit, can be in one place, or may be distributed over multiple networks
On unit.Some or all of units can be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
In addition, each functional unit in various embodiments of the present invention can integrate in one processing unit, it is also possible to
Each unit physically exists alone, and can also be integrated in one unit with two or more units.Above-mentioned integrated unit
Both it can take the form of hardware realization, can also realize in the form of software functional units.
If above-mentioned integrated unit is realized in the form of SFU software functional unit and when sold or used as an independent product,
It can store in a retrievable memory of computer.Based on this understanding, technical solution of the present invention substantially or
Person says all or part of of the part that contributes to existing technology or the technical solution, can be in the form of software products
It embodies, which is stored in a memory, including several requests are with so that a computer is set
Standby (can be personal computer, server or network equipment etc., specifically can be the processor in computer equipment) executes
Some or all of each embodiment above method of the invention step.
In embodiment provided by the present invention, it should be appreciated that " B corresponding with A " indicates that B is associated with A, can be with according to A
Determine B.It is also to be understood that determine that B is not meant to determine B only according to A according to A, it can also be according to A and/or other information
Determine B.
Those of ordinary skill in the art will appreciate that some or all of in the various methods of above-described embodiment step be can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium include read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory,
RAM), programmable read only memory (Programmable Read-only Memory, PROM), erasable programmable is read-only deposits
Reservoir (Erasable Programmable Read Only Memory, EPROM), disposable programmable read-only memory (One-
Time Programmable Read-Only Memory, OTPROM), the electronics formula of erasing can make carbon copies read-only memory
(Electrically-Erasable Programmable Read-Only Memory, EEPROM), CD-ROM (Compact
Disc Read-Only Memory, CD-ROM) or other disc memories, magnetic disk storage, magnetic tape storage or can
For carrying or any other computer-readable medium of storing data.
Above to the multi-dimension testing method and dress of tumour in a kind of digestive endoscope image disclosed by the embodiments of the present invention
It sets and is described in detail, used herein a specific example illustrates the principle and implementation of the invention, above
The explanation of embodiment is merely used to help understand method and its core concept of the invention;Meanwhile for the general skill of this field
Art personnel, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion this
Description should not be construed as limiting the invention.
Claims (12)
1. the multi-dimension testing method of tumour in a kind of digestive endoscope image characterized by comprising
Tumor imaging label is carried out to each digestive endoscope image sample, and according to preset tumour property grade, to being marked
The digestive endoscope image sample for being denoted as tumor imaging carries out tumour property grade mark;
To carry out the digestive endoscope image sample after the tumour property grade mark as foundation, training obtains digestive endoscope
Image detection model;
Obtain scope image to be detected;
According to the digestive endoscope Image detection model, the testing result of the scope image to be detected, the detection are obtained
It as a result include one of the first classification results, the second classification results and multiple dimensioned tumor region testing result or much information;
Wherein, first classification results are for describing whether the scope image to be detected is digestive endoscope image, and described second
Classification results are for describing normal digestive endoscope image, benign lesion or malignant change.
2. the method according to claim 1, wherein described obtain scope image to be detected, comprising:
The detection request and scope image to be detected that real-time reception endoscopy equipment is sent;
And it is described according to the digestive endoscope Image detection model, obtain the testing result of the scope image to be detected
Later, the method also includes:
The testing result is sent to the endoscopy equipment.
3. the method according to claim 1, wherein the digestive endoscope Image detection model includes depth volume
Product baseline neural network, more classification branching networks and multiple scale detecting branching networks;It is described according to the digestive endoscope image
Detection model obtains the testing result of the scope image to be detected, comprising:
The scope image to be detected is inputted into the depth convolution baseline neural network, in the depth convolution baseline nerve net
Multilayer process of convolution is carried out to the scope image to be detected in network, obtains the corresponding characteristic pattern of the scope image to be detected;
By the characteristic pattern input branching networks of classifying, the characteristic pattern is carried out in more classification branching networks more
More classification processings obtain first classification results and second classification results;
If second classification results are for describing digestive endoscope image benign lesion or malignant change, and the characteristic pattern is defeated
Enter the multiple scale detecting network, the characteristic pattern is carried out at tumor region detection in the multiple scale detecting branching networks
Reason, obtains the multiple dimensioned tumor region testing result, and according to first classification results, second classification results and
The multiple dimensioned tumor region testing result obtains the testing result of the scope image to be detected;
If second classification results are normal for describing digestive endoscope image, according to first classification results and described the
Two classification results obtain the testing result of the scope image to be detected.
4. according to the method described in claim 3, it is characterized in that, described input the multiple scale detecting point for the characteristic pattern
Branch network carries out tumor region detection processing to the characteristic pattern in the multiple scale detecting branching networks, obtains described more
Scale tumor region testing result, comprising:
The characteristic pattern is inputted into the multiple scale detecting branching networks, by the spy in the multiple scale detecting branching networks
Sign figure is divided into several super-pixel;
According to second classification results, several target super-pixel are determined, from several described super-pixel to obtain
State multiple dimensioned tumor region testing result.
5. the method according to claim 3 or 4, which is characterized in that the depth convolution baseline neural network includes depth
Separable convolutional layer, expansion convolutional layer and dimensionality reduction layer;It is described that the scope image to be detected is inputted into the depth convolution baseline
Neural network carries out multilayer process of convolution to the scope image to be detected in the depth convolution baseline neural network, obtains
Obtain the corresponding characteristic pattern of the scope image to be detected, comprising:
The scope image to be detected is inputted into the depth and separates convolutional layer, is separated in convolutional layer in the depth to institute
It states scope image to be detected and carries out the separable process of convolution of depth, obtain initial characteristics figure;
The initial characteristics figure is inputted into the expansion convolutional layer, the convolution kernel of different scale is used in the expansion convolutional layer
Concurrent operation processing is carried out to the initial characteristics figure, obtains multidimensional characteristic figure;
The initial characteristics figure is inputted into the dimensionality reduction layer, the multidimensional characteristic figure is carried out at one-dimensional in the dimensionality reduction layer
Reason obtains the corresponding characteristic pattern of the scope image to be detected.
6. according to the method described in claim 5, it is characterized in that, described to carry out disappearing after the tumour property grade mark
Change road scope image sample is foundation, and training obtains digestive endoscope Image detection model, comprising:
The data of digestive endoscope image sample after the progress tumour property grade mark are pre-processed, are trained
Data;
Depth convolutional neural networks are trained using the training data, obtain digestive endoscope Image detection model.
7. the multiple scale detecting device of tumour in a kind of digestive endoscope image characterized by comprising
Marking unit, for carrying out tumor imaging label to each digestive endoscope image sample, and according to preset tumprigenicity
Matter grade carries out tumour property grade mark to the digestive endoscope image sample for being marked as tumor imaging;
Training unit, for carry out the digestive endoscope image sample after the tumour property grade mark as foundation, training
Obtain digestive endoscope Image detection model;
First acquisition unit, for obtaining scope image to be detected;
Second acquisition unit, for obtaining the scope image to be detected according to the digestive endoscope Image detection model
Testing result, the testing result include in the first classification results, the second classification results and multiple dimensioned tumor region testing result
One or more information;Wherein, first classification results are for describing whether the scope image to be detected is alimentary canal
Scope image, second classification results are for describing normal digestive endoscope image, benign lesion or malignant change.
8. device according to claim 7, which is characterized in that the first acquisition unit is specifically used in real-time reception
The detection request and scope image to be detected that spectroscopy equipment is sent;
And described device further include:
Transmission unit, for according to the digestive endoscope Image detection model, obtained in the second acquisition unit it is described to
After the testing result for detecting scope image, the testing result is sent to the endoscopy equipment.
9. device according to claim 7, which is characterized in that the digestive endoscope Image detection model includes depth volume
Product baseline neural network, more classification branching networks and multiple scale detecting branching networks;The second acquisition unit includes:
Convolution subelement, for the scope image to be detected to be inputted the depth convolution baseline neural network, in the depth
It spends in convolution baseline neural network and multilayer process of convolution is carried out to the scope image to be detected, obtain the scope shadow to be detected
As corresponding characteristic pattern;
More classification subelements, for the characteristic pattern to be inputted the branching networks of classifying more, in the branching networks of classifying more
In more classification processings are carried out to the characteristic pattern, obtain first classification results and second classification results;
Multiple scale detecting subelement, for being used to describe digestive endoscope image benign lesion or evil in second classification results
When venereal disease becomes, the characteristic pattern is inputted into the multiple scale detecting branching networks, it is right in the multiple scale detecting branching networks
The characteristic pattern carries out tumor region detection processing, obtains the multiple dimensioned tumor region testing result;
First obtains subelement, in second classification results for describing digestive endoscope image benign lesion or pernicious
When lesion, institute is obtained according to first classification results, second classification results, the multiple dimensioned tumor region testing result
State the testing result of scope image to be detected;
Second obtain subelement, for second classification results for describe digestive endoscope image it is normal when, according to institute
It states the first classification results and second classification results obtains the testing result of the scope image to be detected.
10. device according to claim 9, which is characterized in that the multiple scale detecting subelement includes:
Division module, for being used to describe digestive endoscope image benign lesion or malignant change in second classification results
When, the characteristic pattern is inputted into the multiple scale detecting branching networks, by the spy in the multiple scale detecting branching networks
Sign figure is divided into several super-pixel;
Determining module, for determining that several targets are super from several described super-pixel according to second classification results
Pixel, to obtain the multiple dimensioned tumor region testing result.
11. device according to claim 9 or 10, which is characterized in that the depth convolution baseline neural network includes deep
Spend separable convolutional layer, expansion convolutional layer and dimensionality reduction layer;The convolution subelement includes:
Depth separates convolution module, convolutional layer is separated for the scope image to be detected to be inputted the depth, in institute
It states depth to separate in convolutional layer to the separable process of convolution of scope image progress depth to be detected, obtains initial characteristics
Figure;
Convolution module is expanded, for the initial characteristics figure to be inputted the expansion convolutional layer, is adopted in the expansion convolutional layer
Concurrent operation processing is carried out to the initial characteristics figure with the convolution kernel of different scale, obtains multidimensional characteristic figure;
Dimensionality reduction module, for the initial characteristics figure to be inputted the dimensionality reduction layer, to the multidimensional characteristic in the dimensionality reduction layer
Figure carries out one-dimensional processing, obtains the corresponding characteristic pattern of the scope image to be detected.
12. device according to claim 11, which is characterized in that the training unit includes:
Subelement is handled, is carried out for the data to the digestive endoscope image sample after the progress tumour property grade mark
Pretreatment obtains training data;
Training subelement obtains digestive endoscope for being trained using the training data to depth convolutional neural networks
Image detection model.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910142912.5A CN109919212A (en) | 2019-02-26 | 2019-02-26 | The multi-dimension testing method and device of tumour in digestive endoscope image |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910142912.5A CN109919212A (en) | 2019-02-26 | 2019-02-26 | The multi-dimension testing method and device of tumour in digestive endoscope image |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109919212A true CN109919212A (en) | 2019-06-21 |
Family
ID=66962395
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910142912.5A Pending CN109919212A (en) | 2019-02-26 | 2019-02-26 | The multi-dimension testing method and device of tumour in digestive endoscope image |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109919212A (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110782427A (en) * | 2019-08-19 | 2020-02-11 | 大连大学 | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution |
CN110991561A (en) * | 2019-12-20 | 2020-04-10 | 山东大学齐鲁医院 | Method and system for identifying images of endoscope in lower digestive tract |
CN113288052A (en) * | 2021-05-09 | 2021-08-24 | 华中科技大学同济医学院附属协和医院 | Method for probing digestive tract tumor through FITC-UEA1 marker |
CN113793335A (en) * | 2021-11-16 | 2021-12-14 | 武汉大学 | Method and device for identifying alimentary tract tumor infiltration layer, computer equipment and medium |
CN114511749A (en) * | 2022-04-19 | 2022-05-17 | 武汉大学 | Image processing method, image processing device, computer equipment and storage medium |
CN116740475A (en) * | 2023-08-15 | 2023-09-12 | 苏州凌影云诺医疗科技有限公司 | Digestive tract image recognition method and system based on state classification |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106504255A (en) * | 2016-11-02 | 2017-03-15 | 南京大学 | A kind of multi-Target Image joint dividing method based on multi-tag multi-instance learning |
CN107967946A (en) * | 2017-12-21 | 2018-04-27 | 武汉大学 | Operating gastroscope real-time auxiliary system and method based on deep learning |
CN109118485A (en) * | 2018-08-13 | 2019-01-01 | 复旦大学 | Digestive endoscope image classification based on multitask neural network cancer detection system early |
-
2019
- 2019-02-26 CN CN201910142912.5A patent/CN109919212A/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106504255A (en) * | 2016-11-02 | 2017-03-15 | 南京大学 | A kind of multi-Target Image joint dividing method based on multi-tag multi-instance learning |
CN107967946A (en) * | 2017-12-21 | 2018-04-27 | 武汉大学 | Operating gastroscope real-time auxiliary system and method based on deep learning |
CN109118485A (en) * | 2018-08-13 | 2019-01-01 | 复旦大学 | Digestive endoscope image classification based on multitask neural network cancer detection system early |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110782427A (en) * | 2019-08-19 | 2020-02-11 | 大连大学 | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution |
CN110782427B (en) * | 2019-08-19 | 2023-06-20 | 大连大学 | Magnetic resonance brain tumor automatic segmentation method based on separable cavity convolution |
CN110991561A (en) * | 2019-12-20 | 2020-04-10 | 山东大学齐鲁医院 | Method and system for identifying images of endoscope in lower digestive tract |
CN110991561B (en) * | 2019-12-20 | 2020-12-25 | 山东大学齐鲁医院 | Method and system for identifying images of endoscope in lower digestive tract |
CN113288052A (en) * | 2021-05-09 | 2021-08-24 | 华中科技大学同济医学院附属协和医院 | Method for probing digestive tract tumor through FITC-UEA1 marker |
CN113793335A (en) * | 2021-11-16 | 2021-12-14 | 武汉大学 | Method and device for identifying alimentary tract tumor infiltration layer, computer equipment and medium |
CN113793335B (en) * | 2021-11-16 | 2022-02-08 | 武汉大学 | Method and device for identifying alimentary tract tumor infiltration layer, computer equipment and medium |
CN114511749A (en) * | 2022-04-19 | 2022-05-17 | 武汉大学 | Image processing method, image processing device, computer equipment and storage medium |
CN114511749B (en) * | 2022-04-19 | 2022-06-28 | 武汉大学 | Image processing method, image processing device, computer equipment and storage medium |
CN116740475A (en) * | 2023-08-15 | 2023-09-12 | 苏州凌影云诺医疗科技有限公司 | Digestive tract image recognition method and system based on state classification |
CN116740475B (en) * | 2023-08-15 | 2023-10-17 | 苏州凌影云诺医疗科技有限公司 | Digestive tract image recognition method and system based on state classification |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109919212A (en) | The multi-dimension testing method and device of tumour in digestive endoscope image | |
Ismael et al. | An enhanced deep learning approach for brain cancer MRI images classification using residual networks | |
Wang et al. | [Retracted] PSSPNN: PatchShuffle Stochastic Pooling Neural Network for an Explainable Diagnosis of COVID‐19 with Multiple‐Way Data Augmentation | |
CN106504232B (en) | A kind of pulmonary nodule automatic checkout system based on 3D convolutional neural networks | |
CN109886986A (en) | A kind of skin lens image dividing method based on multiple-limb convolutional neural networks | |
CN108830826A (en) | A kind of system and method detecting Lung neoplasm | |
CN109770932A (en) | The processing method of multi-modal brain neuroblastoma image feature | |
CN107977952A (en) | Medical image cutting method and device | |
Ye et al. | Glioma grading based on 3D multimodal convolutional neural network and privileged learning | |
CN109166104A (en) | A kind of lesion detection method, device and equipment | |
CN111611851B (en) | Model generation method, iris detection method and device | |
CN110390674A (en) | Image processing method, device, storage medium, equipment and system | |
CN113096137B (en) | Adaptive segmentation method and system for OCT (optical coherence tomography) retinal image field | |
Malo et al. | Skin cancer detection using convolutional neural network | |
Zhang et al. | Diagnosis of secondary pulmonary tuberculosis by an eight-layer improved convolutional neural network with stochastic pooling and hyperparameter optimization | |
CN112365980A (en) | Brain tumor multi-target point auxiliary diagnosis and prospective treatment evolution visualization method and system | |
CN109461163A (en) | A kind of edge detection extraction algorithm for magnetic resonance standard water mould | |
CN109949275A (en) | A kind of diagnostic method and device of superior gastrointestinal endoscope image | |
CN108053401A (en) | A kind of B ultrasound image processing method and device | |
Al-Khasawneh et al. | Alzheimer’s Disease Diagnosis Using MRI Images | |
CN111724345A (en) | Pneumonia picture verification device and method capable of adaptively adjusting size of receptive field | |
Shi et al. | Automatic detection of pulmonary nodules in CT images based on 3D Res-I network | |
Li et al. | VBNet: An end-to-end 3D neural network for vessel bifurcation point detection in mesoscopic brain images | |
CN112669319A (en) | Multi-view multi-scale lymph node false positive inhibition modeling method | |
Hwang et al. | An adaptive regularization approach to colonoscopic polyp detection using a cascaded structure of encoder–decoders |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190621 |