CN113139937B - Digestive tract endoscope video image identification method based on deep learning - Google Patents
Digestive tract endoscope video image identification method based on deep learning Download PDFInfo
- Publication number
- CN113139937B CN113139937B CN202110393884.1A CN202110393884A CN113139937B CN 113139937 B CN113139937 B CN 113139937B CN 202110393884 A CN202110393884 A CN 202110393884A CN 113139937 B CN113139937 B CN 113139937B
- Authority
- CN
- China
- Prior art keywords
- lesion
- video image
- image
- endoscope
- digestive tract
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 210000001035 gastrointestinal tract Anatomy 0.000 title claims abstract description 25
- 238000013135 deep learning Methods 0.000 title claims abstract description 13
- 230000003902 lesion Effects 0.000 claims abstract description 82
- 238000001514 detection method Methods 0.000 claims abstract description 51
- 238000013528 artificial neural network Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 9
- 230000036285 pathological change Effects 0.000 claims description 7
- 231100000915 pathological change Toxicity 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 5
- 238000003745 diagnosis Methods 0.000 abstract description 12
- 201000010099 disease Diseases 0.000 description 9
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 9
- 230000002496 gastric effect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 210000002784 stomach Anatomy 0.000 description 2
- 206010028980 Neoplasm Diseases 0.000 description 1
- 208000025865 Ulcer Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 210000001198 duodenum Anatomy 0.000 description 1
- 238000001839 endoscopy Methods 0.000 description 1
- 210000003238 esophagus Anatomy 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000004377 microelectronic Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011369 optimal treatment Methods 0.000 description 1
- 239000011148 porous material Substances 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 231100000397 ulcer Toxicity 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/273—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the upper alimentary canal, e.g. oesophagoscopes, gastroscopes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Surgery (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Radiology & Medical Imaging (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Mathematical Physics (AREA)
- Pathology (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Optics & Photonics (AREA)
- Heart & Thoracic Surgery (AREA)
- Evolutionary Computation (AREA)
- Animal Behavior & Ethology (AREA)
- Artificial Intelligence (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Computing Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Fuzzy Systems (AREA)
- Gastroenterology & Hepatology (AREA)
- Endoscopes (AREA)
Abstract
The invention relates to the technical field of image recognition, and discloses a digestive tract endoscope video image recognition method based on deep learning, which comprises the following steps of S1: acquiring a video image detected by an endoscope; s2: extracting easy-to-lesion key frames of the most common lesion parts and key frames of the rest detection parts from the video images; s3: predicting the key frames of the vulnerable lesions frame by using a neural network prediction model to obtain a primary vulnerable lesion prediction result of each frame, counting the primary prediction results of each frame, and extracting a key frame image with the highest primary vulnerable lesion prediction result if the counted value is higher than a primary preset value; s4: and carrying out auxiliary detection on the highest key frame image by using an auxiliary identification model and outputting a final easy-lesion prediction result. The invention enhances the pertinence of endoscope detection on the premise of ensuring the detection accuracy of all parts, improves the lesion detection accuracy and prevents the occurrence of missed diagnosis.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a digestive tract endoscope video image recognition method based on deep learning.
Background
The endoscope is a detection instrument integrating traditional optics, ergonomics, mathematics, precision machinery, micro-electronic equipment and software. The endoscope is provided with an image sensor, an optical lens, a light source for illumination and the like, can enter a human body through a natural pore canal of the human body or a small incision made by an operation, collects images, and transmits the collected images to a display terminal which can be seen by medical staff, and the medical staff can see pathological changes which cannot be displayed by X rays by using the endoscope, so that the endoscope is very useful for diagnosis of doctors and treatment of patients, for example, the doctors can observe ulcers or tumors in the stomach by means of the endoscope, and an optimal treatment scheme is made according to the pathological changes.
At present, an endoscope utilizes a light source for illumination, utilizes an optical lens and an image sensor to collect images in a human body, and transmits the collected images to a display terminal for medical staff to diagnose. However, the existing endoscope image recognition system has poor pertinence in the whole detection process, so that the energy demand on doctors is high, and meanwhile, when the image is unclear due to the endoscope, the phenomenon of missed diagnosis is easy to occur.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a digestive tract endoscope video image recognition method based on deep learning, which can enhance the pertinence of endoscope detection, improve the lesion detection accuracy and prevent the occurrence of missed diagnosis on the premise of ensuring the detection accuracy of all parts.
In order to achieve the above purpose, the invention provides the following technical scheme:
a digestive tract endoscope video image identification method based on deep learning is characterized by comprising the following steps
S1: acquiring a video image detected by an endoscope;
s2: extracting easy-to-lesion key frames of the most common lesion parts and key frames of the rest detection parts from the video images;
s3: predicting the susceptibility to lesion key frames by using a neural network prediction model to obtain a primary susceptibility to lesion prediction result of each frame, counting the primary susceptibility to lesion prediction results of each frame, and extracting a key frame image with the highest primary susceptibility to lesion prediction result if the statistic value is higher than a primary preset value;
s4: and carrying out auxiliary detection on the highest key frame image by using an auxiliary identification model and outputting a final easy-lesion prediction result.
In the invention, further, the auxiliary recognition model is constructed by acquiring information of the most common lesion part of the digestive tract and utilizing the information of the most common lesion part.
In the present invention, further, the acquiring information of the most frequently diseased region of the digestive tract includes:
s00: acquiring detection data of the digestive tract endoscope, and importing the data into a centralized distributed database;
s01: analyzing and counting the data in the distributed database, and arranging the counting results;
s02: determining the most common lesion part according to the statistical result;
s03: and acquiring the most common lesion part image.
In the present invention, further, the constructing an auxiliary recognition model by using the most common lesion site information includes: and (3) by using a positioning-classifying sub-network, using a multi-branch structure, simultaneously utilizing local information and global information of the most common lesion part image of the training set, and training a local region in a characteristic supervision mode to obtain an auxiliary recognition model.
In the present invention, further, the extracting, in the step S2, a vulnerable key frame and a key frame from the video image according to the result of the vulnerable process includes: and comparing the acquired video image with the most common lesion part image, extracting all easy-lesion key frames with the goodness of fit meeting a preset value, and taking the rest key frames as key frames.
In the present invention, further, the step S3 further includes:
and predicting the key frames by using a neural network prediction model to obtain key prediction results.
In the present invention, the step S4 of performing auxiliary detection on the susceptibility to disease prediction result by using an auxiliary recognition model further includes: and predicting the highest key frame image by using the auxiliary recognition model, determining that the lesion exists if the highest key frame image is higher than a secondary preset value, and outputting a lesion-prone prediction result.
In the present invention, further, the step S1 of acquiring a clear video image for endoscopic detection includes:
s10: acquiring a video image;
s11: and whether the video image meets the condition or not is not met, and pixel adjustment is automatically carried out on the obtained video image.
In the present invention, further, the pixel adjustment includes brightness, contrast, and contour emphasis.
In the present invention, further, the contour emphasis is processed by laplacian sharpening, and the processing formula of the brightness and the contrast is:
g(x)＝αf(x)+β
where α represents a contrast adjustment parameter and β represents a brightness adjustment parameter.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the most common lesion part information of a human body is obtained, an auxiliary detection model is constructed, a video image is obtained through an endoscope, an easily-affected key frame and a key frame are obtained from the video image according to the most common lesion part information, and a frame-by-frame identification method is adopted for the easily-affected key frame so as to enhance the identification accuracy. Meanwhile, in order to further prevent missed diagnosis and false recognition, the scheme utilizes an auxiliary detection model which is specially constructed for the vulnerable part to carry out auxiliary detection on the result predicted by the neural network prediction model, and further verifies the accuracy of lesion.
In addition, the invention can automatically adjust the pixels according to the acquired image, ensure the definition of the acquired image and improve the identification efficiency and accuracy.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic overall flow diagram of the present invention;
FIG. 2 is a flowchart of the present invention, wherein the most common lesion is obtained in step S1;
FIG. 3 is a flowchart of the implementation of step S4 and step S5 in the present invention;
fig. 4 is a flowchart of the implementation of step S2 in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or intervening components may also be present. When a component is referred to as being "disposed on" another component, it can be directly on the other component or intervening components may also be present. The terms "vertical," "horizontal," "left," "right," and the like are used herein for purposes of illustration only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
Since there are many detection channels for endoscope detection, the present invention mainly aims at the most common lesion site obtained from the digestive tract, and certainly, other channel data can be obtained and constructed by using the present solution, which is not described herein in detail. In the process of performing digestive tract endoscopic detection, if an endoscope passes through the mouth and throat of a patient, enters the stomach along the esophagus and then inspects the parts such as duodenum and the like, the process involves a plurality of detection sites.
Referring to fig. 1, a method for recognizing a video image of an endoscope in a digestive tract based on deep learning according to a preferred embodiment of the present invention includes the following steps:
s1: acquiring a video image detected by a sight glass;
s2: extracting easy-to-lesion key frames of the most common lesion parts and key frames of the rest detection parts from the video images;
s3: predicting the vulnerable key frames frame by using a neural network prediction model to obtain a primary vulnerable prediction result;
s4: carrying out auxiliary detection on the susceptibility to lesion prediction result by using an auxiliary identification model and outputting a final prediction result;
according to the invention, the video image is obtained through the endoscope, the video image is subjected to lesion susceptibility key frame and key frame according to the most common lesion part information, the lesion susceptibility key frame and the key frame are respectively predicted by utilizing the existing neural network prediction model, the prediction results are respectively obtained, and the feasibility of the whole lesion detection result is ensured. Meanwhile, in order to further prevent missed diagnosis and false identification, the scheme utilizes a special auxiliary detection model which is constructed aiming at the easy-to-lesion part to carry out auxiliary detection on the result predicted by the neural network prediction model, and further verifies the accuracy of lesion.
The auxiliary recognition model is constructed by acquiring the most common lesion part information of the digestive tract and utilizing the most common lesion part information. Referring to fig. 2, the steps for obtaining the most common lesion site specifically include:
s00: acquiring detection data of the digestive tract endoscope, and storing the data in a centralized distributed database;
s01: analyzing and counting the data in the distributed database, and arranging the counting result;
s02: determining the most common lesion part according to the statistical result;
s03: the most common lesion images are acquired.
Specifically, a large amount of gastrointestinal endoscope detection data is first acquired from an existing endoscope medical case library, the data is a case report table of a patient with a lesion, the case report table includes text information and Image information, the text information includes basic information of the patient, information of an attending doctor, operation information, diagnosis information and the like, and the Image information refers to a lesion Image, and the Image, an Image part and coordinates thereof at the detection part are stored in an Image format. And storing the case report tables in a newly-built centralized distributed database, arranging all the case report tables by taking lesion parts in the diagnosis information as key words, and determining the most common lesion parts according to statistical results, wherein the most common lesion parts are manually selected according to the total amount of the samples and the distribution condition of the data. For example, in the gastrointestinal endoscopy, it is assumed that six detection sites a, B, C, D, E, and F exist in the process, and the detection sites are ranked according to disease incidence, wherein the disease incidence of the site a is 30%, the disease incidence of the site B is 35%, the disease incidence of the site C is 20%, the disease incidence of the site D is 8%, the disease incidence of the site E is 5%, and the disease incidence of the site F is 2%, and the most common disease sites are a, B, and C according to the arrangement of data and the concentration of data distribution.
In the present invention, a medical image of the most common lesion site is acquired in step S03, and the image is used to train a constructed auxiliary model. The method for constructing the auxiliary recognition model by using the most common lesion part information comprises the following steps: and (3) training the local region in a characteristic supervision mode by using a multi-branch structure and simultaneously utilizing local information and global information of the most frequently-diseased part image of the training set through a positioning-classifying sub-network to obtain an auxiliary recognition model.
A location-classification subnetwork comprising a classification network assisted by a location network. The intermediate learning of the classification network is reinforced by the location information (location and segmentation mask) in the location network. And respectively training different lesion parts by determining different most common lesion part images and simultaneously utilizing local information and global information of the most common lesion part images of the training set according to the multi-branch structure in a characteristic supervision mode of a local area so as to construct an auxiliary recognition model. Because the auxiliary model is constructed by training for multiple times according to a large amount of lesion information, the prediction confidence coefficient of lesion detection is high.
In the present invention, further, referring to fig. 4, the acquiring a video image detected by an endoscope in step S1 includes:
s10: acquiring a video image;
s11: and whether the video image meets the condition or not is not met, and pixel adjustment is automatically carried out on the obtained video image.
Specifically, in the invention, the endoscope enters the human body, uploads the acquired video image to obtain the video image in the alimentary canal, and judges the definition and the preset value of the video image, wherein the definition comprises the image resolution and the characteristic outline. If the preset condition is not met, the pixels of the video image need to be adjusted.
Pixel adjustments include brightness, contrast, and contour emphasis. Contour emphasis is processed by laplacian sharpening, which is based on how abrupt the surrounding pixels of a certain pixel of an image change to that pixel, i.e., based on how the pixels of the image change. When the gray level of the central pixel of the neighborhood is lower than the average gray level of other pixels in the domain where the central pixel is located, the gray level of the central pixel should be further reduced, and when the gray level of the central pixel of the neighborhood is higher than the average gray level of other pixels in the neighborhood where the central pixel is located, the gray level of the central pixel should be further improved, specifically, a first-order differential method is adopted to detect whether an edge exists, and a second-order differential method is adopted to determine the position of the edge, so that the sharpening processing of the image is realized.
The brightness and contrast processing formula is:
g(x)＝αf(x)+β
where α represents a contrast adjustment parameter and β represents a brightness adjustment parameter.
Specifically, in the image processing transformation, the value of each output pixel depends on the corresponding input pixel value, and the adjustment of the brightness and contrast of the picture can be directly performed on the image in the RBG space by using the formula g (x) = α f (x) + β, where f (x) is the original pixel, and when the transformed pixel is g (x), α is equal to 1, the transformed pixel represents the original image; when alpha is larger than 1, contrast is enhanced, and the picture looks clearer; when alpha is less than 1, the contrast is weakened, and the picture looks darker; the beta affects the brightness of the image, and as the beta increases or decreases, the gray value of the whole image moves up or down, namely the whole image becomes dark or bright, and the contrast is not changed. The corresponding automatic adjustment process is to compare the alpha beta of the preset clear image picture with the picture to be corrected, and to realize automatic adjustment through programming, so that the definition of the output image meets certain requirements.
In the present invention, further, the extracting, in the step S3, an easy-to-be-detected key frame of the most frequently-detected lesion part and key frames of other detected parts from the video image includes: and comparing the acquired video image with the most frequently-diseased part image, extracting all easy-diseased key frames with the goodness of fit meeting a preset value, and taking the rest key frames as key frames. Specifically, in the present invention, first, a key feature is extracted from the most common lesion image according to the acquired lesion result, the key feature is used as a basis for extracting a lesion-prone key frame, all the lesion-prone key frames in the acquired video image are extracted and classified and stored according to the lesion-prone region, and the rest key frame images are classified and stored by being attributed to the key frames. The lesion-prone key frame and the key frame are predicted through the existing neural network model, so that all detection parts are reliably detected, and missed diagnosis is prevented. However, since the lesion burst rate of the lesion detection position is higher, in order to make the lesion detection more targeted, the scheme adopts a frame-by-frame reading mode for the lesion key frame.
Specifically, as shown in fig. 3, the predicting the vulnerable key frame by using the neural network prediction model in step S4 to obtain a primary vulnerable prediction result includes:
predicting the key frames of the vulnerable lesions frame by using a neural network prediction model, predicting the key frames of the vulnerable lesions frame by frame to obtain a primary vulnerable lesion prediction result of each frame, counting the primary prediction results of each frame, and extracting a key frame image with the highest primary vulnerable lesion prediction result if the statistical value is higher than a primary preset value;
because the risk of pathological changes of the easy-to-pathological-change part is high, in the scheme, each frame in all the easy-to-pathological-change key frames of a certain pathological change part needs to be predicted, the prediction result of each frame is obtained, the prediction results of each frame are counted, and if the counted value is higher than a preset value, the fact that pathological changes exist in the part can be determined.
In order to further improve the accuracy of diagnosis, the scheme also utilizes an auxiliary recognition model to perform auxiliary detection on the prognosis result of the vulnerable lesions, and the method specifically comprises the following steps: and predicting the highest key frame image by using the auxiliary recognition model, determining that the lesion exists if the highest key frame image is higher than a secondary preset value, and outputting a final easy-lesion prediction result. If the image is lower than the secondary preset value, the output result indicates that the pathological change risk exists, and a doctor needs to further compare, confirm and determine the image.
In the present invention, step 4 further includes predicting the key frames by using a neural network prediction model to obtain key prediction results. Specifically, for key frames of other detection parts, the key frames are directly predicted by using the existing neural network prediction model to obtain a prediction result, and although the concurrency rate of the other detection parts is relatively low, detection cannot be excluded, so that the feasibility of the whole lesion detection result is ensured.
In the present embodiment, it is preferred that,
the working principle is as follows:
firstly, a large amount of digestive tract endoscope detection data are taken from an existing endoscope medical case library and stored in a newly-built centralized distributed database, lesion parts in all the data are arranged, the most common lesion parts are determined according to a statistical result, and an auxiliary detection model is constructed by acquiring images of the most common lesion parts of the digestive tract.
Video images are acquired through an endoscope, and pixel adjustment is automatically performed on the acquired images. The video image acquires the easy-to-lesion key frame and the key frame according to the most common lesion part information, the existing neural network prediction model is used for predicting the lesion key frame and the key frame respectively, prediction results are obtained respectively, and the feasibility of the overall lesion detection result is guaranteed. In order to ensure that the diagnosis of the lesion-prone region has higher accuracy, when the neural network model is used for prediction, each frame in all lesion-prone key frames of a certain lesion part needs to be predicted, a prediction result of each frame is obtained, the prediction results of each frame are counted, and if the counted value is higher than a preset value, the lesion in the part can be roughly determined. Meanwhile, in order to further prevent missed diagnosis and false recognition, the auxiliary detection model which is specially constructed for the easy-to-lesion part is utilized in the scheme to perform auxiliary detection on the result predicted by the neural network prediction model, namely the auxiliary recognition model predicts the highest key frame image, if the result is higher than a secondary preset value, the existence of lesion is determined, and the final easy-to-lesion prediction result is output, so that the accuracy of lesion verification is enhanced.
The above description is for the purpose of illustrating the preferred embodiments of the present invention, but the present invention is not limited thereto, and all changes and modifications that can be made within the spirit of the present invention should be included in the scope of the present invention.
Claims (9)
1. A digestive tract endoscope video image recognition method based on deep learning is characterized by comprising the following steps:
s1: acquiring a video image detected by an endoscope;
s2: extracting easy-to-lesion key frames of the most common lesion part and key frames of the rest detection parts from the video image;
s3: predicting the easy-to-lesion key frames frame by using a neural network prediction model to obtain a primary easy-to-lesion prediction result of each frame, counting the primary prediction results of each frame, and extracting a key frame image with the highest primary easy-to-lesion prediction result if the counted value is higher than a primary preset value;
s4: performing auxiliary detection on the key frame image with the highest primary susceptibility prediction result by using an auxiliary identification model and outputting a final susceptibility prediction result;
the method for acquiring the most frequently-diseased part information of the digestive tract comprises the following steps:
s00: acquiring detection data of the digestive tract endoscope, and storing the data in a centralized distributed database;
s01: analyzing and counting the data in the distributed database, and arranging the counting result;
s02: determining the most common lesion part according to the statistical result;
s03: and acquiring the most common lesion part image.
2. The method as claimed in claim 1, wherein the auxiliary recognition model is constructed by acquiring information of most frequently-occurring lesion sites of the digestive tract and using the information of most frequently-occurring lesion sites.
3. The method for recognizing the video image of the endoscope of the digestive tract based on the deep learning as claimed in claim 2, wherein the constructing of the auxiliary recognition model by using the information of the most common lesion part comprises: and (3) by using a positioning-classifying sub-network, using a multi-branch structure, simultaneously utilizing local information and global information of the most common lesion part image of the training set, and training a local region in a characteristic supervision mode to obtain an auxiliary recognition model.
4. The method for recognizing the video image of the endoscope in the digestive tract based on the deep learning as claimed in claim 1, wherein the step S2 of extracting the key frames of the easy lesion and the key frames of the key points in the video image according to the result of the easy lesion comprises: and comparing the acquired video image with the characteristics of the most frequently-diseased part image, extracting all easy-diseased key frames with the goodness of fit meeting a preset value, and taking the rest key frames as key frames.
5. The method for recognizing endoscopic video images of digestive tract based on deep learning according to claim 1, wherein said step S3 further comprises:
and predicting the key frames by using a neural network prediction model to obtain key prediction results.
6. The method for recognizing the video image of the endoscope in the digestive tract based on the deep learning as claimed in claim 2, wherein the step S4 of performing the auxiliary detection on the result of the prediction of the vulnerable lesion by using the auxiliary recognition model comprises: and predicting the highest key frame image by using the auxiliary recognition model, and outputting a final pathological change prediction result if the highest key frame image is higher than a secondary preset value.
7. The method for recognizing the video image of the endoscope in the digestive tract based on the deep learning as claimed in claim 1, wherein the step S1 of acquiring the video image of the endoscope detection comprises the following steps:
s10: acquiring a video image;
s11: and whether the video image meets the condition or not is not met, and pixel adjustment is automatically carried out on the obtained video image.
8. The method of claim 7, wherein the pixel adjustment comprises brightness, contrast and contour emphasis.
9. The method according to claim 8, wherein the contour enhancement is processed by laplacian sharpening, and the processing formula of brightness and contrast is as follows:
g(x)＝αf(x)+β
wherein, alpha represents contrast adjusting parameter, beta represents brightness adjusting parameter; wherein f (x) is the original pixel and g (x) is the converted pixel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110393884.1A CN113139937B (en) | 2021-04-13 | 2021-04-13 | Digestive tract endoscope video image identification method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110393884.1A CN113139937B (en) | 2021-04-13 | 2021-04-13 | Digestive tract endoscope video image identification method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113139937A CN113139937A (en) | 2021-07-20 |
| CN113139937B true CN113139937B (en) | 2022-11-01 |
Family
ID=76811927
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110393884.1A Active CN113139937B (en) | 2021-04-13 | 2021-04-13 | Digestive tract endoscope video image identification method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139937B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113367644B (en) * | 2021-08-11 | 2022-04-15 | 紫东信息科技(苏州)有限公司 | Digestive tract disease recognition system with device migration capability |
| CN114727021B (en) * | 2022-04-19 | 2023-09-15 | 柳州康云互联科技有限公司 | Cloud in-vitro diagnosis image data processing method based on video analysis |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110136106A (en) * | 2019-05-06 | 2019-08-16 | 腾讯科技（深圳）有限公司 | Recognition methods, system, equipment and the endoscopic images system of medical endoscope image |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107730489A (en) * | 2017-10-09 | 2018-02-23 | 杭州电子科技大学 | Wireless capsule endoscope small intestine disease variant computer assisted detection system and detection method |
| CN109460717B (en) * | 2018-10-22 | 2022-04-19 | 青岛美迪康数字工程有限公司 | Digestive tract confocal laser microscopy endoscope lesion image identification method and device |
| CN109447985B (en) * | 2018-11-16 | 2020-09-11 | 青岛美迪康数字工程有限公司 | Colonoscope image analysis method and device and readable storage medium |
| CN110991561B (en) * | 2019-12-20 | 2020-12-25 | 山东大学齐鲁医院 | Method and system for identifying images of endoscope in lower digestive tract |
| CN111128396B (en) * | 2019-12-20 | 2020-11-03 | 山东大学齐鲁医院 | Digestive tract disease auxiliary diagnosis system based on deep learning |
| CN112529892A (en) * | 2020-12-21 | 2021-03-19 | 苏州中科华影健康科技有限公司 | Digestive tract endoscope lesion image detection method, digestive tract endoscope lesion image detection system and computer storage medium |
-
2021
- 2021-04-13 CN CN202110393884.1A patent/CN113139937B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110136106A (en) * | 2019-05-06 | 2019-08-16 | 腾讯科技（深圳）有限公司 | Recognition methods, system, equipment and the endoscopic images system of medical endoscope image |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113139937A (en) | 2021-07-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107451998B (en) | Fundus image quality control method | |
| EP1849402B1 (en) | Medical image processing device, lumen image processing device, lumen image processing method, and programs for them | |
| CN113139937B (en) | Digestive tract endoscope video image identification method based on deep learning | |
| JP5094036B2 (en) | Endoscope insertion direction detection device | |
| CN111275041B (en) | Endoscope image display method and device, computer equipment and storage medium | |
| CN111655116A (en) | Image diagnosis support device, data collection method, image diagnosis support method, and image diagnosis support program | |
| CN104363815B (en) | Image processing apparatus and image processing method | |
| US9947089B2 (en) | Digital specimen manufacturing device, digital specimen manufacturing method, and digital specimen manufacturing server | |
| CN112257704A (en) | Cervical fluid-based cell digital image classification method based on deep learning detection model | |
| US7907775B2 (en) | Image processing apparatus, image processing method and image processing program | |
| CN110738655B (en) | Image report generation method, device, terminal and storage medium | |
| WO2006126384A1 (en) | Abnormal shadow candidate display method, and medical image processing system | |
| CN110930446B (en) | Pretreatment method and storage device for quantitative analysis of fundus images | |
| US8666135B2 (en) | Image processing apparatus | |
| CN110619318B (en) | Image processing method, microscope, system and medium based on artificial intelligence | |
| US8913807B1 (en) | System and method for detecting anomalies in a tissue imaged in-vivo | |
| US20170323445A1 (en) | Morphology identification in tissue samples based on comparison to named feature vectors | |
| KR20160118037A (en) | Apparatus and method for detecting lesion from medical image automatically | |
| CN114022397B (en) | Endoscope image defogging method and device, electronic equipment and storage medium | |
| CN117522719A (en) | Bronchoscope image auxiliary optimization system based on machine learning | |
| JPH03165732A (en) | Detecting method for insertion direction of endoscope | |
| JP5057651B2 (en) | Lumen image processing apparatus, lumen image processing method, and program therefor | |
| CN114359131A (en) | Helicobacter pylori stomach video full-automatic intelligent analysis system and marking method thereof | |
| WO2020246676A1 (en) | System for automatic diagnosis of uterine cervical cancer | |
| CN110110750B (en) | Original picture classification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |