CN113112475A - Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning - Google Patents

Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning Download PDF

Info

Publication number
CN113112475A
CN113112475A CN202110396463.4A CN202110396463A CN113112475A CN 113112475 A CN113112475 A CN 113112475A CN 202110396463 A CN202110396463 A CN 202110396463A CN 113112475 A CN113112475 A CN 113112475A
Authority
CN
China
Prior art keywords
image
ear
images
region
edge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110396463.4A
Other languages
Chinese (zh)
Other versions
CN113112475B (en
Inventor
梁惠珠
冯跃
林卓胜
朱嘉健
李胜可
刘慧琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuyi University
Original Assignee
Wuyi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuyi University filed Critical Wuyi University
Priority to CN202110396463.4A priority Critical patent/CN113112475B/en
Publication of CN113112475A publication Critical patent/CN113112475A/en
Application granted granted Critical
Publication of CN113112475B publication Critical patent/CN113112475B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a traditional Chinese medicine ear five-zang-organ region segmentation method and a device based on machine learning, wherein the method comprises the following steps: collecting ear images and establishing an image data set; labeling the ear image to obtain a label image; respectively expanding the ear images and the tag images; carrying out Canny operator edge detection on the expansion tag image; respectively preprocessing the extended ear image, the extended tag image and the tag edge image; generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by utilizing the gray level images and the binary images; inputting the RGB three-channel image into a learning network to obtain a predicted image; obtaining a prediction image of each single region of the heart, the liver, the spleen, the lung and the kidney from the prediction image; respectively carrying out expansion and corrosion treatment on the predicted image to obtain a single-region rough edge prediction image; calculating a total loss function; and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain the ear five-organ region segmentation image.

Description

Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning
Technical Field
The invention relates to the field of image processing, in particular to a method and a device for segmenting ear five-organ regions in traditional Chinese medicine based on machine learning.
Background
In recent years, machine learning widely encompasses various fields of images and achieves superior performance such as image recognition, object detection, image segmentation, and the like. It is worth mentioning that the deep learning belonging to the machine learning category applied to the research of medical images has important significance for assisting the clinical diagnosis of physicians, and the tongue diagnosis image processing in the inspection of traditional Chinese medicine has achieved a lot of results. However, the digital auxiliary diagnosis of ear diagnosis is still rare, and especially the digitization of ear inspection does not appear in any form, wherein, image segmentation is a necessary step of clinical application, and although the ear biological information identification based on deep learning appears, the ear region segmentation based on deep learning is less, and because the boundary of each segmentation region is based on the theory of traditional Chinese medicine, the region boundary is not the inherent boundary contour of an object, and has certain ambiguity, the segmentation difficulty is brought.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a traditional Chinese medicine ear five-zang-organ region segmentation method and device based on machine learning, and the traditional Chinese medicine ear five-zang-organ region segmentation method and device based on machine learning enable the corresponding region of the traditional Chinese medicine ear five-zang-organ to be automatically segmented.
In a first aspect, an embodiment of the present invention provides a method for segmenting ear regions and five zang organs in traditional Chinese medicine with the above functions. The method comprises the following steps:
collecting ear images; the acquisition of ear images includes:
the existing public ear data set is fully utilized, and ear images meeting the ear diagnosis requirements of the traditional Chinese medicine are screened out to serve as original data of the data set;
another part is according to traditional chinese medical science ear diagnosis requirement, gather original data by oneself, gather original data for by oneself, collection equipment is traditional chinese medical science tongue picture collection appearance, it is close to after gathering the appearance to gather the left (right) side ear of object, the head rotates to about 30 suitable angle left (right), make ear five zang organs region not sheltered from in the acquisition process, include ear, face and collection instrument edge in the original collection image, the original collection image detects out ear position through trained Opencv ear detector model, it does not contain the condition in ear region completely to detect the square frame existence, consequently regard detection square frame as the center, cut out with 1.6 times that detection square frame is wide and high, in the image after cutting out, complete ear region accounts for about half of whole image area, obtain ear image.
Labeling the ear image to obtain a label image;
respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
carrying out Canny operator edge detection on the expansion tag image to obtain a tag edge image;
respectively preprocessing the extended ear image, the extended tag image and the tag edge image to respectively obtain an RGB three-channel image, a gray image and a binary image;
generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
inputting the RGB three-channel image into a learning network to obtain a predicted image;
obtaining a prediction image of each single region of the heart, the liver, the spleen, the lung and the kidney from the prediction image;
respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
calculating a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction image;
and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
The method for segmenting the regions of the five ear zang organs in the traditional Chinese medicine based on machine learning at least has the technical effect that the regions corresponding to the five ear zang organs in the traditional Chinese medicine can be automatically segmented.
According to the machine learning-based traditional Chinese medicine ear five-organ region segmentation method disclosed by the embodiment of the invention, the ear image and the tag image are expanded to obtain an expanded ear image and an expanded tag image, and the method comprises the following steps:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five pixel values corresponding to five areas of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the areas except the five areas are background areas, and the pixel value is 0, so as to obtain the extended tag image.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the ear image is labeled to obtain a label image, and the method comprises the following steps:
labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram;
and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
The ear images and the corresponding label images checked by professional Chinese physicians are divided into a training set, a verification set and a test set according to the proportion of 60%, 20% and 20%, a deep neural network U-Net is used as a training model, and an Intersection over Unit (Intersection) IoU is used as a technical index. On one hand, in the training process, the intersection ratio response sequence of the five labeled regions (heart, liver, spleen, lung and kidney) in the verification set is lung, kidney, liver, spleen and heart, and on the other hand, the average intersection ratio of the five labeled regions on the test set is more than 0.38 after the training is finished. The data set that meets the criteria of both aspects is determined to be the last data set.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the method for preprocessing the extended ear image and the tag edge image respectively comprises the following steps:
and performing position shearing and normalization operation on the extended ear image to obtain the RGB three-channel image with the size of W x H.
And cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W x H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and the gray level pixel values of the six areas are respectively 0,1, 2, 3, 4 and 5.
And cutting the positions of the label edge images corresponding to the extended ear images to obtain the binary image with the size of W x H.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the gray level image and the binary image are used for generating edge images of each single region of a heart, a liver, a spleen, a lung and a kidney, and the method comprises the following steps:
let flabel(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let fedge(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, gi(x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure BDA0003018755690000041
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H being the widths and heights of the grayscale image and the binary image, respectively, g1、g2、g3、g4、g5The single region edge images of heart, liver, spleen, lung, kidney, respectively.
The machine learning-based traditional Chinese medicine ear five-organ region segmentation method of claim 1, wherein inputting the RGB three-channel image into a learning network to obtain a predicted image comprises:
and (3) passing the RGB three-channel image through a convolution layer with two convolution kernels of 3 x 3 to obtain a characteristic F with 64 channels and H multiplied by W sizeorigin；
The feature ForiginObtaining feature F by PAM module1。
The feature F1Obtaining feature F by PAM module2。
The feature F1And said feature F2And performing channel splicing on two results obtained by multiplying the two results by the coefficients of 0.75 and 0.25 respectively, and obtaining the predicted image with the channel of 6 and the size of H multiplied by W through a convolution layer with a convolution kernel of 3 x 3.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the single region prediction images of the heart, the liver, the spleen, the lung and the kidney are obtained from the prediction images, and the method comprises the following steps:
taking the maximum value of the predicted image in the dimension of a channel to obtain a dimension gray image with the size of H multiplied by W and the pixel values only having six pixel values of 0,1, 2, 3, 4 and 5, wherein the regions corresponding to the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, there are
Figure BDA0003018755690000051
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the dimensional grayscale image, respectively, and P (x, y) {0,1, …,5}, P (x, y) }1、p2、p3、p4、p5The single region of heart, liver, spleen, lung, kidney, respectively, predicts the image.
According to the traditional Chinese medicine ear five-heart region segmentation method based on machine learning, the single region prediction images of the heart, the liver, the spleen, the lung and the kidney are respectively subjected to expansion and erosion processing to obtain single region rough edge prediction images of the corresponding regions, and the method comprises the following steps:
defining a structural element B, wherein the size of the structural element B is 3 x 3, the origin is positioned in the center, and the element values are all 1;
expanding and corroding the single-region predicted images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single-region predicted image and the corroded single-region predicted image, and subtracting the corroded single-region predicted image and the single-region predicted image to obtain a corresponding rough edge prediction image of the region;
let pei(x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure BDA0003018755690000052
In the formula, symbol
Figure BDA0003018755690000053
For expanding and transportingComputing and symbol
Figure BDA0003018755690000054
For the erosion operation, i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the image, pe, respectivelyi(x,y)∈{0,1}，，pe1、pe2、pe3、pe4、pe5The single region rough edge prediction images of heart, liver, spleen, lung, kidney, respectively.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, a total loss function is calculated, wherein the total loss function comprises a first loss function of the gray-scale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction image:
the first loss function is
Figure BDA0003018755690000061
Wherein x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are widths and heights of the grayscale image and the prediction image, respectively, C ∈ {1, …, C }, C is a division class number,
Figure BDA0003018755690000062
for the binary image with class c being one-hot encoded,
Figure BDA0003018755690000065
qc(x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure BDA0003018755690000063
Wherein i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the single-region edge image and the single-region rough edge prediction map, respectively，gi(x, y) is the ith area edge image, gi(x, y) is an element {0,1}, AND is an intersection operation, pei(x, y) is the ith area coarse edge prediction image, pei(x,y)∈{0,1}。
The total loss function is
Figure BDA0003018755690000064
In the formula, the parameter N is liThe number of the carbon atoms is not 0.
In a second aspect, an embodiment of the present invention further provides a device for segmenting the five ear regions of traditional Chinese medicine based on machine learning, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the method for segmenting the five ear regions of traditional Chinese medicine based on machine learning according to the first aspect of the present invention when executing the program.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flowchart of a method for segmenting the five ear zang-organ regions according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a process of segmenting the five ear zang-organ regions according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the positioning of the standard auricular point in China according to the embodiment of the present invention;
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
Referring to fig. 1,2 and 3, a method for segmenting the five ear regions of chinese medicine based on machine learning according to a first embodiment of the present invention is described. The method comprises the following steps:
s101: collecting ear images; the acquisition of ear images includes:
the existing public ear data set is fully utilized, and ear images meeting the ear diagnosis requirements of the traditional Chinese medicine are screened out to serve as original data of the data set;
another part is according to traditional chinese medical science ear diagnosis requirement, gather original data by oneself, gather original data for by oneself, collection equipment is traditional chinese medical science tongue picture collection appearance, it is close to after gathering the appearance to gather the left (right) side ear of object, the head rotates to about 30 suitable angle left (right), make ear five zang organs region not sheltered from in the acquisition process, include ear, face and collection instrument edge in the original collection image, the original collection image detects out ear position through trained Opencv ear detector model, it does not contain the condition in ear region completely to detect the square frame existence, consequently regard detection square frame as the center, cut out with 1.6 times that detection square frame is wide and high, in the image after cutting out, complete ear region accounts for about half of whole image area, obtain ear image.
S102: labeling the ear image to obtain a label image;
s103: respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
s104: carrying out Canny operator edge detection on the expansion tag image to obtain a tag edge image;
s105: respectively preprocessing the extended ear image, the extended tag image and the tag edge image to respectively obtain an RGB three-channel image, a gray image and a binary image;
s106: generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
s107: inputting the RGB three-channel image into a learning network to obtain a predicted image;
s108: obtaining a prediction image of each single region of the heart, the liver, the spleen, the lung and the kidney from the prediction image;
s109: respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
s110: calculating a total loss function, wherein the total loss function comprises a first loss function of the gray-scale image and the prediction image and a second loss function of the single-region edge image and the single-region rough edge prediction image;
s111: and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
The method for segmenting the regions of the five ear zang organs in the traditional Chinese medicine based on machine learning at least has the technical effect that the regions corresponding to the five ear zang organs in the traditional Chinese medicine can be automatically segmented.
Referring to fig. 1 and 2, according to a machine learning-based traditional Chinese medicine ear five-organ region segmentation method according to a first embodiment of the present invention, labeling an ear image to obtain a label image, includes: labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram; and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
The ear images and the corresponding label images checked by professional Chinese physicians are divided into a training set, a verification set and a test set according to the proportion of 60%, 20% and 20%, a deep neural network U-Net is used as a training model, and an Intersection over Unit (Intersection) IoU is used as a technical index. In one aspect, during training, the intersection-specific response order of the five labeled regions (heart, liver, spleen, lung, kidney) in the set is verified as lung, kidney, liver, spleen, heart. On the other hand, the average intersection ratio of five labeled regions on the test set after training is over 0.38. The data set that meets the criteria of both aspects is determined to be the last data set.
Referring to fig. 1 and 2, according to a method for segmenting a five-organ ear region in traditional Chinese medicine based on machine learning according to a first embodiment of the present invention, the expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image includes:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five pixel values corresponding to five areas of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the areas except the five areas are background areas, and the pixel value is 0, so as to obtain the extended tag image.
Referring to fig. 1 and 2, a method for segmenting a five-organ ear region in traditional Chinese medicine based on machine learning according to a first embodiment of the present invention respectively preprocesses the extended ear image and the tag edge image, including:
and performing position shearing and normalization operation on the extended ear image to obtain the RGB three-channel image with the size of W x H.
And cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W x H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and the gray level pixel values of the six areas are respectively 0,1, 2, 3, 4 and 5.
And cutting the positions of the label edge images corresponding to the extended ear images to obtain the binary image with the size of W x H.
Referring to fig. 1 and 2, the method for segmenting five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention, which generates edge images of each single region of heart, liver, spleen, lung and kidney by using the gray image and the binary image, includes:
let flabel(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let fedge(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, gi(x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure BDA0003018755690000101
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H being the widths and heights of the grayscale image and the binary image, respectively, g1、g2、g3、g4、g5The single region edge images of heart, liver, spleen, lung, kidney, respectively.
Referring to fig. 1 and 2, the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the embodiment of the present invention generates edge images of each single region of the heart, liver, spleen, lung and kidney by using the gray image and the binary image, including:
referring to fig. 1 and 2, a method for segmenting ear regions of traditional Chinese medicine based on machine learning according to a first embodiment of the present invention is characterized in that the RGB three-channel image is input to a learning network to obtain a predicted image, and the method includes:
and (3) passing the RGB three-channel image through a convolution layer with two convolution kernels of 3 x 3 to obtain a characteristic F with 64 channels and H multiplied by W sizeorigin；
The feature ForiginObtaining feature F by PAM module1；
The feature F1Obtaining feature F by PAM module2；
The feature F1And said feature F2And performing channel splicing on two results obtained by multiplying the two results by the coefficients of 0.75 and 0.25 respectively, and obtaining the predicted image with the channel of 6 and the size of H multiplied by W through a convolution layer with a convolution kernel of 3 x 3.
Referring to fig. 1 and 2, the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention obtains, from the predicted images, predicted images of individual regions of the heart, liver, spleen, lung and kidney, including:
taking the maximum value of the predicted image in the dimension of a channel to obtain a dimension gray image with the size of H multiplied by W and the pixel values only having six pixel values of 0,1, 2, 3, 4 and 5, wherein the regions corresponding to the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, there are
Figure BDA0003018755690000111
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the dimensional grayscale image, respectively, and P (x, y) {0,1, …,5}, P (x, y) }1、p2、p3、p4、p5The single region of heart, liver, spleen, lung, kidney, respectively, predicts the image.
Referring to fig. 1 and fig. 2, according to the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention, the expanding and eroding processing is performed on the predicted images of the single regions of the heart, liver, spleen, lung and kidney respectively to obtain the single-region rough edge prediction map of the corresponding region, including:
defining a structural element B, wherein the size of the structural element B is 3 x 3, the origin is positioned in the center, and the element values are all 1;
expanding and corroding the single-region predicted images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single-region predicted image and the corroded single-region predicted image, and subtracting the corroded single-region predicted image and the single-region predicted image to obtain a corresponding rough edge prediction image of the region;
let pei(x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure BDA0003018755690000121
In the formula, symbol
Figure BDA0003018755690000122
For the operation of dilation, sign
Figure BDA0003018755690000123
For the erosion operation, i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the image, pe, respectivelyi(x,y)∈{0,1}，，pe1、pe2、pe3、pe4、pe5The single region rough edge prediction images of heart, liver, spleen, lung, kidney, respectively.
Referring to fig. 1 and 2, a method for segmenting a five-zang ear region in traditional Chinese medicine based on machine learning according to a first embodiment of the present invention is characterized by comprising: calculating a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction image:
the first loss function is
Figure BDA0003018755690000124
Wherein x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are widths and heights of the grayscale image and the prediction image, respectively, C ∈ {1, …, C }, C is a division class number,
Figure BDA0003018755690000125
for the binary image with class c being one-hot encoded,
Figure BDA0003018755690000126
qc(x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure BDA0003018755690000127
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the single-region edge image and the single-region rough edge prediction map, respectively, gi(x, y) is the ith area edge image, gi(x, y) is an element {0,1}, AND is an intersection operation, pei(x, y) is the ith area coarse edge prediction image, pei(x,y)∈{0,1}。
The total loss function is
Figure BDA0003018755690000131
In the formula, the parameter N is liThe number of the carbon atoms is not 0.
A traditional Chinese medicine ear five-organ region segmentation device based on machine learning is characterized by comprising at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any of the above ear-five zang region segmentation in traditional Chinese medicine.
It should be recognized that the method steps in embodiments of the present invention may be embodied or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method may use standard programming techniques. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.
A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (10)

1. A traditional Chinese medicine ear five-organ region segmentation method based on machine learning is characterized by comprising the following steps:
collecting ear images and establishing an image data set;
labeling the ear image to obtain a label image;
respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
carrying out Canny operator edge detection on the expansion tag image to obtain a tag edge image;
respectively preprocessing the extended ear image, the extended tag image and the tag edge image to respectively obtain an RGB three-channel image, a gray image and a binary image;
generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
inputting the RGB three-channel image into a learning network to obtain a predicted image;
obtaining a prediction image of each single region of the heart, the liver, the spleen, the lung and the kidney from the prediction image;
respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
calculating a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction image;
and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
2. The method for segmenting ear regions of traditional Chinese medicine based on machine learning according to claim 1, wherein labeling the ear images to obtain labeled images comprises:
labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram;
and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
3. The method of machine learning-based segmentation of five ear regions in traditional Chinese medicine according to claim 1, wherein the expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image comprises:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five pixel values corresponding to five areas of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the areas except the five areas are background areas, and the pixel value is 0, so as to obtain the extended tag image.
4. The method of machine learning-based segmentation of five ear regions in traditional Chinese medicine according to claim 1, wherein the pre-processing of the augmented ear image and the tag edge image respectively comprises:
and performing position shearing and normalization operation on the extended ear image to obtain the RGB three-channel image with the size of W x H.
And cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W x H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and the gray level pixel values of the six areas are respectively 0,1, 2, 3, 4 and 5.
And cutting the positions of the label edge images corresponding to the extended ear images to obtain the binary image with the size of W x H.
5. The method of claim 1, wherein the generating the edge image of each single region of heart, liver, spleen, lung and kidney by using the gray image and the binary image comprises:
let flabel(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let fedge(x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, gi(x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure FDA0003018755680000031
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H being the widths and heights of the grayscale image and the binary image, respectively, g1、g2、g3、g4、g5The single region edge images of heart, liver, spleen, lung, kidney, respectively.
6. The method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to claim 1, wherein the inputting of the RGB three-channel image into a learning network to obtain a predicted image comprises:
and (3) passing the RGB three-channel image through a convolution layer with two convolution kernels of 3 x 3 to obtain a characteristic F with 64 channels and H multiplied by W sizeorigin；
The feature ForiginObtaining feature F by PAM module1。
The feature F1Obtaining feature F by PAM module2。
The feature F1And said feature F2And performing channel splicing on two results obtained by multiplying the two results by the coefficients of 0.75 and 0.25 respectively, and obtaining the predicted image with the channel of 6 and the size of H multiplied by W through a convolution layer with a convolution kernel of 3 x 3.
7. The method according to claim 1, wherein obtaining a prediction image of each single region of heart, liver, spleen, lung and kidney from the prediction image comprises:
taking the maximum value of the predicted image in the dimension of a channel to obtain a dimension gray image with the size of H multiplied by W and the pixel values only having six pixel values of 0,1, 2, 3, 4 and 5, wherein the regions corresponding to the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, there are
Figure FDA0003018755680000032
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the dimensional grayscale image, respectively, and P (x, y) {0,1, …,5}, P (x, y) }1、p2、p3、p4、p5The single region of heart, liver, spleen, lung, kidney, respectively, predicts the image.
8. The method according to claim 1, wherein the expanding and eroding the single region prediction images of the heart, liver, spleen, lung and kidney to obtain the single region rough edge prediction map of the corresponding region comprises:
defining a structural element B, wherein the size of the structural element B is 3 x 3, the origin is positioned in the center, and the element values are all 1;
expanding and corroding the single-region predicted images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single-region predicted image and the corroded single-region predicted image, and subtracting the expanded single-region predicted image and the corroded single-region predicted image to obtain the corresponding rough edge prediction image of the region;
let pei(x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure FDA0003018755680000041
In the formula, symbol
Figure FDA0003018755680000042
For the operation of dilation, sign
Figure FDA0003018755680000043
For the erosion operation, i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the image, pe, respectivelyi(x,y)∈{0,1}，，pe1、pe2、pe3、pe4、pe5The single region rough edge prediction images of heart, liver, spleen, lung, kidney, respectively.
9. The machine learning-based traditional Chinese medicine ear five-organ region segmentation method according to claim 1, wherein a total loss function is calculated, the total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction map:
the first loss function is
Figure FDA0003018755680000044
Wherein x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are widths and heights of the grayscale image and the prediction image, respectively, C ∈ {1, …, C }, C is a division class number,
Figure FDA0003018755680000045
for the binary image with class c being one-hot encoded,
Figure FDA0003018755680000046
qc(x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure FDA0003018755680000051
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the single-region edge image and the single-region rough edge prediction map, respectively, gi(x, y) is the ith area edge image, gi(x, y) is an element {0,1}, AND is an intersection operation, pei(x, y) is the ith area coarse edge prediction image, pei(x,y)∈{0,1}。
The total loss function is
Figure FDA0003018755680000052
In the formulaThe parameter N is liThe number of the carbon atoms is not 0.
10. A traditional Chinese medicine ear five-organ region segmentation device based on machine learning is characterized by comprising at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
CN202110396463.4A 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning Active CN113112475B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110396463.4A CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110396463.4A CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Publications (2)

Publication Number Publication Date
CN113112475A true CN113112475A (en) 2021-07-13
CN113112475B CN113112475B (en) 2023-04-18

Family

ID=76716660

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110396463.4A Active CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Country Status (1)

Country Link
CN (1) CN113112475B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114155240A (en) * 2021-12-13 2022-03-08 韩松洁 Ear acupoint detection method and device and electronic equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180276825A1 (en) * 2017-03-23 2018-09-27 Petuum, Inc. Structure Correcting Adversarial Network for Chest X-Rays Organ Segmentation
CN108596193A (en) * 2018-04-27 2018-09-28 东南大学 A kind of method and system for building the deep learning network structure for ear recognition
WO2020119679A1 (en) * 2018-12-14 2020-06-18 深圳先进技术研究院 Three-dimensional left atrium segmentation method and apparatus, terminal device, and storage medium
CN111402268A (en) * 2020-03-16 2020-07-10 苏州科技大学 Method for segmenting liver and focus thereof in medical image
CN111583287A (en) * 2020-04-23 2020-08-25 浙江大学 Deep learning model training method for fine portrait picture segmentation
CN112070752A (en) * 2020-09-10 2020-12-11 杭州晟视科技有限公司 Method, device and storage medium for segmenting auricle of medical image
AU2020103905A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180276825A1 (en) * 2017-03-23 2018-09-27 Petuum, Inc. Structure Correcting Adversarial Network for Chest X-Rays Organ Segmentation
CN108596193A (en) * 2018-04-27 2018-09-28 东南大学 A kind of method and system for building the deep learning network structure for ear recognition
WO2020119679A1 (en) * 2018-12-14 2020-06-18 深圳先进技术研究院 Three-dimensional left atrium segmentation method and apparatus, terminal device, and storage medium
CN111402268A (en) * 2020-03-16 2020-07-10 苏州科技大学 Method for segmenting liver and focus thereof in medical image
CN111583287A (en) * 2020-04-23 2020-08-25 浙江大学 Deep learning model training method for fine portrait picture segmentation
CN112070752A (en) * 2020-09-10 2020-12-11 杭州晟视科技有限公司 Method, device and storage medium for segmenting auricle of medical image
AU2020103905A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ALI ABD ALMISREB ET AL: "Kernel Graph Cut for Robust Ear Segmentation in", 《IEEE》 *
田莹 等: "一种基于边界特征的耳廓提取新方法", 《光电工程》 *
郑冬梅 等: "中医色诊客观化方法研究", 《世界科学技术—中医药现代化★中医研究》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114155240A (en) * 2021-12-13 2022-03-08 韩松洁 Ear acupoint detection method and device and electronic equipment

Also Published As

Publication number Publication date
CN113112475B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
CN110059697B (en) Automatic lung nodule segmentation method based on deep learning
US9737257B2 (en) Estimating and predicting tooth wear using intra-oral 3D scans
US7916919B2 (en) System and method for segmenting chambers of a heart in a three dimensional image
US8009900B2 (en) System and method for detecting an object in a high dimensional space
CN110909618B (en) Method and device for identifying identity of pet
CN112967236B (en) Image registration method, device, computer equipment and storage medium
JP4414401B2 (en) Facial feature point detection method, apparatus, and program
CN102436636B (en) Method and system for segmenting hair automatically
CN111144486B (en) Heart nuclear magnetic resonance image key point detection method based on convolutional neural network
US20090252429A1 (en) System and method for displaying results of an image processing system that has multiple results to allow selection for subsequent image processing
US20230005140A1 (en) Automated detection of tumors based on image processing
CN110705565A (en) Lymph node tumor region identification method and device
Han et al. Robust anatomical landmark detection with application to MR brain image registration
CN112651953A (en) Image similarity calculation method and device, computer equipment and storage medium
Li et al. A visual saliency-based method for automatic lung regions extraction in chest radiographs
Wang et al. Tooth identification based on teeth structure feature
CN111105427A (en) Lung image segmentation method and system based on connected region analysis
CN110992310A (en) Method and device for determining partition where mediastinal lymph node is located
Chen et al. Image segmentation based on mathematical morphological operator
CN113112475B (en) Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning
CN110232684B (en) Automatic three-dimensional medical image segmentation method based on spectrum analysis
CN111275754B (en) Face acne mark proportion calculation method based on deep learning
Mortaheb et al. Metal artifact reduction and segmentation of dental computerized tomography images using least square support vector machine and mean shift algorithm
JP2012048326A (en) Image processor and program
JP2004188202A (en) Automatic analysis method of digital radiograph of chest part

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant