CN117153343B - Placenta multiscale analysis system - Google Patents
Placenta multiscale analysis system Download PDFInfo
- Publication number
- CN117153343B CN117153343B CN202311029039.1A CN202311029039A CN117153343B CN 117153343 B CN117153343 B CN 117153343B CN 202311029039 A CN202311029039 A CN 202311029039A CN 117153343 B CN117153343 B CN 117153343B
- Authority
- CN
- China
- Prior art keywords
- placenta
- image
- analysis
- general
- pathological section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000002826 placenta Anatomy 0.000 title claims abstract description 233
- 238000004458 analytical method Methods 0.000 title claims abstract description 99
- 230000001575 pathological effect Effects 0.000 claims abstract description 89
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000003745 diagnosis Methods 0.000 claims abstract description 27
- 238000010191 image analysis Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 13
- 210000005059 placental tissue Anatomy 0.000 claims abstract description 12
- 238000007405 data analysis Methods 0.000 claims abstract description 9
- 230000003902 lesion Effects 0.000 claims abstract description 9
- 230000007170 pathology Effects 0.000 claims description 29
- 210000003954 umbilical cord Anatomy 0.000 claims description 17
- 230000003169 placental effect Effects 0.000 claims description 10
- 210000003754 fetus Anatomy 0.000 claims description 9
- 230000002452 interceptive effect Effects 0.000 claims description 7
- 230000035935 pregnancy Effects 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 7
- 210000004204 blood vessel Anatomy 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 6
- 206010028980 Neoplasm Diseases 0.000 claims description 5
- 201000011510 cancer Diseases 0.000 claims description 5
- 230000001605 fetal effect Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 230000008774 maternal effect Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 2
- 238000010827 pathological analysis Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 7
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 201000010099 disease Diseases 0.000 description 3
- 230000036541 health Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 229920002678 cellulose Polymers 0.000 description 2
- 239000001913 cellulose Substances 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 208000001951 Fetal Death Diseases 0.000 description 1
- 206010055690 Foetal death Diseases 0.000 description 1
- 210000001015 abdomen Anatomy 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004064 dysfunction Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002124 endocrine Effects 0.000 description 1
- 231100000479 fetal death Toxicity 0.000 description 1
- 230000004578 fetal growth Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000012010 growth Effects 0.000 description 1
- 238000002513 implantation Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000002207 metabolite Substances 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 230000008506 pathogenesis Effects 0.000 description 1
- 230000007310 pathophysiology Effects 0.000 description 1
- 208000025934 placenta disease Diseases 0.000 description 1
- 230000002028 premature Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003777 tissue processing method Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 210000004291 uterus Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/67—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for remote operation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30044—Fetus; Embryo
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Biomedical Technology (AREA)
- Radiology & Medical Imaging (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- General Business, Economics & Management (AREA)
- Business, Economics & Management (AREA)
- Pathology (AREA)
- Databases & Information Systems (AREA)
- Investigating Or Analysing Biological Materials (AREA)
Abstract
The invention discloses a placenta multi-scale analysis system, which comprises: placenta general analysis module: the method comprises the steps of uniformly numbering placenta general samples, and obtaining the thickness, weight and image data of the placenta general samples; placenta pathological section scanning module: the method comprises the steps of scanning and analyzing a placenta general sample, obtaining placenta tissues of suspicious lesions, and processing the placenta tissues to obtain placenta pathological section images; placenta image analysis module: the system comprises a remote consultation client and a mobile client, wherein the remote consultation client is used for performing medical record data analysis on placenta general images and placenta pathological section images, generating an analysis report and sending the analysis report to the remote consultation client or the mobile client; remote consultation client: the method is used for a pathologist to browse analysis reports, placenta general images or placenta pathological section images and fill in diagnosis reports; the mobile client: for browsing images or analysis reports and viewing diagnostic reports; the invention can improve the accuracy and efficiency of placenta pathological diagnosis and analysis.
Description
Technical Field
The invention relates to the technical field of biomedical instruments, in particular to a placenta multi-scale analysis system.
Background
Placenta is an organ of a fetus that is co-involved in the formation of the components of the fetus and the mother. Placenta is a channel for the fetus to absorb nutrients and excrete metabolites, and is an important endocrine organ. When the placenta is dysfunctional due to disease or various causes, the growth and development of the fetus can be severely affected, even with the risk of fetal death in the abdomen. Briefly, placenta is the window for understanding the overall process from gestation to delivery.
Analysis of placenta can (1) assess fetal health: placental pathology analysis helps to understand the fetal growth environment in utero and factors that may affect fetal health; (2) found complications: placenta pathological analysis can find some placenta-related abnormalities and complications, such as premature placenta peeling, placenta implantation abnormalities, placenta dysfunction and the like, which are helpful for doctors to know the health condition of mother and infant and provide better medical advice and intervention for patients; (3) guiding future pregnancy risk management: the method is helpful for evaluating the risk of the pregnant woman in future pregnancy, and through diagnosis of placenta lesions, doctors can provide personalized advice and preventive measures for the pregnant woman, so that the risk of the future pregnancy is reduced; (4) facilitating scientific research and education: placenta pathology analysis is of great importance for deep understanding of pathogenesis, pathophysiology and etiology of placenta diseases, and the information is of great value for developing new diagnosis and treatment methods and improving medical education quality.
However, there are a number of disadvantages in the current placenta analysis process, such as insufficient resources of pathologists (registered medical doctors in China are seriously lacking, and resource allocation is unbalanced, so that the pathologists in primary hospitals are more lacking); the placenta pathological diagnosis is usually given by the manual interpretation of pathologists, and the accuracy and consistency of the analysis and judgment conclusion are not high; the clinical need for placenta is not strong enough compared to other cancer diseases; placenta pathology related research is relatively lacking, and data sets and targeted analysis software and algorithms are relatively scarce; the placenta pathology has relatively poor specificity and small difference between different pathological tissues.
Disclosure of Invention
In order to solve the technical problems, the invention provides a placenta multi-scale analysis system for improving accuracy and efficiency of placenta pathological diagnosis analysis.
To achieve the above object, the present invention provides a placenta multiscale analysis system, comprising:
placenta general analysis module: the method comprises the steps of uniformly numbering placenta general samples, and obtaining the thickness, weight and image data of the placenta general samples;
placenta pathological section scanning module: the method comprises the steps of scanning and analyzing a placenta general sample to obtain placenta tissues of suspicious lesions, and processing the placenta tissues of the suspicious lesions to obtain placenta pathological section images;
placenta image analysis module: the system comprises a remote consultation client and a mobile client, wherein the remote consultation client is used for performing pathological data analysis on placenta general images and placenta pathological section images, generating an analysis report and sending the analysis report to the remote consultation client or the mobile client;
remote consultation client: the method comprises the steps that a pathologist browses the analysis report, placenta general image or placenta pathological section image and fills in a diagnosis report;
the mobile client: for viewing said placenta general image, said placenta pathological section image or said analysis report, and viewing said diagnosis report;
the placenta general analysis module, the placenta pathological section scanning module, the placenta image analysis module, the remote consultation client and the mobile client are sequentially connected.
Preferably, the placenta general analysis module comprises:
placenta data acquisition unit: the method comprises the steps of acquiring macroscopic image, weight and thickness data of a placenta general sample, and carrying out pathological numbering on the placenta general sample as a sample tracing basis;
placenta general analysis unit: the placenta comprises a placenta main sample, a placenta main sample and a placenta image processing device.
Preferably, the placenta details include: the net weight of placenta, volume, density, gestational week, number of fetuses, percentage of gestation Zhou Chongliang, length of umbilical cord, number of umbilical cord helices, direction of umbilical cord helices, color of fetal face, and whether maternal face is intact.
Preferably, the placenta pathological section scanning module comprises:
placenta pathological section acquisition unit: the method comprises the steps of slicing placenta tissues of suspicious lesions through placenta images to obtain placenta pathological section specimens, wherein the placenta pathological section specimens contain identification information of placenta pathological numbers;
placenta pathological section analysis unit: the method comprises the steps of previewing a placenta pathological section specimen through special software of a placenta pathological section scanner, and shooting a label image and a section image of a pathological section;
placenta pathological section image acquisition unit: and the method is used for extracting the pathological numbers in the label images, identifying the slice images, taking the slice images as a navigation chart to determine a scanning area, and controlling a scanner to scan so as to obtain high-definition placenta pathological slice images.
Preferably, the placenta image analysis module comprises:
user interaction unit: the data reading and processing operation is carried out through the interaction page;
a pathology data analysis unit: the method comprises the steps of analyzing a placenta general image or a placenta pathological section image through the interactive page to obtain an analysis result;
database server: for reading placenta general data, placenta pathological section data and storing sample analysis results.
Preferably, the data reading and processing operation through the interactive page includes:
reading and processing data through an interactive page of a placenta image analysis subunit, wherein the left side of the placenta image analysis subunit lists all data in a working path in a tree view form, and a right side main working area is used for displaying all files under a current node; wherein the analyzed pathology number or the display icon of the image is marked with an analyzed mark symbol; the manner in which pathology numbers are analyzed includes batch analysis and individual analysis.
Preferably, the pathology data analysis unit includes:
an image labeling subunit: the marking tool is used for manually marking the target area in the placenta general image and the placenta pathological section image; wherein the target area in the placenta general image is placenta, umbilical cord and blood vessel area, and the target area in the placenta pathological section image is cancer area;
image-text analysis subunit: the method comprises the steps of arranging index parameters and analysis reports corresponding to all pathology numbers, inputting placenta general images, placenta pathological section images and diagnosis reports into an image-text analysis model, and extracting image characteristic data and text data by using a frozen image encoder;
an index calculation subunit: the method comprises the steps of inputting the image characteristic data and the text data into different transformation models respectively, obtaining a target region segmentation result of an image, and calculating index parameters;
an analysis report generation unit: and the analysis report is used for splicing the result output by the transducer model and inputting the result into an image-text matcher, inputting the spliced image characteristics and text characteristics into an image-text comparison learning subunit, learning the relation between image data and text data, and inputting the spliced text characteristics into an image-based text generator to generate the analysis report.
Preferably, the image encoder is a VIT model-based image encoder, and the characteristic information of the placenta general image and the placenta pathological section image are respectively extracted by using two identical image encoders;
the image-text comparison learning subunit selects the highest value as the similarity of the image-text comparison learning subunit by comparing the similarity between all image features and text features of one placenta image;
and calculating the matching score of the image-text pair through the image-text matcher, and finally combining the matching score with the similarity of the image-text comparison learning subunit and the loss value of the text generator of the image to optimize the transducer model.
Preferably, the remote consultation client includes:
auxiliary diagnosis and result acquisition unit: when the method is used for training an auxiliary image-text multi-mode model, the placenta general image and the placenta pathological section image are marked, and a diagnosis report is filled in below the analysis report; when the diagnosis result of the placenta is needed to be obtained, selecting placenta data to be analyzed to generate a list to be analyzed, and then clicking for analysis to obtain an analysis report and a diagnosis report.
Compared with the prior art, the invention has the following advantages and technical effects:
the invention collects macroscopic data such as appearance, weight and the like of placenta by a placenta general analysis module, and collects microscopic images of pathological sections of the placenta by a full-glass scanner; adopting an image-text multi-mode model based on deep learning to macroscopically identify the major placenta and the abnormal umbilical cord at a pixel level, and identifying the central point of the umbilical cord insertion position; and (3) carrying out pixel-level identification analysis on placenta villi, blood vessels and cellulose on microcosmic basis to obtain parameters such as target area ratio and the like, and realizing the predictive diagnosis of common diseases of automatic fetuses and parents based on placenta.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application, illustrate and explain the application and are not to be construed as limiting the application. In the drawings:
FIG. 1 is a schematic diagram of a placenta multi-scale analysis system according to an embodiment of the present invention;
FIG. 2 is a diagram of a placenta general analyzer in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a placenta image analysis software interface according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a list to be analyzed in placenta image analysis software according to an embodiment of the present invention;
FIG. 5 is a functional schematic diagram of a placenta image analysis system according to an embodiment of the present invention;
FIG. 6 is a diagram of a multi-scale based image-text model structure in an embodiment of the invention;
fig. 7 is a diagram showing a placenta pathological analysis report in the embodiment of the present invention.
Detailed Description
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
The invention provides a placenta multi-scale analysis system, as shown in figure 1, comprising:
placenta general analysis module: the method comprises the steps of uniformly numbering placenta general samples, and obtaining the thickness, weight and image data of the placenta general samples;
macroscopic images of the placenta, weight and thickness data of the placenta are collected through placenta general analysis software, and a series of index parameters of the placenta general are obtained through software analysis.
As shown in fig. 1-4, the method specifically comprises:
(1) Collecting macroscopic image, weight and thickness data of the placenta, and collecting two-dimensional code image of the placenta general sample;
the scanner software generates a data storage path according to the two-dimension code information, and stores the relevant data of all the samples into a database server;
the placenta general analysis software analyzes all data, obtains the information of the net weight, volume, density, gestational week, number of fetuses, percentage of gestation Zhou Chongliang, umbilical cord length, umbilical cord spiral number, umbilical cord spiral direction, color of fetal surface, whether maternal surface is complete and the like of the placenta, stores the information in a database server path associated with pathological numbering, and generates an analysis report;
the scanning process allows multiple measurement and analysis of the same placenta, and the images collected multiple times are named and distinguished in a pathological number and suffix mode;
placenta general analysis software processes placenta data singly or in batches according to a list to be analyzed selected by a user.
(2) Placenta pathological section scanning module: the placenta tissue processing method comprises the steps of scanning and analyzing a placenta general sample, obtaining placenta tissues of suspicious lesions, and processing the placenta tissues to obtain placenta pathological section images;
the method for acquiring the pathological section image of the placenta by placenta pathological section scanning software specifically comprises the following steps:
after carrying out macroscopic scanning analysis on placenta, a doctor carries out slicing treatment on placenta tissues which can be diseased to obtain placenta pathological section samples, and prints or pastes two-dimensional codes containing placenta pathological numbers;
the placenta pathological section scanner and special software preview the placenta pathological section first, shoot the label image and slice image of the pathological section;
extracting a pathology number in the tag image for identifying the slice image, and the pathology number is also used for correlating the general data of the placenta;
placenta pathology section scanner software takes a section image as a navigation map to determine a scanning area, and controls the scanner to scan the area to obtain a high-definition section image, and the section image is stored in a database server path associated with a pathology number of the sample.
(3) Placenta image analysis module: the system comprises a remote consultation client and a mobile client, wherein the remote consultation client is used for performing medical record data analysis on placenta general images and placenta pathological section images, generating an analysis report and sending the analysis report to the remote consultation client or the mobile client;
and analyzing the placenta general data and the slice images by placenta image analysis software to obtain a diagnosis report of the placenta general.
As shown in fig. 5-7, the placenta image analysis module includes:
user interaction unit: the data reading and processing operation is carried out through the interaction page;
the left side of placenta image analysis software lists all data in a working path in a tree view mode, and a large-area main working area on the right side is used for displaying all files under the current node;
the doctor selects a certain node, and if the node is not a pathological number, the child nodes below the selected node are displayed in the main working area;
if the node is a pathology number, displaying all placenta general images and placenta pathology section images of the pathology number in a main working area;
the doctor can select a plurality of pathology numbers for batch analysis, and can select a single pathology number for independent analysis;
the pathology number or image that has been analyzed, the icon displayed will have the sign that has been analyzed.
A pathology data analysis unit: the method comprises the steps of analyzing a placenta general image or a placenta pathological section image through the interactive page to obtain an analysis result;
the method specifically comprises the following steps:
an image labeling subunit: the marking tool is used for manually marking the target area in the placenta general image and the placenta pathological section image; wherein the target area in the placenta general image is placenta, umbilical cord and blood vessel area, and the target area in the placenta pathological section image is cancer area;
the general images and pathological section images corresponding to all pathological numbers are arranged;
labeling placenta, umbilical cord and blood vessel regions of the general image with labeling software;
the cancer area of the pathological section image is marked by marking software.
Image-text analysis subunit: the method comprises the steps of arranging index parameters and analysis reports corresponding to all pathology numbers, inputting placenta general images, placenta pathological section images and diagnosis reports into an image-text analysis model, and extracting image characteristic data and text data by using a frozen image encoder;
arranging placenta index parameters of all pathological numbers in a database server and corresponding diagnosis reports;
an image encoder based on a VIT model is selected, the encoder is frozen to reduce the calculation amount, and the characteristic information of a general image and a pathological section image is respectively extracted by using two identical encoders.
An index calculation subunit: the method comprises the steps of inputting the image characteristic data and the text data into different transformation models respectively, obtaining a target region segmentation result of an image, and calculating index parameters;
three transducer modules are designed to respectively learn information of two types of image data and one type of text data, wherein the two transducer modules for image feature learning have the same architecture: the text feature learning system comprises a self-attention layer, a cross-attention layer and a feedforward layer, wherein a transducer module for text feature learning consists of the self-attention layer and the feedforward layer, and the self-attention layers of the three transducer modules learn each other to maximize information utilizing images and text data;
the bidirectional self-attention layer enables all image feature blocks and texts to learn each other, so that the output features can extract multi-mode information.
An analysis report generation unit: the method comprises the steps of splicing results output by a transducer model, inputting the spliced results into an image-text matcher, inputting spliced image features and text features into an image-text comparison learning subunit, learning the relation between image data and text data, and inputting the spliced text features into an image-based text generator to generate an analysis report.
Splicing the input of the two image transformation modules, and taking the spliced image features as the input of an image-text matcher to learn the relation between the image and the text features; the matcher is a task of two categories, namely judging whether the image-text pairs are matched or not, and calculating matching scores to optimize the whole model;
taking the output of the text transformation module as the input of an image-based text generator which takes image features as learning conditions and controls the association between the image features and the text by utilizing a multi-modal transformation layer;
taking the output of the spliced image features and the output of the text transformation module as the input of an image-text comparison learning module, and learning and aligning the image features and the text features by the module so as to maximize mutual information between the two features;
the image-text comparison learning module selects the highest value as the final similarity by comparing the similarity between all image features and text features of one placenta image so as to assist in optimizing the whole model;
and (3) carrying out the analysis on each pathology number, namely completing sample analysis corresponding to the pathology number in the whole queue, generating an analysis report and presenting the analysis report to a pathologist in a form of a table.
Database server: for reading placenta general data, placenta pathological section data and storing sample analysis results.
Based on MySQL, oracle, SQL Server and the like, a database Server is built for reading placenta general data, placenta pathological section data and storing sample analysis results.
(4) Remote consultation client: the method comprises the steps that a pathologist browses the analysis report, placenta general image or placenta pathological section image and fills in a diagnosis report;
the pathologist logs in the remote consultation client;
browsing placenta general images and pathological section images;
when the auxiliary image-text multi-modal model is trained, a pathologist marks the placenta general image and the pathological section image, and fills in a diagnosis report below the analysis report;
when the general diagnosis result of the placenta is required to be obtained, placenta data to be analyzed is selected to generate a list to be analyzed, and then the analysis report and the diagnosis report are obtained by clicking analysis.
(5) The mobile client: for viewing said placenta general image, said placenta pathological section image or said analysis report, and viewing said diagnosis report;
the patient first needs to log in to the mobile client, and then performs the following series of operations:
browsing placenta general images;
browsing digital images of placenta pathological sections;
the analysis report is reviewed.
The placenta general analyzer collects macroscopic data such as appearance, weight and the like of the placenta, and the whole-slide scanner collects microscopic images of pathological sections of the placenta. Adopting an image-text multi-mode model based on deep learning to macroscopically identify the major placenta and the abnormal umbilical cord at a pixel level, and identifying the central point of the umbilical cord insertion position; and (3) carrying out pixel-level identification analysis on placenta villi, blood vessels and cellulose on microcosmic basis to obtain parameters such as target area ratio and the like, and realizing the predictive diagnosis of common diseases of automatic fetuses and parents based on placenta.
The foregoing is merely a preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions easily conceivable by those skilled in the art within the technical scope of the present application should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (7)
1. A placental multiscale analysis system, comprising:
placenta general analysis module: the method comprises the steps of uniformly numbering placenta general samples, and obtaining the thickness, weight and image data of the placenta general samples;
placenta pathological section scanning module: the method comprises the steps of scanning and analyzing a placenta general sample to obtain placenta tissues of suspicious lesions, and processing the placenta tissues of the suspicious lesions to obtain placenta pathological section images;
placenta image analysis module: the system comprises a remote consultation client and a mobile client, wherein the remote consultation client is used for performing pathological data analysis on placenta general images and placenta pathological section images, generating an analysis report and sending the analysis report to the remote consultation client or the mobile client;
wherein, the placenta image analysis module includes:
user interaction unit: the data reading and processing operation is carried out through the interaction page;
a pathology data analysis unit: the method comprises the steps of analyzing a placenta general image or a placenta pathological section image through the interactive page to obtain an analysis result;
database server: the method comprises the steps of reading placenta general data, placenta pathological section data and storing sample analysis results;
the pathology data analysis unit includes:
an image labeling subunit: the marking tool is used for manually marking the target area in the placenta general image and the placenta pathological section image; wherein the target area in the placenta general image is placenta, umbilical cord and blood vessel area, and the target area in the placenta pathological section image is cancer area;
image-text analysis subunit: the method comprises the steps of arranging index parameters and analysis reports corresponding to all pathology numbers, inputting placenta general images, placenta pathological section images and diagnosis reports into an image-text analysis model, and extracting image characteristic data and text data by using a frozen image encoder;
an index calculation subunit: the method comprises the steps of inputting the image characteristic data and the text data into different transformation models respectively, obtaining a target region segmentation result of an image, and calculating index parameters;
an analysis report generation unit: the analysis report is used for splicing the result output by the transducer model and inputting the result into an image-text matcher, inputting the spliced image characteristics and text characteristics into an image-text comparison learning subunit, learning the relation between image data and text data, and inputting the spliced text characteristics into an image-based text generator to generate the analysis report;
remote consultation client: the method comprises the steps that a pathologist browses the analysis report, placenta general image or placenta pathological section image and fills in a diagnosis report;
the mobile client: for viewing said placenta general image, said placenta pathological section image or said analysis report, and viewing said diagnosis report;
the placenta general analysis module, the placenta pathological section scanning module, the placenta image analysis module, the remote consultation client and the mobile client are sequentially connected.
2. The placental multiscale analysis system according to claim 1, wherein the placental general analysis module comprises:
placenta data acquisition unit: the method comprises the steps of acquiring macroscopic image, weight and thickness data of a placenta general sample, and carrying out pathological numbering on the placenta general sample as a sample tracing basis;
placenta general analysis unit: the placenta comprises a placenta main sample, a placenta main sample and a placenta image processing device.
3. The placental multiscale analysis system according to claim 2, wherein the placental detailed information comprises: the net weight of placenta, volume, density, gestational week, number of fetuses, percentage of gestation Zhou Chongliang, length of umbilical cord, number of umbilical cord helices, direction of umbilical cord helices, color of fetal face, and whether maternal face is intact.
4. The placental multiscale analysis system according to claim 1, wherein the placental pathology section scanning module comprises:
placenta pathological section acquisition unit: the method comprises the steps of slicing placenta tissues of suspicious lesions through placenta images to obtain placenta pathological section specimens, wherein the placenta pathological section specimens contain identification information of placenta pathological numbers;
placenta pathological section analysis unit: the method comprises the steps of previewing a placenta pathological section specimen through special software of a placenta pathological section scanner, and shooting a label image and a section image of a pathological section;
placenta pathological section image acquisition unit: and the method is used for extracting the pathological numbers in the label images, identifying the slice images, taking the slice images as a navigation chart to determine a scanning area, and controlling a scanner to scan so as to obtain high-definition placenta pathological slice images.
5. The placental multiscale analysis system according to claim 1, wherein the data reading and processing operations via interactive pages comprise:
reading and processing data through an interactive page of a placenta image analysis subunit, wherein the left side of the placenta image analysis subunit lists all data in a working path in a tree view form, and a right side main working area is used for displaying all files under a current node; wherein the analyzed pathology number or the display icon of the image is marked with an analyzed mark symbol; the manner in which pathology numbers are analyzed includes batch analysis and individual analysis.
6. The placenta multi-scale analysis system of claim 1, wherein the image encoder is a VIT model-based image encoder, and the feature information of the placenta general image and the placenta pathological section image are extracted by two identical image encoders respectively;
the image-text comparison learning subunit selects the highest value as the similarity of the image-text comparison learning subunit by comparing the similarity between all image features and text features of one placenta image;
and calculating the matching score of the image-text pair through the image-text matcher, and finally combining the matching score with the similarity of the image-text comparison learning subunit and the loss value of the text generator of the image to optimize the transducer model.
7. The placental multiscale analysis system according to claim 1, wherein the remote consultation client comprises:
auxiliary diagnosis and result acquisition unit: when the method is used for training an auxiliary image-text multi-mode model, the placenta general image and the placenta pathological section image are marked, and a diagnosis report is filled in below the analysis report; when the diagnosis result of the placenta is needed to be obtained, selecting placenta data to be analyzed to generate a list to be analyzed, and then clicking for analysis to obtain an analysis report and a diagnosis report.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311029039.1A CN117153343B (en) | 2023-08-16 | 2023-08-16 | Placenta multiscale analysis system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311029039.1A CN117153343B (en) | 2023-08-16 | 2023-08-16 | Placenta multiscale analysis system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117153343A CN117153343A (en) | 2023-12-01 |
| CN117153343B true CN117153343B (en) | 2024-04-05 |
Family
ID=88903576
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311029039.1A Active CN117153343B (en) | 2023-08-16 | 2023-08-16 | Placenta multiscale analysis system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117153343B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368283A (en) * | 2011-02-21 | 2012-03-07 | 麦克奥迪实业集团有限公司 | Digital slice-based digital remote pathological diagnosis system and method |
| CN112582076A (en) * | 2020-12-07 | 2021-03-30 | 广州金域医学检验中心有限公司 | Method, device and system for placenta pathology submission assessment and storage medium |
| CN112992308A (en) * | 2021-03-25 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Training method of medical image report generation model and image report generation method |
| CN114298974A (en) * | 2021-11-30 | 2022-04-08 | 北京赛迈特锐医疗科技有限公司 | Intelligent diagnosis method and system for MR (magnetic resonance) image of fetal head |
| CN114550915A (en) * | 2022-02-22 | 2022-05-27 | 深圳市医未医疗科技有限公司 | Method and system for automatically generating report in image diagnosis |
| CN114661933A (en) * | 2022-03-08 | 2022-06-24 | 重庆邮电大学 | Cross-modal retrieval method based on fetal congenital heart disease ultrasonic image-diagnosis report |
| CN114867418A (en) * | 2019-12-16 | 2022-08-05 | 皇家飞利浦有限公司 | System and method for evaluating placenta |
| CN114882978A (en) * | 2022-07-12 | 2022-08-09 | 紫东信息科技(苏州)有限公司 | Stomach image processing method and system introducing picture translation information |
| CN115087867A (en) * | 2019-12-26 | 2022-09-20 | 公益财团法人癌研究会 | Pathological diagnosis assistance method and assistance device using AI |
| CN116230154A (en) * | 2023-03-03 | 2023-06-06 | 华东师范大学 | Chest X-ray diagnosis report generation method based on memory strengthening transducer |
| CN116363081A (en) * | 2023-03-16 | 2023-06-30 | 北京大学 | Placenta implantation MRI sign detection classification method and device based on deep neural network |
| CN116386803A (en) * | 2023-03-15 | 2023-07-04 | 南京大学 | Cytopathology report generation method based on graph |
| CN116563224A (en) * | 2023-04-12 | 2023-08-08 | 东莞市人民医院 | Image histology placenta implantation prediction method and device based on depth semantic features |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11244450B2 (en) * | 2019-08-19 | 2022-02-08 | The Penn State Research Foundation | Systems and methods utilizing artificial intelligence for placental assessment and examination |
| WO2022158908A1 (en) * | 2021-01-22 | 2022-07-28 | 주식회사 루닛 | Method and system for analyzing pathological image |
| EP4210070A1 (en) * | 2022-01-11 | 2023-07-12 | Siemens Healthcare GmbH | Machine learning based on radiology report |
-
2023
- 2023-08-16 CN CN202311029039.1A patent/CN117153343B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102368283A (en) * | 2011-02-21 | 2012-03-07 | 麦克奥迪实业集团有限公司 | Digital slice-based digital remote pathological diagnosis system and method |
| CN114867418A (en) * | 2019-12-16 | 2022-08-05 | 皇家飞利浦有限公司 | System and method for evaluating placenta |
| CN115087867A (en) * | 2019-12-26 | 2022-09-20 | 公益财团法人癌研究会 | Pathological diagnosis assistance method and assistance device using AI |
| CN112582076A (en) * | 2020-12-07 | 2021-03-30 | 广州金域医学检验中心有限公司 | Method, device and system for placenta pathology submission assessment and storage medium |
| CN112992308A (en) * | 2021-03-25 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Training method of medical image report generation model and image report generation method |
| CN114298974A (en) * | 2021-11-30 | 2022-04-08 | 北京赛迈特锐医疗科技有限公司 | Intelligent diagnosis method and system for MR (magnetic resonance) image of fetal head |
| CN114550915A (en) * | 2022-02-22 | 2022-05-27 | 深圳市医未医疗科技有限公司 | Method and system for automatically generating report in image diagnosis |
| CN114661933A (en) * | 2022-03-08 | 2022-06-24 | 重庆邮电大学 | Cross-modal retrieval method based on fetal congenital heart disease ultrasonic image-diagnosis report |
| CN114882978A (en) * | 2022-07-12 | 2022-08-09 | 紫东信息科技(苏州)有限公司 | Stomach image processing method and system introducing picture translation information |
| CN116230154A (en) * | 2023-03-03 | 2023-06-06 | 华东师范大学 | Chest X-ray diagnosis report generation method based on memory strengthening transducer |
| CN116386803A (en) * | 2023-03-15 | 2023-07-04 | 南京大学 | Cytopathology report generation method based on graph |
| CN116363081A (en) * | 2023-03-16 | 2023-06-30 | 北京大学 | Placenta implantation MRI sign detection classification method and device based on deep neural network |
| CN116563224A (en) * | 2023-04-12 | 2023-08-08 | 东莞市人民医院 | Image histology placenta implantation prediction method and device based on depth semantic features |
Non-Patent Citations (3)
| Title |
|---|
| AI-PLAX:AI-based placental assessment and examination using photos;Yukun Chen et al.;Computerized Medical Imaging and Graphics;第84卷;第1-15页 * |
| 基于iOS手机客户端的病理图像分析软件;王睿乔;《中国优秀硕士学位论文全文数据库 (医药卫生科技辑)》(第02期);第E059-140页 * |
| 磁共振技术在胎盘异常诊断中的应用进展;廖卫彬等;《影像研究与医学应用》;第4卷(第10期);第7-10页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117153343A (en) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110993064B (en) | Deep learning-oriented medical image labeling method and device | |
| CN108596884B (en) | Esophagus cancer segmentation method in chest CT image | |
| CN110245657B (en) | Pathological image similarity detection method and detection device | |
| CN112086197B (en) | Breast nodule detection method and system based on ultrasonic medicine | |
| CN111161822A (en) | Medical image artificial intelligence diagnosis data management system and management method thereof | |
| CN107330263A (en) | A kind of method of area of computer aided breast invasive ductal carcinoma histological grading | |
| CN106056595A (en) | Method for automatically identifying whether thyroid nodule is benign or malignant based on deep convolutional neural network | |
| Beddiar et al. | Automatic captioning for medical imaging (MIC): a rapid review of literature | |
| EP2996058A1 (en) | Method for automatically generating representations of imaging data and interactive visual imaging reports | |
| CN111079901A (en) | Acute stroke lesion segmentation method based on small sample learning | |
| CN112508884A (en) | Comprehensive detection device and method for cancerous region | |
| CN112614573A (en) | Deep learning model training method and device based on pathological image labeling tool | |
| Ungureanu et al. | Learning deep architectures for the interpretation of first-trimester fetal echocardiography (LIFE)-a study protocol for developing an automated intelligent decision support system for early fetal echocardiography | |
| CN115471512A (en) | Medical image segmentation method based on self-supervision contrast learning | |
| CN110660477A (en) | System and method for automatically screening and labeling helicobacter pylori | |
| CN117237351B (en) | Ultrasonic image analysis method and related device | |
| CN112200810B (en) | Multi-modal automated ventricle segmentation system and method of use thereof | |
| CN116779093B (en) | Method and device for generating medical image structured report and computer equipment | |
| CN114283406A (en) | Cell image recognition method, device, equipment, medium and computer program product | |
| CN111340807B (en) | Nidus positioning core data extraction method, system, electronic equipment and storage medium | |
| CN117153343B (en) | Placenta multiscale analysis system | |
| Do et al. | Supporting thyroid cancer diagnosis based on cell classification over microscopic images | |
| CN116933135A (en) | Modeling system and method for cancer stage prediction model based on cross-modal fusion cascade | |
| CN116759076A (en) | Unsupervised disease diagnosis method and system based on medical image | |
| CN116486156A (en) | Full-view digital slice image classification method integrating multi-scale feature context |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |