CN117576321A - Auxiliary liver operation planning modeling system - Google Patents

Auxiliary liver operation planning modeling system Download PDF

Info

Publication number
CN117576321A
CN117576321A CN202311654269.7A CN202311654269A CN117576321A CN 117576321 A CN117576321 A CN 117576321A CN 202311654269 A CN202311654269 A CN 202311654269A CN 117576321 A CN117576321 A CN 117576321A
Authority
CN
China
Prior art keywords
image
liver
lesion
standard
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311654269.7A
Other languages
Chinese (zh)
Other versions
CN117576321B (en
Inventor
郭闻渊
滕飞
毛家玺
鲁欣翼
李静静
赵渊宇
傅宏
朱鲤烨
钟瀚翔
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Changzheng Hospital
Original Assignee
Shanghai Changzheng Hospital
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Changzheng Hospital filed Critical Shanghai Changzheng Hospital
Priority to CN202311654269.7A priority Critical patent/CN117576321B/en
Publication of CN117576321A publication Critical patent/CN117576321A/en
Application granted granted Critical
Publication of CN117576321B publication Critical patent/CN117576321B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10Computer-aided planning, simulation or modelling of surgical operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/101Computer-aided simulation of surgical operations
    • A61B2034/105Modelling of the patient, e.g. for ligaments or bones
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/10Computer-aided planning, simulation or modelling of surgical operations
    • A61B2034/107Visualisation of planned trajectories or target regions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • G06T2207/10012Stereo images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30056Liver; Hepatic
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30096Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Surgery (AREA)
  • Evolutionary Computation (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Robotics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses an auxiliary liver operation planning modeling system which comprises data acquisition equipment, a three-dimensional reconstruction module, a correction module, a segmentation module, an identification module, a preoperative planning module, an evaluation module and display equipment, wherein the three-dimensional reconstruction module is used for reconstructing the three-dimensional reconstruction module; the data acquisition equipment acquires a two-dimensional liver image of an acquired person; the three-dimensional reconstruction module carries out three-dimensional modeling on the two-dimensional liver image to generate a first liver three-dimensional image; the correction module corrects the three-dimensional liver image to generate a corrected second three-dimensional liver image; the segmentation module segments the liver segments of the second liver three-dimensional image to obtain segmented images of the liver; the identification module identifies the split images, an identification result is obtained, and the preoperative planning module plans an operation path based on the identification result; the evaluation module maps the operation path to the two-dimensional liver image for evaluation to obtain an evaluation result; the display device displays the identification result and the evaluation result.

Description

Auxiliary liver operation planning modeling system
Technical Field
The invention relates to the technical field of computers, in particular to an auxiliary liver operation planning modeling system.
Background
In liver surgery, accurate surgical planning and accurate surgical manipulation are critical to the success of the surgery. The existing liver operation planning method mainly depends on experience of doctors and medical image data. Although these methods may provide some degree of reference for surgical planning, they are often limited by physician experience, skill and vision, and lack intuitive three-dimensional models, making the accuracy and effectiveness of surgical planning difficult to guarantee.
In the prior art, three-dimensional modeling is carried out on the liver by means of a computer 3D imaging system, a doctor can clearly, intuitively and stereoscopically grasp the relation among tissues in the liver before operation, so that the doctor can plan in detail before operation, the planning of a three-dimensional operation path is very complex, a plurality of factors such as the morphology of the liver, pathological change parts, vascularity and the like need to be considered, and the evaluation of the operation path is only stopped on a three-dimensional image and is not comprehensive enough.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides an auxiliary liver operation planning modeling system.
The technical scheme adopted by the invention is as follows:
in a first aspect, the present application provides an assisted liver surgery planning modeling system, including a data acquisition device, a three-dimensional reconstruction module, a collation module, a segmentation module, an identification module, a preoperative planning module, an evaluation module and a display device;
the data acquisition equipment acquires a two-dimensional liver image of an acquired person;
the three-dimensional reconstruction module carries out three-dimensional modeling on the two-dimensional liver image to generate a first liver three-dimensional image;
the correction module corrects the liver three-dimensional image to generate a corrected second liver three-dimensional image;
the segmentation module segments the liver segments of the second liver three-dimensional image to obtain a tail She Sanwei image of the liver, a left She Sanwei image of the liver and a right She Sanwei image of the liver;
the identification module respectively identifies the tail She Sanwei image, the left She Sanwei image and the right She Sanwei image to obtain an identification result, wherein the identification result comprises a lesion area and a lesion type on the tail She Sanwei image, the left She Sanwei image of the liver and the right She Sanwei image of the liver;
the preoperative planning module plans an operative path based on the identification result;
the evaluation module maps the operation path to the two-dimensional liver image for evaluation to obtain an evaluation result;
the display device displays the identification result and the evaluation result.
Preferably, the identification module includes:
the storage unit is used for storing liver lobe standard lesion images, wherein the liver lobe standard lesion images comprise standard tail lobe three-dimensional images, standard left She Sanwei images and standard right lobe three-dimensional images;
the first comparison unit is used for comparing the tail She Sanwei image with a standard tail leaf three-dimensional image, comparing the left She Sanwei image with a standard left She Sanwei image and comparing the right She Sanwei image with a standard right leaf three-dimensional image to generate a first comparison result;
a first judging unit that judges whether or not a lesion region exists in the tail-shaped leaf three-dimensional image or the left She Sanwei image or the right She Sanwei image based on the first comparison result;
a first identifying unit that identifies a lesion type based on a lesion area image when a lesion area exists in the tail leaf three-dimensional image or the left She Sanwei image or the right She Sanwei image.
Preferably, the first identifying unit includes:
a first calculation subunit calculating a similarity between the lesion area image and the liver leaf standard lesion image;
a first judging subunit, configured to judge whether a similarity between the lesion area image and at least one standard lesion image of the liver lobe is greater than a preset first threshold;
a first screening subunit, configured to screen, when the similarity between the lesion area image and at least one liver lobe standard lesion image is greater than a preset first threshold, a liver lobe standard lesion image with the highest similarity to the lesion area image from at least one liver lobe standard lesion image with the similarity greater than the preset first threshold;
the device comprises a first transmitting subunit, a lesion type visualizing module and a second transmitting subunit, wherein the first transmitting subunit transmits a liver leaf standard lesion image with the highest similarity between a lesion area image and the lesion area image to the lesion type visualizing module of the display device;
a second judging subunit, configured to judge whether the similarity between the lesion area image and at least two standard lesion images of liver lobes is greater than a preset second threshold when the similarity between the lesion area image and any one standard lesion image of liver lobes is less than or equal to a preset first threshold, where the second threshold is less than the first threshold;
and the second transmission subunit is used for transmitting the lesion area image and the at least two liver leaf standard lesion images with the similarity larger than a preset second threshold value to a to-be-selected visualization module of the display device when the similarity of the lesion area image and the at least two liver leaf standard lesion images is larger than the preset second threshold value.
Preferably, the segmentation module comprises a segmentation unit;
the segmentation unit segments the three-dimensional liver image when the tail leaf three-dimensional image, the left She Sanwei image and the right She Sanwei image do not have a lesion region, and generates a plurality of liver segment images.
Preferably, the storage unit further stores a standard liver segment image, and the identification module further includes:
the second comparison unit is used for comparing the standard liver segment images corresponding to each liver segment image in the plurality of liver segment images to generate a second comparison result;
a second judging unit that judges whether or not each liver segment image has a lesion region based on the second comparison result;
and a second identifying unit that identifies a lesion type based on the lesion area image of the liver segment when the lesion area exists in the at least one liver segment image.
Preferably, the second identifying unit includes:
a second calculating subunit, configured to calculate a similarity between the liver segment lesion region image and a liver segment standard lesion image;
a third judging subunit, configured to judge whether a similarity between the liver segment lesion area image and at least one liver segment standard lesion image is greater than a preset first threshold;
a second screening subunit, configured to screen out, from at least one liver segment standard lesion image with a similarity greater than a preset first threshold, a liver segment standard lesion image with a highest similarity to the liver segment lesion image when the similarity between the liver segment lesion image and the at least one liver segment standard lesion image is greater than a preset first threshold
A third transmitting subunit, configured to transmit the liver segment lesion region image and a liver segment standard lesion image with the highest similarity to the liver segment lesion region image to a lesion type visualization module of a display device; the method comprises the steps of carrying out a first treatment on the surface of the
A fourth judging subunit, configured to judge whether the similarity between the liver segment lesion area image and at least two liver segment standard lesion images is greater than a preset second threshold when the similarity between the liver segment lesion area image and any one liver segment standard lesion image is less than or equal to a preset first threshold, where the second threshold is less than the first threshold;
and the fourth transmission subunit is used for transmitting the liver segment lesion region image and the at least two liver segment standard lesion images with the similarity of the liver segment lesion region image and the liver segment standard lesion images being greater than a preset second threshold value to a to-be-selected visualization module of the display device if the similarity of the liver segment lesion region image and the at least two liver segment standard lesion images is greater than the preset second threshold value.
Preferably, the liver image segmentation and the liver image segmentation both adopt a Couinaud division method. .
The beneficial effects of the invention are as follows: through the three-dimensional reconstruction and correction module, the system can convert the two-dimensional liver image into a three-dimensional model and provide a corrected three-dimensional image, so that doctors can evaluate the lesion areas and the lesion types more accurately. The preoperative planning module can help doctors to make more accurate operation paths, and accuracy and success rate of operations are improved. Through the identification module, the system can automatically identify lesion areas and lesion types on tail leaves, zuo She and right leaves of the liver, help doctors to better know the illness state of patients, and formulate personalized operation schemes according to identification results. The evaluation module can map the operation path to the two-dimensional liver image for evaluation, and an evaluation result is obtained, so that operation risk is reduced. Through the display device, the system can display the identification result and the evaluation result in real time, so that doctors can communicate operation schemes and risks with patients and family members of the patients more clearly, and communication efficiency is improved.
Drawings
FIG. 1 is a block diagram of a system in accordance with an embodiment of the present application;
fig. 2 is a block diagram of an identification module according to an embodiment of the present application
Fig. 3 is a block diagram of a first identification unit according to an embodiment of the present application;
fig. 4 is a block diagram of a segmentation module and an identification module according to a second embodiment of the present application;
fig. 5 is a block diagram of a second identifying unit according to a second embodiment of the present application.
Description of the drawings: 10. data acquisition equipment: 20. a three-dimensional reconstruction module; 30. a checking module; 40. a segmentation module; 401. a segmentation unit; 50. an identification module; 501. a storage unit; 502. a first comparison unit; 503. a first judgment unit; 504. a first identification unit; 505. a first computing subunit; 506. a first judgment subunit; 507. a first screening subunit; 508. a first transmitting subunit; 509. a second judgment subunit; 510. a second transmitting subunit; 511. a second comparison unit; 512. a second judgment unit; 513. a second recognition unit; 514. a second computing subunit; 515. a third judgment subunit; 516. a second screening subunit; 517. a third transmitting subunit; 518. a fourth judgment subunit; 519. a fourth transmitting subunit; 60. a preoperative planning module; 70. an evaluation module; 80. a display device; 801. a lesion type visualization module; 802. and a visual module to be selected.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
Embodiment one:
as shown in fig. 1, an auxiliary liver operation planning modeling system comprises a data acquisition device 10, a three-dimensional reconstruction module 20, a correction module 30, a segmentation module 40, an identification module 50, a preoperative planning module 60, an evaluation module 70 and a display device 80;
the data acquisition device 10 acquires a two-dimensional liver image of an acquired person;
the data acquisition device 10 may be a medical imaging device, such as a CT, MRI, or the like, among others.
The three-dimensional reconstruction module 20 is connected to the data acquisition device 10, the three-dimensional reconstruction module 20 performs three-dimensional modeling on the two-dimensional liver image, generates a first liver three-dimensional image,
these two-dimensional images are input to the three-dimensional reconstruction module 20, and these image data are three-dimensionally reconstructed by conventional computer image processing techniques. In the three-dimensional reconstruction process, various algorithms and techniques, such as voxel reconstruction, surface reconstruction, volume rendering, etc., may be employed to generate a high quality three-dimensional liver model.
The calibration module 30 is connected to the three-dimensional reconstruction module 20, and the calibration module 30 calibrates the liver three-dimensional image to generate a calibrated second liver three-dimensional image.
Specifically, the three-dimensional liver model is input to the calibration module 30, and the model is primarily calibrated and corrected by using a medical image analysis technology, for example, whether the overall shape and structure of the model are correct, whether abnormality exists, and the like are checked. And carrying out subsequent processing on the corrected three-dimensional liver model, such as removing noise, optimizing the model, repairing errors and the like, and finally generating a corrected second liver three-dimensional image.
The segmentation module 40 is connected to the calibration module 30, and the segmentation module 40 segments the liver segment of the second three-dimensional liver image to obtain a tail She Sanwei image of the liver, a left She Sanwei image of the liver and a right She Sanwei image of the liver.
Specifically, liver image segmentation was performed using the Couinaud scoring method, which was first proposed by french surgeons, decrealogists, claude Couinaud. After the foliar treatment, a tail She Sanwei image of the liver, a left She Sanwei image of the liver and a right She Sanwei image of the liver can be obtained. These segmented three-dimensional images can be stored separately and subsequently planned and evaluated for surgery.
The recognition module 50 is connected to the segmentation module 40, and the recognition module recognizes the tail shape She Sanwei image, the left She Sanwei image and the right She Sanwei image respectively, so as to obtain a recognition result, where the recognition result includes the tail shape She Sanwei image, the left She Sanwei image of the liver, and the lesion area and the lesion type on the right She Sanwei image of the liver.
In one possible implementation, as shown in fig. 2, the identification module 50 includes:
a storage unit 501, wherein the storage unit 501 stores a liver lobe standard lesion image, wherein the liver lobe standard lesion image includes a standard tail lobe three-dimensional image, a standard left She Sanwei image, and a standard right lobe three-dimensional image.
The storage unit 501 stores standard lesion images of liver lobes, which are known and standardized lesion images, such as standardized lesion images of tumors, inflammations, and the like. These images may provide references and comparisons for subsequent alignment and identification.
The first comparing unit 502 is connected with the segmentation module 40 and the storage unit, and the first comparing unit 502 compares the tail She Sanwei image with the standard tail leaf three-dimensional image, compares the left She Sanwei image with the standard left She Sanwei image, and compares the right She Sanwei image with the standard right leaf three-dimensional image to generate a first comparing result.
The first comparing unit 502 compares the liver She Sanwei image with the standard three-dimensional liver leaf image by using an image processing technique and algorithm. For example, shape matching, texture analysis, etc. techniques may be used to compare the shape, texture, etc. characteristics of the image to find the lesion area.
A first judging unit 503, where the first judging unit 503 is connected to the first comparing unit 502, and the first judging unit judges whether a lesion area exists in the caudate three-dimensional image or the left She Sanwei image or the right She Sanwei image based on the first comparison result.
It should be noted that the first determination unit 503 determines whether or not there is a lesion region in the three-dimensional image of the liver lobe according to the comparison result. For example, if the comparison result shows that there is a region of the liver She Sanwei image similar to the standard lesion image, it can be judged that there is a lesion region in the image.
A first identifying unit 504, the first identifying unit 504 is connected to the first judging unit 503, and the identifying unit identifies a lesion type based on a lesion area image when there is a lesion area in the tail-shaped leaf three-dimensional image or the left She Sanwei image or the right She Sanwei image.
In one possible implementation, as shown in fig. 3, the first identifying unit 504 includes:
a first calculating subunit 505, where the first calculating subunit 505 is connected to the storage unit 501, the first comparing unit 502, and the first judging unit 503; the first calculating subunit 505 calculates a similarity of the lesion area image and the liver lobe standard lesion image.
The first calculating subunit 505 is responsible for calculating the similarity between the lesion area image and the standard lesion image of the liver lobe. Such similarity calculation may be performed based on features of the image, such as shape, texture, color, etc. For example, an image registration technique may be employed to calculate the similarity between the lesion area image and the standard lesion image by comparing their feature points.
The first judging subunit 506, the first judging subunit 506 and the first calculating subunit 505, where the first judging subunit 506 judges whether the similarity between the lesion area image and the at least one standard lesion image of liver lobe is greater than a preset first threshold.
It should be noted that, the first determining subunit 506 is responsible for determining whether the similarity between the lesion area image and the at least one standard lesion image of the liver lobe is greater than a preset first threshold. This threshold may be an empirical value or a statistical value that represents the lowest degree of similarity that can be judged to be a lesion. If the similarity of the lesion area image to any one of the standard lesion images is greater than the threshold value, then the lesion area image may be considered to be diseased.
The first screening subunit 507 is connected with the first judging subunit 506, and when the similarity between the lesion area image and at least one standard lesion image of liver leaf is greater than a preset first threshold, the first screening unit screens out the standard lesion image of liver leaf with the highest similarity between the lesion area image and at least one standard lesion image of liver leaf with the similarity greater than the preset first threshold.
When the similarity between the lesion area image and at least one liver lobe standard lesion image is greater than a preset first threshold value, the unit screens out the liver lobe standard lesion image with the highest similarity with the lesion area image from at least one liver lobe standard lesion image with the similarity greater than the preset first threshold value.
The first sending subunit 508 is connected with the first screening subunit 507 and the lesion type visualizing module 801, and the first sending subunit 508 sends the lesion type visualizing module 801 of the display device with the liver leaf standard lesion image with the highest similarity between the lesion area image and the lesion area image.
The first transmitting subunit 508 transmits the lesion region image and the liver leaf standard lesion image having the highest similarity with the lesion region image to the lesion type visualization module of the display device. This transmission may be by transmitting the data directly to the display device via a data line or network, or by some dedicated medical imaging system or surgical navigation system.
The second judging subunit 509 is connected to the first judging subunit 506, where when the similarity between the lesion area image and any one of the liver lobe standard lesion images is smaller than or equal to a preset first threshold, the second judging subunit 509 judges whether the similarity between the lesion area image and at least two liver lobe standard lesion images is greater than a preset second threshold, where the second threshold is smaller than the first threshold.
When the similarity between the lesion area image and any one of the liver leaf standard lesion images is smaller than or equal to a preset first threshold, it is indicated that the system cannot accurately judge the specific lesion type of the lesion area image at this time, so that two or more lesion types possibly corresponding to the lesion area image can be obtained by judging whether the similarity between the lesion area image and at least two liver leaf standard lesion images is larger than a preset second threshold at this time.
The second sending subunit 510, where the second sending subunit 510 is connected to the second judging subunit 510 and the visualization module to be selected 802; when the similarity between the lesion area image and at least two standard lesion images of liver lobes is greater than a preset second threshold, the second sending subunit sends the lesion area image and at least two standard lesion images of liver lobes, the similarity between which is greater than the preset second threshold, to the to-be-selected visualization module 802 of the display device.
When the similarity between the lesion area image and at least two standard lesion images of liver lobes is larger than a preset second threshold value, the second sending subunit sends the lesion area image and at least two standard lesion images of liver lobes, the similarity between the standard lesion images and the lesion area image of which is larger than the preset second threshold value, to a to-be-selected visualization module of the display device. The goal of this transmission is to provide a more comprehensive view to the physician, allowing the physician to view and analyze the possible types and conditions of the lesion from multiple angles. And judging again from two or more lesion types possibly corresponding to the lesion area image.
The pre-operation planning module 60 is connected to the identification module 50, and the pre-operation planning module 60 plans a surgical path based on the identification result.
Specifically, the preoperative planning module 60 acquires the lesion area image and the lesion type, which are from the lesion area image transmitted by the first transmitting subunit 508 and the liver lobe standard lesion image having the highest similarity with the lesion area image. Depending on the type and location of the lesion, the target of the procedure is determined, such as the lesion area to be resected, the organ or vessel to be protected, etc. According to the surgical targets, a surgical path is designed, including surgical approach, surgical sequence, operative steps, etc.
The evaluation module 70 is connected to the preoperative planning module 60; the evaluation module 70 maps the surgical path to the two-dimensional liver image for evaluation, and obtains an evaluation result.
Specifically, mapping the designed surgical path (e.g., surgical approach, surgical order, procedure, etc.) into a two-dimensional liver image may be accomplished by a three-dimensional to two-dimensional projection or mapping algorithm.
Various techniques in computer graphics, such as view clipping, projective transformation, and the like, may be used to transform a three-dimensional surgical path into a path on a two-dimensional plane.
In the mapping process, the morphology, position and position of the lesion area of the liver need to be considered to ensure the accuracy and feasibility of the surgical path. Based on the mapped surgical path, the feasibility of the surgical path is assessed, including whether the path is capable of reaching the surgical target, whether it passes through vital organs or blood vessels, whether postoperative complications may result, and so forth. A medical knowledge base or expert system may be used to assist in the assessment. The feasibility of the surgical path can be comprehensively evaluated according to factors such as the importance of the region, the organ or the blood vessel through which the surgical path passes, the difficulty of the surgical operation and the like. According to the evaluation result, the operation effect is predicted and evaluated, including operation time, bleeding amount in operation, recovery after operation and the like. Medical simulation techniques or virtual reality techniques may be used to simulate the surgical procedure and evaluate the surgical effect. The method can predict and evaluate factors such as operation time, bleeding amount in operation, recovery condition after operation and the like according to simulation results or experience data. And generating a judging result of the surgical path according to the evaluation result and the prediction result, wherein the judging result comprises whether the surgical path is feasible, whether the surgical path needs to be optimized and the like.
The display device 80 is connected with the identification module 50 and the evaluation module 70; the display device 80 displays the identification result and the evaluation result.
In the embodiment, by mapping the operation path to the two-dimensional liver image for evaluation, a doctor can more intuitively understand the condition of the operation path, reduce the risk of operation and the occurrence of complications, and improve the safety of operation. By evaluating the feasibility and effect of the surgical path, a doctor can better plan the surgical procedure, reduce unnecessary operations and time, and improve the efficiency of the surgery. By generating a judgment result on the surgical path, the physician can more fully understand the risk and effect of the surgery, thereby making a more informed decision. Mapping the surgical path to the two-dimensional liver image may facilitate a physician in assessing the surgical path in multiple angles and views, improving accuracy and repeatability of the assessment. This facilitates more efficient communication and collaboration between doctors, ensuring the safety and effectiveness of the procedure. By optimizing the surgical path and reducing unnecessary operations, the medical cost can be reduced and the utilization efficiency of medical resources can be improved.
Embodiment two:
as shown in fig. 4, the segmentation module 40 is further connected to the first judging unit 503, and the segmentation module 40 further includes a segmentation unit 401;
the segmentation unit 401 segments the three-dimensional liver image when there is no lesion region in the tail leaf three-dimensional image, the left She Sanwei image, and the right She Sanwei image, and generates a plurality of liver segment images.
Specifically, the segmentation also uses the Couinaud division method. The Couinaud scoring method leaves liver images: the liver is divided into left, right and tail lobes. Left She Zaifen is taken as left inner leaf, left outer leaf and left tail leaf. Right She Zaifen is right front leaf, right back leaf and right inner leaf, and if the lesion area in the segmented image is still not accurately identified, the segmented image can be further segmented. For example, the left inner She Zaifen is an upper left inner segment, a lower left inner segment, or the like, and the right front She Zaifen is an upper right front segment, a lower right front segment, or the like.
The storage unit 501 also stores standard liver segment images. Template images of each liver segment are predefined and stored in a storage unit. The template images can be manually marked lesion areas or standard liver segment images automatically extracted by other algorithms.
The identification module 50 further includes: the second comparing unit 511 is connected to the segmenting unit 401, and the second comparing unit 511 compares the standard liver segment images corresponding to each of the plurality of liver segment images to generate a second comparison result.
The second comparing unit 511 compares each liver segment image with the corresponding standard liver segment image. Specifically, image processing techniques such as feature extraction and similarity calculation are used to compare the similarity of two images.
And a second judging unit 512, wherein the second judging unit 512 is connected to the second comparing unit 511, and the second judging unit 512 judges whether each liver segment image has a lesion area based on the second comparing result.
A second identifying unit 513, the second identifying unit 513 being connected to the second judging unit 512; the second identifying unit 513 identifies a lesion type based on the lesion area image of the liver segment when there is a lesion area in at least one liver segment image.
In one possible implementation, as shown in fig. 5, the second identifying unit 513 includes:
the second calculating subunit 514 is connected to the storage unit 501 and the second comparing unit 511, and the second calculating subunit 514 calculates the similarity between the liver segment lesion area image and the liver segment standard lesion image.
It should be noted that, the second calculating subunit 514 is responsible for calculating the similarity between the liver segment lesion region image and the liver segment standard lesion image. This may be achieved by image processing techniques such as feature extraction and similarity calculation.
And a third judging subunit 515, where the third judging subunit 515 is connected to the second calculating subunit 514, and the third judging subunit 515 judges whether the similarity between the liver segment lesion area image and at least one liver segment standard lesion image is greater than a preset first threshold.
The third judging subunit 515 judges whether the similarity between the liver segment lesion area image and the at least one liver segment standard lesion image is greater than a preset first threshold. If the similarity is greater than the first threshold, the lesion area image is considered to match a standard lesion image.
The second screening subunit 516 is connected to the third judging subunit 515, where when the similarity between the liver segment lesion area image and the at least one liver segment standard lesion image is greater than a preset first threshold, the second screening subunit screens out the liver segment standard lesion image with the highest similarity between the liver segment lesion area image and the at least one liver segment standard lesion image with the similarity greater than the preset first threshold.
When the similarity between the liver segment lesion region image and at least one liver segment standard lesion image is greater than a preset first threshold value, the unit is responsible for screening the liver segment standard lesion image with the highest similarity with the liver segment lesion region image from at least one liver segment standard lesion image with the similarity greater than the first threshold value. This may help find the best matching lesion type.
And a third transmitting subunit 517, where the third transmitting subunit 517 is connected to the second screening subunit 516, and the third transmitting subunit 517 transmits the liver segment lesion region image and the liver segment standard lesion image with the highest similarity of the liver segment lesion region image to a lesion type visualization module of the display device.
The third transmitting subunit 517 is responsible for transmitting the lesion type visualization module 801 of the display device with the liver segment lesion region image having the highest similarity to the liver segment standard lesion image. This allows the physician to intuitively see the matching of the lesion area with the standard lesion, thereby diagnosing the lesion type more accurately.
And a fourth judging subunit 518, where the fourth judging subunit 518 is connected to the third judging subunit 515, and when the similarity between the liver segment lesion area image and any one liver segment standard lesion image is smaller than or equal to a preset first threshold, the fourth judging subunit judges whether the similarity between the liver segment lesion area image and at least two liver segment standard lesion images is greater than a preset second threshold, where the second threshold is smaller than the first threshold.
When the similarity between the liver segment lesion area image and any liver segment standard lesion image is smaller than or equal to a preset first threshold value, the unit judges whether the similarity between the liver segment lesion area image and at least two liver segment standard lesion images is larger than a preset second threshold value, wherein the second threshold value is smaller than the first threshold value. This may reveal other similar lesion types that may be present.
A fourth transmitting sub-unit 519, wherein the fourth transmitting sub-unit 519 is connected to the fourth judging sub-unit 518; if the similarity between the liver segment lesion area image and the at least two liver segment standard lesion images is greater than a preset second threshold, the fourth sending subunit sends the liver segment lesion area image and the at least two liver segment standard lesion images with the similarity between the liver segment lesion area image and the liver segment standard lesion images is greater than the preset second threshold to the to-be-selected visualization module 802 of the display device.
If the similarity between the liver segment lesion area image and at least two liver segment standard lesion images is larger than a preset second threshold value, the unit is responsible for sending the liver segment lesion area image and at least two liver segment standard lesion images with the similarity between the liver segment lesion area image and the liver segment standard lesion images being larger than the preset second threshold value to a to-be-selected visualization module of the display device. This allows the physician to see multiple possible lesion types simultaneously, thereby providing a more comprehensive understanding of the condition.
In this embodiment, if the segmented image cannot identify the lesion region, then the segmentation process is performed, and each subdivided liver segment is identified, so that the lesion region can be more accurately located, and the accuracy of diagnosis is improved.
The foregoing examples merely illustrate specific embodiments of the invention, which are described in greater detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention.

Claims (7)

1. An auxiliary liver surgery planning modeling system, which is characterized in that: comprising the following steps:
the data acquisition equipment acquires a two-dimensional liver image of an acquired person;
the three-dimensional reconstruction module is connected with the data acquisition equipment and is used for carrying out three-dimensional modeling on the two-dimensional liver image to generate a first liver three-dimensional image;
the correction module is connected with the three-dimensional reconstruction module and corrects the liver three-dimensional image to generate a corrected second liver three-dimensional image;
the segmentation module is connected with the correction module and is used for carrying out leaf segmentation on the liver segments of the second liver three-dimensional image to obtain a tail She Sanwei image of the liver, a left She Sanwei image of the liver and a right She Sanwei image of the liver;
the identification module is connected with the segmentation module and is used for respectively identifying the tail shape She Sanwei image, the left She Sanwei image and the right She Sanwei image to obtain an identification result, wherein the identification result comprises a lesion area and a lesion type on the tail shape She Sanwei image, the left She Sanwei image of the liver and the right She Sanwei image of the liver;
the preoperative planning module is connected with the identification module and is used for planning a surgical path based on the identification result;
the evaluation module is connected with the preoperative planning module; the evaluation module maps the operation path to the two-dimensional liver image for evaluation to obtain an evaluation result;
the display device is connected with the identification module and the evaluation module; the display device displays the identification result and the evaluation result.
2. An assisted liver surgery planning modeling system according to claim 1 wherein: the identification module comprises:
the storage unit is used for storing liver lobe standard lesion images, wherein the liver lobe standard lesion images comprise standard tail lobe three-dimensional images, standard left She Sanwei images and standard right lobe three-dimensional images;
the first comparison unit is connected with the segmentation module and the storage unit, and is used for comparing the tail She Sanwei image with a standard tail leaf three-dimensional image, comparing the left She Sanwei image with a standard left She Sanwei image and comparing the right She Sanwei image with a standard right leaf three-dimensional image to generate a first comparison result;
a first judging unit connected to the first comparing unit, the first judging unit judging whether there is a lesion region in the caudate leaf three-dimensional image or the left She Sanwei image or the right She Sanwei image based on the first comparison result;
and the first identification unit is connected with the first judgment unit, and is used for identifying the lesion type based on the lesion area image when the three-dimensional image of the tail leaf or the left She Sanwei image or the right She Sanwei image has a lesion area.
3. An assisted liver surgery planning modeling system according to claim 2 in which the first recognition unit comprises:
the first calculating subunit is connected with the storage unit, the first comparison unit and the first judging unit; the first calculating subunit calculates the similarity between the lesion area image and the liver leaf standard lesion image;
the first judging subunit judges whether the similarity between the lesion area image and at least one standard liver leaf lesion image is larger than a preset first threshold value or not;
the first screening subunit is connected with the first judging subunit, and when the similarity between the lesion area image and at least one liver leaf standard lesion image is larger than a preset first threshold value, the first screening subunit screens out the liver leaf standard lesion image with the highest similarity with the lesion area image from at least one liver leaf standard lesion image with the similarity larger than the preset first threshold value;
the first transmitting subunit is connected with the first screening subunit and the lesion type visualizing module, and transmits the liver leaf standard lesion image with the highest similarity between the lesion area image and the lesion area image to the lesion type visualizing module of the display device;
the second judging subunit is connected with the first judging subunit, and judges whether the similarity between the lesion area image and at least two liver leaf standard lesion images is larger than a preset second threshold value or not when the similarity between the lesion area image and any liver leaf standard lesion image is smaller than or equal to a preset first threshold value, wherein the second threshold value is smaller than the first threshold value;
the second sending subunit is connected with the second judging subunit and the to-be-selected visualization module; when the similarity between the lesion area image and at least two liver lobe standard lesion images is larger than a preset second threshold value, the second sending subunit sends the lesion area image and at least two liver lobe standard lesion images with the similarity between the lesion area image and the at least two liver lobe standard lesion images being larger than the preset second threshold value to a to-be-selected visualization module of a display device.
4. An assisted liver surgery planning modeling system according to claim 2 wherein: the segmentation module is also connected with the first judgment unit and further comprises a segmentation unit;
the segmentation unit segments the three-dimensional liver image when the tail leaf three-dimensional image, the left She Sanwei image and the right She Sanwei image do not have a lesion region, and generates a plurality of liver segment images.
5. An assisted liver surgery planning modeling system according to claim 4 in which: the storage unit further stores a standard liver segment image, and the identification module further includes:
the second comparison unit is connected with the segmentation unit and is used for comparing standard liver segment images corresponding to each liver segment image in the plurality of liver segment images to generate a second comparison result;
the second judging unit is connected with the second comparing unit and judges whether each liver segment image has a lesion area or not based on the second comparing result;
the second identification unit is connected with the second judgment unit; the second identifying unit identifies a lesion type based on the liver segment lesion area image when the lesion area exists in the at least one liver segment image.
6. An assisted liver surgery planning modeling system according to claim 5 wherein: the second recognition unit includes:
the second calculating subunit is connected with the storage unit and the second comparing unit and calculates the similarity between the liver segment lesion area image and the liver segment standard lesion image;
the third judging subunit is connected with the second calculating subunit and judges whether the similarity between the liver segment lesion region image and at least one liver segment standard lesion image is larger than a preset first threshold value or not;
the second screening subunit is connected with the third judging subunit, and when the similarity between the liver segment lesion region image and at least one liver segment standard lesion image is larger than a preset first threshold value, the second screening subunit screens out the liver segment standard lesion image with the highest similarity between the liver segment lesion region image and the at least one liver segment standard lesion image with the similarity larger than the preset first threshold value.
The third transmission subunit is connected with the second screening subunit, and transmits the liver segment lesion region image and the liver segment standard lesion image with the highest similarity to the liver segment lesion region image to a lesion type visualization module of display equipment;
a fourth judging subunit, connected to the third judging subunit, for judging whether the similarity between the liver segment lesion region image and at least two liver segment standard lesion images is greater than a preset second threshold value when the similarity between the liver segment lesion region image and any one liver segment standard lesion image is less than or equal to a preset first threshold value, wherein the second threshold value is smaller than the first threshold value;
the fourth sending subunit is connected with the fourth judging subunit; and if the similarity between the liver segment lesion area image and the at least two liver segment standard lesion images is larger than a preset second threshold value, the fourth sending subunit sends the liver segment lesion area image and the at least two liver segment standard lesion images with the similarity between the liver segment lesion area image and the liver segment standard lesion images being larger than the preset second threshold value to a to-be-selected visualization module of the display device.
7. An assisted liver surgery planning modeling system according to any of claims 1-6 in which: the liver image segmentation and the liver image segmentation both adopt a Couinaud division method.
CN202311654269.7A 2023-12-05 2023-12-05 Auxiliary liver operation planning modeling system Active CN117576321B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311654269.7A CN117576321B (en) 2023-12-05 2023-12-05 Auxiliary liver operation planning modeling system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311654269.7A CN117576321B (en) 2023-12-05 2023-12-05 Auxiliary liver operation planning modeling system

Publications (2)

Publication Number Publication Date
CN117576321A true CN117576321A (en) 2024-02-20
CN117576321B CN117576321B (en) 2024-07-19

Family

ID=89862328

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311654269.7A Active CN117576321B (en) 2023-12-05 2023-12-05 Auxiliary liver operation planning modeling system

Country Status (1)

Country Link
CN (1) CN117576321B (en)

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007053676A2 (en) * 2005-11-01 2007-05-10 Edda Technology, Inc. Method and system for liver lobe segmentation and pre-operative surgical planning
US20150063668A1 (en) * 2012-03-02 2015-03-05 Postech Academy-Industry Foundation Three-dimensionlal virtual liver surgery planning system
CN106780728A (en) * 2017-02-28 2017-05-31 成都金盘电子科大多媒体技术有限公司 A kind of single organ method for splitting and system based on medical image
CN108074242A (en) * 2018-01-29 2018-05-25 北京大学人民医院 A kind of inflammation recognition methods and device
CN108305255A (en) * 2017-01-12 2018-07-20 浙江京新术派医疗科技有限公司 The generation method and generating means of operation on liver cut surface
CN108335282A (en) * 2017-01-20 2018-07-27 浙江京新术派医疗科技有限公司 The cut surface generation method and generating means of operation on liver
CN109925053A (en) * 2019-03-04 2019-06-25 杭州三坛医疗科技有限公司 Determination methods, devices and systems, the readable storage medium storing program for executing of operation pathway
US20190197688A1 (en) * 2017-03-10 2019-06-27 Fujitsu Limited Similar case image search program, similar case image search apparatus, and similar case image search method
CN110136095A (en) * 2019-03-28 2019-08-16 青岛大学附属医院 Liver three-dimensional CT image lesion region automatic extracting system and method
CN110310726A (en) * 2019-08-12 2019-10-08 杨晓鹏 Hepatic segmental resection auxiliary system based on medical image three-dimensional reconstruction and holographic display
CN110428488A (en) * 2019-07-09 2019-11-08 浙江工贸职业技术学院 One kind being based on area of computer aided Hepatic CT diagnostic system and method
CN110610181A (en) * 2019-09-06 2019-12-24 腾讯科技(深圳)有限公司 Medical image identification method and device, electronic equipment and storage medium
CN114532965A (en) * 2022-03-01 2022-05-27 广西国际壮医医院 Real-time lung cancer focus recognition system under thoracoscope
WO2023139086A1 (en) * 2022-01-24 2023-07-27 Koninklijke Philips N.V. Analysing liver lesions in medical images

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007053676A2 (en) * 2005-11-01 2007-05-10 Edda Technology, Inc. Method and system for liver lobe segmentation and pre-operative surgical planning
CN107358595A (en) * 2005-11-01 2017-11-17 美国医软科技公司 For liver lobe segmentation and the method and system of pre-operative surgical planning
US20150063668A1 (en) * 2012-03-02 2015-03-05 Postech Academy-Industry Foundation Three-dimensionlal virtual liver surgery planning system
CN108305255A (en) * 2017-01-12 2018-07-20 浙江京新术派医疗科技有限公司 The generation method and generating means of operation on liver cut surface
CN108335282A (en) * 2017-01-20 2018-07-27 浙江京新术派医疗科技有限公司 The cut surface generation method and generating means of operation on liver
CN106780728A (en) * 2017-02-28 2017-05-31 成都金盘电子科大多媒体技术有限公司 A kind of single organ method for splitting and system based on medical image
US20190197688A1 (en) * 2017-03-10 2019-06-27 Fujitsu Limited Similar case image search program, similar case image search apparatus, and similar case image search method
CN108074242A (en) * 2018-01-29 2018-05-25 北京大学人民医院 A kind of inflammation recognition methods and device
CN109925053A (en) * 2019-03-04 2019-06-25 杭州三坛医疗科技有限公司 Determination methods, devices and systems, the readable storage medium storing program for executing of operation pathway
CN110136095A (en) * 2019-03-28 2019-08-16 青岛大学附属医院 Liver three-dimensional CT image lesion region automatic extracting system and method
CN110428488A (en) * 2019-07-09 2019-11-08 浙江工贸职业技术学院 One kind being based on area of computer aided Hepatic CT diagnostic system and method
CN110310726A (en) * 2019-08-12 2019-10-08 杨晓鹏 Hepatic segmental resection auxiliary system based on medical image three-dimensional reconstruction and holographic display
CN110610181A (en) * 2019-09-06 2019-12-24 腾讯科技(深圳)有限公司 Medical image identification method and device, electronic equipment and storage medium
WO2023139086A1 (en) * 2022-01-24 2023-07-27 Koninklijke Philips N.V. Analysing liver lesions in medical images
CN114532965A (en) * 2022-03-01 2022-05-27 广西国际壮医医院 Real-time lung cancer focus recognition system under thoracoscope

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
祝文;方驰华;范应方;杨剑;项楠;曾宁;方兆山;陈青山;: "原发性肝癌三维可视化诊治平台的构建及临床应用", 中华肝脏外科手术学电子杂志, no. 05, 10 October 2015 (2015-10-10), pages 268 - 273 *
陆声等: "医学3D打印技术基础与应用", vol. 1, 31 July 2021, 山东科学技术出版社, pages: 134 - 135 *
黄志强: "黄志强胆道外科", vol. 1, 30 November 1998, 山东科学技术出版社, pages: 13 - 14 *
黄骁: "图像参照对比在超声诊断中的应用", 广西医学, no. 10, 15 October 2003 (2003-10-15), pages 1912 - 1913 *

Also Published As

Publication number Publication date
CN117576321B (en) 2024-07-19

Similar Documents

Publication Publication Date Title
CN108520519B (en) Image processing method and device and computer readable storage medium
CN110010249B (en) Augmented reality operation navigation method and system based on video superposition and electronic equipment
CN109785303B (en) Rib marking method, device and equipment and training method of image segmentation model
Collins et al. Augmented reality guided laparoscopic surgery of the uterus
CN105228548B (en) The designing system and method for operative correction for abnormal bone
US9129362B2 (en) Semantic navigation and lesion mapping from digital breast tomosynthesis
CN114129240B (en) Method, system and device for generating guide information and electronic equipment
CN107067398B (en) Completion method and device for missing blood vessels in three-dimensional medical model
US10635924B2 (en) System and method for surgical guidance and intra-operative pathology through endo-microscopic tissue differentiation
EP2901934B1 (en) Method and device for generating virtual endoscope image, and program
EP2901935B1 (en) Method and device for generating virtual endoscope image, and program
JP5399225B2 (en) Image processing apparatus and method, and program
CN107106128B (en) Ultrasound imaging apparatus and method for segmenting an anatomical target
Varnavas et al. Increasing the automation of a 2D-3D registration system
Debarba et al. Efficient liver surgery planning in 3D based on functional segment classification and volumetric information
KR102433473B1 (en) Method, apparatus and computer program for providing augmented reality based medical information of patient
CN111260704A (en) Vascular structure 3D/2D rigid registration method and device based on heuristic tree search
CN112331311B (en) Method and device for fusion display of video and preoperative model in laparoscopic surgery
WO2023232129A1 (en) Three-dimensional-model processing method and apparatus, and electronic device
KR20230013041A (en) How to determine the ablation site based on deep learning
CN117576321B (en) Auxiliary liver operation planning modeling system
KR102213412B1 (en) Method, apparatus and program for generating a pneumoperitoneum model
CN115886999A (en) Operation guiding method, device and control system based on simulation virtual technology
US11452566B2 (en) Pre-operative planning for reorientation surgery: surface-model-free approach using simulated x-rays
KR20230013042A (en) Method for predicting recurrence of lesions through image analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant