CN112700551A - Virtual choledochoscope interventional operation planning method, device, equipment and storage medium - Google Patents
Virtual choledochoscope interventional operation planning method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112700551A CN112700551A CN202011626331.8A CN202011626331A CN112700551A CN 112700551 A CN112700551 A CN 112700551A CN 202011626331 A CN202011626331 A CN 202011626331A CN 112700551 A CN112700551 A CN 112700551A
- Authority
- CN
- China
- Prior art keywords
- biliary tract
- dimensional
- choledochoscope
- path
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Geometry (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Magnetic Resonance Imaging Apparatus (AREA)
Abstract
According to the planning method, the device, the equipment and the storage medium for the virtual choledochoscope interventional operation, provided by the embodiment of the invention, a two-dimensional medical image sequence is obtained by scanning the medical image of the abdomen of a target object; performing three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model; according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model, carrying out biliary tract path planning in the three-dimensional biliary tract model; and selecting a target biliary tract path meeting the indication requirement from the planned biliary tract paths according to the biliary tract path selection indication requirement. The three-dimensional model can be used by a doctor for diagnosis and preoperative training, and compared with the two-dimensional medical image for diagnosis and preoperative training, the three-dimensional model is more visual and stereoscopic, the disease diagnosis and treatment difficulty of the doctor is reduced, the shortening of the operation time is facilitated, and the operation risk is reduced.
Description
Technical Field
The invention relates to the technical field of medical instruments, in particular to a planning method, a planning device, planning equipment and a storage medium for virtual choledochoscope interventional operations.
Background
With the influence of factors such as improvement of living standard of people in China, adjustment of dietary structure and the like, the prevalence rate of gallstones in China is gradually increased. At present, the incidence rate of gallstones in China is more than 10%. Gallstones become common diseases and frequently encountered diseases in China. Because gallstones can cause symptoms of digestive tracts such as abdominal distension and dyspepsia, severe patients induce biliary colic, complicated bile duct infection, obstructive jaundice, pancreatitis and the like, and are closely related to bile duct tumors, enough attention needs to be paid to treatment of the gallstones, and early discovery and early treatment are achieved.
At present, the main methods for treating gallstones comprise cholecystectomy, gallbladder protection and stone removal. Compared with the cholecystectomy, the gallbladder-protecting calculus-removing operation can not only avoid complications caused by the cholecystectomy, but also meet the gallbladder-protecting requirements of patients. With the rapid progress of medical apparatus and instruments technologies such as endoscopes, more advanced choledochoscope calculus removal operation methods have been developed. The choledochoscope calculus-removing operation is to remove calculus in the biliary tract system under the direct vision of a fiber choledochoscope. The choledochoscope calculus removing operation has the advantages of high calculus removing rate, small wound and quick recovery while realizing gallbladder protection, and can effectively reduce the recurrence rate of the cholelithiasis.
However, at present, the choledochoscope calculus removing operation can only be performed by a doctor to judge the position of a calculus according to a Computed Tomography (CT) image of the abdomen of a patient. Doctors can only imagine three-dimensional biliary tract space and calculus position according to two-dimensional image sequences, and the requirements on the space imagination capability of the doctors are extremely high. On the other hand, the biliary system of the human body is in a tree state, and has more biliary branches, so that a doctor is easy to lose directions in the operation process. The traditional Chinese medicine repeatedly searches for calculus and has great damage to patients.
Disclosure of Invention
The embodiment of the invention provides a planning method, a device, equipment and a storage medium for a virtual choledochoscope interventional operation, which are used for solving the problems that in the prior art, when a biliary tract disease is diagnosed and treated, a doctor can only imagine a three-dimensional biliary tract space and a focus position according to a two-dimensional medical image sequence, the requirement on the space imagination capacity of the doctor is extremely high, and the doctor repeatedly searches for great damage to a patient in the biliary tract operation.
The embodiment of the invention provides a virtual choledochoscope interventional operation planning method, which comprises the following steps:
performing medical image scanning on the abdomen of a target object to obtain a two-dimensional medical image sequence;
performing three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model, planning a biliary tract path by using the three-dimensional biliary tract model;
and selecting a target biliary tract path meeting the indication requirement from the planned biliary tract paths according to the biliary tract path selection indication requirement.
Optionally, selecting, according to a biliary tract path selection indication requirement, a target biliary tract path meeting the indication requirement from the planned biliary tract paths, including:
and selecting the biliary tract with the shortest length from the planned biliary tract routes according to the optimal biliary tract route selection indication requirement.
Optionally, performing a biliary tract path planning by using the three-dimensional biliary tract model according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model, including:
calculating to obtain a skeleton topological structure of a central line of the biliary tract by using an average curvature flow algorithm on the three-dimensional biliary tract model;
and planning a biliary tract path in the central line skeleton topological structure according to the position of the entrance of the choledochoscope and the position of the focus marked in the three-dimensional biliary tract model. Optionally, performing a biliary tract path planning in the centerline skeleton topology according to the choledochoscope entrance position and the calculus position marked in the three-dimensional biliary tract model, including:
and planning a biliary tract path in the central line skeleton topological structure by using a breadth-first search algorithm or a depth-first search algorithm according to the position of the entrance of the choledochoscope and the position of the calculus marked in the three-dimensional biliary tract model.
Optionally, the medical image scan is an electron computed tomography CT or magnetic resonance imaging scan.
Optionally, after selecting the biliary tract path meeting the indication requirement, the method further includes:
generating and displaying a simulated three-dimensional image of the biliary tract environment at a designated position in the target biliary tract path according to the three-dimensional biliary tract model, wherein the designated position is determined according to display control information;
or generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a moving mode according to the three-dimensional biliary tract model.
Optionally, generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a moving manner according to the three-dimensional biliary tract model, further includes:
and when the simulated three-dimensional image of the biliary tract environment at the branch position of the target biliary tract path is displayed in a moving mode, the displaying of the moving mode is suspended.
Optionally, the virtual choledochoscope intervention operation planning method further includes:
synchronously displaying a simulated three-dimensional image of the biliary tract environment at the position corresponding to the biliary tract three-dimensional model according to the acquired position of the choledochoscope in the biliary tract of the target object;
and displaying the motion navigation information of the choledochoscope according to the structural information of the biliary tract path meeting the indication requirement and the acquired position of the choledochoscope in the biliary tract of the target object.
Based on the same inventive concept, the embodiment of the present invention further provides a virtual choledochoscope intervention operation planning device, including:
the medical image scanning module is used for scanning medical images of the abdomen of the target object to obtain a two-dimensional medical image sequence;
the three-dimensional modeling module is used for carrying out three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
the path planning module is used for planning a biliary tract path by utilizing the three-dimensional biliary tract model according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model;
and the path selection module is used for selecting a target biliary tract path meeting the indication requirement from the planned biliary tract paths according to the biliary tract path selection indication requirement.
Optionally, selecting, according to a biliary tract path selection indication requirement, a target biliary tract path meeting the indication requirement from the planned biliary tract paths, including:
and selecting the biliary tract with the shortest length from the planned biliary tract routes according to the optimal biliary tract route selection indication requirement.
Optionally, performing a biliary tract path planning by using the three-dimensional biliary tract model according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model, including:
calculating to obtain a skeleton topological structure of a central line of the biliary tract by using an average curvature flow algorithm on the three-dimensional biliary tract model;
and planning a biliary tract path in the central line skeleton topological structure according to the position of the entrance of the choledochoscope and the position of the focus marked in the three-dimensional biliary tract model.
Optionally, performing a biliary tract path planning in the centerline skeleton topology according to the choledochoscope entrance position and the calculus position marked in the three-dimensional biliary tract model, including:
and planning a biliary tract path in the central line skeleton topological structure by using a breadth-first search algorithm or a depth-first search algorithm according to the position of the entrance of the choledochoscope and the position of the calculus marked in the three-dimensional biliary tract model.
Optionally, the medical image scan is an electron computed tomography CT or magnetic resonance imaging scan.
Optionally, after selecting the biliary tract path meeting the indication requirement, the method further includes:
the static display module is used for generating and displaying a simulated three-dimensional image of the biliary tract environment at a specified position in the target biliary tract path according to the three-dimensional biliary tract model, and the specified position is determined according to display control information;
or the motion display module is used for generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a motion mode according to the three-dimensional biliary tract model.
Optionally, the virtual choledochoscope intervention operation planning apparatus further includes:
and the pause motion display module is used for pausing the motion display when the simulated three-dimensional image of the biliary tract environment at the branch position of the target biliary tract path is displayed in a motion mode.
Optionally, the virtual choledochoscope intervention operation planning apparatus further includes:
the auxiliary contrast navigation module is used for synchronously displaying a simulated three-dimensional image of the biliary tract environment at the position corresponding to the biliary tract three-dimensional model according to the acquired position of the choledochoscope in the biliary tract of the target object; and displaying the motion navigation information of the choledochoscope according to the structural information of the biliary tract path meeting the indication requirement and the acquired position of the choledochoscope in the biliary tract of the target object.
Based on the same inventive concept, the embodiment of the present invention further provides a virtual choledochoscope intervention operation planning device, including: a processor and a memory for storing processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the virtual choledochoscope interventional procedure planning method.
Based on the same inventive concept, the embodiment of the present invention further provides a computer storage medium, where a computer program is stored, and the computer program is used to implement the virtual choledochoscope intervention operation planning method.
The invention has the following beneficial effects:
according to the virtual choledochoscope intervention operation planning method, the virtual choledochoscope intervention operation planning device, the virtual choledochoscope intervention operation planning equipment and the storage medium, a three-dimensional biliary tract three-dimensional model is obtained by performing three-dimensional modeling on a two-dimensional medical image sequence of the abdomen of a target object, biliary tract path planning is performed in the biliary tract three-dimensional model according to the entrance position and the focus position of a choledochoscope, and a target biliary tract path is selected, so that a doctor can use the three-dimensional model to perform diagnosis and pre-operation training.
Drawings
Fig. 1 is a flowchart of a virtual choledochoscope interventional procedure planning method according to an embodiment of the present invention;
FIG. 2A is a three-dimensional model of a biliary tract with gallstones according to an embodiment of the present invention;
fig. 2B is a three-dimensional model planning path diagram of a biliary tract with gallstones according to an embodiment of the present invention;
FIG. 2C is a three-dimensional biliary tract model of a biliary tract with gallstones according to an embodiment of the present invention;
FIG. 2D is a simulated three-dimensional image of the biliary environment within a target biliary tract having a biliary stone, provided in accordance with an embodiment of the present invention;
FIG. 3A is a schematic view of a search sequence of a breadth-first search algorithm;
FIG. 3B is a schematic diagram of a search sequence of a depth-first search algorithm;
fig. 4 is a schematic structural diagram of a virtual choledochoscope intervention operation planning device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a virtual choledochoscope intervention operation planning apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, the present invention is further described with reference to the accompanying drawings and examples. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus their repetitive description will be omitted. The words expressing the position and direction described in the present invention are illustrated in the accompanying drawings, but may be changed as required and still be within the scope of the present invention. The drawings of the present invention are for illustrative purposes only and do not represent true scale.
It should be noted that in the following description, specific details are set forth in order to provide a thorough understanding of the present invention. The invention can be implemented in a number of ways different from those described herein and similar generalizations can be made by those skilled in the art without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The description which follows is a preferred embodiment of the present application, but is made for the purpose of illustrating the general principles of the application and not for the purpose of limiting the scope of the application. The protection scope of the present application shall be subject to the definitions of the appended claims.
The following describes a virtual choledochoscope intervention operation planning method, device, equipment and storage medium provided by the embodiments of the present invention with reference to the accompanying drawings.
The embodiment of the invention provides a virtual choledochoscope interventional operation planning method, as shown in fig. 1-2C, comprising the following steps:
s101, medical image scanning is carried out on the abdomen of a target object to obtain a two-dimensional medical image sequence;
s102, performing three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
s103, according to the marked position of the entrance RK of the choledochoscope and the marked position of the focus in the three-dimensional biliary tract model, planning a biliary tract path by using the three-dimensional biliary tract model;
and S104, selecting a target biliary tract path MBLJ meeting the indication requirement from the planned biliary tract paths LJ according to the biliary tract path selection indication requirement.
In a specific implementation, the lesion may be a biliary calculus, a biliary nodule, a biliary tumor, or the like, which is not limited herein. The virtual choledochoscope intervention operation planning method can be applied to operation simulation of operations such as biliary calculus removal operation, biliary calculus breaking operation, biliary biopsy, biliary tumor ablation operation and the like. Fig. 2A, 2B, and 2C respectively illustrate the step S102, the step S103, and the step S104, respectively, by taking the gallstone JS as an example.
In a specific implementation process, in step S101, medical image scanning is performed on the abdomen of the target object to obtain a two-dimensional medical image sequence, specifically, continuous medical image perspective scanning with different cross sections is sequentially performed on the abdomen of the target object to obtain the two-dimensional medical image sequence of each cross section. Wherein the separation distance between said sections is small.
In a specific implementation process, in the step S102, three-dimensional modeling is performed according to the medical image sequence, and a basic principle of obtaining a three-dimensional biliary tract model is as follows: firstly, performing two-dimensional image processing on an input medical image sequence, wherein the two-dimensional image processing comprises the steps of segmenting by means of gray level image binarization and the like, smoothing and noise elimination on the medical image sequence by means of median filtering and the like, smoothing by means of low-pass filtering and the like, sharpening by means of high-pass filtering and the like, and finally performing edge detection and edge enhancement to extract relevant features. And then, modeling a three-dimensional biliary tract model by adopting a surface drawing method or a volume drawing method for the medical image sequence after the two-dimensional image processing is finished. The three-dimensional biliary tract model may be a surface three-dimensional model represented by a planar sheet (e.g., a triangular planar sheet) in an approximation manner. The surface drawing method is also called a surface-based three-dimensional surface drawing method or an indirect drawing method, and is a method for describing a three-dimensional structure of an object by splicing and fitting the surface of the object through geometric units based on two-dimensional image edge or contour line extraction. The volume rendering method, also referred to as a volume rendering method based on volume data or a direct rendering method, is a method of projecting voxels to a display plane by directly applying a visual principle.
In a specific implementation process, the position of the choledochoscope entrance RK is determined by a doctor according to medical experience according to the actual physical condition of a target object.
Therefore, a three-dimensional biliary tract model is obtained by three-dimensional modeling of a two-dimensional medical image sequence of the abdomen of the target object, a biliary tract path is planned in the three-dimensional biliary tract model according to the entrance position and the focus position of the choledochoscope, and the target biliary tract path is selected, so that a doctor can use the three-dimensional biliary tract model to perform diagnosis and pre-operation training, and compared with the diagnosis and pre-operation training by using a two-dimensional medical image, the three-dimensional biliary tract model is more visual and three-dimensional, and the disease diagnosis and treatment difficulty of the doctor is reduced.
Optionally, the step S104 of selecting, according to the biliary tract path selection instruction requirement, a target biliary tract path MBLJ meeting the instruction requirement from the planned biliary tract paths LJ includes:
and selecting the biliary tract LJ with the shortest biliary tract path length from the planned biliary tract paths LJ according to the optimal biliary tract path selection indication requirement.
In this way, by selecting the biliary tract having the shortest length as the optimal route, it is possible to reduce the damage to the target object and alleviate the pain of the target object when the doctor performs a choledochoscopy or a choledochoscopy operation on the target object.
Optionally, in the step S102, performing three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model, including:
calculating to obtain a skeleton topological structure of a central line of the biliary tract by using an average curvature flow algorithm on the three-dimensional biliary tract model;
and planning a biliary tract path in the central line skeleton topological structure according to the position of the choledochoscope entrance RK and the position of the focus marked in the three-dimensional biliary tract model.
Specifically, the curve skeleton represents the shape abstraction of the geometric structure and the topological relation of the three-dimensional model, can also be interpreted as a degenerated manifold with infinitesimal small cross-section and zero area and volume, and can be obtained through the evolution of a curved surface. The invention adopts a mean curvature flow method, pushes the curvature flow to an extreme value by utilizing the characteristic of minimized area, thereby folding the geometry of an input grid, and moves each point of the model surface along the reverse normal line of the model surface in an iteration mode at a speed proportional to the local mean curvature, so that the three-dimensional model surface gradually collapses to obtain a skeleton structure. Mean curvature is an "extrinsic" bending measure in differential geometry, and describes locally the curvature of a curved surface embedded in the surrounding space (e.g., a two-dimensional surface embedded in a three-dimensional euclidean space). The mean curvature is the average of any two orthogonal curvatures perpendicular to each other at a point on the spatially curved surface. If a set of orthogonal curvatures that are orthogonal to each other can be represented as K1, K2, then the average curvature is: k ═ K1+ K2)/2.
In a specific implementation process, an average curvature flow algorithm iteratively calls an implicit constraint Laplace solver, triangulation is optimized through local subdivision until the volume of the shape disappears, and finally the topological structure of the center line skeleton of the biliary tract is obtained.
In this way, the biliary tract path planning process is facilitated by using the centerline skeleton topology of the biliary tract obtained by the mean curvature flow algorithm.
Optionally, performing a biliary tract path planning in the centerline skeleton topology according to the choledochoscope entrance position and the calculus position marked in the three-dimensional biliary tract model, including:
and planning a biliary tract path in the central line skeleton topological structure by using a breadth-first search algorithm or a depth-first search algorithm according to the position of the entrance of the choledochoscope and the position of the calculus marked in the three-dimensional biliary tract model.
Specifically, the specific working principle of the breadth-first search algorithm is as follows: traversing a primary path from a starting point, recording all secondary paths branched from all the primary nodes of the primary path at primary nodes of the primary path, and traversing all the secondary paths to secondary nodes respectively; all tertiary paths … … branching from all secondary nodes are recorded for the secondary path, and so on, and finally all paths are planned through traversal. As shown in fig. 3A, the circles shown in the figure are nodes of paths of each level, and circles in the same row represent nodes of the same level. Starting from the starting node 1, all the primary paths are traversed to the corresponding primary nodes 2, 3 and 4. Secondly, all secondary paths of the primary node 2 are traversed to the secondary nodes 5 and 6 respectively, all secondary paths of the primary node 3 are traversed to the secondary nodes 7 and 8 respectively, and all secondary paths of the primary node 4 are traversed to the secondary nodes 9 and 10 respectively. After traversing all the secondary paths, respectively traversing all the tertiary paths of all the secondary nodes: all the three-level paths of the second-level node 5 traverse to the third- level nodes 11 and 12, the second-level node 6 is an end point, the second-level node 7 is an end point, the second-level node 8 is an end point, the second-level node 9 is an end point, and all the three-level paths of the second-level node 10 traverse to the third- level nodes 13 and 14. After the traversal of all the three-level paths is completed, respectively traversing all the four-level paths of all the three-level nodes: the third level node 11 is a destination, all the four-level paths to the third level node 12 traverse to the four- level nodes 15 and 16, the third level node 13 is a destination, and the third level node 14 is a destination. The traversal is then ended since both of the four- level nodes 15, 16 are end points. The final traversal order for the path is then the order of the ordinal numbers in the circles in the figure. And correspondingly combining all levels of paths to obtain all paths planned finally.
Specifically, the specific working principle of the depth-first search algorithm is as follows: starting from the starting node, an undiversad path is selected to move to the end node. And returning to the node at the upper level of the end point node after reaching the end point node, and selecting another non-traversed path to move to the end point node. And when the destination node is reached, returning to the previous-stage node of the destination node, repeating the process until all paths corresponding to the previous-stage node are traversed, then returning to the next previous-stage node, repeating the process … … and so on until the nodes return to the starting point, and all paths are traversed to plan all paths. As shown in fig. 3B, the circles shown in the figure are nodes of the path, and the circles of the same row represent nodes of the same level. Starting from the starting node 1, one of the branch paths (2-3, 3-4) is selected for each branch node and moves to the end node 4. Returning to the previous node 3, one of the branch paths (3-5, 5-6) which is not traversed is selected and moved to the end node 6. Returning to the previous node 5, one of the branch paths (5-7) which is not traversed is selected and moved to the end node 7. And returning to the previous-stage node 5, and returning to the next previous-stage node 3 because the traversal of all the branch paths corresponding to the node 5 is completed. Since the traversal of all the branch paths of the node 3 has been completed, the node 2 at the next higher stage is returned. One branch path (2-8) is selected that is not traversed and moves to the end node 8. Returning to the node 2 at the previous stage, and returning to the node 1 at the next previous stage because the traversal of all the branch paths of the node 2 is completed. One branch path (1-9, 9-10) is selected to be not traversed, the path is moved to the end node 10 and returned to the previous node 9 … …, and the final traversal order of the paths is the order of the sequence numbers marked in the circles in the figure.
In the specific implementation process, the lengths of all levels of branch paths are calculated, and the lengths of all levels of branch paths in the final planned path are accumulated, so that the total length of each final planned path can be obtained.
In this way, the path planning is performed by using the depth-first search algorithm, so that the occupied memory is less, and the traversal speed is slower. The path planning is carried out by using the breadth-first algorithm, so that the occupied memory is large, and the traversal speed is high.
Optionally, the medical image scan is an electron computed tomography CT or magnetic resonance imaging scan.
Optionally, as shown in fig. 1 and fig. 2D, after selecting the biliary tract path meeting the indication requirement, the method further includes:
and S105, generating and displaying a simulated three-dimensional image of the biliary tract environment at a designated position in the target biliary tract path according to the three-dimensional biliary tract model, wherein the designated position is determined according to display control information.
Therefore, the corresponding simulated three-dimensional image is displayed by manually selecting the biliary tract environment, and a doctor can conveniently check the simulated three-dimensional image of the biliary tract environment at a specific position.
Or, optionally, after selecting the biliary tract path meeting the indication requirement, the method further includes:
and S106, generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a moving mode according to the three-dimensional biliary tract model.
Therefore, the simulated three-dimensional image of the biliary tract environment in the target biliary tract path is displayed through automatic movement, and a doctor can conveniently check the simulated three-dimensional image of the whole target biliary tract path.
Optionally, the step S106 of generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a moving manner according to the three-dimensional biliary tract model further includes:
and S107, when the simulated three-dimensional image of the biliary tract environment at the branch position of the target biliary tract path is displayed in a moving mode, the displaying of the moving mode is suspended.
Therefore, the simulated three-dimensional image is displayed by pausing the motion at the branch position, so that the doctor can conveniently check the simulated three-dimensional image at the biliary branch.
Optionally, the virtual choledochoscope intervention operation planning method further includes:
s108, synchronously displaying a simulated three-dimensional image of the biliary tract environment at the position corresponding to the biliary tract three-dimensional model according to the acquired position of the choledochoscope in the biliary tract of the target object; and displaying the motion navigation information of the choledochoscope according to the structural information of the biliary tract path meeting the indication requirement and the collected position of the choledochoscope in the biliary tract of the target object.
In a specific implementation process, the step S108 may be to synchronously display a real image acquired by the choledochoscope and a simulated three-dimensional image of a corresponding position of the real image during a real surgery. The position of the choledochoscope in the biliary tract of the target object may be acquired by mechanical positioning, image positioning, electromagnetic positioning, and the like, which is not limited herein. The mechanical positioning is to collect the motion path of the surgical instrument by controlling the control equipment of the surgical instrument such as the choledochoscope and the like to position. The image positioning is to perform medical image scanning on the abdomen of the target object, perform image analysis or three-dimensional modeling on the obtained two-dimensional medical image, and then determine the position of the choledochoscope for positioning. The electromagnetic positioning is to collect the induction signal of the external magnetic field outside the target object body through a magnetic field sensor arranged on the surgical instruments such as the choledochoscope and the like to position.
Therefore, by synchronously displaying the simulated three-dimensional image corresponding to the position of the choledochoscope, a doctor can conveniently check the simulated three-dimensional image in the operation. By displaying the motion navigation information of the choledochoscope, the doctor can be assisted to control the motion direction of the choledochoscope.
Based on the same inventive concept, an embodiment of the present invention further provides a virtual choledochoscope intervention operation planning apparatus, as shown in fig. 4, including:
the medical image scanning module M101 is used for performing medical image scanning on the abdomen of the target object to obtain a two-dimensional medical image sequence;
the three-dimensional modeling module M102 is used for carrying out three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
a path planning module M103, configured to plan a biliary tract path in the three-dimensional biliary tract model according to a position of a choledochoscope entrance and a position of a lesion marked in the three-dimensional biliary tract model;
and the path selection module M104 is configured to select, according to the biliary tract path selection indication requirement, a target biliary tract path meeting the indication requirement from the planned biliary tract paths.
Optionally, the selecting module M104 selects a target biliary tract path meeting the indication requirement from planned biliary tract paths according to the biliary tract path selection indication requirement, including:
and selecting the biliary tract with the shortest length from the planned biliary tract routes according to the optimal biliary tract route selection indication requirement.
Optionally, in the path planning module M103, performing a biliary path planning in the three-dimensional biliary tract model according to a position of a choledochoscope entrance and a position of a lesion marked in the three-dimensional biliary tract model, including:
calculating to obtain a skeleton topological structure of a central line of the biliary tract by using an average curvature flow algorithm on the three-dimensional biliary tract model;
and planning a biliary tract path in the central line skeleton topological structure according to the position of the entrance of the choledochoscope and the position of the focus marked in the three-dimensional biliary tract model. Optionally, in the path planning module M103, performing a biliary path planning in the centerline skeleton topology according to the entrance position and the calculus position of the choledochoscope marked in the three-dimensional biliary tract model, including:
and planning a biliary tract path in the central line skeleton topological structure by using a breadth-first search algorithm or a depth-first search algorithm according to the position of the entrance of the choledochoscope and the position of the calculus marked in the three-dimensional biliary tract model.
Optionally, the medical image scan is an electron computed tomography CT or magnetic resonance imaging scan.
Optionally, the virtual choledochoscope intervention operation planning apparatus further includes:
the static display module M105 is used for generating and displaying a simulated three-dimensional image of the biliary tract environment at a specified position in the target biliary tract path according to the three-dimensional biliary tract model, wherein the specified position is determined according to display control information;
or the motion display module M106 is configured to generate and display a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a motion manner according to the three-dimensional biliary tract model.
Optionally, the virtual choledochoscope intervention operation planning apparatus further includes:
a pause motion display module M107 for pausing the motion display when the simulated three-dimensional image of the biliary tract environment at the branch position of the target biliary tract path is motion displayed.
Optionally, the virtual choledochoscope intervention operation planning apparatus further includes:
the auxiliary contrast navigation module M108 is used for synchronously displaying a simulated three-dimensional image of the biliary tract environment at the position corresponding to the biliary tract three-dimensional model according to the acquired position of the choledochoscope in the biliary tract of the target object; and displaying the motion navigation information of the choledochoscope according to the structural information of the biliary tract path meeting the indication requirement and the acquired position of the choledochoscope in the biliary tract of the target object.
In a specific implementation process, a specific working principle of the virtual choledochoscope intervention operation planning device is substantially consistent with a principle of the virtual choledochoscope intervention operation planning method, and a specific implementation manner of the virtual choledochoscope intervention operation planning device can refer to an implementation manner of the virtual choledochoscope intervention operation planning method, so that details are not repeated.
Based on the same inventive concept, the embodiment of the present invention further provides a virtual choledochoscope intervention operation planning device, including: as shown in fig. 5, includes: a processor 110 and a memory 120 for storing instructions executable by the processor 110; wherein the processor 110 is configured to execute the instructions to implement the virtual choledochoscope interventional procedure planning method.
In particular implementations, the apparatus may vary widely depending on configuration or performance, and may include one or more processors 110 and memory 120, one or more storage media 130 storing applications 131 or data 132. Memory 120 and storage medium 130 may be, among other things, transient or persistent storage. The application 131 stored in the storage medium 130 may include one or more units (not shown in fig. 5) described above, and each module may include a series of instruction operations in the information processing apparatus. Further, the processor 110 may be configured to communicate with the storage medium 130 to execute a series of instruction operations in the storage medium 130 on the device. The apparatus may also include one or more power supplies (not shown in FIG. 5); one or more transceivers 140, the transceivers 140 comprising a wired or wireless network interface 141, one or more input-output interfaces 142; and/or one or more operating systems 133, such as Windows, Mac OS, Linux, IOS, Android, Unix, FreeBSD, etc.
Based on the same inventive concept, the embodiment of the present invention further provides a computer storage medium, where a computer program is stored, and the computer program is used to implement the virtual choledochoscope intervention operation planning method.
According to the virtual choledochoscope intervention operation planning method, the virtual choledochoscope intervention operation planning device, the virtual choledochoscope intervention operation planning equipment and the storage medium, a three-dimensional biliary tract three-dimensional model is obtained by performing three-dimensional modeling on a two-dimensional medical image sequence of the abdomen of a target object, biliary tract path planning is performed in the biliary tract three-dimensional model according to the entrance position and the focus position of a choledochoscope, and a target biliary tract path is selected, so that a doctor can use the three-dimensional model to perform diagnosis and pre-operation training.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A virtual choledochoscope intervention operation planning method is characterized by comprising the following steps:
performing medical image scanning on the abdomen of a target object to obtain a two-dimensional medical image sequence;
performing three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model, planning a biliary tract path by using the three-dimensional biliary tract model;
and selecting a target biliary tract path meeting the indication requirement from the planned biliary tract paths according to the biliary tract path selection indication requirement.
2. The virtual choledochoscope interventional procedure planning method of claim 1, wherein selecting a target biliary tract path meeting an indication requirement from planned biliary tract paths according to the indication requirement for biliary tract path selection comprises:
and selecting the biliary tract with the shortest length from the planned biliary tract routes according to the optimal biliary tract route selection indication requirement.
3. The virtual choledochoscope intervention operation planning method according to claim 1, wherein performing a biliary tract path planning using the three-dimensional biliary tract model according to a choledochoscope entrance position and a lesion position marked in the three-dimensional biliary tract model comprises:
calculating to obtain a skeleton topological structure of a central line of the biliary tract by using an average curvature flow algorithm on the three-dimensional biliary tract model;
and planning a biliary tract path in the central line skeleton topological structure according to the position of the entrance of the choledochoscope and the position of the focus marked in the three-dimensional biliary tract model.
4. The virtual choledochoscope interventional procedure planning method of claim 3, wherein performing a biliary path planning in the centerline skeleton topology based on the choledochoscope entry location and the stone location marked in the three-dimensional model of the biliary tract comprises:
and planning a biliary tract path in the central line skeleton topological structure by using a breadth-first search algorithm or a depth-first search algorithm according to the position of the entrance of the choledochoscope and the position of the calculus marked in the three-dimensional biliary tract model.
5. The virtual choledochoscope interventional surgical planning method of claim 1, further comprising, after selecting the biliary path that meets the indicated requirements:
generating and displaying a simulated three-dimensional image of the biliary tract environment at a designated position in the target biliary tract path according to the three-dimensional biliary tract model, wherein the designated position is determined according to display control information;
or generating and displaying a simulated three-dimensional image of the biliary tract environment in the target biliary tract path in a moving mode according to the three-dimensional biliary tract model.
6. The virtual choledochoscope interventional surgical planning method of claim 5, wherein generating and displaying a simulated three-dimensional image of the biliary tract environment within the target biliary tract path in motion from the three-dimensional biliary tract model, further comprises:
and when the simulated three-dimensional image of the biliary tract environment at the branch position of the target biliary tract path is displayed in a moving mode, the displaying of the moving mode is suspended.
7. The virtual choledochoscope interventional procedure planning method of claim 1, further comprising:
synchronously displaying a simulated three-dimensional image of the biliary tract environment at the position corresponding to the biliary tract three-dimensional model according to the acquired position of the choledochoscope in the biliary tract of the target object;
and displaying the motion navigation information of the choledochoscope according to the structural information of the biliary tract path meeting the indication requirement and the acquired position of the choledochoscope in the biliary tract of the target object.
8. A virtual choledochoscope interventional procedure planning device, comprising:
the medical image scanning module is used for scanning medical images of the abdomen of the target object to obtain a two-dimensional medical image sequence;
the three-dimensional modeling module is used for carrying out three-dimensional modeling according to the medical image sequence to obtain a three-dimensional biliary tract model;
the path planning module is used for planning a biliary tract path by utilizing the three-dimensional biliary tract model according to the entrance position and the lesion position of the choledochoscope marked in the three-dimensional biliary tract model;
and the path selection module is used for selecting a target biliary tract path meeting the indication requirement from the planned biliary tract paths according to the biliary tract path selection indication requirement.
9. A virtual choledochoscope interventional procedure planning device, comprising: a processor and a memory for storing processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the virtual choledochoscope interventional procedure planning method of any one of claims 1-7.
10. A computer storage medium, characterized in that the computer storage medium stores a computer program for implementing the virtual choledochoscope interventional procedure planning method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011626331.8A CN112700551A (en) | 2020-12-31 | 2020-12-31 | Virtual choledochoscope interventional operation planning method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011626331.8A CN112700551A (en) | 2020-12-31 | 2020-12-31 | Virtual choledochoscope interventional operation planning method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112700551A true CN112700551A (en) | 2021-04-23 |
Family
ID=75513244
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011626331.8A Withdrawn CN112700551A (en) | 2020-12-31 | 2020-12-31 | Virtual choledochoscope interventional operation planning method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112700551A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113628346A (en) * | 2021-06-30 | 2021-11-09 | 上海市胸科医院 | Method for freely browsing VB (visual basic) and method and system for marking |
| CN113876420A (en) * | 2021-09-08 | 2022-01-04 | 上海微创医疗机器人（集团）股份有限公司 | Path planning method, system, device and medium for planning surgical path |
| CN114550943A (en) * | 2022-04-21 | 2022-05-27 | 武汉烽火凯卓科技有限公司 | Shock wave incident point simulation planning method and system based on medical image |
| WO2024096084A1 (en) * | 2022-11-04 | 2024-05-10 | 富士フイルム株式会社 | Medical assistance device, endoscope, medical assistance method, and program |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1823349A (en) * | 2003-07-11 | 2006-08-23 | 西门子共同研究公司 | System and method for endoscopic path planning |
| CN102920510A (en) * | 2012-10-29 | 2013-02-13 | 清华大学 | Path planning method for flexible puncture needle |
| CN102961187A (en) * | 2012-10-26 | 2013-03-13 | 深圳市旭东数字医学影像技术有限公司 | Surgical planning method and system for percutaneous puncture |
| CN104050313A (en) * | 2013-03-15 | 2014-09-17 | 柯惠有限合伙公司 | Pathway planning system and method |
| CN104323860A (en) * | 2014-11-07 | 2015-02-04 | 刘弘毅 | Navigation path planning device and method |
| CN104463965A (en) * | 2014-12-17 | 2015-03-25 | 中国科学院自动化研究所 | Training scene simulation system and method for minimally invasive cardiovascular interventional operation |
| CN204364122U (en) * | 2014-11-07 | 2015-06-03 | 刘弘毅 | Navigation path planning device |
| CN105167849A (en) * | 2015-05-19 | 2015-12-23 | 上海大学 | Three-dimensional vascular path planning method based on ant colony algorithm |
| CN107126258A (en) * | 2017-06-29 | 2017-09-05 | 哈尔滨理工大学 | A kind of paths planning method of Minimally Invasive Surgery sleeve pipe flexible needle |
| CN109259806A (en) * | 2017-07-17 | 2019-01-25 | 云南师范大学 | A method of the accurate aspiration biopsy of tumour for image guidance |
| CN110507415A (en) * | 2019-08-13 | 2019-11-29 | 广州新脉科技有限公司 | The path simulation method and device that microtubular is virtually intervened |
| CN110604616A (en) * | 2019-09-10 | 2019-12-24 | 中国科学院深圳先进技术研究院 | Interventional operation path planning method and system based on graph search and electronic equipment |
| CN111798451A (en) * | 2020-06-16 | 2020-10-20 | 北京理工大学 | 3D guide wire tracking method and device based on blood vessel 3D/2D matching |
| CN112053400A (en) * | 2020-09-09 | 2020-12-08 | 北京柏惠维康科技有限公司 | Data processing method and robot navigation system |
| CN112057165A (en) * | 2020-09-22 | 2020-12-11 | 上海联影医疗科技股份有限公司 | Path planning method, device, equipment and medium |
| CN112116575A (en) * | 2020-09-18 | 2020-12-22 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
-
2020
- 2020-12-31 CN CN202011626331.8A patent/CN112700551A/en not_active Withdrawn
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1823349A (en) * | 2003-07-11 | 2006-08-23 | 西门子共同研究公司 | System and method for endoscopic path planning |
| CN102961187A (en) * | 2012-10-26 | 2013-03-13 | 深圳市旭东数字医学影像技术有限公司 | Surgical planning method and system for percutaneous puncture |
| CN102920510A (en) * | 2012-10-29 | 2013-02-13 | 清华大学 | Path planning method for flexible puncture needle |
| CN104050313A (en) * | 2013-03-15 | 2014-09-17 | 柯惠有限合伙公司 | Pathway planning system and method |
| CN104323860A (en) * | 2014-11-07 | 2015-02-04 | 刘弘毅 | Navigation path planning device and method |
| CN204364122U (en) * | 2014-11-07 | 2015-06-03 | 刘弘毅 | Navigation path planning device |
| CN104463965A (en) * | 2014-12-17 | 2015-03-25 | 中国科学院自动化研究所 | Training scene simulation system and method for minimally invasive cardiovascular interventional operation |
| CN105167849A (en) * | 2015-05-19 | 2015-12-23 | 上海大学 | Three-dimensional vascular path planning method based on ant colony algorithm |
| CN107126258A (en) * | 2017-06-29 | 2017-09-05 | 哈尔滨理工大学 | A kind of paths planning method of Minimally Invasive Surgery sleeve pipe flexible needle |
| CN109259806A (en) * | 2017-07-17 | 2019-01-25 | 云南师范大学 | A method of the accurate aspiration biopsy of tumour for image guidance |
| CN110507415A (en) * | 2019-08-13 | 2019-11-29 | 广州新脉科技有限公司 | The path simulation method and device that microtubular is virtually intervened |
| CN110604616A (en) * | 2019-09-10 | 2019-12-24 | 中国科学院深圳先进技术研究院 | Interventional operation path planning method and system based on graph search and electronic equipment |
| CN111798451A (en) * | 2020-06-16 | 2020-10-20 | 北京理工大学 | 3D guide wire tracking method and device based on blood vessel 3D/2D matching |
| CN112053400A (en) * | 2020-09-09 | 2020-12-08 | 北京柏惠维康科技有限公司 | Data processing method and robot navigation system |
| CN112116575A (en) * | 2020-09-18 | 2020-12-22 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN112057165A (en) * | 2020-09-22 | 2020-12-11 | 上海联影医疗科技股份有限公司 | Path planning method, device, equipment and medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113628346A (en) * | 2021-06-30 | 2021-11-09 | 上海市胸科医院 | Method for freely browsing VB (visual basic) and method and system for marking |
| CN113876420A (en) * | 2021-09-08 | 2022-01-04 | 上海微创医疗机器人（集团）股份有限公司 | Path planning method, system, device and medium for planning surgical path |
| CN113876420B (en) * | 2021-09-08 | 2024-04-26 | 上海微创微航机器人有限公司 | Path planning method, system, device and medium for planning surgical path |
| CN114550943A (en) * | 2022-04-21 | 2022-05-27 | 武汉烽火凯卓科技有限公司 | Shock wave incident point simulation planning method and system based on medical image |
| CN114550943B (en) * | 2022-04-21 | 2022-07-29 | 武汉烽火凯卓科技有限公司 | Shock wave incident point simulation planning method and system based on medical image |
| WO2024096084A1 (en) * | 2022-11-04 | 2024-05-10 | 富士フイルム株式会社 | Medical assistance device, endoscope, medical assistance method, and program |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112700551A (en) | Virtual choledochoscope interventional operation planning method, device, equipment and storage medium | |
| CN107067398B (en) | Completion method and device for missing blood vessels in three-dimensional medical model | |
| JP2007144177A (en) | Method and apparatus for semi-automatic segmentation technique for low-contrast tubular shaped objects | |
| JP4748843B2 (en) | RECORDING MEDIUM CONTAINING PROGRAM FOR CAUSING COMPUTER TO EXECUTE SPECIFIC AREA EXTRACTION DISPLAY AND DEVICE | |
| CN108420529A (en) | The surgical navigational emulation mode guided based on image in magnetic tracking and art | |
| JP5685605B2 (en) | System for navigating an interventional device and its operating method, computer program element and computer readable medium | |
| JP6912168B2 (en) | Methods and equipment to improve 3D + time reconstruction | |
| JP4688361B2 (en) | Organ specific area extraction display device and display method thereof | |
| KR101700847B1 (en) | Method for Providing Training of Image Guided Surgery and Computer-readable Recording Medium for the same | |
| US20120288173A1 (en) | Surgical assistance planning method using lung motion analysis | |
| CN110432986B (en) | System and method for constructing virtual radial ultrasound images from CT data | |
| JP2006246941A (en) | Image processing apparatus and vessel tracking method | |
| CN111798451A (en) | 3D guide wire tracking method and device based on blood vessel 3D/2D matching | |
| EP3102141A1 (en) | Visualization of depth and position of blood vessels and robot guided visualization of blood vessel cross section | |
| CN102843972A (en) | Instrument-based image registration for fusing images with tubular structures | |
| CN102982531B (en) | Bronchial dividing method and system | |
| EP3795108A1 (en) | Systems and methods for planning medical procedures | |
| WO2019073681A1 (en) | Radiation imaging device, image processing method, and image processing program | |
| JP2017143872A (en) | Radiation imaging apparatus, image processing method, and program | |
| JP4675509B2 (en) | Apparatus and method for extracting and displaying specific region of organ | |
| CN106952264B (en) | Method and device for cutting three-dimensional medical target | |
| CN104011724B (en) | For the method and system for the multi-dimensional surface model for generating geometry | |
| JP2023552577A (en) | Dynamic deformation tracking for navigational bronchoscopy | |
| CN108694007B (en) | Unfolding ribs from magnetic resonance images | |
| CN110025376A (en) | A kind of interventional cardiac procedures analogy method, terminal device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210423 |