CN115778554A - Catheter robot, registration method thereof and readable storage medium - Google Patents

Abstract

The application provides a catheter robot, a registration method thereof and a readable storage medium, wherein the method comprises the following steps: acquiring actual path points of the catheter and/or the instrument by using a sensor arranged on the catheter and/or the instrument carried by the catheter; acquiring simulated path points of the catheter and/or the instrument in the anatomical model, which correspond to the actual path points, according to the first transformation matrix; determining a target pipeline center line segment corresponding to the simulation path point from a plurality of pipeline center line segments into which the pipeline center line is divided; calculating the centroid of the central line segment of the target pipeline; if the centroid is different from a plurality of skeleton points included in the target pipeline central line segment, determining a connecting line between the centroid and the simulated path point, and selecting one of the skeleton points with the shortest distance to the connecting line as a matching point of the actual path point. Through the mode, the accuracy of point cloud registration can be improved, and therefore the navigation accuracy in the catheter operation is improved.

Description

Catheter robot, registration method thereof and readable storage medium

Technical Field

The present application relates to the field of surgical robotics, and in particular, to a catheter robot, a registration method of the catheter robot, and a readable storage medium.

Background

Minimally invasive medical techniques are intended to reduce the amount of tissue damaged during a medical procedure to reduce patient recovery time, discomfort, and harmful side effects. In minimally invasive medical techniques it is often necessary to insert a catheter through a natural orifice in the patient's anatomy or through a surgical incision, through the complex structure of the body duct to reach or be adjacent to the target object in cooperation with an intraoperative navigation system.

In most cases, the catheter is moved along a body duct (e.g. a bronchus, a blood vessel, a ureter, etc.) without being shuttled freely in a manner that damages the duct, so that the path of travel of the catheter/instrument can be considered to be limited within the body duct, with similarity to the centerline of the body duct, especially for smaller ducts, which has a higher similarity to the centerline due to the limited radial range of motion.

According to this principle, the intraoperative navigation system can employ point cloud registration to detect and display the location of the catheter tip and/or the catheter-carried instrument tip in real-time. Specifically, path data of catheter/instrument movement can be acquired through a sensor, matched with pipeline centerline data in an anatomical model of a patient, and coordinate transformation between the two is determined through point cloud registration, so that the real-time position of the catheter/instrument is obtained.

In the related art, the point cloud registration adopts the criterion of the shortest Euclidean distance to search for corresponding point pairs, and then coordinate conversion is calculated according to the corresponding point pairs. However, due to the complexity of the human body duct and the influence of the motion of the patient, the adoption of the criterion of the shortest euclidean distance may often cause wrong corresponding point pairs, for example, a point on the center line of the duct corresponding to the path point of the catheter is not in the duct where the catheter is currently located, so that the precision of point cloud registration is not high, and the accuracy of navigation in the catheter operation is influenced.

Disclosure of Invention

The embodiment of the application provides a method and a device for registering a catheter robot and terminal equipment, and can solve the problem that accurate registration is difficult to perform on path data in the related technology.

In a first aspect, embodiments of the present application provide a catheter robot, the catheter robot including a robotic arm, a catheter instrument engaged with a power portion of the robotic arm, and a processor communicatively connected to the robotic arm, the catheter instrument including an instrument box configured to engage with the power portion and a catheter connected to the instrument box, the catheter and/or a catheter-mounted instrument having a sensor disposed thereon for measuring a position of the catheter and/or the instrument, the processor configured to perform the following steps: acquiring actual waypoints of the catheter and/or instrument using the sensor; acquiring a simulated path point of the catheter and/or the instrument in the anatomical model corresponding to the actual path point according to the first transformation matrix, wherein the anatomical model comprises a pipeline central line; determining a target pipeline center line segment corresponding to the simulation path point in a plurality of pipeline center line segments divided by the pipeline center line; calculating the centroid of the central line segment of the target pipeline; and if the centroid is different from a plurality of skeleton points included in the target pipeline central line segment, determining a connecting line between the centroid and the simulated path point, and selecting one of the skeleton points with the shortest distance to the connecting line as a matching point of the actual path point.

Wherein the processor is configured to perform the following steps after determining a connection line between the centroid and the simulation waypoint: if the centroid is one of the plurality of skeleton points, acquiring a first foot from the simulation path point to the projection line, and acquiring a second foot from each skeleton point to the projection line, wherein the projection line is a straight line passing through two end points of the center line segment of the target pipeline; and selecting a skeleton point corresponding to a second drop foot with the shortest distance between the first drop foot and the second drop foot as a matching point of the actual path point.

Wherein the processor is configured to perform the steps of: updating the point pair set by using a first point pair consisting of the actual path point and the matching point thereof; and if the point pair set meets a first condition, updating the first transformation matrix by using the point pair set, wherein the first condition comprises that the number of the point pairs in the point pair set is greater than a preset value.

Wherein the processor is configured to perform the following steps in updating the point pair set with matching point pairs consisting of actual waypoints and their matching points: judging whether a second point pair comprising the matching points exists in the point pair set or not; if the first distance is smaller than the second distance, replacing the second point pair with the first point pair.

Wherein the processor is configured to perform the following steps in updating the first transformation matrix using the set of point pairs: substituting the point pairs in the point pair set into a target function to calculate a latest second transformation matrix; the first transformation matrix is updated according to the latest second transformation matrix.

The first transformation matrix is a weighted average of the latest second transformation matrix and at least part of the historical second transformation matrix, wherein the weight of the latest second transformation matrix is greater than that of the historical second transformation matrix.

Wherein the processor is configured to perform the following steps prior to determining a target pipeline centerline segment corresponding to the simulated path point among the plurality of pipeline centerline segments into which the pipeline centerline is divided: extracting characteristic skeleton points from a plurality of skeleton points included by a pipeline central line, wherein the positions of the skeleton points are represented by codes, and the codes of the skeleton points are s = x ₁ +x ₂ R ₁ +x ₃ R ₁ R ₂ Wherein x is ₁ 、x ₂ 、x ₃ Respectively the coordinates of the skeleton point in the first, second and third dimension of the coding sequence, R ₁ 、R ₂ The sizes of the anatomical model in the first dimension and the second dimension of the coding order are respectively; and dividing the pipeline center line into a plurality of pipeline center line segments according to the characteristic skeleton point.

Wherein the processor is configured to perform the following steps in extracting feature skeleton points from a plurality of skeleton points comprised by the pipeline centerline: determining a starting point; from the starting point, searching skeleton points by using a region growing method and counting the number of connected skeleton points, wherein the number of connected skeleton points is the number of all child nodes in the neighborhood of the skeleton points; if the number of connected skeleton points is greater than 1 and the number of connected parent nodes of the skeleton points is equal to 1, the skeleton points are bifurcation points in the characteristic skeleton points, and if the number of connected skeleton points is equal to 1 and the number of connected parent nodes of the skeleton points is equal to 0, the skeleton points are terminal points in the characteristic skeleton points.

Wherein the processor is configured to perform the following steps in selecting a starting point: randomly selecting an aggregate layer; counting the total number of skeleton points in a specified number of layers on two sides of the body layer in the direction perpendicular to the cross section so as to determine the extending direction of the air passage; and finding the skeleton points of the trachea entrance as a starting point according to the total number of the skeleton points in the body layer and the extending direction of the airway.

Wherein the processor is configured to perform the steps of: counting the number of skeleton points in the pipeline center line segment; and deleting the pipeline central line segments of which the number is less than a preset threshold value.

In a second aspect, an embodiment of the present application provides a registration method for a catheter robot, where the method includes: acquiring actual waypoints of the catheter and/or the instrument using a sensor disposed on the catheter and/or the instrument carried by the catheter; acquiring simulated path points of the catheter and/or the instrument in the anatomical model corresponding to the actual path points according to the first transformation matrix, wherein the anatomical model comprises a pipeline central line; determining a target pipeline center line segment corresponding to the simulation path point in a plurality of pipeline center line segments into which the pipeline center line is divided, wherein the target pipeline center line segment comprises a plurality of skeleton points; calculating the centroid of the central line segment of the target pipeline; and if the centroid is different from a plurality of skeleton points included in the target pipeline central line segment, determining a connecting line between the centroid and the simulated path point, and selecting one of the skeleton points with the shortest distance to the connecting line as a matching point of the actual path point.

In a third aspect, embodiments of the present application provide a computer-readable storage medium, on which computer program instructions are stored, the computer program instructions being executed by a processor to implement the method described in the second aspect of the present application.

Compared with the prior art, the embodiment of the application has the beneficial effects that: with the embodiments of the present application, actual waypoints of the catheter and/or instrument are acquired with the sensor; acquiring simulated path points of the catheter and/or the instrument in the anatomical model corresponding to the actual path points according to the first transformation matrix, wherein the anatomical model comprises a pipeline central line; determining a target pipeline center line segment corresponding to the simulation path point from a plurality of pipeline center line segments into which the pipeline center line is divided; calculating the centroid of the central line segment of the target pipeline; determining a connecting line between the centroid and the simulation path point; and if the centroid is different from a plurality of skeleton points included in the target pipeline central line segment, selecting one of the skeleton points with the shortest distance to the connecting line as a matching point of the actual path point. In order to adapt to a complex human body environment, the method firstly determines a target pipeline central line segment corresponding to a simulation path point so as to reduce errors that a matching point is not in a pipeline where the catheter is located at present, selects a skeleton point which is shortest in distance from a connecting line between the simulation path point and a centroid of the target pipeline central line segment, can approximately consider the skeleton point as an intersection point of the connecting line and the target pipeline central line segment, takes the intersection point as a matching point of an actual path point, can more accurately reflect the matching relation between the path point of the catheter moving in the pipeline and the pipeline central line, improves the accuracy of point cloud registration, namely obtains a more accurate first transformation matrix, and further improves the accuracy of navigation in the catheter operation.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained based on the provided drawings without creative efforts.

Fig. 1 is a schematic view of a catheter robot provided in an embodiment of the present application;

FIG. 2 is a schematic view of a catheter instrument and a power section provided in accordance with an embodiment of the present application;

fig. 3 is a schematic flowchart of a registration method of a catheter robot according to an embodiment of the present application;

FIG. 4 is a schematic diagram of determining a target pipeline centerline segment corresponding to a simulated path point in an embodiment of the present application;

FIG. 5 is a diagram illustrating the determination of matching points in the case where the centroid is not a skeleton point on the target pipeline centerline segment in an embodiment of the present application;

FIG. 6 is a schematic illustration of a centroid as a skeletal point on a centerline segment of a target pipe in an embodiment of the present application;

FIG. 7 is a diagram illustrating the determination of matching points for the case where the centroid is a skeleton point on the target pipe centerline segment in an embodiment of the present application;

FIG. 8 is a schematic view of a detailed process of S17 in FIG. 3;

fig. 9 is a detailed flowchart of S18 in fig. 3;

FIG. 10 is a schematic flow chart of intraoperative navigation provided in accordance with an embodiment of the present application;

FIG. 11 is a schematic flow chart illustrating a process for segmenting a centerline of a pipeline according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an embodiment of the present application in which a pixel location is represented by a single number;

FIG. 13 is a detailed flowchart of S21 in FIG. 11;

fig. 14 is a detailed flowchart of S211 in fig. 12;

fig. 15 is a schematic structural diagram of a control system of a catheter robot according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application.

Detailed Description

Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

It is to be noted that, unless otherwise specified, technical terms or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which this application belongs.

A registration method of a catheter robot, and a computer-readable storage medium according to embodiments of the present application are described below with reference to the accompanying drawings.

Fig. 1 illustrates a catheter system 1000 provided by an embodiment of the present application. Catheter system 1000 includes a video cart 100, a cart 200 and a master controller 300 respectively connected to video cart 100, a catheter instrument 400 that can be coupled to cart 200, a sensor system 500 connected to cart 200, and a control system 600 for effecting control among catheter instrument 400, master controller 300, sensor system 500, and video cart 100. The main controller 300 may be connected to the trolley 200 by wire or wirelessly. When the operator performs various procedures on the patient beside the trolley 200, the operator can operate the main controller 300 to trigger control instructions, and the catheter instrument 400 is controlled to advance, retract, bend, turn and the like through the driving of the trolley 200.

The trolley 200 may be generally moved to the side of the operating bed for engaging the catheter instrument 400 and, under control commands, controlling the catheter instrument 400 to be raised and lowered in a vertical direction, or translated in a horizontal direction, or moved in non-vertical and non-horizontal directions, to provide a better pre-operative preparation angle for the operation of the catheter instrument 400. The control command may be a command triggered by an operator operating the master controller 300, or may be a command triggered by an operator directly clicking or pressing a key provided on the cart 200. Of course, in other embodiments, the control instructions may also be voice-controlled or triggered instructions through a force feedback mechanism.

As shown in fig. 1, further, the trolley 200 may include a base 210, a slide base 220 capable of performing lifting movement along the base 210, and 2 robot arms 230 fixedly connected to the slide base 220. The robotic arm 230 may include a plurality of arm segments coupled at joints that provide the robotic arm 230 with multiple degrees of freedom, for example, seven degrees of freedom corresponding to seven arm segments. The distal end of the mechanical arm 230 is provided with a power portion (not shown), and the power portion of the mechanical arm 230 is used for engaging the catheter instrument 400 and controlling the distal end of the catheter instrument 400 to correspondingly bend and turn under the driving action of the power portion. Wherein the 2 robotic arms 230 may be identical or partially identical in structure, one robotic arm 230 for engaging the inner catheter instrument 410 and another robotic arm 230 for engaging the outer catheter instrument 420. During installation, the outer catheter device 420 may be installed first, and after the outer catheter device 420 is installed, the catheter of the inner catheter device 410 is inserted into the catheter of the outer catheter device 420.

The sensor system 500 has one or more subsystems for receiving information about the catheter instrument 400. The subsystem may include: a position sensor system; a shape sensor system for determining the position, orientation, speed, velocity, pose, and/or shape of the tip of the catheter instrument 400 and/or one or more sections along a catheter that may comprise the catheter instrument 400; and/or a visualization system for capturing images from the tip of the catheter instrument 400.

The imaging cart 100 may be provided with a display system 110, a flushing system (not shown), and the like. The display system 110 is used to display images or representations of the surgical site and catheter instrument 400 generated by the subsystems of the sensor system 500. Real-time images of the surgical site and catheter instrument 400 captured by the visualization system may also be displayed. Images of the surgical site recorded preoperatively or intraoperatively may also be presented using image data from imaging techniques such as Computed Tomography (CT), magnetic Resonance Imaging (MRI), optical Coherence Tomography (OCT), and ultrasound, among others. The preoperative or intraoperative image data may be presented as two-dimensional, three-dimensional or four-dimensional (e.g., time-based or rate-based information) images and/or as images from a model created from a preoperative or intraoperative image dataset, and virtual navigation images may also be displayed. In the virtual navigation image, the actual position of the catheter instrument 400 is registered with the preoperative image to externally present the operator with a virtual image of the catheter instrument 400 within the surgical site.

The control system 600 includes at least one memory and at least one processor. It is understood that the control system 600 may be integrated into the trolley 200 or the video cart 100, or may be independent. The control system 600 may support wireless communication protocols such as IEEE 802.11, irDA, bluetooth, homeRF, DECT, and wireless telemetry, among others. The control system 600 may transmit one or more signals indicative of movement of the catheter instrument 400 by the motive portion to move the catheter instrument 400. The catheter instrument 400 may be extended to a surgical site in the body via an opening or surgical incision in the patient's natural orifice.

Further, the control system 600 may comprise a mechanical control system (not shown) for controlling the movement of the catheter instrument 400 and an image processing system (not shown) and may thus be integrated in the trolley 200. The image processing system is used for virtual navigation path planning and, therefore, may be integrated into the charter 100. Of course, the subsystems of the control system 600 are not limited to the specific cases listed above, and may be configured according to practical situations. Wherein the image processing system may image the surgical site using the imaging techniques described above based on images of the surgical site recorded preoperatively or intra-operatively. Software used in conjunction with manual input may also convert the recorded images into a two-dimensional or three-dimensional composite image of part or the entire anatomical organ or segment. During the virtual navigation procedure, the sensor system 500 may be used to calculate the position of the catheter instrument 400 relative to the patient's anatomy, which may be used to generate an external tracking image and an internal virtual image of the patient's anatomy, enabling registration of the actual position of the catheter instrument 400 with the pre-operative images, so that the operator may be externally presented with a virtual image of the catheter instrument 400 within the surgical site.

The inner catheter device 410 and the outer catheter device 420 have substantially the same structure and are respectively provided with an inner catheter 41 and an outer catheter 42, wherein the outer catheter 42 has a diameter slightly larger than that of the inner catheter 41, so that the inner catheter 41 can pass through the outer catheter 42 and provide a certain support for the inner catheter 41, and thus the inner catheter 41 can reach a target position in a patient body, so as to facilitate tissue or cell sampling and other operations from the target position.

Certain movements of the master control 300 may cause corresponding movements of the catheter instrument 400. For example, as an operator manipulates steering rod of master 300 to move up or down, the movement of steering rod of master 300 may be mapped to a corresponding pitch movement of the distal end of catheter instrument 400; when the operator operates the direction stick of master 300 to the left or right, the movement of the direction stick of master 300 may be mapped to a corresponding yaw movement of the tip of catheter instrument 400. In this embodiment, the master controller 300 may control the distal end of the catheter instrument 400 to move through a 360 ° spatial range.

Fig. 2 illustrates a catheter instrument 400 provided by an embodiment of the present application. The catheter instrument 400 is configured to engage with the motive portion 240 of the robotic arm 230, the catheter instrument 400 including an instrument box 45 configured to engage with the motive portion 240 and a catheter 48 connected to the instrument box 45. The term "engaged" refers to a state in which the driving force of the power unit 240 can be transmitted into the instrument case 45 and the catheter 48 can be normally moved when the instrument case 45 is attached to the power unit 240. For example, the tip of the guide tube 48 may be bent or turned by the driving force of the power unit 240.

The tip, which may also be referred to as the distal end or head in this application, refers to the end distal to the cartridge 45; the leading end, which may also be referred to as the proximal end or the tail, refers to the end near the instrument box 45.

The processor of the control system 600 is configured to perform the following steps to implement the registration method of the catheter robot provided by an embodiment of the present application. As shown in fig. 3, the method includes:

s11: actual waypoints of the catheter and/or instrument are acquired using the sensor.

The actual waypoints are the waypoints of the tip of the catheter and/or the instrument carried by the catheter acquired by the sensor and are generally described in terms of the coordinates of the tip of the catheter and/or the instrument in the world coordinate system, or in terms of the surgical environment coordinate system. The sensors may include position sensors and/or shape sensors.

The position sensor may be disposed on the catheter and/or the instrument. A position sensor may be a component of an electromagnetic positioning system. The electromagnetic positioning system may further comprise a magnetic field generating component and a magnetic field detecting component. The magnetic field generating component is used for generating a magnetic field, the position sensor can cause magnetic field change in the magnetic field, the magnetic field detecting component can detect the change of the magnetic field so as to detect the position and posture of the position sensor relative to the magnetic field/magnetic field generating component, the position of the catheter end/instrument end relative to the magnetic field/magnetic field generating component can be calculated by combining the preset or calibrated coordinate conversion relation between the position sensor and the catheter end/instrument end, and the position of the catheter end/instrument end in a world coordinate system, namely an actual path point, can be calculated by combining the coordinate conversion relation between the magnetic field/magnetic field generating component and the world coordinate system.

The position of the catheter tip/instrument tip can be acquired with a shape sensor system. For example, the shape sensor may comprise an optical fiber aligned with the catheter, and the optical fiber formed by the optical fiber may provide feedback on the shape of the catheter from which the position of the catheter tip/instrument tip relative to the base of the shape sensor may be calculated, and in combination with the position of the base of the shape sensor in the world coordinate system, the position of the catheter tip/instrument tip in the world coordinate system, i.e. the actual waypoint, may be calculated.

One actual path point reflects the position of the catheter tip/instrument tip when the sensor feeds back, and the actual path points obtained by multiple times of feedback of the same sensor are arranged according to the feedback time, so that the moving path of the catheter tip/instrument tip can be obtained. The collection of these actual path points may be referred to as a set of path points or a path point cloud.

S12: and acquiring simulated path points of the catheter and/or the instrument in the anatomical model, which correspond to the actual path points, according to the first transformation matrix.

A medical image (e.g., CT, MRI, etc.) may be obtained by scanning a target region of a patient before surgery, a three-dimensional preoperative image may be obtained by three-dimensionally reconstructing the preoperative image, and an anatomical model may be obtained by appropriately performing image processing on the three-dimensional preoperative image. The image processing can comprise image segmentation and skeleton extraction, wherein the image segmentation is used for extracting a human body pipeline involved in the operation from the three-dimensional image, and in order to facilitate navigation in the operation, the skeleton extraction is carried out on the human body pipeline to obtain a pipeline central line. The pipeline center line is the center line of the human body pipeline, can also be called as the skeleton of the human body pipeline, is a curve for describing partial geometric characteristics of the human body pipeline, is positioned in the middle of the human body pipeline, has the same topological structure as the human body pipeline, and generally has a single pixel width. The pipeline central line can provide information required by navigation in the operation, and compared with the original human body pipeline, the pipeline central line has the advantages of obviously reduced data volume, more convenient processing and real-time navigation.

The pipeline centerline is a three-dimensional curve, and in practical applications, the pipeline centerline is often stored and used in the form of a plurality of three-dimensional points, in other words, the pipeline centerline includes these three-dimensional points, these three-dimensional points may be referred to as skeleton points, and the set of skeleton points may be referred to as skeleton point set or skeleton point cloud.

The anatomical model is a three-dimensional binary image in which each pixel (which may also be referred to as a voxel) has a pixel value of 0 or 1, and is used to indicate whether the pixel is a skeleton point. In the embodiments of the present application, an example is described in which the pixel value of the skeleton point is 1 and the pixel value of the non-skeleton point is 0. In fact, the values may be reversed in other embodiments, which is not limited.

The value range of the pixel value is determined by the image bit depth supported by the display system, and the common value range is 0 to 255 or 0 to 1023. When displaying an anatomical model, in order to visually distinguish between skeletal points and non-skeletal points, it is common to map pixel values of the anatomical model, for example, the pixel values of the skeletal points are mapped to a maximum value (255 or 1023) and displayed in white, and the pixel values of the non-skeletal points are mapped to 0 and displayed in black. Of course the reverse could be true, i.e. skeletal points are shown in black and non-skeletal points in white.

To facilitate subsequent registration, the pipeline centerline may be preprocessed. The preprocessing may include a resampling that extracts feature skeleton points and/or the pipeline centerline from the set of skeleton points. The characteristic skeleton points generally include bifurcation and end points for segmenting the centerline of the pipe. Whether the skeleton point is a characteristic skeleton point can be judged by counting the number of the skeleton points in the neighborhood of a certain skeleton point. The application provides a feature skeleton point extraction method based on coding, which is specifically described in the following embodiments. The bifurcation point and the end point reflect the structural characteristics of the human body duct, and besides, for the segmentation with too long length/too large bending degree, new characteristic points can be introduced to further segment the segmentation so as to improve the registration accuracy.

In order to avoid destroying the connectivity of the pipeline center line, the resampling is generally an upsampling, and specific resampling parameters, such as resampling interval and interpolation number, can be determined according to actual needs. If the preprocessing comprises extracting the characteristic skeleton points and resampling, the sequence of the characteristic skeleton points and the resampling is not limited.

The first transformation matrix may be a transformation matrix between a world coordinate system and the anatomical model, and the actual path points may be transformed from the world coordinate system into a coordinate system of the anatomical model using the first transformation matrix, resulting in simulated path points.

The simulated path points reflect the position of the catheter end/instrument end in the anatomical model, and the position of the catheter end/instrument end in the displayed surgical site can be obtained by combining the display parameters of the surgical site, so that the surgical site and the catheter/instrument in the surgical site are displayed in a fusion mode, and intraoperative navigation is achieved. The surgical site may be displayed in the form of a preoperative image and/or an intra-operative image. If the displayed surgical site uses preoperative image data, since the anatomical model is obtained by processing a three-dimensional preoperative image, the position of the catheter tip/instrument tip in the surgical site can be obtained according to the coordinate transformation relationship of the displayed surgical site relative to the three-dimensional preoperative image. If the surgical site is displayed using intraoperative image data, the intraoperative image needs to be registered with the three-dimensional preoperative image/anatomical model to obtain a coordinate transformation relationship of the displayed surgical site with respect to the three-dimensional preoperative image, and then the position of the catheter tip/instrument tip in the surgical site.

To achieve intraoperative navigation, the present application computes a transformation matrix between the world coordinate system and the anatomical model, i.e., a first transformation matrix, by point cloud registration of a set of skeleton points and a set of path points.

The representative algorithm of point cloud registration is an Iterative Closest Point (ICP) algorithm, and the core idea is to minimize the distance between two point sets, so that the two point sets are close to each other through iteration. The principle of ICP is briefly described below.

The basic approach to ICP involves two steps: 1. matching the point clouds Q and P to find out corresponding point pairs between the point clouds Q and P; 2. and calculating a transformation matrix between the point clouds Q and P according to the corresponding point pairs. A corresponding pair of points consists of two points, one from point cloud P and the other from point cloud Q, which are matching points of each other and are considered to be corresponding, i.e. the two points are substantially identical.

If the true and accurate corresponding point pairs can be directly found out, the above process only needs to be carried out once, and the transformation matrix with sufficient accuracy can be directly calculated. However, in practical applications, it is difficult to directly find an accurate corresponding point pair, so that the ICP matches according to the principle of closest distance (generally, euclidean distance), that is, for each point in the point cloud P, a point closest to the point cloud Q is searched as a matching point. A transformation matrix is then calculated from the pairs of points. And after one round of calculation is finished, the ICP judges whether an iteration stop condition is met or not, if not, the conversion matrix obtained by the round of calculation is used for carrying out conversion updating on the point cloud P, and then the point cloud Q and the converted point cloud P are used for repeating the process until the iteration is stopped.

Because ICP is sensitive to initial values, if the initial difference between point clouds P and Q is large, convergence may not be guaranteed by directly using ICP. Therefore, point cloud registration can be divided into a coarse registration part and a fine registration part, an initial transformation matrix is obtained by adopting the coarse registration, and then fine registration is carried out by using ICP (inductively coupled plasma), and at the moment, point cloud Q and point cloud P transformed by using the initial transformation matrix are matched by the ICP in the first round.

In the process that the conduit moves along the human body pipeline, the radial direction of the pipeline often has a movable range, so that the conduit can deviate from the central line of the pipeline, the thicker the pipeline is, the larger the movable range is, and the influence of organ movement causes the shape of the moving path of the conduit and the central line of the framework to be possibly different. Therefore, for the registration of the path point set and the skeleton point set, the optimal matching point pair is difficult to accurately find by adopting the classical ICP, and the obtained registration result is often in local optimization. For this purpose, the present application proposes an improved ICP fine registration method, which is described in detail later. Before the ICP fine registration method provided by the present application is performed, coarse registration may be performed to obtain initial values of the first transformation matrix.

Since it is not necessary to match each actual waypoint during the procedure, for example, when ICP iteration is over, only virtual waypoints need to be obtained to achieve intra-operative navigation, and a subsequent registration step does not need to be performed. Therefore, after the step, whether the registration condition is met or not can be judged, if the registration condition is met, the subsequent step is executed, and if not, the subsequent step is not executed. Registration conditions may include ICP iteration in progress, initiating ICP iteration, etc.

S13: and determining a target pipeline centerline segment corresponding to the simulated path point from the plurality of pipeline centerline segments into which the pipeline centerline is divided.

The pipeline centerline may be divided into a plurality of pipeline centerline segments according to the characteristic skeleton points. Each pipeline central line segment has two end points, one of which can be set as a starting point, the other as an end point, and the specific one of which is the starting point and which is the end point can be set according to the anatomical characteristics, the motion path of the catheter robot, the actual requirements and the like. For example, if the body duct is an airway, since the airway has a multi-level branched tree structure, a branch point with a higher level (i.e. closer to the trachea) of the two end points is generally used as a starting point, and a branch point/end point with a lower level (i.e. closer to the alveoli) is used as an end point, which may be set in reverse according to the requirement.

Based on the characteristics of the motion of the catheter robot in the human body pipeline, the target pipeline center line segment can be determined firstly to improve the accuracy of subsequent matching, and the target pipeline center line segment is the pipeline center line segment where the estimated simulation path point is located.

First, the simulated waypoint is within the target pipe centerline segment, meaning that the simulated waypoint must be between the two endpoints of the target pipe centerline segment. Converted into a spatial relative positional relationship, and segmented with respect to the centerline of the pipeline

Only satisfied simulation waypoints

At the starting point

Is biased to the end point

While at the end point

Is deviated from the starting point

The condition of one side of (a) may be a target pipe centerline segment. From this, it can be calculated according to the following formula

And

：

(1)

(2)

only satisfy

And is

Is the simulation path point

And (4) performing primary screening on the pipeline centerline segment according to the possible segment (the possible segment for short). If not found to satisfy

And is

The pipeline central line is segmented, then the former simulation path point is

As a simulated waypoint

Is segmented by the target pipeline centerline.

If the number of the possible sections screened out preliminarily is 1, the possible sections are directly used as the center line sections of the target pipeline; if the number is greater than 1, further screening is required. In particular, the simulated path points may be calculated

Third distance between projected line segments to possible segmentation

And selecting the smallest third distance as the target pipeline centerline segment. For pipeline centerline segmentation

The projected line segment is the starting point

And end point

The straight line segment in between.

For example, referring to the drawings, as shown in FIG. 4, the dashed lines indicate the pipeline centerlines, and the a, b, c pipeline centerline segments satisfy the simulated path point A

And is provided with

Comparing the third distance between the projection line segments of the simulated path point A to a and b, and obtaining a target pipeline centerline segment corresponding to the simulated path point A as a.

If the simulated waypoint is close to the bifurcation of the body duct, there may be more than one minimum third distance, or there may be a third distance very close to the minimum third distance, and there may be some error in the above manner of determining the centerline segment of the target duct. For this purpose, the above determination may be supplemented.

Specifically, whether the possible segments preliminarily screened need to be screened again is judged, if so, candidate segments are obtained by screening again from the possible segments, the moving direction of the conduit is calculated, and one segment closest to the moving direction is selected from the candidate segments as a target pipeline center line segment; otherwise, directly selecting the smallest third distance as the target pipeline centerline segment.

When judging whether to need to filter again, there can be two ways based on the third distance and based on the position of the simulation path point. These two methods can be used independently, or can be used in a combined manner or a combined manner.

And if the difference between other third distances and the minimum third distance falls into the range except the minimum third distance, re-screening is required, and the pipeline centerline segment corresponding to the third distance with the difference falling into the range is a candidate segment. The difference here can be an absolute value, i.e. the difference itself, or a relative value, i.e. the ratio of the difference to the minimum third distance.

Based on the positions of the simulated path points, it can be determined whether an end point exists in the end points of the possible segments, which belongs to the bifurcation point and has a distance to the simulated path point smaller than a threshold value, and if so, all the segments to which the end point belongs are selected as candidate segments. The distance between the simulated path point and the end point may be the euclidean distance between the simulated path point and the end point, or the ratio of the euclidean distance to the length of the projection line segment to which the end point belongs, or the projection distance between the simulated path point and the end point (the euclidean distance between the foot of the simulated path point on the projection line segment and the end point), or the ratio of the projection distance to the length of the projection line segment to which the end point belongs, or the euclidean distance or the projection distance between the corresponding point of the simulated path point (i.e., the intersection point of the connecting line of the simulated path point to the centroid of the pipeline centerline segment and the pipeline centerline segment) and the end point, or the ratio of the distance to the length of the projection line segment to which the end point belongs.

The prior simulation path point can be calculated

Pointing to the current simulated path point

The vector of (b) is a moving direction of the catheter, and h is a positive integer, and can be set by experiments, experience, and the like. The direction of a candidate segment may be a vector pointing from its start point to its end point. And then calculating the included angle between the moving direction of the catheter and the candidate segment in a vector inner product mode, and selecting one with the smallest included angle as the central line segment of the target pipeline.

S14: and calculating the centroid of the central line segment of the target pipeline.

The centroid, i.e., the center of the shape, may be the average of the coordinates of all the skeleton points that make up the target pipe centerline segment. Since the target pipeline centerline segment is obtained by processing the medical image, the density of the segment can be considered uniform, and therefore the centroid of the target pipeline centerline segment is the centroid.

If the centroid is different from a plurality of skeleton points forming the target pipeline central line segment, jumping to S15; if the centroid is one of the skeleton points, then S16 is skipped.

S15: and determining a connecting line between the centroid and the simulated path point, and selecting one of the plurality of skeleton points with the shortest distance to the connecting line as a matching point of the actual path point.

Since the pipeline center line reflects the shape of the human body pipeline and is generally stored and used in a point set manner, the pipeline center line cannot be expressed by an analytical expression, an analytical solution of the intersection point cannot be obtained, and even if the pipeline center line can be obtained, the pipeline center line is not necessarily exactly the skeleton point. Therefore, in the present application, a skeleton point on the target pipeline centerline segment, which has the shortest distance to the connecting line, is approximately used as the intersection point between the target pipeline centerline segment and the connecting line.

In the application, the intersection point between the target pipeline central line segment and the connecting line is mainly used as the matching point, the registering mode is more in line with the motion mode of the catheter robot in the human body pipeline, and the found matching point is more accurate.

For example, as shown in FIG. 5, the dashed line in the figure represents the target pipeline centerline segment, s _c For the centroid of the target tract centerline segment, the solid line represents the path of the catheter robot in the anatomical model, simulating the path point t _n And the centroid s _c The intersection point of the connecting line and the target pipeline central line segment is approximate to a skeleton point q _k Skeleton point q _k I.e. the matching point of the actual path point corresponding to tn.

There is a special case: the centroid of a certain target pipeline central line segment is just one skeleton point on the target pipeline central line segment. As shown in FIG. 6, the dashed lines in the figure represent the target tract centerline segments, the solid lines represent the path of the catheter robot in the anatomical model, s _c Centroid of target pipe centerline segment and s _c For a skeleton point on the centerline segment of the target pipelineIn this case, no matter where the simulated waypoint is, the intersection of the connecting line and the target pipe centerline segment must be the centroid s _c That is, the matching points of all the path points corresponding to the target pipeline centerline segment are the centroids s _c This clearly violates the intention of the match. In this case, therefore, the intersection cannot be adopted as the matching point.

S16: acquiring a first foot from a simulation path point to a projection line, and acquiring a second foot from each framework point to the projection line, wherein the projection line is a straight line passing through two end points of a central line segment of a target pipeline; and selecting a skeleton point corresponding to a second drop foot with the shortest distance between the second drop foot and the first drop foot as a matching point of the actual path point.

And if the centroid of the target pipeline central line segment is just one skeleton point on the centroid, selecting a matching point according to the principle that the projection distance is closest. Specifically, a first foot from a simulation path point to a projection line is obtained, the projection line is a straight line passing through two end points of a central line segment of the target pipeline, and a second foot from each framework point to the projection line is obtained. And selecting a skeleton point corresponding to a second drop foot with the shortest distance between the first drop foot and the second drop foot as a matching point of the actual path point. After the segmentation is completed, the second feet and the fourth distance of each framework point are determined, and in order to shorten the calculation time, the fourth distances between the second feet of all framework points on the pipeline central line and the starting point/the end point of the pipeline central line segment to which the second feet belong can be calculated and stored in advance. When the projection distance is required to be used subsequently, the projection distance can be obtained only by taking the fourth distances of all framework points on the target pipeline center line segment, calculating the fifth distance between the first drop foot and the starting point/end point of the target pipeline center line segment and calculating the difference between the fourth distance and the fifth distance.

For example, as shown in FIG. 7, the solid lines in the figure represent the path of the catheter robot in the anatomical model, the dashed lines represent the target tract centerline segments, the dotted lines represent projection lines, and the simulated path points

The first foot to the projection line is

And the starting point

A fifth distance therebetween of

Skeleton point

To the projection line is

And the starting point

A fourth distance therebetween of

Simulating a waypoint

And skeleton point

A projected distance therebetween of

。

S17: and updating the point pair set by using a first point pair consisting of the actual path point and the matching point thereof.

Since the transform matrix generally has 6 degrees of freedom, a single point pair fails to find a unique solution, and therefore a set of point pairs is required to temporarily store a plurality of point pairs to calculate the transform matrix. After the matching is completed, the point pair set may be attempted to be updated with the resulting point pairs. One way of updating is to directly join the first point pair into the set of point pairs. However, due to the complexity of the motion of the catheter robot, it may happen that multiple path points are matched to the same skeleton point, which is not favorable for subsequent calculation. To solve this problem, the first point pair may be determined before adding the point pair set, as follows.

As shown in fig. 8, in some embodiments, the step specifically includes:

s171: and judging whether a second point pair comprising the matching points exists in the point pair set.

If not, jumping to S172; if so, the process jumps to S173.

S172: the first point pair is added to the point pair set.

S173: and calculating a first distance according to the actual path point and the matching point, calculating a second distance according to the path point and the matching point in the second point pair, and replacing the second point pair with the first point pair if the first distance is smaller than the second distance.

The first distance may be referred to as a distance of the first point pair and the second distance may be referred to as a distance of the second point pair. The first distance may be a current actual waypoint

Corresponding simulation path point

And the second distance may be a euclidean distance between the simulated path point corresponding to the path point in the second point pair and the matching point. Alternatively, the first distance may be the current actual path point

Corresponding simulation path point

And the second distance may be a projection distance between the simulated path point corresponding to the path point in the second point pair and the matching point.

If the first distance is greater than or equal to the second distance, the set of point pairs is not modified.

It may then be determined whether the updated set of point pairs satisfies a first condition, the first condition being a condition for updating the transformation matrix. The first condition may include a number of point pairs in the set of point pairs being greater than a preset value. Theoretically, only 3 point pairs are needed to calculate the unique solution of the transformation matrix, and in order to reduce the influence of errors, the optimal solution of the transformation matrix is estimated by a least square method through a plurality of point pairs in practical application. The preset value is the minimum value of the number of point pairs required for estimating the transformation matrix. The preset value must be greater than or equal to 3, and specific values can be set as required, such as 100, 200, and the like.

If the point pair set satisfies the first condition, it jumps to S18.

S18: the first transformation matrix is updated with the set of point pairs.

As shown in fig. 9, in some embodiments, this step specifically includes:

s181: and substituting the point pairs in the point pair set into the objective function to calculate the latest second transformation matrix.

An objective function may be pre-designed, which generally reflects the error of the matching point pairs, and then each point pair in the point pair set is substituted into the objective function, and the transformation matrix that can minimize the objective function is estimated as the latest second transformation matrix, or the second transformation matrix of the iteration.

An objective function commonly used for ICP can be used:

（3）

wherein N is the total number of the point pairs in the point pair set, i is the serial number of the point pairs in the point pair set, i =1,2, \ 8230, N,

for the coordinates of the matching points in the ith pair of points in the anatomical model,

and T is the coordinate of the actual path point in the ith point pair in a world coordinate system, and is a transformation matrix.

Because the human body duct is thick or thin, the greater the offset range of the path from the centerline in thicker ducts. In order to obtain a more accurate transformation matrix, the radius parameter of the human body pipeline can be introduced into an objective function, and the improved objective function is as follows:

（4）

wherein the content of the first and second substances,

and the average radius of the target pipeline center line segment n corresponding to the simulated path point corresponding to the actual path point in the ith point pair.

S182: the first transformation matrix is updated according to the latest second transformation matrix.

The latest second transformation matrix may be directly used as the updated first transformation matrix. Alternatively, to accommodate the case of dynamic local registration, a historical second transformation matrix may be introduced when computing the first transformation matrix. Optionally, the first transformation matrix is a weighted average of the latest second transformation matrix and at least part of the historical second transformation matrix, wherein the weight of the latest second transformation matrix is greater than that of the historical second transformation matrix.

For example, a specific calculation formula of the first transformation matrix is:

（5）

wherein, M represents the serial number of the completed latest iteration, M represents the serial number of the history iteration (the iteration before the latest iteration), and M =1,2, \ 8230;, and M-1.

After each iteration of ICP is complete, the actual path points are next transformed using the updated first transformation matrix.

Through the implementation of the embodiment, in order to adapt to a complex human body environment, the method determines the target pipeline center line segment corresponding to the simulation path point first to reduce errors that the matching point is not in the pipeline where the catheter is currently located, selects a skeleton point which is shortest from a connecting line between the simulation path point and the centroid of the target pipeline center line segment, can approximately consider the skeleton point as the intersection point of the connecting line and the target pipeline center line segment, takes the intersection point as the matching point of the actual path point, can more accurately reflect the matching relation between the path point of the catheter moving in the pipeline and the pipeline center line, improves the accuracy of point cloud registration, namely obtains a more accurate first transformation matrix, and accordingly improves the accuracy of navigation in the catheter operation.

The complete procedure of intraoperative navigation is illustrated below based on an anatomical model extracted from a three-dimensional lung CT image (obtained by three-dimensional reconstruction of a plurality of lung tomographic images) and an airway centerline therein, with reference to the drawings, wherein the same parts as those in the previous embodiments are not repeated.

The air passage comprises an air pipe and a bronchus, and the whole body is of a tree structure with step-by-step bifurcation, so the air passage can also be called as an air passage tree. The trachea is divided into left and right main bronchi at the end, the main bronchi are distributed deep into the left and right lungs, and are divided into lobar bronchi (usually three branches in the left lung and two branches in the right lung), and the lobar bronchi are divided into segmental bronchi and are gradually divided to the alveoli.

As shown in fig. 10, the intraoperative navigation process in an embodiment of the present application includes:

s101: an anatomical model is obtained that includes an airway centerline.

S102: the airway centerline is resampled.

S103: bifurcation and end points are extracted from a plurality of skeletal points included in the airway centerline.

S104: the airway centerline is divided into a plurality of segments using bifurcation and end points.

S105: the world coordinate system and the anatomical model are coarsely registered.

The coarse registration is typically performed by acquiring coordinates of a small number of distinct feature points (e.g., the main carina, the first, second-level carina of the left and right lobes, etc.) in the world coordinate system as well as the anatomical model.

S106: the actual waypoints of the catheter tip are acquired using the sensor.

S107: and performing coordinate transformation on the actual path points by using the first transformation matrix to obtain corresponding simulation path points.

S108: and judging whether the registration condition is met.

If yes, jumping to S109; otherwise, the process jumps to S106.

S109: a target segment corresponding to the simulated path point is determined.

S110: the centroid of the target segment is calculated.

S111: and judging whether the centroid is a skeleton point on the target segment.

If so, jumping to S112, otherwise, jumping to S113.

S112: acquiring a first foot from a simulation path point to a projection line, and acquiring a second foot from each framework point included in a target segment to the projection line, wherein the projection line is a straight line passing through two end points of the target segment; and selecting a skeleton point corresponding to a second drop foot with the shortest distance between the first drop foot and the second drop foot as a matching point of the actual path point.

Jumping to S114.

S113: and determining a connecting line between the centroid and the simulated path point, and selecting one of a plurality of skeleton points included in the target segment, which has the shortest distance with the connecting line, as a matching point of the actual path point.

S114: and judging whether a second point pair comprising the matching points exists in the point pair set.

If not, jumping to S115; otherwise, the process jumps to S116.

S115: and adding a first point pair consisting of the actual path point and the matching point thereof into the point pair set.

Jumping to S118.

S116: and calculating a first distance according to the actual path point and the matching point, calculating a second distance according to the path point and the matching point in the second point pair, and comparing the first distance with the second distance.

If the first distance is smaller than the second distance, jumping to S117; otherwise, the process jumps to S118. In some embodiments of the present invention, the,

s117: the second pair of points is replaced with the first pair of points.

S118: and judging whether the point pair set meets a first condition or not.

If yes, jumping to S119; otherwise, the process jumps to S106.

S119: and substituting the point pairs in the point pair set into the objective function to calculate the latest second transformation matrix.

S120: and calculating a weighted average of the latest second transformation matrix and at least part of the historical second transformation matrix as the updated first transformation matrix.

Jumping to S106.

The process of S106-S120 may be referred to as dynamic fine registration of the world coordinate system and the anatomical model.

As shown in fig. 11, in an embodiment of the present invention, the process of segmenting the center line of the pipeline specifically includes:

s21: characteristic skeleton points are extracted from a plurality of skeleton points constituting a pipeline centerline.

In the conventional three-dimensional image, the position of the pixel is represented by three-dimensional coordinates (x, y, z), 3 numbers are required to represent the position of the pixel, a large storage space is occupied, and the searching speed is also adversely affected due to the need of comparing the 3 numbers in the searching process. In order to reduce the space occupied by storing three-dimensional images such as anatomical models and to increase the query speed, the positions of skeleton points in this embodiment are represented by a single number of codes.

Specifically, the code s = x for each pixel (including the skeleton point) in the anatomical model ₁ +x ₂ R ₁ +x ₃ R ₁ R ₂ Wherein x is ₁ 、x ₂ 、x ₃ Respectively the coordinates of the pixel in the first, second and third dimension of the coding order, R ₁ 、R ₂ The sizes of the anatomical model in the first and second dimension of the coding order, respectively.

For example, referring to fig. 12, if three dimensions of the anatomical model are X, Y, and Z, and the coding order is X, Y, and Z, s = X ₁ +x ₂ R ₁ +x ₃ R ₁ R ₂ ，x ₁ Is the coordinate of the pixel on the X axis, X ₂ Is the coordinate of the pixel on the Y axis, x ₃ As the coordinate of the pixel on the Z-axis, R ₁ Is the size of the anatomical model on the X-axis, i.e. the total number of pixels, R ₂ Is the size of the anatomical model in the Y-axis.

In the related art, extracting feature points from a two-dimensional skeleton image generally includes screening candidate feature points by a table look-up method and extracting feature points from the candidate feature points. The principle of the table lookup method is as follows: the two-dimensional skeleton image is a binary image, and then a total of 2 is considered in an 8 neighborhood of one pixel, including its own 3 × 3 region ⁹ =512 possible arrangements, each representing a distribution of skeleton points, where it is known which arrangements represent that the pixel is an end point/intersection. A 3 x 3 convolution kernel is designed in which each position is set to a different value, for example, 1,2, 4, 8, 16, 32, 64, 128, 256, the value after convolution is a scalar, one-to-one correspondence with the arrangement of 3 x 3 regions centered on the pixel, and the convolution values corresponding to the end points and the intersections are calculated and stored as the feature point correspondence table. When the method is used, only the convolution kernel is needed to be used for performing convolution on the two-dimensional skeleton image, and then the convolution value in the characteristic point corresponding table is searched in the obtained convolution image, so that all candidate characteristic points can be found out at one time.

However, when a table lookup is applied to a three-dimensional skeleton image (e.g., an anatomical model in the present application), the class of possible arrangements is from 2 since the neighborhood extends from a two-dimensional 8 neighborhood to a 26 neighborhood, and the 3 x 3 region to a 3 x 3 region ⁹ Becomes 2 ²⁷ =134217728, the table generation and lookup will become very complicated, and the table lookup method is not suitable for extracting characteristic skeleton points from three-dimensional skeleton images.

As shown in fig. 13, in an embodiment of the present application, a method for extracting feature skeleton points based on a region growing method is provided, which specifically includes:

s211: a starting point is determined.

The starting point, also called a seed point, is a skeleton point that is the starting point for region growing. According to the structural characteristics of the airway tree, the starting point of the airway tree, namely the skeleton point of the trachea entrance, is generally selected as the starting point. Generally, a starting point needs to be manually selected in the related art, and a method for automatically selecting a starting point is provided in the application, which is as follows.

As shown in fig. 14, in an embodiment of the present application, the present step specifically includes:

s2111: a hierarchy of layers is randomly selected.

The three-dimensional CT image is reconstructed from a plurality of tomographic CT images, wherein each tomographic CT image corresponds to one tomographic layer. The anatomical model is obtained by image processing of a three-dimensional CT image, and accordingly, the anatomical model is composed of a plurality of body layers, and each body layer corresponds to a processing result of a tomographic CT image. One dimension of the anatomical model (typically z) is used to distinguish the different body layers.

S2112: the total number of skeleton points in a specified number of layers on both sides of the body layer in the direction perpendicular to the cross section is counted to determine the extending direction of the air passage.

Tomographic CT images are cross-sectional images imaged by projection reconstruction. According to the structural characteristics of the air passage, the position of the air inlet of the air passage is only provided with a small number of skeleton points (only one skeleton point is arranged on each layer of the air passage) at the beginning of the side with the extending direction of the air passage, then the number of the skeleton points is rapidly increased (including the part of the bronchus), and the side opposite to the extending direction of the air passage is not provided with the skeleton points or is provided with the small number of the skeleton points at the beginning and then is changed into the position without the skeleton points; the number of skeleton points near the end of the airway tree is significantly reduced, but the number of skeleton points in a single layer is generally greater than the trachea entrance position. From which the direction in which the airway extends can be determined.

S2113: and finding the skeleton points of the trachea entrance as a starting point according to the total number of the skeleton points in the body layer and the extending direction of the airway.

Starting from a randomly selected body layer, the trachea entrance is searched according to the extending direction of the airway until a skeleton point of the trachea entrance is found.

S212: and from the starting point, searching skeleton points by using a region growing method and counting the number of connected skeleton points.

The number of connected nodes is the number of all child nodes in the neighborhood of the skeleton point. In a traditional three-dimensional coordinate system, skeleton points and pointsThe relative coordinate relationship between the pixel points in the neighborhood is known, namely the three-dimensional coordinate of a certain framework point is known, the three-dimensional coordinates of all the pixel points in the neighborhood can be determined, and the relative coordinate relationship is substituted into a coding calculation formula s = x ₁ +x ₂ R ₁ +x ₃ R ₁ R ₂ And obtaining the relative coding relation between the skeleton point and the pixel points in the neighborhood.

And searching skeleton points in the neighborhood of the starting point from the starting point, wherein the searched skeleton points are child nodes of the starting point, and the starting point is a father node of the skeleton points. By analogy, for each skeleton point except the initial point, a father node is required in the neighborhood, the child nodes except the father node in the skeleton points found in the neighborhood are the child nodes of the skeleton point, and the number of the child nodes is counted to obtain the connection number of the skeleton point.

S213: and determining the characteristic skeleton points according to the connection number of the skeleton points and the father nodes thereof.

And if the number of connected nodes of the skeleton point is greater than 1 and the number of connected nodes of the father node is equal to 1, the skeleton point is a bifurcation point in the characteristic skeleton point. If the number of connected nodes of the skeleton point is equal to 1 and the number of connected nodes of the parent node is equal to 0, the skeleton point is the terminal point in the characteristic skeleton point.

S22: and dividing the pipeline center line into a plurality of pipeline center line segments according to the characteristic skeleton point.

Determining the father characteristic point of each characteristic skeleton point except the initial point, wherein two end points of each pipeline central line segment are one characteristic skeleton point and the father characteristic point thereof. The process of determining the parent feature point for a feature skeleton point includes: searching along the direction of the father node of the characteristic skeleton point, namely searching the father node of the characteristic skeleton point, then searching the father node of the father node, and so on, wherein the first found characteristic skeleton point is the father characteristic point of the characteristic skeleton point.

Since the pipeline centerline may have burrs, resulting in errors in the extracted feature skeleton points and the divided pipeline centerline segments, optionally, the following two steps are performed to filter the erroneous segments and feature skeleton points.

S23: and counting the number of skeleton points in the pipeline centerline segment.

S24: and deleting the pipeline central line segments of which the number is less than a preset threshold value.

A parameter for representing the level may be set for the feature skeleton point, where the level parameter of each feature skeleton point is the sum of the level parameter of its parent feature point and 1, and the smaller the level parameter is, the higher the level of the feature skeleton point is, the closer it is to the trachea. The preset threshold may be determined according to a level parameter of an end point of the central segment of the pipe. Generally, the larger the level parameter, the shorter the bronchial segment corresponding to the characteristic skeleton point, and correspondingly, the smaller the preset threshold value. Optionally, a level range is set, the level range generally excludes larger level parameters, and only pipeline centerline segments with endpoints within the level range are filtered.

If a pipeline central line segment is deleted, dividing the segment by a starting point s _i Other skeleton points except for the one with the smaller level parameter in the two endpoints are changed into non-skeleton points, and all pipeline centerline segments with starting points of the non-skeleton points are deleted. If after deletion s _i The number of pipe centerline segments as starting points equals 1, s will be _i Modifying the skeleton point into a common skeleton point, and if the skeleton point is deleted, s _i The number of pipe centerline segments as starting points is equal to 0, s is then _i Modified to an end point.

The embodiment of the application also provides a control system of the catheter robot. Referring to fig. 15, a schematic structural diagram of a control system of a catheter system according to an embodiment of the present application is shown. As shown in fig. 15, the control system 600 includes: a processor 60, a memory 61, a bus 62 and a communication interface 63, wherein the processor 60, the communication interface 63 and the memory 61 are connected through the bus 62; the memory 61 stores computer program instructions executable by the processor 60, and the processor 60 executes the computer program instructions to perform the registration method of the catheter robot provided in any one of the embodiments described above.

The Memory 61 may include a Random Access Memory (RAM) and a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the apparatus and at least one other network element is realized through at least one communication interface 63 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like may be used.

The bus 62 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. Wherein, the memory 61 is used for storing a program, and the processor 60 executes the program after receiving the execution instruction, and the registration method of the catheter robot disclosed in any embodiment of the foregoing application can be applied in the processor 60, or implemented by the processor 60.

The processor 60 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 60. The Processor 60 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; but may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 61, and the processor 60 reads the information in the memory 61 and, in combination with its hardware, performs the steps of the above method.

The control system of the catheter robot provided by the embodiment of the application and the registration method of the catheter robot provided by the embodiment of the application have the same inventive concept and have the same beneficial effects as the method adopted, operated or realized by the control system.

The embodiment of the present application further provides a computer readable storage medium corresponding to the registration method of the catheter robot provided in the foregoing embodiment, please refer to fig. 16, which shows a computer readable storage medium 6 having computer program instructions stored thereon, where the computer program instructions, when executed by a processor, implement the registration method of the catheter robot provided in any of the foregoing embodiments.

It should be noted that examples of the computer-readable storage medium may include, but are not limited to, optical disks, phase-change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory, or other optical and magnetic storage media, which are not described in detail herein.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the registration method of the catheter robot provided by the embodiments of the present application have the same beneficial effects as the method adopted, run or implemented by the application program stored in the computer-readable storage medium.

The embodiments of the present application provide a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the above method embodiments when executed.

It should be noted that:

in the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted to reflect the following schematic: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

It will be apparent to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional units and modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional units and modules as needed, that is, the internal structure of the apparatus may be divided into different functional units or modules to perform all or part of the above described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one type of logical function division, and other division manners may be available in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected based on actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated module/unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, read-Only Memory (ROM), random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is appropriately increased or decreased based on the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, the computer readable medium does not include electrical carrier signals and telecommunication signals based on legislation and patent practice.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a described condition or event is detected" may be interpreted, depending on the context, to mean "upon determining" or "in response to determining" or "upon detecting a described condition or event" or "in response to detecting a described condition or event".

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing a relative importance or importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically stated.

The above-mentioned embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (12)

1. A catheter robot comprising a robotic arm, a catheter instrument engaged with a motive portion of the robotic arm, a processor communicatively coupled to the robotic arm, the catheter instrument comprising an instrument cartridge configured to engage with the motive portion and a catheter coupled to the instrument cartridge, the catheter and/or an instrument carried by the catheter having a sensor disposed thereon for measuring a position of the catheter and/or the instrument, the processor configured to perform the steps of:

acquiring actual waypoints of the catheter and/or the instrument using the sensor;

acquiring simulated path points of the catheter and/or the instrument, which correspond to the actual path points, in an anatomical model according to a first transformation matrix, wherein the anatomical model comprises a pipeline central line;

determining a target pipeline centerline segment corresponding to the simulated path point from among a plurality of pipeline centerline segments into which the pipeline centerline is divided;

calculating the centroid of the target pipeline central line segment;

if the centroid is different from a plurality of skeleton points included in the target pipeline center line segment, determining a connecting line between the centroid and the simulated path point, and selecting one of the skeleton points with the shortest distance to the connecting line as a matching point of the actual path point.

2. The catheter robot of claim 1, wherein the processor is configured to perform the following steps after the calculating the centroid of the target tube centerline segment:

if the centroid is one of the plurality of skeleton points, acquiring a first foot from the simulation path point to a projection line, and acquiring a second foot from each skeleton point to the projection line, wherein the projection line is a straight line passing through two end points of the center line segment of the target pipeline; and selecting a skeleton point corresponding to a second drop foot with the shortest distance between the first drop foot and the second drop foot as a matching point of the actual path point.

3. The catheter robot of claim 2, wherein the processor is configured to perform the steps of:

updating the point pair set by utilizing a first point pair consisting of the actual path point and the matching point thereof;

if the point pair set meets a first condition, updating the first transformation matrix by using the point pair set, wherein the first condition comprises that the number of point pairs in the point pair set is greater than a preset value.

4. A catheter robot according to claim 3, wherein the processor is configured to perform the following steps in the matching point pair update point pair set consisting of the actual waypoints and their matching points:

judging whether a second point pair comprising the matching point exists in the point pair set or not;

if the first point pair does not exist, adding the first point pair into the point pair set, if the first point pair exists, calculating a first distance according to the actual path point and the matching point, calculating a second distance according to the path point in the second point pair and the matching point, and if the first distance is smaller than the second distance, replacing the second point pair with the first point pair.

5. The catheter robot of claim 3, wherein the processor is configured to perform the following steps in the updating the first transformation matrix using the set of point pairs:

substituting the point pairs in the point pair set into a target function to calculate a latest second transformation matrix;

updating the first transformation matrix according to the latest second transformation matrix.

6. The catheter robot of claim 5, wherein the first transformation matrix is a weighted average of the most recent second transformation matrix and at least a portion of a historical second transformation matrix, wherein the most recent second transformation matrix has a greater weight than the historical second transformation matrix.

7. The catheter robot of claim 1, wherein the processor is configured to perform the following steps prior to determining a target pipe centerline segment corresponding to the simulated path point among a plurality of pipe centerline segments into which the pipe centerline is divided:

extracting characteristic skeleton points from a plurality of skeleton points comprised by the pipeline centerline, wherein positions of the skeleton points are represented by codes, the codes of the skeleton points s = x ₁ +x ₂ R ₁ +x ₃ R ₁ R ₂ Wherein x is ₁ 、x ₂ 、x ₃ Respectively the coordinates, R, of said skeleton points in the first, second and third dimension of the coding sequence ₁ 、R ₂ The sizes of the anatomical model in the first dimension and the second dimension of the coding sequence are respectively;

and dividing the pipeline central line into a plurality of pipeline central line segments according to the characteristic skeleton points.

8. The catheter robot of claim 7, wherein the processor is configured to perform the following steps in the extracting feature skeleton points from a plurality of skeleton points of the pipeline centerline:

determining a starting point;

searching the skeleton points by using a region growing method from the initial point and counting the number of connected skeleton points, wherein the number of connected skeleton points is the number of all child nodes in the neighborhood of the skeleton points;

if the number of connected nodes of the skeleton point is greater than 1 and the number of connected nodes of a parent node of the skeleton point is equal to 1, the skeleton point is a bifurcation point in the characteristic skeleton point, and if the number of connected nodes of the skeleton point is equal to 1 and the number of connected nodes of the parent node of the skeleton point is equal to 0, the skeleton point is a tail end point in the characteristic skeleton point.

9. The catheter robot of claim 8, wherein the processor is configured to perform the following steps in the selected starting point:

randomly selecting an aggregate layer;

counting the total number of the skeleton points in a specified number of layers on two sides of the body layer in the direction perpendicular to the cross section so as to determine the extending direction of the air passage;

and searching a skeleton point of the trachea inlet as the starting point according to the total number of the skeleton points in the body layer and the extending direction of the airway.

10. The catheter robot of claim 7, wherein the processor is configured to perform the steps of:

counting the number of skeleton points in the pipeline center line segment;

and deleting the pipeline center line segments of which the number is smaller than a preset threshold value.

11. A method of registration of a catheter robot, comprising:

acquiring actual waypoints of a catheter and/or an instrument carried by the catheter using a sensor disposed on the catheter and/or the instrument;

acquiring a simulated path point of the catheter and/or the instrument in an anatomical model corresponding to the actual path point according to a first transformation matrix, wherein the anatomical model comprises a pipeline central line;

determining a target pipeline centerline segment corresponding to the simulated path point from a plurality of pipeline centerline segments into which the pipeline centerline is divided, wherein the target pipeline centerline segment comprises a plurality of skeleton points;

calculating the centroid of the target pipeline central line segment;

if the centroid is different from the skeleton points, determining a connecting line between the centroid and the simulated path point, and selecting one of the skeleton points with the shortest distance to serve as a matching point of the actual path point.

12. A computer readable storage medium storing computer program instructions configured to be loaded by a processor and to execute steps implementing the method of claim 11.

Images