CN115607275A

CN115607275A - Image display mode, device, storage medium and electronic equipment

Info

Publication number: CN115607275A
Application number: CN202211253083.6A
Authority: CN
Inventors: 楼珊珊; 虞跃洋
Original assignee: Neusoft Medical Systems Co Ltd
Current assignee: Neusoft Medical Systems Co Ltd
Priority date: 2022-10-13
Filing date: 2022-10-13
Publication date: 2023-01-17

Abstract

The application discloses an image display mode, an image display device, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring a real-time tomographic image of a target object; acquiring a predicted depth image corresponding to the real-time tomographic image based on a preset corresponding relation between the tomographic image and the depth image; acquiring a current depth image corresponding to a real-time endoscopic image acquired by endoscopic equipment; and acquiring and displaying a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image. According to the method and the device, the real-time tomographic image can be rapidly predicted by utilizing the preset corresponding relation between the tomographic image and the depth image, so that the corresponding predicted depth image can be rapidly obtained, the predicted depth image and the current depth image can be subsequently fused and matched in real time based on the predicted depth image, and the fusion and display of the two-dimensional image (endoscope image) and the three-dimensional position information (CT image) can be accurately realized in real time.

Description

Image display mode, device, storage medium and electronic equipment

Technical Field

The present invention relates to the field of medical image processing, and in particular, to an image display method, an image display apparatus, a storage medium, and an electronic device.

Background

The operation refers to the treatment of excision, suturing, etc. of the body of a patient by a doctor with medical instruments. The health of the patient is maintained by the operation of a knife, scissors, a needle and other instruments on the local part of the human body. Aims to cure or diagnose diseases, such as removing pathological tissues, repairing injuries, transplanting organs, improving the functions and the forms of organisms and the like. The conventional surgery is limited by manipulation techniques, visual fields of operation and other factors to a great extent, so that the success of the surgery cannot be ensured, and an imaging technology for guiding the surgery is required.

At present, the existing image-guided surgery navigation device and medical imaging equipment are independent devices in physical position and working time, so that the coordinate systems of the two devices are independent. Moreover, the working time difference between the two is large, so that the position of the focus may be changed due to the overlong time difference between the human body state during medical image scanning and the human body state during minimally invasive surgery. Moreover, the prior image-guided surgery device cannot transmit the coordinate and position information of the medical image of the human body to the surgery navigation device at the same time after the patient completes the medical image scanning, and the unified calibration of the coordinate system of the medical image of the patient and the coordinate system of the surgery navigation device must be completed again through an independent position sensor such as an optical position positioning device before the surgery, so that the position information of the medical image of the patient can be accurately transmitted to the surgery navigation device, which can prolong the preparation time of the surgery because the step of optical position calibration must be completed in advance when the patient uses the navigation device, increase the possibility of optical calibration failure, and simultaneously reduce the accuracy of the final position of the surgery navigation because the conversion times of the position coordinate information are increased.

Therefore, an image display method is needed to solve the problem that the image fusion and display cannot be performed accurately in real time and further the operation cannot be guided accurately in the prior art.

Disclosure of Invention

In view of the above, the present invention provides an image display method, an image display apparatus, a storage medium and an electronic device, and mainly aims to solve the problem that image fusion and display cannot be performed accurately in real time, and thus an operation cannot be guided accurately.

In order to solve the above problem, the present application provides an image display method, comprising:

acquiring a real-time tomographic image of a target object;

acquiring a predicted depth image corresponding to the real-time tomographic image based on a preset corresponding relation between the tomographic image and the depth image;

acquiring a current depth image corresponding to a real-time endoscopic image acquired by endoscopic equipment;

and acquiring and displaying a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image.

Optionally, before acquiring the real-time tomographic image of the target object, the method includes:

acquiring sample three-dimensional images of a plurality of sample objects;

acquiring a global depth map of each sample object based on each sample three-dimensional image and a virtual camera technology;

and establishing a corresponding relation between the tomographic image and the depth image based on each sample three-dimensional image and each global depth map.

Optionally, establishing a corresponding relationship between the tomographic image and the depth image based on each sample three-dimensional image and each global depth map includes:

acquiring a plurality of tomographic images and a plurality of depth images corresponding to the plurality of tomographic image positions based on each of the sample three-dimensional images and each of the global depth maps;

and taking the plurality of tomographic images as a training set, taking the plurality of depth images as a label set, training a relation model, and taking the trained relation model as the corresponding relation between the tomographic images and the depth images.

Optionally, obtaining a global depth map of each sample object based on each sample three-dimensional image and the virtual camera technology includes:

obtaining a central line of each sample object based on each sample three-dimensional image;

placing a virtual camera at the position of a central line and at a random position outside the central line for roaming to obtain a virtual endoscope image of each sample object;

and carrying out depth estimation on the virtual endoscope image to obtain each global depth map.

Optionally, acquiring a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image, includes:

performing global pixel level registration based on the predicted depth image and the current depth image to acquire deformation field information;

and fusing the real-time tomographic image and the real-time endoscopic image based on the deformation field information to obtain a real-time fused image.

Optionally, the image display method further includes:

obtaining a target three-dimensional image of the target object based on real-time tomographic image reconstruction;

acquiring position information of the real-time tomographic image on the target three-dimensional image based on the real-time tomographic image and the target three-dimensional image;

and marking and displaying the position of the real-time fusion image on the target three-dimensional image based on the position information.

Optionally, the image display method further includes:

acquiring real-time pose information of the endoscope or the surgical instrument based on the real-time tomographic image;

and displaying the real-time pose information on the target three-dimensional image.

In order to solve the above problem, the present application provides an image display device comprising:

the first acquisition module is used for acquiring a real-time tomographic image of the target object;

the prediction module is used for acquiring a prediction depth image corresponding to the real-time tomographic image based on the corresponding relation between a preset tomographic image and a depth image;

the second acquisition module is used for acquiring a current depth map corresponding to a real-time endoscopic image acquired by the endoscopic equipment;

and the display module is used for acquiring and displaying a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image.

In order to solve the above problem, the present application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the image display methods described above.

In order to solve the above problem, the present application provides an electronic device, which at least includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the image display modes when executing the computer program on the memory.

According to the image display mode, the image display device, the storage medium and the electronic equipment, the real-time tomographic image can be rapidly predicted by utilizing the preset corresponding relation between the tomographic image and the depth image, so that the corresponding predicted depth image can be rapidly obtained, the predicted depth image and the current depth image can be subsequently subjected to real-time fusion matching based on the predicted depth image, and fusion and display of a two-dimensional image (an endoscopic image) and three-dimensional position information (a CT image) can be achieved in real time, accurately and intuitively. Therefore, the method can assist a doctor to accurately plan the operation path and lay a foundation for realizing accurate guided operation.

The above description is only an overview of the technical solutions of the present invention, and the present invention can be implemented in accordance with the content of the description so as to make the technical means of the present invention more clearly understood, and the above and other objects, features, and advantages of the present invention will be more clearly understood.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:

FIG. 1 is a flowchart illustrating an image display method according to an embodiment of the present disclosure;

FIG. 2 is a schematic block diagram of an image display method according to an embodiment of the present disclosure;

FIG. 3 is a block diagram of an image display device according to another embodiment of the present application;

fig. 4 is a block diagram of an electronic device according to another embodiment of the present application.

Detailed Description

Various aspects and features of the present application are described herein with reference to the drawings.

It will be understood that various modifications may be made to the embodiments of the present application. Accordingly, the foregoing description should not be construed as limiting, but merely as exemplifications of embodiments. Those skilled in the art will envision other modifications within the scope and spirit of the application.

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and, together with a general description of the application given above and the detailed description of the embodiments given below, serve to explain the principles of the application.

These and other characteristics of the present application will become apparent from the following description of preferred forms of embodiment, given as non-limiting examples, with reference to the attached drawings.

It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the present application.

The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

Specific embodiments of the present application are described hereinafter with reference to the drawings; however, it is to be understood that the disclosed embodiments are merely exemplary of the application, which can be embodied in various forms. Well-known and/or repeated functions and constructions are not described in detail to avoid obscuring the application of unnecessary or unnecessary detail. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present application in virtually any appropriately detailed structure.

The description may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may each refer to one or more of the same or different embodiments in accordance with the application.

An embodiment of the present application provides an image display method, as shown in fig. 1, including the following steps:

step S101, acquiring a real-time tomographic image of a target object;

the target object in this step refers to an organ object or a site object, etc., and may be, for example, an organ, a site, a tissue, etc., where the region to be operated is located, such as a right coronary artery, an anterior descending branch, a circumflex branch, etc., in a coronary artery of the heart. Specifically, each organ and part of the human body can be divided according to actual needs, so that a plurality of organ objects or part objects are obtained.

Step S102, acquiring a predicted depth image corresponding to the real-time tomographic image based on a preset corresponding relation between the tomographic image and the depth image;

the preset corresponding relation between the tomographic image and the depth image in the step can be specifically a depth map prediction model, namely, the preset model is used for predicting the real-time tomographic image to obtain a predicted depth map;

step S103, acquiring a current depth image corresponding to a real-time endoscopic image acquired by endoscopic equipment;

in the specific implementation process of the step, the endoscopic equipment is provided with a camera which can be a common monocular camera endoscope; therefore, the camera can be used for shooting in real time in the operation process to obtain video images. And then processing the video image by adopting a preset image depth method, and obtaining the current depth image based on the depth map of the RGB image. The specific preset image depth method may be an image SFM depth method. In the implementation process, the depth image of the endoscopic equipment can be directly obtained by using the lens with RGB-D.

And step S103, acquiring a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image, and displaying the real-time fusion image.

In the step, after the image fusion is completed, a doctor can perform intuitive focus position judgment and operation path planning according to the fused image;

taking an operation guidance scene in which the CT device and the endoscopic device are applied cooperatively as an example, in a specific implementation process of this step, the current depth map and the predicted depth image may be specifically matched in a pixel-level global matching manner, so as to determine the position information in the global depth map corresponding to the current depth map. And then, the position information of the surgical instrument is determined through real-time CT scanning in the operation, so that a doctor can track the surgical oblique cutting position conveniently, and the doctor can plan, judge and the like the surgical path more accurately and intuitively according to the position information of the surgical instrument provided in real time and the fusion image of the endoscopic equipment and the CT image.

In the image display mode in this embodiment, the real-time tomographic image can be rapidly predicted by using the preset corresponding relationship between the tomographic image and the depth image, so that a corresponding predicted depth image can be rapidly obtained, and thus, the predicted depth image and the current depth image can be subsequently fused and matched in real time, and fusion and display of a two-dimensional image (endoscopic image) and three-dimensional position information (CT image) can be accurately and intuitively realized in real time. Thereby assisting the doctor in accurately planning the operation path. Real-time CT scanning may be used simultaneously as a tracking of the position of the surgical instrument to identify and plan deviations in the surgical path.

Another embodiment of the present application provides an image display method, including:

step S201, obtaining sample three-dimensional images of a plurality of sample objects

In the specific implementation process of the step, a sample CT tomographic image of a sample person, namely the sample CT image, can be obtained in advance. Then, each sample CT image is segmented to obtain a sub-sample CT image corresponding to each part object. Specifically, different organs and tissues are classified, and a specific organ or tissue is segmented from an image sequence. In the implementation process, the following CT image segmentation method can be adopted: region growing segmentation algorithms, threshold segmentation algorithms, deep learning based segmentation algorithms, and the like. After the plurality of sub-sample CT images are obtained, three-dimensional reconstruction processing can be performed on each sub-sample CT image to obtain a three-dimensional image of each sample. The three-dimensional reconstruction can be implemented by using an open source kit, commercial software of each company, and the like.

Step S202, acquiring a global depth map of each sample object based on each sample three-dimensional image and a virtual camera technology;

in the specific implementation process of the step, after the sample three-dimensional image is obtained, the depth map acquisition can be carried out on the basis of the sample three-dimensional image/three-dimensional model obtained by three-dimensional reconstruction. When acquiring the depth map, the following method may be specifically adopted: obtaining a central line of each sample object based on each sample three-dimensional image; placing a virtual camera at a central line position and a random position outside the central line for roaming to obtain a virtual endoscope image of each sample object; and carrying out depth estimation on the virtual endoscope image to obtain each global depth map.

Specifically, the virtual camera may be used to roam through the three-dimensional reconstruction structure, so as to obtain an internal scene image of the tissue or organ and corresponding depth information, i.e., obtain a tag depth image (global depth map). And further, a data set of the CT image and the RGB-D depth image can be constructed, and a foundation is laid for the subsequent training of the corresponding relation (depth map model) of the tomographic image and the depth image based on the data set. In this step, virtual camera roaming technology may be used to roam some critical regions of interest or focus along the centerline inside the organ. When depth information is obtained, the richness of the data set is ensured. This step may also take the form of a combination of placing the virtual camera at a centerline position and a random position. Therefore, the virtual image/label depth image of the organ can be obtained in multiple directions, multiple degrees of freedom and multiple visual angles.

Step S204, establishing a corresponding relation between a tomographic image and a depth image based on each sample three-dimensional image and each global depth map;

in a specific implementation process of the step, a plurality of tomographic images and a plurality of depth images corresponding to the plurality of tomographic images can be obtained based on each sample three-dimensional image and each global depth map; and then training a relation model by taking the plurality of tomograms as a training set and the plurality of depth images as a label set, wherein the trained relation model is taken as the corresponding relation between the tomograms and the depth images. That is, model training is performed on an initial encoder-decoder (coding and decoding network) model based on the tomogram and the depth image corresponding to the tomogram, so as to obtain a target encoder-decoder (coding and decoding network) model, so as to obtain a relational model, which may also be referred to as a target depth map prediction model.

In the specific implementation process of the step, the network structure of the model adopts an encoder-decoder network (a coding and decoding network) with jump connection. An encoder-decoder (codec network) with hopping connections is used to fuse features between layers, typically a U-Net network. Depth error loss and structural similarity loss. The model estimate is denoted by y' and the true depth value is denoted by y. The adopted Loss function is shown as the following formula: l1 is the root mean square error loss function of the depth map, and L2 is the structural similarity loss error. L is the overall loss function.

L＝L2+λL1

Step S205, acquiring a real-time tomographic image of a target object;

step S206, acquiring a prediction depth image corresponding to the real-time tomographic image based on a preset corresponding relation between the tomographic image and the depth image;

in the specific implementation process of this step, the real-time tomographic image may be three-dimensionally reconstructed to obtain the target three-dimensional image. The three-dimensional reconstruction can be implemented using an open source toolkit, and commercially available software of various companies. Then, the corresponding relation (namely a target depth map prediction model) is utilized to carry out depth map prediction on the three-dimensional image of the target object, and a prediction depth map of the target object is obtained;

in the step, when the depth map is obtained, the depth estimation is performed on the virtual-style image by performing regression on the pixels by adopting a depth neural network, so that the predicted depth map of the target object is obtained.

Step S207, carrying out global pixel level registration based on the predicted depth image and the current depth image to obtain deformation field information; based on the deformation field information, fusing the real-time tomographic image and the real-time endoscopic image to obtain a real-time fused image, and displaying the real-time fused image;

in the step, during surgery, by means of the real-time image fed back by the endoscope device, the real-time image of the endoscopic device and the real-time CT image in the surgery are fused together through the fusion technology, the fused image (or a single image, such as an endoscope and a CT image respectively) is displayed on the display device, and a doctor can easily find out the corresponding surgery point and the corresponding surgery path from the fused image in the surgery; in the operation, the spatial position information of the surgical instrument can be obtained through real-time CT scanning, so that the position error between the surgical instrument and a focus can be accurately judged, and the operation is guided.

In a specific implementation process, acquiring position information of the real-time tomographic image on the target three-dimensional image based on the real-time tomographic image and the target three-dimensional image; and marking and displaying the position of the real-time fusion image on the target three-dimensional image based on the position information.

Specifically, the display mode in this embodiment may further acquire real-time pose information of the endoscope or the surgical instrument based on the real-time tomographic image; and displaying the real-time pose information on the target three-dimensional image.

In this embodiment, after obtaining the fusion image of the current depth map and the predicted depth map, a step of determining current position information of the surgical instrument based on the CT scan image obtained in real time, or obtaining pose information of the endoscope or the surgical instrument based on a positioning unit or other positioning modules of the endoscope or the surgical instrument, and then determining a target surgical path based on the current position information and a target position of a lesion in the fusion image, so as to determine an offset of the planned path. Specifically, the target surgical path and the offset of the planned path can be prompted. The deviation of the position and the planned path is prompted, for example, in a predetermined prompting manner, to guide the procedure. For example, the deviation of the planned path may be displayed by a display device, or the deviation of the planned path may be prompted by voice in a voice broadcast manner.

The process of the image display mode in this embodiment may specifically be as shown in fig. 2. The embodiment performs real-time fusion by using the video image and the CT image acquired by the common camera of the endoscopic equipment. The problem of inaccurate guide position caused by real-time operation guide only through a common camera image is solved. In addition, the method in the embodiment can also display the position of the focus by utilizing rich content information in the CT image. The invention can overcome the deformation, improve the guiding precision of the operation and improve the real-time and accuracy of the operation.

Another embodiment of the present application provides an image display device, and as shown in fig. 3, an image display device 1 in the present embodiment includes:

a first acquisition module 11, configured to acquire a real-time tomographic image of a target object;

the prediction module 12 is configured to obtain a predicted depth image corresponding to the real-time tomographic image based on a preset correspondence between the tomographic image and the depth image;

the second acquisition module 13 is configured to acquire a current depth map corresponding to a real-time endoscopic image acquired by an endoscopic device;

and the display module 14 is configured to acquire a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image, and display the real-time fusion image.

In a specific implementation process of this embodiment, the image display apparatus further includes an establishing module, where the establishing module includes: a first acquisition subunit configured to acquire a sample three-dimensional image of a plurality of sample objects before acquiring a real-time tomographic image of a target object; the second acquisition subunit is used for acquiring a global depth map of each sample object based on each sample three-dimensional image and a virtual camera technology; and the establishing subunit is used for establishing the corresponding relation between the tomographic image and the depth image on the basis of each sample three-dimensional image and each global depth map.

In a specific implementation process of this embodiment, the establishing subunit is specifically configured to: acquiring a plurality of tomographic images and a plurality of depth images corresponding to the plurality of tomographic image positions based on each of the sample three-dimensional images and each of the global depth maps; and taking the plurality of tomographic images as a training set, taking the plurality of depth images as a label set, training a relation model, and taking the trained relation model as the corresponding relation between the tomographic images and the depth images.

In a specific implementation process of this embodiment, the second obtaining subunit is specifically configured to: obtaining a central line of each sample object based on each sample three-dimensional image; placing a virtual camera at a central line position and a random position outside the central line for roaming to obtain a virtual endoscope image of each sample object; and carrying out depth estimation on the virtual endoscope image to obtain each global depth map.

In a specific implementation process of this embodiment, the pre-display module includes a registration subunit and a fusion subunit; the registration subunit is configured to perform global pixel-level registration based on the predicted depth image and the current depth image, and acquire deformation field information; and the fusion subunit is used for fusing the real-time tomographic image and the real-time endoscopic image based on the deformation field information to obtain a real-time fusion image.

In a specific implementation process of this embodiment, the image display device further includes a position information obtaining module, configured to obtain a target three-dimensional image of a target object based on real-time tomographic image reconstruction; acquiring position information of the real-time tomographic image on the target three-dimensional image based on the real-time tomographic image and the target three-dimensional image; and marking and displaying the position of the real-time fusion image on the target three-dimensional image based on the position information.

In a specific implementation process of this embodiment, the image display device further includes a pose information acquiring module, configured to acquire real-time pose information of the endoscope or the surgical instrument based on the real-time tomographic image; and displaying the real-time pose information on the target three-dimensional image.

In the specific implementation process, the real-time tomographic image can be rapidly predicted by using the preset corresponding relationship between the tomographic image and the depth image, so that the corresponding predicted depth image can be rapidly obtained, and then the predicted depth image and the current depth image can be fused and matched in real time, and the fusion and display of the two-dimensional image (endoscope image) and the three-dimensional position information (CT image) can be achieved in real time, accurately and intuitively. Therefore, the method can assist a doctor to accurately plan the operation path and lay a foundation for realizing accurate guided operation.

Another embodiment of the present application provides a system for guiding a procedure, comprising:

the CT scanning system is used for acquiring preoperative and intraoperative CT images. The data of preoperative scanning can be reconstructed, stored and displayed by a CT image-building machine and image processing software. The CT scanning system comprises a main control console, a camera and a graphic workstation. The system can perform organ or tissue segmentation and reconstruction on the CT image sequence to obtain a three-dimensional RGB image of the CT image and central line information of a scanning position.

And the positioning module is used for acquiring pose information of surgical instruments and endoscopic equipment during CT scanning in an operation and transmitting the information to the data processing module.

The endoscopic equipment is used for acquiring a video image in the surgical human body in real time based on the camera and then sending the video image to the data processing module; and processing the video image through the data processing model to obtain a depth map of the RGB image, namely obtaining the current depth map. The depth map/current depth map of the endoscopic device may also be obtained directly using a lens with RGB-D.

The data processing module is used for carrying out depth map prediction on the intraoperative CT scanning image based on the depth map prediction model to obtain a prediction depth map (global depth map); the system is also used for carrying out process processing on the video image sent by the endoscopic equipment based on an image SFM depth algorithm to obtain a current depth map, or directly receiving the current depth map sent by the endoscopic equipment; the depth image prediction method is also used for carrying out pixel level global matching on the current depth image and the predicted depth image to obtain deformation field information; and fusing the real-time tomographic image and the real-time endoscopic image based on the deformation field information to obtain a real-time fused image.

The display module is used for displaying the intraoperative CT image and the image obtained by fusing and matching the current depth map and the prediction depth map; and is used to display three-dimensional information of surgical instruments and human bodies, etc.

The embodiment performs real-time fusion on the video image and the CT image acquired by the common camera of the endoscopic equipment. The problem of inaccurate guide position caused by real-time operation guide only through a common camera image is solved. In addition, the method in the embodiment can also display the focus position by utilizing rich content information in the CT image. The invention can overcome the deformation, improve the guiding precision of the operation and improve the real-time and accuracy of the operation.

Another embodiment of the present application provides a storage medium storing a computer program which, when executed by a processor, performs the method steps of:

step one, acquiring a real-time sectional image of a target object;

and fourthly, acquiring a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image, and displaying the real-time fusion image.

The specific implementation process of the above method steps can refer to any embodiment of the image display method, and this embodiment is not repeated herein.

The storage medium in the application can rapidly predict the real-time tomographic image by utilizing the corresponding relation between the preset tomographic image and the depth image, so that the corresponding predicted depth image can be rapidly obtained, the predicted depth image and the current depth image can be fused and matched in real time subsequently, and fusion and display of a two-dimensional image (endoscopic image) and three-dimensional position information (CT image) can be achieved in real time, accurately and intuitively. Therefore, the method can assist a doctor to accurately plan the operation path and lay a foundation for realizing accurate guided operation.

Another embodiment of the present application provides an electronic device, as shown in fig. 4, at least including a memory 1 and a processor 2, where the memory 1 stores a computer program thereon, and the processor 2, when executing the computer program on the memory 1, implements the following method steps:

step one, acquiring a real-time sectional image of a target object;

The specific implementation process of the above method steps can refer to any embodiment of the image display method, and details are not repeated here.

The electronic equipment can rapidly predict the real-time tomographic image by utilizing the preset corresponding relation between the tomographic image and the depth image, so that the corresponding predicted depth image can be rapidly obtained, the predicted depth image and the current depth image can be subsequently fused and matched in real time based on the predicted depth image, and the fusion and display of the two-dimensional image (endoscope image) and the three-dimensional position information (CT image) can be achieved in real time, accurately and intuitively. Therefore, the method can assist doctors in accurately planning the surgical path and lay a foundation for realizing accurate guided surgery.

The above embodiments are only exemplary embodiments of the present application, and are not intended to limit the present application, and the protection scope of the present application is defined by the claims. Various modifications and equivalents may be made by those skilled in the art within the spirit and scope of the present application and such modifications and equivalents should also be considered to be within the scope of the present application.

Claims

1. An image display method, comprising:

acquiring a real-time tomographic image of a target object;

2. An image display method according to claim 1, wherein before acquiring the real-time tomographic image of the target object, the method comprises:

acquiring sample three-dimensional images of a plurality of sample objects;

3. The image display method according to claim 2, wherein establishing a correspondence relationship between the tomographic image and the depth image based on each of the sample three-dimensional images and each of the global depth maps includes:

4. The method of claim 2, wherein obtaining a global depth map of each sample object based on each of the sample three-dimensional images and a virtual camera technique comprises:

placing a virtual camera at a central line position and a random position outside the central line for roaming to obtain a virtual endoscope image of each sample object;

5. The image display method according to claim 1, wherein acquiring a real-time fusion image of the real-time tomographic image and the real-time endoscopic image based on the predicted depth image and the current depth image includes:

6. The image display method according to claim 1, further comprising:

7. The image display method according to claim 6, further comprising:

acquiring real-time pose information of the endoscope or surgical instrument based on the real-time tomographic image;

8. An image display apparatus comprising:

9. A storage medium storing a computer program which, when executed by a processor, implements the image display method steps of any one of claims 1 to 7.

10. An electronic device, comprising at least a memory having a computer program stored thereon, and a processor, wherein the processor implements the steps of the image display method according to any one of claims 1 to 7 when executing the computer program stored in the memory.