WO2015044433A1

WO2015044433A1 - Merging vessel maps

Info

Publication number: WO2015044433A1
Application number: PCT/EP2014/070832
Authority: WO
Inventors: Fransciscus Joannes Leonardus Everaerts; Peter Maria Johannes Rongen; Vincent Maurice André AUVRAY; Raoul Florent
Original assignee: Koninklijke Philips N.V.
Priority date: 2013-09-27
Filing date: 2014-09-29
Publication date: 2015-04-02

Abstract

An examination apparatus (12) for merging vessel maps comprising means for: receiving a first 2D image series (40a), comprising X-ray images of a vessel system of an object of interest; receiving a second 2D image series (40b), comprising X-ray images of the vessel system of the object of interest; extracting first vessel map data (42a) of the vessel system from the first 2D image series (40a); extracting second vessel map data (42b) of the vessel system from the second 2D image series (40b); temporally matching the second vessel map data (42b) to the first 2D image series (40a) by analyzing artifacts in the first and second 2D image series (40a, 40b); and generating a merged 2D image series (44) by projecting the first vessel map data (42a) and the matched second vessel map data (42b) into the first 2D image series (40a).

Description

Merging vessel maps

FIELD OF THE INVENTION

The invention relates to an examination apparatus, a method, a computer program and a computer-readable medium for merging vessel maps. BACKGROUND OF THE INVENTION

In certain applications, such as chronic total occlusion (CTO) PCI

(percutaneous coronary intervention), it is required to review two image series of vessels acquired with an X-ray imaging device and "virtually" merge them. That is, a physician needs to visualize a merged image in his mind. For example, two angiograms involving contrast agent injections firstly in the left coronary artery (LCA) and then in the right coronary artery (RCA), or vice versa, need to be merged.

In contrary to general PCI where the physician often prepares just prior to the procedure, a CTO specialist may prepare a case well ahead. For the so-called retrograde technique it may be important to know if the patient has suited collateral arteries for conducting such a procedure. To decide this, usually a dual injection angiogram is needed. However, in most cases, a patient with a CTO is transferred to a CTO specialist without a dual injection angiogram available. Therefore, the CTO specialist has to rely on two angiograms obtained after one injection of contrast agent in the LCA and one injection in the RCA. He mentally merges the two image series to obtain information needed to prepare for the case.

In US 2012/0020462 Al volume data of the left and right coronary arteries is registered in relation to time and space with the aid of a gating signal provided by an electrocardiogram. SUMMARY OF THE INVENTION

There may be a need to facilitate the work of a CTO PCI specialist.

This need is met by the subject-matter of the independent claims. Further exemplary embodiments are evident from the dependent claims and the following description. A first aspect of the invention relates to a method for merging vessel maps. A vessel map may comprise 2D (two-dimensional) image data or may comprise a two- dimensional graph indicating the presence of a vessel in a two-dimensional image.

According to an embodiment of the invention, the method comprises the steps of: receiving a first 2D image series, comprising X-ray images of a vessel system of an object of interest; receiving a second 2D image series, comprising X-ray images of the vessel system of the object of interest; extracting first vessel map data of the vessel system from the first 2D image series; extracting second vessel map data of the vessel system from the second 2D image series; temporally matching the second vessel map data to the first 2D image series by analyzing artifacts in the first and second 2D image series; and generating a merged 2D image series by projecting the first vessel map data and the matched second vessel map data into the first 2D image series.

In other words, two sets, series and/or sequences of 2D X-ray images may be received in an examination apparatus, which, for example, may be acquired at different times, different places and/or different X-ray devices. For example, a 2D image series may be a film/movie of the object of interest such as an angiogram. For example, the first and second vessel systems may be the left coronary arteries (LCA) and the right coronary arteries of a heart of a patient (the object of interest) and the two image series may be angiograms. With the method, the LCA and the RCA angiograms may be merged and displayed to a physician.

For each image series, vessel map data is extracted by the examination apparatus from the respective image series. It may be possible that the vessels in the two image series are made visible by a contrast agent in these vessels. The two image series and/or the extracted vessel map data are temporally matched with each other by the examination apparatus, which analyzes artifacts, such as vessels and/or metal objects, in the image series. This analysis may be performed solely based on the image data encoded into the 2D image series. Temporally matching may mean that the temporally relationships of the images and/or the vessel map data of one series with the other is determined. In the end, a merged 2D image series is generated by the examination apparatus by, for example, projecting the vessel map data from one of the image series into the other image series. The two image series may be matched and registered both temporally (i.e. synchronization through cardiac cycle analysis) and spatially (for example using image anchors / landmarks or other image content). The merged image series may be displayed on a screen or display of the examination apparatus. The merged image series may include an overlay of the matched angiograms and/or the vessel maps extracted therefrom.

The method may further comprise advanced image processing, used in image stitching and blending of the two image series. The image processing may include a targeted segmentation process which may take into account the transparent nature of the 2D X-ray images. The segmented objects or vessels of interest may be superimposed on other anatomical layers, a dominant part of which is constituted by the background. These other anatomical layers or structures may move independently from the objects of interest.

Artifacts need to be taken into account for the segmentation task, , wherein the pixel data may be analyzed and classified to allow an accurate separation of background and object information.

Further, background subtraction or object recognition may be performed during image processing.

The targeted segmentation task as a further step of the method may be performed during or preceding to or subsequent to the step of extracting first vessel map data of the vessel system from the first 2D image series and extracting second vessel map data of the vessel system from the second 2D image series.

When extracting the first vessel map data and/or second vessel map data, artifacts on the first and second 2D image series may be recognized and may be assigned to objects and/or assigned to the background. Generally, the term "artifacts" refers to any device or object that is identifiable by the examination apparatus and that appears in both 2D image series. For example, artifacts may be anatomical features such as a collateral vessel or diaphragm border, or external objects such as a catheter, a sternal wire, or a metal object.

Further, a motion parameter describing a movement of at least one of the artifact objects during the heart phase and/or to a breathing phase may be calculated. The first and/or second vessel map data may then be updated according to the calculated motion parameter of this object. For instance, the artifact may be an object with a prominent back and forth movement during the cardiac and/or breathing cycle.

For each identified artifact object, the calculated motion parameter may provide an improved merging of the two series of images. For instance, if the two series of images were recorded at different phases of the cardiac and/or of the breathing cycle, the motion parameter may allow to extrapolate the motion of the identified object during the time difference of the two image series, and compensate for movements in the corresponding vessel map accordingly.

The first and second 2D image series may be filtered in order to comprise a binary or quasi-binary appearance based on the background-objects or background-vessel or background-object-of-interest segmentation. This may advantageously improve the non- binary appearance of the 2D image series, for instance of non-binary 2D angiograms.

The two image series may be matched and registered temporally with respect to a heart phase and/or to a breathing phase at which each one of the two image series was recorded. This advantageously allows synchronizing the two images series which were recorded at different time points through the cardiac and/or breathing cycle. According to an exemplary embodiment of the present invention, at first a temporal synchronization is performed, which cancels out the difference in heart phases and, secondly, a spatial registration, for instance by motion compensation, is performed to cancel out the difference in breathing phase.

A further aspect of the invention relates to an examination apparatus for merging vessel maps, which is adapted for performing the method as described in the above and in the following. For example, the examination apparatus may be a computer connected via a data communication connection to one or more X-ray imaging devices providing the X- ray image series. The examination apparatus also may be a controller of an X-ray imaging device, which is controlled for one or both of the X-ray image series.

It has to be understood that features of the method as described in the above and in the following may be features of the examination apparatus as described in the above and in the following.

A further aspect of the invention relates to an X-ray imaging system comprising such an examination apparatus.

A further aspect of the invention relates to a computer program element, which, when being executed by a processing unit, is adapted to carry out the steps of the method as described in the above and in the following.

A further aspect of the invention relates to a computer-readable medium in which such a computer program element is stored. A (non-volatile) computer-readable medium may be a floppy disk, a hard disk, an USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory) or a FLASH memory. A computer-readable medium may also be a data communication network, e.g. the Internet, which allows downloading a program code.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Below, embodiments of the present invention are described in more detail with reference to the attached drawings.

FIG. 1 shows an X-ray imaging system with an integrated examination apparatus according to an embodiment the invention.

FIG. 2 shows a flow diagram for a method for merging vessel maps according to an embodiment the invention.

FIG. 3 shows a flow diagram for a method for merging vessel maps according to an embodiment the invention.

FIG. 4 shows an X-ray image of LCA to be used in a method for merging vessel maps according to an embodiment the invention.

FIG. 5 shows an X-ray image of RCA to be used in a method for merging vessel maps according to an embodiment the invention.

FIG. 6 shows a merged image generated by a method for merging vessel maps according to an embodiment the invention.

In principle, identical parts are provided with the same reference symbols in the figures.

DETAILED DESCRIPTION OF EMBODIMENTS FIG. 1 schematically shows an X-ray imaging system 10 with an examination apparatus 12 for mapping vessel maps. The X-ray imaging system 10 comprises an X-ray image acquisition device (C-arm device) 14 with a source of X-ray radiation 16. A table 18 is provided to receive an object of interest to be examined. Further, an X-ray image detection device 20 is located opposite the source of X-ray radiation 16, i.e. during the radiation procedure the object is located between the source of X-ray radiation 16 and the detection device 20.

The movement of the C-arm device 14, the source of radiation 16, the table 18 and the detection device 20 are controlled by a controller 22 of the examination apparatus 12, which receives (raw) 2D X-ray image data from the detection device 20 for further processing. The examination apparatus 12 furthermore comprises a display 24 to display information to a user (such as a physician) operating the X-ray imaging system 10, which can be a clinician such as a cardiologist or cardiac surgeon. For example, the processed images may be displayed on the display 24. The examination apparatus 12 may also comprise a user interface 26 (such as a keyboard) to input information by the user.

Basically, the image detection device 20 acquires (raw) 2D X-ray images by exposing the object of interest to X-ray radiation generated by the source of radiation 16. By acquiring a sequence of X-ray images, also a film/movie of the object of interest, such as a beating heart, may be generated.

It is noted that the example shown in FIG. 1 is a so-called C-type X-ray image acquisition device 10. The X-ray image acquisition device 14 comprises an arm 28 in form of a C where the image detection device 20 is arranged at one end of the C-arm 28 and the source of X-ray radiation 16 is located at the opposite end of the C-arm 28. The C-arm 28 is moveably mounted and can be rotated around the object of interest located on the table 18. In other words, it is possible to acquire X-ray images with different directions of view.

FIG. 2 shows a flow diagram for a method for merging vessel maps that may be performed by the X-ray imaging system 10 of FIG. 1. For example, the controller 22 may comprise a processing unit (like a microprocessor) that executes software that controls the movement of the C-arm 28, the acquisition of the X-ray images with the detector 20 and the processing of the acquired images as described in the following.

In particular, FIG. 2 relates to a method for merging vessel map data 42a, 42b from two angiograms 40a, 40b into an overlay image 44. An example of an image of a LCA (left coronary arteries) angiogram is shown in FIG. 4 and an example of an image of a RCA (right coronary arteries) angiogram is shown in FIG. 5. An overlay image 44 that may be seen as a virtual dual injection image 44 is shown in FIG. 6. It has to be understood that the angiograms 42 a, 42b as well as the merged virtual angiogram 44 are sequences or series of images, i.e. are movies or films.

The two angiograms 40a, 40b may be obtained after injection of contrast agent in the LCA as well as the RCA. The two angiograms 40a, 40b may be matching in the sense that the images are acquired from the same patient (object of interest) and/or at the same projection angle and/or other equivalent system settings such as SID (source-image-distance, magnification), FOV (field of view) and/or fps (frames per second). However, as explained in the following, automated correction for these parameters may be performed as well. The two angiograms 40a, 40b may be acquired by the X-ray imaging device 12. After that in first processing steps 46a and 46b, vessel map data 42a, 42b is extracted from the two angiograms 40a, 40b and in second processing steps 48a and 48b, a cardiac cycle analysis may be performed based on the two angiograms 40a, 40b. Furthermore, preprocessed overlay images 50a, 50b may be generated from the vessel map data 42a, 42b. In step 52, the two sets of overlay images 50a, 50b are temporally matched and registered (or merged) with each other based on the cardiac cycle analysis of steps 48a, 48b. In step 54, the matched and merged overlay images 50a, 50b are blended into one of the angiograms 40b, for generating the virtual angiogram 44.

In general, the examination apparatus 12 may merge two vessel systems of an object of interest, i.e. may process first and second 2D image series 40a, 40b as indicated in FIG. 3. As indicated in FIG. 4 to 6, the 2D image series 40a, 40b may be angiograms and/or the two 2D image series 40a, 40b are image series of left and right coronary arteries 80a, 80b with injected contrast agent. As another example, the object of interest may be a brain and the two vessel systems may be interconnected vessels in the brain.

In general, each of the two image series 40a, 40b may comprise X-ray images, which comprise pixel data, that for example may be directly generated by the detection device 16 (i.e. raw image data).

According to an embodiment, the two 2D image series 40a, 40b may be acquired by the same X-ray imaging device 14. For example, the X-ray imaging system 10 may generate the two 2D image series immediately after each other and also may generate the merged 2D image series with the examination apparatus 12.

According to an embodiment, the two 2D image series 40a, 40b may be acquired by different X-ray imaging devices and/or at different times. For example, one of the 2D image series 40a may be acquired at a different hospital/doctor's office and may have been sent to the examination apparatus 12, where it is merged with a 2D image series acquired with the X-ray imaging devices 14.

In general (either if the two image series 40a, 40b are acquired locally or remotely), the two image series 40a, 40b are received in the examination apparatus 12 for further processing. In the case, one image series 40a, 40b is acquired locally, raw data obtained from the image detection device 20 directly may be used, since this may provide the highest possible image quality.

However, it is also possible to use image series 40a, 40b that are stored in a database and/or in standard formats such as the DICOM format. In step 46a, the examination apparatus 12 extracts first vessel map data 42a of the vessel system from the first 2D image series 40a, and in step 46b, the examination apparatus 12 extracts second vessel map data 42b of the vessel system from the second 2D image series 40b. For example, the first and/or second vessel map data 42a, 42b may be extracted by ridge filtering, thresholding and/or background subtraction. It has to be understood that the first and/or second vessel map data 42a, 42b may comprise 2D image data, i.e. pixel data that may be projected directly into the image series 40a, 40b.

In step 52a, the examination apparatus 12 temporally matches both image series 40a, 40b with each other by analyzing artifacts in both image series 40a, 40b. In this context, temporally matching may mean that a timeline is generated, which indicates the temporal allocation of the images of the first and second 2D image series 40a, 40b with respect to each other. In other words, both image series 40a, 40b may be synchronized. It has to be noted that also a cyclic temporal matching may be possible (for example with a cardiac cycle analysis).

With the temporally matching of the image series 40a, 40b, also the second vessel map data 42b (which may be have a vessel map for each image of the image series 40b) may be temporally matched to the first 2D image series 40a.

Artifacts may be all kind of objects in the image series that are visible in both image series 40a, 40b and that are identifiable by the examination apparatus 12. For example, an artifact used for matching the two image series 40a, 40b may be a collateral vessel, a catheter, a wire, a metal object. For example, in case of a true CTO, it may be expected that collateral arteries are visible on both angiograms and these may be used as anchoring points for the matching.

At the end of step 52a, temporal matching data may have been generated, which indicates the temporal correspondence of the images of the two image series 40a, 40b. For example, the temporal matching data may comprise a table with time points that are associated with the images of the two image series 40a, 40b.

When the two image series have not been acquired at exactly the same projection angle and/or field of view, the examination apparatus 12 also may spatially match the two image series 40a, 40b with each other. This also may be done based on the identified artifact(s) that also have been used for temporally matching.

Spatial matching may mean generating (temporally dependent) matching data, which indicates the spatial allocation of the images of the first and second 2D image series 40a, 40b with respect to each other. In particular with the spatially matching of the two image series 40a, 40b, also the second vessel map data 42b (which may have the same projection angle and the same field of view as the second image series 40b) may be spatially matched to the first 2D image series 40a.

At the end of step 52b, spatial matching data may have been generated, which indicates the spatial correspondence of image pairs of the two image series 40a, 40b. For example, the spatial matching data may comprise translations, scalings and/or rotations that map an image from the one image series 40a to an image of the other image series 40b.

In step 54, at least one of the two series 40a, 40b and the two sets of vessel map data 42a, 42b are merged based on the temporal and/or spatial matching determined in steps 52a, 52b. For example, a vessel map 42a from an image series may be projected into its image series 40a and the other vessel map 42b may be projected into the same image series 40a in a spatially and temporally transformed way based on the matching data determined in steps 52a and 52b. A merged 2D image series 44 may be generated by projecting the first vessel map data 42a and the matched second vessel map data 42b into the first 2D image series 40a.

It may be possible that the two image series 40a, 40b correspond to each other in the sense that they have been acquired and/or generated with the same parameters/formats that may be of importance for the image quality (for example, the same projection angle, same FOV, SID, fps, patient position, heart rate and breath rate).

It may also be possible that the two image series 40a, 40b have different formats. In this case, the examination apparatus may compensate for changes of some of those parameters by image processing (fps, heart rate, breath rate, FOV, SID, slight patient position change). For example, at least the second vessel map data 42b may be converted to the same format based on the temporally and/or spatially matching.

It has to be noted that the merged 2D image series 44 may be generated simultaneously to an acquisition of the first or second 2D image series 40a, 40b. For example, the first image series 40a may be acquired before a CTO PCI and the second image series 40b may be acquired during the CTO PCI. Simultaneously, the merged image series 44 may be generated by the examination apparatus 12.

There are different possibilities, how the two sets of vessel set data 42a, 42b are merged with one or both of the image series 40a, 40b:

In one embodiment, the two sets of vessel map data 42a, 42b are merged and thereafter blended (overlaid) on the first or the second image series 40a, 40b. In another embodiment the two sets of vessel map data 42a, 42b are used for calculating the optimal position of the two vessel maps needed for blending. Thereafter, by using this information, the vessel map data 42b made out of image series 40b is blended (overlaid) with image series 40a or vice versa.

In an example, a virtual dual injection angiogram 44 may be constructed based on two single injection angiograms 40a, 40b. With computer software running in the examination apparatus 12, vessel map data 42a, 42b may be extracted from the angiograms 40a, 40b and may be merged/projected onto a matching angiogram. The software may create vessel map data 42a, 42b using the image data of both angiograms 40a, 40b, may match the frames/images i) temporally, for example using cardiac cycle analysis, and ii) may match the frames/images spatially using a motion compensation method, and finally may project/merge these on one of the two angiograms 40a, 40b.

The method as described in the above may be used in the preparing phase of a CTO procedure. It may help a physician in the decision making process for the treatment options (typically antegrade, retrograde or dissection/re-entry). However, it may also be used for more general diagnostics as well as interventional PCI; vascular cases and for neuro applications.

While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art and practising the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or controller or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. An examination apparatus (12) for merging vessel maps, the examination apparatus (12) comprising means for:

receiving a first 2D image series (40a), comprising X-ray images of a first vessel system of an object of interest;

receiving a second 2D image series (40b), comprising X-ray images of a second vessel system of the object of interest;

extracting first vessel map data (42a) of the first vessel system from the first 2D image series (40a);

extracting second vessel map data (42b) of the second vessel system from the second 2D image series (40b);

temporally matching the second vessel map data (42b) to the first 2D image series (40a) by analyzing artifacts in the first and second 2D image series (40a, 40b);

generating a merged 2D image series (44) by projecting the first vessel map data (42a) and the matched second vessel map data (42b) into the first 2D image series (40a).

2. An X-ray imaging system (10), comprising:

an X-ray imaging device (14) for acquiring at least one of the first 2D image series (40a) and second 2D image series (40b);

an examination apparatus (12) according to claim 1.

3. A method for merging vessel maps, the method comprising:

extracting second vessel map data (42b) of the second vessel system from the second 2D image series (40b); temporally matching the second vessel map data (42b) to the first 2D image series (40a) by analyzing artifacts in the first and second 2D image series (40a, 40b);

4. The method of claim 3, further comprising:

spatially matching the second vessel map data (42b) to the first 2D image series (40a) by analyzing the artifacts in the first and second 2D image series;

5. The method of claim 3 or 4, further comprising:

temporally and/or spatially matching the first and second 2D image series (40a, 40b) by analyzing artifacts in the first and second 2D image series (40a, 40b);

temporally and/or spatially matching the second vessel map data (42b) based on the matching of the first and second 2D image series (40a, 40b).

6. The method of one of claims 3 to 5,

wherein the first 2D image series (40a) and the second 2D image series (40b) have different formats, the method further comprising:

converting the second vessel map data (42b) to the same format based on the temporally and/or spatially matching.

7. The method of one of claims 3 to 6,

wherein the artifacts comprises at least one of: a collateral vessel, a catheter, a wire, a metal object.

8. The method of one of claims 3 to 7,

wherein the first vessel system is a left coronary artery (LCA) and the second vessel system is a right coronary artery (RCA).

9. The method of one of claims 3 to 8,

wherein at least one artifact in the first and second 2D image series is identified by image processing and at least one motion parameter for at least one of the identified artifacts is determined.

10. The method of claim 9,

further comprising the step of extrapolating a motion of the at least one identified artifact for the generating of the merged 2D image series (44).

11. The method of one of claims 3 to 10,

wherein the merged 2D image series (44) is generated simultaneously to an acquisition of the first or second 2D image series (40a, 40b).

12. The method of one of claims 3 to 11,

wherein the first and/or second vessel map data (42a, 42b) comprises 2D image data; and/or

wherein the first and/or second vessel map data is extracted by at least one of: ridge filtering, thresholding, background subtraction.

13. The method of one of claims 3 to 12,

wherein the first and second 2D image series (40a, 40b) are angiograms; and/or

wherein the first and/or second 2D image series (40a, 40b) are image series of the left and/or right coronary arteries with injected contrast agent.

14. A computer program element, which, when being executed by a processing unit, is adapted to carry out the steps of the method of one of claims 3 to 13.

15. A computer-readable medium in which a computer program according claim 14 is stored.