CN113693724A - Irradiation method, device and storage medium suitable for fluorescence image navigation operation - Google Patents
Irradiation method, device and storage medium suitable for fluorescence image navigation operation Download PDFInfo
- Publication number
- CN113693724A CN113693724A CN202110954082.3A CN202110954082A CN113693724A CN 113693724 A CN113693724 A CN 113693724A CN 202110954082 A CN202110954082 A CN 202110954082A CN 113693724 A CN113693724 A CN 113693724A
- Authority
- CN
- China
- Prior art keywords
- fluorescence
- image
- illumination
- test
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000002073 fluorescence micrograph Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012360 testing method Methods 0.000 claims abstract description 118
- 238000005286 illumination Methods 0.000 claims abstract description 92
- 239000002131 composite material Substances 0.000 claims abstract description 19
- 238000012545 processing Methods 0.000 claims description 21
- 230000004927 fusion Effects 0.000 claims description 16
- 238000002675 image-guided surgery Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000004020 luminiscence type Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 206010028980 Neoplasm Diseases 0.000 abstract description 11
- 230000003902 lesion Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 230000005284 excitation Effects 0.000 description 7
- 201000011510 cancer Diseases 0.000 description 6
- 238000004891 communication Methods 0.000 description 3
- 239000003814 drug Substances 0.000 description 3
- 238000000799 fluorescence microscopy Methods 0.000 description 3
- 238000002271 resection Methods 0.000 description 3
- 238000001356 surgical procedure Methods 0.000 description 3
- 238000002512 chemotherapy Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000001678 irradiating effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 208000007660 Residual Neoplasm Diseases 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000007850 fluorescent dye Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0071—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B2090/364—Correlation of different images or relation of image positions in respect to the body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
- A61B2090/373—Surgical systems with images on a monitor during operation using light, e.g. by using optical scanners
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Animal Behavior & Ethology (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Pathology (AREA)
- Robotics (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- Gynecology & Obstetrics (AREA)
- Radiology & Medical Imaging (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The invention provides an irradiation method, an irradiation device and a storage medium suitable for a fluorescence image navigation operation, wherein the irradiation method, the irradiation device and the storage medium comprise the following steps: presetting a test interval time period, and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period; acquiring a plurality of test images obtained by different irradiation equipment in a test interval time period, wherein the plurality of test images comprise an illumination test image and a plurality of fluorescence test images; determining the fluorescence area of a fluorescence area in each fluorescence test image, and determining first irradiation duration of different irradiation equipment based on the fluorescence area; different irradiation equipment irradiates the focus based on different first irradiation duration to obtain an illumination working image and a plurality of first-stage fluorescence images, and the illumination working image and the plurality of first-stage fluorescence images are fused to obtain a composite image. The irradiation time of each light path can be controlled according to the actual tumor, focus and tissue conditions, so that the fluorescence area of the focus can be acquired more accurately.
Description
Technical Field
The present invention relates to image processing technologies, and in particular, to an illumination method, an illumination device, and a storage medium suitable for fluoroscopic image navigation surgery.
Background
The most therapeutically effective methods of cancer treatment are surgical resection and chemotherapy. The number of cancer cells left after surgical resection is an index for determining the success or failure of surgical resection: the extent of residual cancer cells and their complete elimination greatly affect the later stages of the disease, and if too many cancer cells remain in the organ after surgery, the subsequent chemotherapy effect and the survival probability of the patient may be directly reduced.
For cancer tissues or tumor boundaries of small size, it is difficult for the physician to give an accurate judgment from the naked eye and from the experience of touch. In addition, in the focal region of a patient, cancer tissues are staggered with normal tissues such as blood vessels, nerves and the like, so that the tissues cannot be distinguished by naked eyes.
Therefore, a device capable of providing a doctor with a real-time image of a focal region and accurately distinguishing cancer tissues from normal tissues becomes a hot spot of research.
Granted patent 1, chinese patent application with granted publication number CN105982645B, discloses a fluorescence imaging apparatus and a method for photographing a focus area, wherein the fluorescence imaging apparatus comprises: a camera adapted to acquire images of the focal zone; an illumination source adapted to provide illumination light to illuminate the focal zone; the time-sharing multiplexing excitation light source is suitable for providing at least two excitation light sources with different wave bands, the excitation light is suitable for exciting the target drugs corresponding to the focal region to fluoresce, the excitation light sources with different wave bands share the light path of the time-sharing multiplexing excitation light source, and the time sequences of the excitation lights with different wave bands for exciting the corresponding target drugs are different.
The granted patent 1 can multiplex the light paths of the excitation light sources, effectively reducing the volume of the fluorescence imaging instrument, but in the imaging process, the irradiation time of each light path cannot be controlled according to the actual conditions of tumors, lesions and tissues, so that the tumors, lesions and tissues in the generated composite image have inaccurate conditions.
Disclosure of Invention
The embodiment of the invention provides an irradiation method, an irradiation device and a storage medium suitable for a fluorescence image navigation operation, which can control the irradiation time of each light path according to the actual conditions of tumors, focuses and tissues, so that the fluorescence region of the focus is more accurately collected.
In a first aspect of the embodiments of the present invention, an irradiation method suitable for a fluoroscopic image guided surgery is provided, including:
presetting a test interval time period, and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period;
acquiring a plurality of test images obtained by different illumination devices in a test interval time period, wherein the plurality of test images comprise an illumination test image and a plurality of fluorescence test images;
determining the fluorescence area of a fluorescence area in each fluorescence test image, and determining first irradiation duration of different irradiation equipment based on the fluorescence area;
different irradiation equipment irradiates the focus based on different first irradiation duration to obtain an illumination working image and a plurality of first-stage fluorescence images, and the illumination working image and the plurality of first-stage fluorescence images are fused to obtain a composite image.
Optionally, in one possible implementation manner of the first aspect, determining a fluorescence area of a fluorescence region in each fluorescence test image, and determining the first illumination duration of different illumination apparatuses based on the fluorescence area includes:
acquiring the number x of pixel points in the illumination test image and the number k of luminous pixel points in the y fluorescent test imagey1；
Calculating the first irradiation duration t of the irradiation equipment corresponding to the y-th fluorescence test image by the following formulay1，
Wherein k isiThe number of luminous pixel points in the ith fluorescence test image, c the number of all fluorescence test images, p the conversion coefficient, tmAs a reference irradiation time period, tαIs the minimum irradiation duration.
Optionally, in a possible implementation manner of the first aspect, the method further includes:
setting a secondary acquisition time point, and acquiring a plurality of first-stage fluorescence images of the secondary acquisition time point closest to the moment;
acquiring the number k of luminous pixel points in the y-th first-stage fluorescence image at the time closest to the secondary acquisition time pointy2；
Based on the number k of luminous pixel points in the y-th fluorescence test imagey1And the number k of luminous pixel points in the first-stage fluorescence imagey2And calculating a second irradiation duration corresponding to the acquisition of the second-stage fluorescence image.
Optionally, in a possible implementation manner of the first aspect, if k is determinedy1Is not equal to ky2；
Then, the second illumination time t of the illumination device corresponding to the y-th fluorescence test image is calculated by the following formulay2，
Wherein A is a constant greater than 1, and s is an attenuation conversion coefficient.
Optionally, in one possible implementation manner of the first aspect, the determining the fluorescence area of the fluorescence area in each fluorescence test image includes:
presetting a light emitting threshold value of each fluorescence test image, and carrying out binarization processing on each fluorescence test image based on the light emitting threshold value;
and extracting the area of the fluorescence area in the fluorescence test image after the binarization processing.
Optionally, in a possible implementation manner of the first aspect, the extracting the area of the fluorescence region in the fluorescence test image after the binarization processing includes:
displaying the pixel points within the luminescence threshold value in a form of 1 in the fluorescence test image, and displaying the pixel points outside the luminescence threshold value in a form of 0 in the fluorescence test image;
and acquiring the area of the pixel point which is 1 in all the fluorescence test images.
Optionally, in one possible implementation manner of the first aspect, the fusing the illumination working image and the plurality of first-stage fluorescence images to obtain a composite image includes:
registering the illumination working image and the plurality of first-stage fluorescent images so as to enable the coordinates of each pixel point in the illumination working image and the plurality of first-stage fluorescent images to correspond to each other;
respectively extracting regions of pixel points which are 1 in the first-stage fluorescence image, fusing the regions of the pixel points which are 1 in different first fluorescence images with the illumination working image, and displaying the fused regions in the illumination working image with preset pixel values, wherein the preset pixel values corresponding to the regions of the pixel points which are 1 in different first fluorescence images are different;
and outputting and displaying the generated composite image after judging that all the first-stage fluorescence images are respectively fused with the illumination working image.
Optionally, in a possible implementation manner of the first aspect, the method further includes:
judging whether a repeated fusion area exists in the composite image, wherein the repeated fusion area is an area which is respectively fused with different first-stage fluorescent images in the illumination working image;
and performing label display on the repeated fusion area, wherein the label display at least comprises the information of each fused first-stage fluorescence image.
In a second aspect of the embodiments of the present invention, there is provided an irradiation device suitable for fluoroscopic image guided surgery, including:
the device comprises a setting module, a control module and a control module, wherein the setting module is used for presetting a test interval time period and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period;
the system comprises an obtaining module, a processing module and a control module, wherein the obtaining module is used for obtaining a plurality of test images obtained by different irradiation equipment in a test interval time period, and the plurality of test images comprise an illumination test image and a plurality of fluorescence test images;
the determination module is used for determining the fluorescence area of the fluorescence area in each fluorescence test image and determining first illumination duration of different illumination devices based on the fluorescence area;
and the fusion module is used for enabling different irradiation equipment to irradiate the focus based on different first irradiation durations to obtain an illumination working image and a plurality of first-stage fluorescence images, and fusing the illumination working image and the plurality of first-stage fluorescence images to obtain a composite image.
In a third aspect of the embodiments of the present invention, a readable storage medium is provided, in which a computer program is stored, which, when being executed by a processor, is adapted to carry out the method according to the first aspect of the present invention and various possible designs of the first aspect of the present invention.
The irradiation method, the irradiation device and the storage medium suitable for the fluorescence image navigation operation provided by the invention can be used for testing an actual working scene before synthesizing an image, obtaining the first irradiation duration of irradiation equipment of a corresponding type according to the fluorescence area of a fluorescence area in fluorescence test images of different types and types, respectively controlling the irradiation equipment of the corresponding type to irradiate through a plurality of first irradiation durations, and respectively collecting the fluorescence areas of different types to obtain a first-stage fluorescence image.
According to the invention, when the first irradiation time lengths of different irradiation devices are calculated, the number of the pixel points in the illumination test image and the number of the pixel points in the other fluorescence test images are fully considered, and the first irradiation time length of each irradiation device is determined according to the number relation. The first irradiation duration of each irradiation device is different under different scenes, and the first irradiation duration is longer when the fluorescence area is larger and the fluorescence area is larger, so that the matching degree of the fluorescence area and the actual form of human tissue in the obtained composite image is higher.
According to the method, when the area of the pixel point is determined and the synthetic image is generated, binarization processing is carried out on the first-stage fluorescence image according to the light emitting threshold values of different fluorescence test images, so that the image processing amount and the calculated amount are reduced when the area of the pixel point is calculated and the synthetic image is generated, the problem of inaccurate identification of a fluorescence region caused by the light emitting attenuation of the targeted medicine can be solved by setting the light emitting threshold value, the accuracy of determining the area of the pixel point and generating the synthetic image is improved, and the image data processing amount is reduced.
Drawings
FIG. 1 is a flow chart of an illumination method suitable for fluoroscopic image guided surgery;
FIG. 2 is a schematic diagram of an irradiation apparatus suitable for fluoroscopic image guided surgery.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.
It should be understood that, in various embodiments of the present invention, the sequence numbers of the processes do not mean the execution sequence, and the execution sequence of the processes should be determined by the functions and the internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
It should be understood that in the present application, "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that, in the present invention, "a plurality" means two or more. "and/or" is merely an association describing an associated object, meaning that three relationships may exist, for example, and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "comprises A, B and C" and "comprises A, B, C" means that all three of A, B, C comprise, "comprises A, B or C" means that one of A, B, C comprises, "comprises A, B and/or C" means that any 1 or any 2 or 3 of A, B, C comprises.
It should be understood that in the present invention, "B corresponding to a", "a corresponds to B", or "B corresponds to a" means that B is associated with a, and B can be determined from a. Determining B from a does not mean determining B from a alone, but may be determined from a and/or other information. And the matching of A and B means that the similarity of A and B is greater than or equal to a preset threshold value.
As used herein, "if" may be interpreted as "at … …" or "when … …" or "in response to a determination" or "in response to a detection", depending on the context.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The invention provides an irradiation method suitable for a fluorescence image navigation operation, as shown in a flow chart of fig. 1, comprising the following steps:
step S110, presetting a test interval time period, and controlling different irradiation devices to irradiate the focus in sequence based on the test interval time period. The test interval time period may be 15 ms, 20 ms, 50 ms, or the like, and may be selected according to the form of the lesion, where the test interval time period may be longer if the form of the lesion is large, and may be smaller if the form of the lesion is small, and may reflect the irradiation time of each irradiation device on the lesion.
Step S120, obtaining a plurality of test images obtained by different illumination devices in a test interval time period, where the plurality of test images include an illumination test image and a plurality of fluorescence test images. The fluoroscopic image navigation apparatus includes a plurality of irradiation devices, for example, an illumination device for irradiating visible light, and a radiation irradiation device for irradiating light rays of different wavelengths.
Step S130, determining the fluorescence area of the fluorescence area in each fluorescence test image, and determining the first irradiation duration of different irradiation devices based on the fluorescence area.
Wherein the step S130 includes, in step S,
acquiring the number x of pixel points in the illumination test image and the number k of luminous pixel points in the y fluorescent test imagey1. In the actual operation process, a plurality of images are required to be acquired to obtain a composite image, the acquired images comprise an illumination image and a fluorescence image, and different naming modes are respectively adopted for distinguishing according to the functions of the acquired illumination image and the fluorescence image and different acquisition time periods.
The first illumination duration of different illumination devices is calculated, the method mainly aims to distinguish the different illumination devices, and control the different illumination devices for different illumination durations according to the distinguishing result, wherein the first illumination duration of the illumination device corresponding to the illumination test image can be preset.
The y-th fluorescence test was calculated by the following formulaFirst irradiation duration t of irradiation equipment corresponding to imagey1，
Wherein k isiThe number of luminous pixel points in the ith fluorescence test image, c the number of all fluorescence test images, p the conversion coefficient, tmAs a reference irradiation time period, tαIs the minimum irradiation duration.
When calculating the first irradiation duration, the invention firstly determines the number k of luminous pixel points in the fluorescence test imagey1Number of pixels emitting light on average
Performing comparison only when
Then, the reference irradiation duration is adjusted to be high, and when the reference irradiation duration is adjusted to be high, the reference irradiation duration is adjusted to be high
In time, t will be guaranteedy1＝tαI.e. the number k of luminous pixel pointsy1Smaller, the corresponding first irradiation duration can be ensured, and through the above mode, the effect of enhancing the irradiation of the fluorescent image with more luminous pixel points is realized.
According to the invention, when the first irradiation time lengths of different irradiation devices are calculated, the number of the pixel points in the illumination test image and the number of the pixel points in the other fluorescence test images are fully considered, and the first irradiation time length of each irradiation device is determined according to the number relation. The first irradiation duration of each irradiation device is different under different scenes, and the first irradiation duration is longer when the fluorescence area is larger and the fluorescence area is larger, so that the matching degree of the fluorescence area and the actual form of human tissue in the obtained composite image is higher.
The invention also includes:
setting a secondary acquisition time point, and acquiring a plurality of first-stage fluorescence images of the secondary acquisition time point closest to the moment. The secondary acquisition time point can be set according to the operation plan and can be a plurality of equal values of the time of the operation plan. For example, if a procedure is planned to be 60 minutes, the secondary collection time point may be the fifth minute, the tenth minute, the tertiary collection time point may be the tenth minute, the twentieth minute, and so on. And acquiring a first-stage fluorescence image closest to the time at the second acquisition time point, wherein the current surgical condition can be reflected by the first-stage fluorescence image at the time, whether the focus is cut or not is judged, and if the focus is cut, the number of fluorescence areas, fluorescence areas and fluorescence pixels in one first-stage fluorescence image is reduced.
Acquiring the number k of luminous pixel points in the y-th first-stage fluorescence image at the time closest to the secondary acquisition time pointy2. The invention can carry out the number k of luminous pixel points in the first-stage fluorescence image acquired at the secondary acquisition time pointy2And (6) carrying out statistics.
Based on the number k of luminous pixel points in the y-th fluorescence test imagey1And the number k of luminous pixel points in the first-stage fluorescence imagey2And calculating a second irradiation duration corresponding to the acquisition of the second-stage fluorescence image. Through ky2And k isy1The quantitative relation between the two parts can reflect whether the part of the focus in the first-stage fluorescence image is cut and processed, if the part of the focus in the first-stage fluorescence image is cut and processed, the first irradiation duration of the irradiation equipment corresponding to the fluorescence image can be adjusted, so that the irradiation equipment can be dynamically adjusted according to the condition of the focus.
Wherein, if yes, determine ky1Is not equal to ky2；
Then, the second illumination time t of the illumination device corresponding to the y-th fluorescence test image is calculated by the following formulay2，
Wherein A is a constant greater than 1, and s is an attenuation conversion coefficient.
The invention will be based on ky1And ky2To determine the second irradiation period ty2Through a second irradiation period ty2To guide the irradiation of the irradiation device in the second stage of the operation.
In a possible scenario, a doctor needs to excise two lesion parts and tissue parts with different fluorescence, and then the corresponding fluorescence part does not exist in the corresponding first-stage fluorescence image or second-stage fluorescence image after the lesion part and the tissue part with the first fluorescence are excised, and at this time, the first irradiation duration or the second irradiation duration of the corresponding irradiation equipment needs to be adjusted, and at this time, ty2May be equal to tα. Through the above manner, each irradiation apparatus is dynamically adjusted.
Step S140, different irradiation devices irradiate the focus based on different first irradiation durations to obtain an illumination working image and a plurality of first-stage fluorescence images, and the illumination working image and the plurality of first-stage fluorescence images are fused to obtain a composite image.
Wherein, the different fluorescence areas represent the forms of the corresponding tissues, if the fluorescence area in a first-stage fluorescence image is smaller, the actual volumes of the tissues and the lesions are proved to be smaller; conversely, if the fluorescence area in a first-stage fluorescence image is large, the actual volume of the tissue, lesion, is demonstrated to be large.
According to the technical scheme provided by the invention, an actual working scene can be tested before images are synthesized, the first irradiation duration of the irradiation equipment of the corresponding type is obtained according to the fluorescence area of the fluorescence area in the fluorescence test images of different types and types, the irradiation of the irradiation equipment of the corresponding type is respectively controlled through a plurality of first irradiation durations, and the fluorescence areas of different types are respectively collected to obtain the fluorescence images of the first stage.
In one possible embodiment, determining the fluorescence area of the fluorescence area in each fluorescence test image comprises:
and presetting a light emitting threshold value of each fluorescence test image, and carrying out binarization processing on each fluorescence test image based on the light emitting threshold value. For example, if the fluorescence in a fluorescence test image is green, the corresponding threshold value is the RGB value corresponding to green, and green includes dark green, light green, and the like, so the invention uses the threshold value to respectively include colors in the form of dark green, light green, and the like. In an actual surgical procedure, the color of the fluorescent portion may shift slightly because the fluorescent stain may be metabolized by the cells over time, causing the color of the fluorescent portion to change from light to dark and then light. According to the invention, the offset part can be covered by setting the luminous threshold value, so that the invention is more accurate in determining the form of the focus.
And extracting the area of the fluorescence area in the fluorescence test image after the binarization processing. The area statistics of the binaryzation fluorescence area is convenient by the method, the calculated amount is reduced, the fluorescence pixel point statistics can be realized by judging two possibilities in the pixel points, the data processing amount is reduced, and the data processing efficiency is improved.
In one possible embodiment, the extracting the area of the fluorescence region in the fluorescence test image after the binarization processing comprises:
and displaying the pixel points within the light-emitting threshold value in a form of 1 in the fluorescence test image, and displaying the pixel points outside the light-emitting threshold value in a form of 0 in the fluorescence test image. In the invention, the fluorescence test image after binaryzation is displayed and displayed in a specific form, so that the area statistics of a computer system is facilitated.
And acquiring the area of the pixel point which is 1 in all the fluorescence test images. The pixel point of 1 can be considered as a fluorescence region, and as shown below, 1 fluorescence test image can be displayed as follows.
In one possible embodiment, step S140 includes:
and registering the illumination working image and the plurality of first-stage fluorescent images so as to enable the coordinates of each pixel point in the illumination working image and the plurality of first-stage fluorescent images to correspond. The present invention first performs registration so that the illumination working image and the plurality of first-stage fluorograms are matched when combined. The registration belongs to the conventional technical scheme, and the invention is not repeated.
Respectively extracting regions of pixel points which are 1 in the first-stage fluorescence image, fusing the regions of the pixel points which are 1 in different first fluorescence images with the illumination working image, and displaying the fused regions in the illumination working image with preset pixel values, wherein the preset pixel values corresponding to the regions of the pixel points which are 1 in different first fluorescence images are different.
When a plurality of images are fused, the area of the pixel point which is 1 in the first-stage fluorescent image is extracted, the illumination working image is used as a substrate, and the area of the pixel point which is 1 in each first-stage fluorescent image is respectively added to the illumination working image to obtain a corresponding fused image.
And outputting and displaying the generated composite image after judging that all the first-stage fluorescence images are respectively fused with the illumination working image.
When the image is fused, the invention only collects the region of the pixel point which is 1 in the first-stage fluorescence image, so that the data processing efficiency is higher, the data processing amount is less and the image fusion speed is higher during the image fusion.
The invention also includes:
and judging whether a repeated fusion area exists in the composite image, wherein the repeated fusion area is respectively fused with different first-stage fluorescence images in the illumination working image. In the operation process, different fluorescence in the same region is likely to occur, and the condition of the focus is complicated at the moment, so the invention needs to specifically determine and lock the region.
And performing label display on the repeated fusion area, wherein the label display at least comprises the information of each fused first-stage fluorescence image. After determining that the repeated fusion area exists, the area is marked and displayed, so that medical personnel can know the fluorescence information in the repeated fusion area according to the marked and displayed area.
The technical scheme of the invention provides an irradiation device suitable for a fluorescence image navigation operation, as shown in a schematic structural diagram of fig. 2, the irradiation device comprises:
the device comprises a setting module, a control module and a control module, wherein the setting module is used for presetting a test interval time period and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period;
the system comprises an obtaining module, a processing module and a control module, wherein the obtaining module is used for obtaining a plurality of test images obtained by different irradiation equipment in a test interval time period, and the plurality of test images comprise an illumination test image and a plurality of fluorescence test images;
the determination module is used for determining the fluorescence area of the fluorescence area in each fluorescence test image and determining first illumination duration of different illumination devices based on the fluorescence area;
and the fusion module is used for enabling different irradiation equipment to irradiate the focus based on different first irradiation durations to obtain an illumination working image and a plurality of first-stage fluorescence images, and fusing the illumination working image and the plurality of first-stage fluorescence images to obtain a composite image.
The readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, a readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Additionally, the ASIC may reside in user equipment. Of course, the processor and the readable storage medium may also reside as discrete components in a communication device. The readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
The present invention also provides a program product comprising execution instructions stored in a readable storage medium. The at least one processor of the device may read the execution instructions from the readable storage medium, and the execution of the execution instructions by the at least one processor causes the device to implement the methods provided by the various embodiments described above.
In the above embodiments of the terminal or the server, it should be understood that the Processor may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. An irradiation method suitable for fluoroscopic image guided surgery, comprising:
presetting a test interval time period, and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period;
acquiring a plurality of test images obtained by different illumination devices in a test interval time period, wherein the plurality of test images comprise an illumination test image and a plurality of fluorescence test images;
determining the fluorescence area of a fluorescence area in each fluorescence test image, and determining first irradiation duration of different irradiation equipment based on the fluorescence area;
different irradiation equipment irradiates the focus based on different first irradiation duration to obtain an illumination working image and a plurality of first-stage fluorescence images, and the illumination working image and the plurality of first-stage fluorescence images are fused to obtain a composite image.
2. The illumination method suitable for fluoroscopic image guided surgery according to claim 1,
determining a fluorescence area of a fluorescence region in each fluorescence test image, determining a first illumination duration for different illumination devices based on the fluorescence area comprising:
acquiring the number x of pixel points in the illumination test image and the number k of luminous pixel points in the y fluorescent test imagey1；
Calculating the first irradiation duration t of the irradiation equipment corresponding to the y-th fluorescence test image by the following formulay1，
Wherein k isiThe number of luminous pixel points in the ith fluorescence test image, c the number of all fluorescence test images, p the conversion coefficient, tmAs a reference irradiation time period, tαIs the minimum irradiation duration.
3. The illumination method suitable for fluoroscopic image guided surgery according to claim 2, further comprising:
setting a secondary acquisition time point, and acquiring a plurality of first-stage fluorescence images of the secondary acquisition time point closest to the moment;
obtaining the yth time which is closest to the time of the secondary acquisition time pointNumber k of luminous pixel points in one-stage fluorescence imagey2；
Based on the number k of luminous pixel points in the y-th fluorescence test imagey1And the number k of luminous pixel points in the first-stage fluorescence imagey2And calculating a second irradiation duration corresponding to the acquisition of the second-stage fluorescence image.
4. The illumination method suitable for fluoroscopic image guided surgery according to claim 3,
if so, judge ky1Is not equal to ky2；
Then, the second illumination time t of the illumination device corresponding to the y-th fluorescence test image is calculated by the following formulay2，
Wherein A is a constant greater than 1, and s is an attenuation conversion coefficient.
5. The illumination method suitable for fluoroscopic image guided surgery according to claim 3,
determining the fluorescence area of the fluorescence area in each fluorescence test image comprises:
presetting a light emitting threshold value of each fluorescence test image, and carrying out binarization processing on each fluorescence test image based on the light emitting threshold value;
and extracting the area of the fluorescence area in the fluorescence test image after the binarization processing.
6. The illumination method suitable for fluoroscopic image guided surgery according to claim 5,
extracting the area of the fluorescence area in the fluorescence test image after the binarization processing comprises the following steps:
displaying the pixel points within the luminescence threshold value in a form of 1 in the fluorescence test image, and displaying the pixel points outside the luminescence threshold value in a form of 0 in the fluorescence test image;
and acquiring the area of the pixel point which is 1 in all the fluorescence test images.
7. The illumination method suitable for fluoroscopic image guided surgery according to claim 6,
fusing the illumination working image and the plurality of first-stage fluorescence images to obtain a composite image comprises:
registering the illumination working image and the plurality of first-stage fluorescent images so as to enable the coordinates of each pixel point in the illumination working image and the plurality of first-stage fluorescent images to correspond to each other;
respectively extracting regions of pixel points which are 1 in the first-stage fluorescence image, fusing the regions of the pixel points which are 1 in different first fluorescence images with the illumination working image, and displaying the fused regions in the illumination working image with preset pixel values, wherein the preset pixel values corresponding to the regions of the pixel points which are 1 in different first fluorescence images are different;
and outputting and displaying the generated composite image after judging that all the first-stage fluorescence images are respectively fused with the illumination working image.
8. The illumination method suitable for fluoroscopic image guided surgery according to claim 6, further comprising:
judging whether a repeated fusion area exists in the composite image, wherein the repeated fusion area is an area which is respectively fused with different first-stage fluorescent images in the illumination working image;
and performing label display on the repeated fusion area, wherein the label display at least comprises the information of each fused first-stage fluorescence image.
9. An illumination device suitable for fluoroscopic image guided surgery, comprising:
the device comprises a setting module, a control module and a control module, wherein the setting module is used for presetting a test interval time period and controlling different irradiation equipment to irradiate the focus in sequence based on the test interval time period;
the system comprises an obtaining module, a processing module and a control module, wherein the obtaining module is used for obtaining a plurality of test images obtained by different irradiation equipment in a test interval time period, and the plurality of test images comprise an illumination test image and a plurality of fluorescence test images;
the determination module is used for determining the fluorescence area of the fluorescence area in each fluorescence test image and determining first illumination duration of different illumination devices based on the fluorescence area;
and the fusion module is used for enabling different irradiation equipment to irradiate the focus based on different first irradiation durations to obtain an illumination working image and a plurality of first-stage fluorescence images, and fusing the illumination working image and the plurality of first-stage fluorescence images to obtain a composite image.
10. A readable storage medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110954082.3A CN113693724B (en) | 2021-08-19 | 2021-08-19 | Irradiation method, device and storage medium suitable for fluorescence image navigation operation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110954082.3A CN113693724B (en) | 2021-08-19 | 2021-08-19 | Irradiation method, device and storage medium suitable for fluorescence image navigation operation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113693724A true CN113693724A (en) | 2021-11-26 |
| CN113693724B CN113693724B (en) | 2022-10-14 |
Family
ID=78653452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110954082.3A Active CN113693724B (en) | 2021-08-19 | 2021-08-19 | Irradiation method, device and storage medium suitable for fluorescence image navigation operation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113693724B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115719415A (en) * | 2022-03-28 | 2023-02-28 | 南京诺源医疗器械有限公司 | Visual field adjustable double-video fusion imaging method and system |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5528046A (en) * | 1992-11-10 | 1996-06-18 | Hamamatsu Photonics K.K. | Method and device for determining the location and number of fluorescent molecules |
| CN102204811A (en) * | 2010-03-31 | 2011-10-05 | 索尼公司 | Fluorescence image acquisition method, fluorescence image acquisition program, and fluorescence image acquisition apparatus |
| CN102370462A (en) * | 2010-07-13 | 2012-03-14 | 索尼公司 | Imaging apparatus, imaging system, surgical navigation system, and imaging method |
| JP2013015325A (en) * | 2011-06-30 | 2013-01-24 | Tottori Univ | Apparatus and method for analyzing dna double-strand cutting damage |
| CN105982645A (en) * | 2015-02-13 | 2016-10-05 | 王平 | Fluorescent imaging instrument and focal zone image shooting method |
| CN108185974A (en) * | 2018-02-08 | 2018-06-22 | 北京数字精准医疗科技有限公司 | A kind of endoscopic fluorescence ultrasound merges radiography navigation system |
| CN108961250A (en) * | 2018-07-20 | 2018-12-07 | 中国科学院深圳先进技术研究院 | A kind of object statistical method, device, terminal and storage medium |
| CN109124768A (en) * | 2018-08-15 | 2019-01-04 | 南京航空航天大学 | A kind of fluorescence imaging operation guiding system |
| CN110706206A (en) * | 2019-09-11 | 2020-01-17 | 深圳先进技术研究院 | Fluorescent cell counting method, fluorescent cell counting device, terminal equipment and storage medium |
| CN111968105A (en) * | 2020-08-28 | 2020-11-20 | 南京诺源医疗器械有限公司 | Method for detecting salient region in medical fluorescence imaging |
| WO2021024314A1 (en) * | 2019-08-02 | 2021-02-11 | 株式会社島津製作所 | Medical treatment assistance device and medical treatment assistance method |
| CN113208567A (en) * | 2021-06-07 | 2021-08-06 | 上海微创医疗机器人（集团）股份有限公司 | Multispectral imaging system, imaging method and storage medium |
-
2021
- 2021-08-19 CN CN202110954082.3A patent/CN113693724B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5528046A (en) * | 1992-11-10 | 1996-06-18 | Hamamatsu Photonics K.K. | Method and device for determining the location and number of fluorescent molecules |
| CN102204811A (en) * | 2010-03-31 | 2011-10-05 | 索尼公司 | Fluorescence image acquisition method, fluorescence image acquisition program, and fluorescence image acquisition apparatus |
| CN102370462A (en) * | 2010-07-13 | 2012-03-14 | 索尼公司 | Imaging apparatus, imaging system, surgical navigation system, and imaging method |
| JP2013015325A (en) * | 2011-06-30 | 2013-01-24 | Tottori Univ | Apparatus and method for analyzing dna double-strand cutting damage |
| CN105982645A (en) * | 2015-02-13 | 2016-10-05 | 王平 | Fluorescent imaging instrument and focal zone image shooting method |
| CN108185974A (en) * | 2018-02-08 | 2018-06-22 | 北京数字精准医疗科技有限公司 | A kind of endoscopic fluorescence ultrasound merges radiography navigation system |
| CN108961250A (en) * | 2018-07-20 | 2018-12-07 | 中国科学院深圳先进技术研究院 | A kind of object statistical method, device, terminal and storage medium |
| CN109124768A (en) * | 2018-08-15 | 2019-01-04 | 南京航空航天大学 | A kind of fluorescence imaging operation guiding system |
| WO2021024314A1 (en) * | 2019-08-02 | 2021-02-11 | 株式会社島津製作所 | Medical treatment assistance device and medical treatment assistance method |
| CN110706206A (en) * | 2019-09-11 | 2020-01-17 | 深圳先进技术研究院 | Fluorescent cell counting method, fluorescent cell counting device, terminal equipment and storage medium |
| CN111968105A (en) * | 2020-08-28 | 2020-11-20 | 南京诺源医疗器械有限公司 | Method for detecting salient region in medical fluorescence imaging |
| CN113208567A (en) * | 2021-06-07 | 2021-08-06 | 上海微创医疗机器人（集团）股份有限公司 | Multispectral imaging system, imaging method and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115719415A (en) * | 2022-03-28 | 2023-02-28 | 南京诺源医疗器械有限公司 | Visual field adjustable double-video fusion imaging method and system |
| CN115719415B (en) * | 2022-03-28 | 2023-11-10 | 南京诺源医疗器械有限公司 | Visual field adjustable double-video fusion imaging method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113693724B (en) | 2022-10-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6657480B2 (en) | Image diagnosis support apparatus, operation method of image diagnosis support apparatus, and image diagnosis support program | |
| CN103491850B (en) | Fluorescence monitoring apparatus and method for displaying image thereof | |
| US20140085686A1 (en) | Fluorescence observation device | |
| EP3590413A1 (en) | Endoscope system and method for operating same | |
| CN112105284B (en) | Image processing device, endoscope system, and image processing method | |
| JP2005192944A (en) | Remote diagnostic system for pigmentation region | |
| KR20160037834A (en) | Medical imaging device and methods of use | |
| WO2016174911A1 (en) | Photodynamic diagnosis apparatus and photodynamic diagnosis method | |
| CN103975364B (en) | Selection of images for optical examination of the cervix | |
| CN114298980A (en) | Image processing method, device and equipment | |
| JP6941233B2 (en) | Image processing equipment, endoscopic system, and image processing method | |
| JPWO2019198637A1 (en) | Image processing equipment, endoscopic system, and image processing method | |
| CN112656369B (en) | AR technology-based near-infrared fluorescent parathyroid gland identification enhancement method | |
| CN215305781U (en) | Apparatus for identifying parathyroid gland position and evaluating parathyroid gland survival rate | |
| CN112566540A (en) | Processor for endoscope, information processing device, endoscope system, program, and information processing method | |
| WO2020054543A1 (en) | Medical image processing device and method, endoscope system, processor device, diagnosis assistance device and program | |
| CN113693724B (en) | Irradiation method, device and storage medium suitable for fluorescence image navigation operation | |
| JPWO2020039931A1 (en) | Endoscopic system and medical image processing system | |
| CN116745861A (en) | Control method, device and program of lesion judgment system obtained through real-time image | |
| CN114343555A (en) | Colposcope artificial intelligence auxiliary quality control management method and device | |
| KR102141541B1 (en) | Endoscope apparatus for measuring lesion volume using depth map and method for measuring lesion volume using the apparatus | |
| CN115668296A (en) | Globally processing multiple luminescence images for mapping and/or segmentation thereof | |
| CN113842212A (en) | Fluorescence scattering optical tomography processing method and system | |
| CN112704473A (en) | Parathyroid gland position enhancement equipment | |
| CN112704474A (en) | AR detection glasses for detecting parathyroid gland position |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Illumination methods, devices, and storage media suitable for fluorescence imaging navigation surgery Granted publication date: 20221014 Pledgee: China Postal Savings Bank Co.,Ltd. Nanjing Liuhe Branch Pledgor: NANJING NUOYUAN MEDICAL DEVICES CO.,LTD. Registration number: Y2024980005021 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |