CN115719415B - Visual field adjustable double-video fusion imaging method and system - Google Patents

Visual field adjustable double-video fusion imaging method and system Download PDF

Info

Publication number
CN115719415B
CN115719415B CN202210313868.1A CN202210313868A CN115719415B CN 115719415 B CN115719415 B CN 115719415B CN 202210313868 A CN202210313868 A CN 202210313868A CN 115719415 B CN115719415 B CN 115719415B
Authority
CN
China
Prior art keywords
pixel
white light
pixel point
value
point set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210313868.1A
Other languages
Chinese (zh)
Other versions
CN115719415A (en
Inventor
蔡惠明
钱露
卢露
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Nuoyuan Medical Devices Co Ltd
Original Assignee
Nanjing Nuoyuan Medical Devices Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Nuoyuan Medical Devices Co Ltd filed Critical Nanjing Nuoyuan Medical Devices Co Ltd
Priority to CN202210313868.1A priority Critical patent/CN115719415B/en
Publication of CN115719415A publication Critical patent/CN115719415A/en
Application granted granted Critical
Publication of CN115719415B publication Critical patent/CN115719415B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Endoscopes (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a visual field adjustable double-video fusion imaging method and a visual field adjustable double-video fusion imaging system, which are used for respectively obtaining a focus first fluorescent video image and a focus first white light video image under fluorescent and white light irradiation, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set; extracting pixel points in a first preset pixel interval in the fluorescent pixel point set to generate a fluorescent pixel point set of interest; comparing the fluorescence interest pixel point set with the white light pixel point set to determine a white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set; generating a fusion video image according to the replaced white light pixel point set; the invention can embody the pathological change condition in the focus area and improve the distinguishing degree of the focus area and the periphery.

Description

Visual field adjustable double-video fusion imaging method and system
Technical Field
The invention relates to the technical field of image data processing, in particular to a visual field adjustable double-video fusion imaging system.
Background
The double video fusion imaging is a process of automatically matching the interested areas of the two cameras, and the automatic matching is to generate a new image under a specified common space coordinate system through automatic calibration.
In the prior art, single-camera video imaging is mainly used, and the existing double-camera generally carries out automatic calibration on two graphs through a processor and carries out fusion processing on the images through a data superposition processing module.
However, due to the problems of setting and placing the lenses of the dual video and the like, the processor is prone to generating errors when superposing two graphic data, and the degree of distinguishing the superposed pictures is low, especially in the fields of medical treatment and the like with extremely high requirements on image precision, for example: minimally invasive treatment surgery aiming at minimizing the trauma of a patient can cause damage to healthy tissues of the patient, and the postoperative wound is too large, so that the recovery time of the patient is delayed, and therefore, how to improve the distinguishing degree of a focus area and a normal tissue area is particularly important.
Disclosure of Invention
The embodiment of the invention provides a visual field adjustable double-video fusion imaging method, which can increase the contrast ratio of a focus area and a healthy tissue area, so that a doctor can accurately remove the focus area according to the pixel change of fusion imaging during operation, reduce the trauma of healthy tissue of a patient to the maximum extent and reduce the rehabilitation time of the patient.
The embodiment of the invention provides a visual field adjustable double-video fusion imaging method, which comprises the following steps of:
respectively obtaining a focus first fluorescent video image and a focus first white light video image under fluorescent light and white light irradiation, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set;
extracting pixel points in the first preset pixel interval in the fluorescence pixel point set to generate a fluorescence interested pixel point set;
comparing the fluorescence interest pixel point set with the white light pixel point set to determine a white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set;
and generating a fusion video image according to the replaced white light pixel point set.
Optionally, in one possible implementation manner of the first aspect, in the step of acquiring a first fluorescence video image and a first white light video image of a focus under fluorescence and white light illumination, respectively, decomposing the first fluorescence image and the first white light image based on a preset origin of coordinates to obtain a fluorescence pixel point set and a white light pixel point set, specifically includes:
Selecting a first pixel point with the same first fluorescent image and first white light image as a preset origin of coordinates;
decomposing the first fluorescent image according to a preset origin of coordinates to obtain coordinate values of each pixel point to obtain a fluorescent pixel point setWherein a is n For the nth fluorescent pixel point coordinate in the fluorescent pixel point set,/for the fluorescent pixel point coordinate>Is the abscissa of the nth fluorescent pixel,>is the ordinate of the nth fluorescent pixel point;
decomposing the first white light image according to a preset origin of coordinates to obtain coordinate values of each pixel point to obtain a white light pixel point setWherein b n Is the coordinates of the nth white light pixel point in the white light pixel point set, and is +.>Is the abscissa of the nth white light pixel,>is the ordinate of the nth white light pixel point.
Optionally, in a possible implementation manner of the first aspect, in the step of extracting a pixel point in the set of fluorescent pixel points that is located in the first preset pixel interval and generating the set of fluorescent pixel points of interest, the method specifically includes:
traversing a fluorescent pixel value of each fluorescent pixel point in a fluorescent pixel point set, and extracting the fluorescent pixel point if the fluorescent pixel value is positioned in the first preset pixel interval;
Acquiring all extracted fluorescent pixel points to generate a fluorescent interested pixel point setWherein c n For the nth fluorescence interest pixel point coordinate in the fluorescence interest pixel point set,/-for the fluorescence interest pixel point coordinate>For the abscissa of the nth fluorescence pixel of interest, +.>Is the ordinate of the nth fluorescence pixel of interest.
Optionally, in one possible implementation manner of the first aspect, in the step of comparing the fluorescent pixel-of-interest set with the white light pixel-of-interest set to determine the white light pixel-of-interest set, replacing a pixel value of a pixel in the white light pixel-of-interest set with a pixel value of a corresponding pixel in the fluorescent pixel-of-interest set to obtain a replaced white light pixel-of-interest set, the method specifically includes:
comparing the coordinates of each pixel point in the fluorescence interest pixel point set with the coordinates of each pixel point in the white light pixel point set to determine the white light interest pixel point setWherein d n For the coordinates of the nth white light interest pixel point in the white light interest pixel point set,/-for the coordinates of the nth white light interest pixel point in the white light interest pixel point set>The abscissa of the pixel of interest for the nth white light, +.>An ordinate of an nth white light pixel of interest;
replacing the pixel value of the pixel point with the same coordinate in the white light pixel point set to obtain a replaced white light pixel point set;
Acquiring a non-white light interested pixel point set in a white light pixel point setWherein e n Is the coordinates of the nth fluorescence interest pixel point in the non-white light interest pixel point set,/for the non-white light interest pixel point set>For the abscissa of the nth non-white light pixel of interest, +.>Is the ordinate of the nth non-white light pixel of interest;
generating first pixel adjustment information according to the pixel value of each pixel in the white light interested pixel set and the pixel value of each pixel in the non-white light interested pixel set;
and processing the white light interested pixel points according to the first pixel adjustment information to obtain a replaced white light pixel point set.
Optionally, in a possible implementation manner of the first aspect, in the step of generating the first pixel adjustment information according to the pixel value of each pixel in the white light interesting pixel point set and the pixel value of each pixel in the non-white light interesting pixel point set, the method specifically includes:
generating first reference pixel information according to the pixel value of each pixel point in the white light interested pixel point set;
generating second reference pixel information according to the pixel value of each pixel point in the non-white light interesting pixel point set;
Calculating the first reference pixel information and the second reference pixel information to obtain first pixel adjustment information;
the first reference pixel information, the second reference pixel information and the first pixel adjustment information are calculated by the following formulas,
wherein p is Base 1 For the first reference pixel information,r value of pixel value RGB of ith pixel point in white light interested pixel point set,/and/or>B value of pixel value RGB of ith pixel point in white light interested pixel point set,/and/or>Blue value, n, of pixel value RGB for the ith pixel in the set of pixel points of interest for white light 1 Reference weight value for first pixel, q Base 2 For the second reference pixel information +.>R value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>B value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>Blue value, n, of pixel value RGB for the ith pixel in the set of non-white light pixels of interest 2 For the reference weight value of the second pixel, i is the ith pixel point, n is the upper limit of the pixel point, and k 1 For the pixel deviation value weight, W is the first pixel adjustment information, +.>Is the sum of pixel values of the pixel points of interest for white light,/for>Is the sum of the pixel values of the non-white light pixel of interest.
Optionally, in a possible implementation manner of the first aspect, in the step of processing the white light pixel of interest according to the first pixel adjustment information to obtain a replaced white light pixel set, the method specifically includes:
if the first pixel adjustment information is larger than the preset adjustment information, directly taking the pixel value of the pixel point of interest of the white light as the pixel value of the corresponding pixel point in the set of the white light pixel points after replacement;
if the first pixel adjustment information is smaller than the preset adjustment information, comparing the second reference pixel information with the preset standard information to obtain first conversion trend information;
and adjusting the white light interested pixel points according to the first conversion trend information and the first pixel adjustment information to obtain a replaced white light pixel point set.
Optionally, in one possible implementation manner of the first aspect, in the step of performing adjustment processing on the white light pixel of interest according to the first conversion trend information and the first pixel adjustment information to obtain a replaced white light pixel set, the method specifically includes:
if the first transformation trend information is positive, the pixel value of each pixel point in the white light interested pixel point set is reduced, the specific formula is as follows,
If the first transformation trend information is negative, the pixel value of each pixel point in the white light interested pixel point set is increased, the specific formula is as follows,
wherein I is 1 Reducing correction value for pixel value, I 2 Adding correction value, k, to pixel value r K is the correction value of R value of pixel value RGB g Correction value k for G value of pixel value RGB b A correction value for the B value of the pixel value RGB,for the R value of the pixel value RGB of the ith pixel point in the reduced white light interesting pixel point set,/-degree>G value of pixel value RGB of ith pixel point in reduced white light interest pixel point set,/-degree>For the reduced B value of the pixel value RGB of the ith pixel point in the white light pixel point set of interest,/->R value of pixel value RGB of ith pixel point in increased white light interest pixel point set is +.>G value of pixel value RGB of ith pixel point in increased white light interest pixel point set is +.>And B value of pixel value RGB of the ith pixel point in the increased white light interested pixel point set.
Optionally, in one possible implementation manner of the first aspect, according to the second pixel adjustment information actively input by the doctor, adjusting a pixel value of each pixel in the white light interesting pixel set to obtain an adjusted white light interesting pixel set;
Obtaining a pixel adjustment trend according to the white light interesting pixel point set before adjustment and the white light interesting pixel point set before adjustment;
correcting the pixel deviation value weight according to the pixel adjustment trend;
the corrected pixel deviation value weight is obtained by the following formula,
wherein W is 2 Adjusting information, k, for the second pixel 2 For corrected pixel deviation value weight, U 1 Increasing trend correction value for pixel adjustment information, U 2 The trend correction value is reduced for the pixel adjustment information.
Optionally, in a possible implementation manner of the first aspect, before the step of acquiring the first fluorescence video image and the first white light video image of the lesion under fluorescence and white light illumination, respectively, specifically includes: randomly extracting N pixel points in a fluorescent pixel point set, and if the horizontal coordinate value and the vertical coordinate value of the N pixel points are equal to the horizontal coordinate value and the vertical coordinate value in the corresponding white pixel point set, finishing the calibration;
if the abscissa and the ordinate value of a certain pixel point in the N pixel points are not equal to the abscissa and the ordinate value in the corresponding white light pixel point set, the calibration fails, and the equipment needs to be debugged.
In a second aspect of an embodiment of the present invention, a field-of-view adjustable dual video fusion imaging system includes:
the acquisition module is used for acquiring a focus first fluorescent video image and a focus first white light video image under the irradiation of fluorescence and white light respectively, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set;
the extraction module is used for extracting pixel points positioned in a first preset pixel interval in the fluorescent pixel point set to generate a fluorescent pixel point set of interest;
the replacing module is used for comparing the fluorescence interest pixel point set with the white light pixel point set to determine the white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set;
and the fusion module is used for generating a fusion video image according to the replaced white light pixel point set. In a third aspect of the embodiments of the present invention, there is provided a field-of-view adjustable dual video fusion imaging apparatus, including: a memory, a processor and a computer program stored in the memory, the processor running the computer program to perform the first aspect of the invention and the methods that the first aspect may relate to.
In a fourth aspect of embodiments of the present invention, there is provided a readable storage medium having stored therein a computer program for implementing the method of the first aspect and the various possible aspects of the first aspect when executed by a processor.
According to the visual field adjustable double-video fusion imaging method provided by the invention, pixel information of the first fluorescent image and pixel information of the first white light image can be fused, so that a disease focus area is visible to the naked eye, and compared with the field of fluorescent imaging which is invisible to the naked eye, a doctor needs to constantly change a sheet and turn off a lamp to observe, continuous use and working efficiency are affected, and the working efficiency of the doctor is improved.
According to the technical scheme provided by the invention, the pixel values of the pixels in the white light interested pixel point set are replaced with the pixel values of the corresponding pixels in the fluorescence interested pixel point set by establishing the coordinate system, so that the white light interested pixel has the density change trend of lesion cells, and medical staff can determine the segmentation depth of the scalpel in the operation process according to different internal change degrees.
According to the technical scheme provided by the invention, the distinguishing degree standard is determined through preset adjustment information which is preset, and when the distinguishing degree meets the standard when the first pixel adjustment information is larger than the preset adjustment information, direct display is performed; when the first pixel adjustment information is smaller than the preset adjustment information and the time division does not meet the standard, the pixel value of each pixel in the white light interested pixel set is adjusted through the first pixel adjustment information, so that the white light interested pixel set and the non-white light interested pixel set are obviously different.
According to the technical scheme provided by the invention, the pixel value of the white light pixel point set of interest can be adjusted by receiving the second pixel adjustment information actively input by medical staff, so that the pixel value of the white light pixel point set of interest and the pixel value of the non-white light pixel point set of interest can be more suitable for personal habits of the medical staff; according to the invention, the pixel deviation value weight is corrected according to the first pixel adjustment information which is automatically calculated and the second pixel adjustment information which is actively input by the medical staff, so that the first pixel adjustment information is more attached to the use habit of the medical staff in the next calculation, and the working efficiency of the medical staff is improved.
According to the technical scheme provided by the invention, after the step that the first pixel point with the same first fluorescent image and the first white light image is used as the preset origin of coordinates, the coordinates of the images are calibrated, the pixel points are randomly extracted for verification through a random sampling method, if the abscissa and the ordinate corresponding to each pixel point of the extracted first fluorescent image are the same as the values of the abscissa and the ordinate of the first white light image, the subsequent step is carried out, the accuracy of the subsequent step is ensured through calibration, and the damage to healthy tissues of a patient caused by offset generated in the using process of the device is avoided.
Drawings
Fig. 1 is a schematic view of an application scenario of the technical scheme provided by the invention;
FIG. 2 is a flow chart of a first embodiment of a field-of-view tunable dual video fusion imaging method;
FIG. 3 is a flow chart of a second embodiment of a field-of-view tunable dual video fusion imaging method;
FIG. 4 is a schematic diagram of a dual video fusion imaging system with adjustable field of view;
fig. 5 is a schematic diagram of a hardware configuration of a field-of-view adjustable dual video fusion imaging apparatus.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein.
It should be understood that, in various embodiments of the present invention, the sequence number of each process does not mean that the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
It should be understood that in the present invention, "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements that are expressly listed or inherent to such process, method, article, or apparatus.
It should be understood that in the present invention, "plurality" means two or more. "and/or" is merely an association relationship describing an association object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. "comprising A, B and C", "comprising A, B, C" means that all three of A, B, C comprise, "comprising A, B or C" means that one of the three comprises A, B, C, and "comprising A, B and/or C" means that any 1 or any 2 or 3 of the three comprises A, B, C.
It should be understood that in the present invention, "B corresponding to a", "a corresponding to B", or "B corresponding to a" means that B is associated with a, from which B can be determined. Determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information. The matching of A and B is that the similarity of A and B is larger than or equal to a preset threshold value.
As used herein, "if" may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to detection" depending on the context.
The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
As shown in fig. 1, in an application scenario of the technical solution provided by the present invention, a doctor uses a video acquisition device, such as a dual-light-source camera, where the working principle of the dual-light-source camera is that the camera respectively shoots pictures under fluorescent illumination of a fluorescent emission device and white light illumination of a white light emission device according to a fixed frequency to perform fusion to form a video through a processor, so that the camera shoots a focus area under the fluorescent light source and the white light source respectively, acquires an image, establishes a coordinate system for fusion processing of the acquired image through the processor, renders the image to protrude through a GPU to display the focus area, and finally displays the image by a display device, for example: the display device can be a computer display screen, a television screen and the like, and the images acquired each time are fused immediately, and the fusion time interval is less than 0.01 second.
The video capture device may be coupled to a processor coupled to an adjustment device such as an adjustment knob, the adjustment knob rotated by an angle related to the brightness of the lesion area, for example: the device is turned to the first direction to lighten, turned to the second direction to darken, a doctor can adjust the brightness of a focus area through the turning of the adjusting knob, the adjusting knob sends an adjusting signal to the processor, the processor receives the processing signal, renders the picture by the GPU, and the focus area is highlighted and displayed through the display device. The processor processes and records the change trend from the first pixel adjustment information to the second pixel adjustment information and influences the pixel deviation value weight, so that the pixel deviation value weight value is more suitable for actual situations.
According to the technical scheme provided by the invention, the advantage that the fluorescence image can display the lesion degree in the focus area is combined with the advantage that the white light image can directly obtain the range of the focus area observed by naked eyes.
The invention provides a visual field adjustable double-video fusion imaging method, wherein the visual field of a double-light source camera is adjustable, the visual field adjustment is carried out through different amplifying positions of lenses, the visual field adjustment is carried out, and the image processing is carried out through the following steps, as shown in fig. 2, the method specifically comprises the following steps:
Step S110, a focus first fluorescent video image and a focus first white light video image are obtained under the irradiation of fluorescence and white light respectively, and the first fluorescent image and the first white light image are decomposed based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set.
According to the technical scheme provided by the invention, in daily diagnosis, after a doctor adjusts the visual field of the focus area by the video acquisition device, the focus area is acquired by the fluorescent video acquisition device, but in the diagnosis process, because the illumination area of the focus area of a patient is limited, the lamp is required to be switched off and replaced, the process is very complicated, and the process is required to be under the irradiation of the film viewer.
Therefore, the invention respectively acquires the fluorescence video image and the white light video image of the focus under the irradiation of fluorescence and white light, establishes a two-dimensional coordinate system through a preset origin of coordinates, obtains the coordinates of each pixel point of the fluorescence video image according to the preset origin of coordinates and the two-dimensional coordinate system, forms a fluorescence pixel point set, and obtains the coordinates of each pixel point of the white light video image according to the preset origin of coordinates and the two-dimensional coordinate system, thus forming a white light pixel point set. The coordinates of each pixel of the fluorescent video image and the coordinates of each pixel of the white light video image are formed in order to achieve the subsequent pixel replacement and contrast steps.
In one possible implementation manner, the step S110 specifically includes:
and selecting a first pixel point with the same first fluorescent image and the same first white light image as a preset origin of coordinates. The preset origin of coordinates may be coordinates of pixels in the first row and the first column in the first fluorescent image and the first white light image as the origin of coordinates to establish a coordinate system, which is not limited herein.
Decomposing the first fluorescent image according to a preset origin of coordinates to obtain coordinate values of each pixel point to obtain a fluorescent pixel point setWherein a is n For the nth fluorescent pixel point coordinate in the fluorescent pixel point set,/for the fluorescent pixel point coordinate>Is the abscissa of the nth fluorescent pixel,>is the ordinate of the nth fluorescent pixel point. Wherein the coordinates of the fluorescent pixel point set are obtained by a coordinate system established by a preset origin of coordinates, such as the pixel point coordinates a of the first row and the second column 2 For (1, 0), the pixel point coordinate a of the third column of the first row 3 For (2, 0), etc., the coordinates of all the pixels are combined to obtain a set of fluorescent pixels.
Decomposing the first white light image according to a preset origin of coordinates to obtain coordinate values of each pixel point to obtain a white light pixel point set Wherein b n Is the coordinates of the nth white light pixel point in the white light pixel point set, and is +.>Is the abscissa of the nth white light pixel,>is the ordinate of the nth white light pixel point. Wherein the white light pixel point set is a coordinate corresponding to all the pixel points obtained according to the coordinate system, such as the pixel point coordinate b of the first row and the second column at this time 2 And combining the coordinates of all the pixels to obtain a white light pixel point set, wherein the white light pixel point set is (1, 0) and the like.
According to the technical scheme provided by the invention, the processor can select the same pixel point at any position in the fluorescent image and the white light image as the origin of the coordinate system to establish a two-dimensional coordinate system, for example: establishing a coordinate system through the origin of the coordinate system, and distributing a coordinate value to each pixel point on the fluorescent image to form a fluorescent pixel point setSimilarly, white light is usedEach pixel point on the image is allocated a coordinate value to form a white light pixel point set +.> Wherein a is n For the nth fluorescent pixel point coordinate in the fluorescent pixel point set,/for the fluorescent pixel point coordinate>Is the abscissa of the nth fluorescent pixel,>is the ordinate, b, of the nth fluorescent pixel point n Is the coordinates of the nth white light pixel point in the white light pixel point set, and is +.>Is the abscissa of the nth white light pixel,>is the ordinate of the nth white light pixel point; for example: a, a 1 The coordinates may be (111 ), b 1 Coordinates are (111 ), the pixel points with the same coordinate point are obtained, a coordinate system origin (0, 0) is correspondingly established, the direction corresponding to the abscissa X can be north-facing, the direction corresponding to the ordinate Y is east-facing, the method is not limited, and then the coordinates of each pixel point in the fluorescent image and the white light image are obtained according to the coordinate origin, so that later replacement and comparison operation are facilitated.
And step S120, extracting pixel points positioned in a first preset pixel interval in the fluorescence pixel point set to generate a fluorescence interested pixel point set.
According to the technical scheme provided by the invention, a doctor observes a focus area on a film viewer through own experience, and only human eyes are used for identifying a lesion part and a non-lesion part, so that a large error exists, a processor counts focus area pixel point values of past historical pictures, a first preset pixel interval is set according to focus areas caused by different types of lesions, only coordinates of pixels meeting the first preset pixel interval in a fluorescent pixel point set are selected, coordinates of pixels meeting conditions form a set, and a fluorescent interested pixel point set is generated, for example: the RGB value of the wart mole is a green interval, then a first preset pixel interval is set to be (0, 50-255, 0), a focus area pixel point set is correspondingly generated, the pixel value is from dark green to turquoise, wherein the dark green represents that the lesion cell density is higher, the turquoise represents that the lesion cell density is lower, and a doctor can determine the depth of an operation according to the color change.
In one possible implementation manner, the step S120 specifically includes:
traversing the fluorescent pixel value of each fluorescent pixel point in the fluorescent pixel point set, and extracting the fluorescent pixel point if the fluorescent pixel value is positioned in the first preset pixel interval. The technical scheme provided by the invention, wherein the first preset pixel interval is a pixel value of a focus area, such as: the pixel interval value of the wart is (0, 50-255, 0), and the doctor can set the pixel value of the first preset pixel interval to be (0, 50-255, 0) when diagnosing the wart, and then traverse the fluorescent pixel value of each fluorescent pixel point in the fluorescent pixel point set, so that the focus area can be accurately obtained.
Acquiring all extracted fluorescent pixel points to generate a fluorescent interested pixel point setWherein c n For the nth fluorescence interest pixel point coordinate in the fluorescence interest pixel point set,/-for the fluorescence interest pixel point coordinate>For the abscissa of the nth fluorescence pixel of interest, +.>Is the ordinate of the nth fluorescence pixel of interest.
The invention providesAccording to the technical scheme, in the past, doctors judge that the pathological change part of a patient is estimated based on own experience and then the pathological change region of an image is estimated, but in reality, the pathological change types are diversified, for example, doctors encounter skin surface pathological changes, for example: warty nevi, papules, sweat duct tumors, keloids, basal cell carcinomas of nevi, cutaneous lymphomas, mycosis fungoides, lymphomatoid papules, malignant histiocytosis, sweat duct tumors, sweat gland carcinomas, knuckle pads, hemangiomas, warty hemangiomas, lipomas, malignant melanomas, etc., and lesions in lesion areas are not limited to lesions on the skin surface, and it is obviously unreasonable to have a huge experience for doctors, so the invention obtains a better first preset pixel interval by collecting and recording pixel values of lesion parts of each case in the past by a processor, such as: presetting an RGB pixel interval as (125 ～ 224,110 ～ 220, 60-128), and traversing fluorescent RGB pixel values of each fluorescent pixel point in a fluorescent pixel point set, wherein the range of the RGB pixel values is as follows: the R value is 0-255, the G value is 0-255, the B value is 0-255, if the pixel value of the pixel point in the fluorescence pixel point set meets the preset pixel interval, the coordinates corresponding to the pixel point are recorded, the coordinates of all the pixel points meeting the preset RGB pixel interval (125 ～ 224,110 ～ 220, 60-128) are obtained and combined, and the fluorescence pixel point set is generated
And S130, comparing the fluorescence interest pixel point set with the white light pixel point set to determine the white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set.
According to the technical scheme provided by the invention, it can be understood that the internal pathological changes of the focus area can not be known as the macroscopic white light video image only irradiates the surface of the focus area, a doctor needs to compare the white light video image with the fluorescent video image, the internal pathological changes are known through fluorescence, and the pathological changes of the surface are checked through the white light naked eyes. After the replacement is finished, the change of the lesion degree inside the lesion cells can be displayed on the white light video image; the replaced white light pixel point set of interest belongs to the replaced white light pixel point set subset, and the white light pixel point set of interest belongs to the white light pixel point set subset; the purpose of replacing the pixel values of the white light pixel of interest set with the pixel values of the fluorescent pixel of interest set C is that the first white light video image under the original white light is not capable of displaying pixel variations inside the lesion area, which are representative of the degree of cytopathy, for example: the degree of tightness of lesion cells, the degree of depth of lesion cells and the like, a doctor can determine the depth of an operation wound according to the change of the pixel value in the lesion cells after replacement by naked eyes, the accuracy of the operation is improved, the possibility of postoperative recurrence of a patient is prevented, and on the basis that the change is needed to be seen in the lesion, the degree of distinction between a lesion part and a non-lesion part is also needed to be increased so as to improve the accuracy of the operation, avoid damage of healthy tissues and reduce the time for repairing the wound of the patient.
In one possible implementation manner of the technical solution provided in the present invention, as shown in fig. 3, step S130 specifically includes:
step S1301, comparing the coordinates of each pixel in the fluorescence interest pixel set with the coordinates of each pixel in the white light pixel set to determine the white light interest pixel set According to the technical scheme provided by the invention, firstly, the white light interested pixel point set is positioned through the coordinate value of the fluorescent interested pixel point set, for example: the fluorescence interest pixel point set is C= { C 1 |(10,15),c 2 |(11,17),c 3 The corresponding white light pixel point set of interest is d= { D = (12, 19) } 1 |(10,15),d 2 |(11,17),d 3 (12, 19) }, by which the lesion area in the first white light image can be located directly.
The technical proposal provided by the invention uses the coordinate system and the origin of coordinates established before to coordinate each pixel point in the fluorescence interest pixel point set CCoordinates with each pixel in white light pixel set BComparing, selecting the point with the same coordinates in the white light pixel point set B and the fluorescence pixel point set C, thereby determining the pixel point set +.>For example: fluorescence interest pixel point set C= { C 1 |(25,27),c 2 |(25,28),c 3 I (25, 29) }, then the white light pixel point set b= { B is selected correspondingly 1 |(25,27),b 2 |(25,28),b 3 |(25,29),…,b n The coordinate in (26,99) is b 1 (25,27)，b 2 |(25,28),b 3 |(25,29)，b 4 |(26,27),b 5 Pixels of (26, 28) to determine a white light pixel of interest set d= { D 1 |(25,27),d 2 |(25,28),d 3 (25, 29) a white light set of pixels of interest is determined in preparation for facilitating subsequent pixel value replacement and highlighting of the lesion.
Step S1302, replacing the pixel value of each pixel coordinate in the fluorescence interest pixel set with the pixel value of the pixel with the same coordinate in the white light interest pixel set to obtain a replaced white light interest pixel set.
According to the technical scheme provided by the invention, in general, doctor and patient communication needs to combine fluorescence and white light images, the pathological change degree is illustrated by the fluorescence images, and the pathological change range is illustrated by the white light images, but the continuous comparison and the illustration processes are too complicated, so that the invention integrates the fluorescence interested pixel point set C= { C 1 |(25,27),c 2 |(25,28),c 3 Pixel value RGB { c) corresponding to coordinates of each pixel point in (25, 29) 1 |(50,50,50),c 2 |(200,150,200),c 3 Pixel point set of interest for white light d= { D = (50,80,150) } 1 |(25,27),d 2 |(25,28),d 3 Pixel values RGB { d) of pixel points of the same coordinates in (25, 29) } 1 |(50,100,150),d 2 |(50,100,50),d 3 | (200,200,200) } substitution, where: c 1 The coordinates of (d) and d 1 Will d if the coordinates of (a) are the same 1 The pixel value of (c) is changed to c 1 Is replaced by d 1 The pixel value of (5) is (50,100,150), d can be obtained by the same method 2 The pixel value of (5) is (50,150,150), d 2 The pixel value of (50,100,100) is changed to obtain a white light interested pixel point set, the coordinate value of the white light interested pixel point set after the change is unchanged, but the corresponding pixel value is changed to the pixel value RGB { d } of the fluorescent interested pixel point set 1 |(50,50,50),d 2 |(200,150,200),d 3 The original white light video image can only shoot the surface of the pathological change part quite inaccurately, a doctor can not know the inside of the pathological change part, but after the pixel is changed, the doctor can generate different imaging and pixel values according to the difference of the cell density and the volume of the fluorescent image, the replaced image is more convenient for the doctor to operate, for example, the pixel value of the white light interested pixel point set after the replacement is RGB { d }, the pixel value of the white light interested pixel point set is the RGB { d }, the pixel value of the white light interested pixel point set is the pixel point set 1 |(50,50,50),d 2 |(200,150,200),d 3 | (50,80,150) }, it can be known that the pixel value isThe pixel point d can be known from the RGB (0, 0) to (255 ) which is the black-to-white process, i.e. the dark-to-bright process 1 Comparison with d 2 And d 3 The darkness, the most corresponding pixel point coordinate lesion is serious, and because the white light video image only shoots the surface of the lesion part and only can refer to the surface graph, but after replacement, a doctor can see the change of the inside of the lesion part to determine the depth of the operation, thereby not only facilitating the observation of the doctor, but also improving the operation effect and reducing the possibility of postoperative recurrence caused by the fact that part of the lesion area is not removed.
Step S1303, obtaining a non-white light interested pixel point set in the replaced white light pixel point set
According to the technical scheme provided by the invention, the non-white light interested pixel point set E in the white light pixel point set is obtained in advance, so that the subsequent distinction of the white light interested pixel point set D and the non-white light interested pixel point set E is facilitated, whether the distinction degree is enough or not is judged, the subsequent adjustment of the non-white light interested pixel point is facilitated if the distinction degree is insufficient, the non-white light interested pixel point set E belongs to the white light pixel point set, the non-white light interested pixel point set E and the white light interested pixel point set D are both sub-sets of the white light pixel point set B, and meanwhile, the non-white light interested pixel point set E and the white light interested pixel point set D are mutually exclusive, so that a doctor can conveniently distinguish a range area from a non-focus expectation by virtue of the doctor.
Step S1304, generating first pixel adjustment information according to the pixel value of each pixel in the white light interested pixel set and the pixel value of each pixel in the non-white light interested pixel set.
According to the technical scheme provided by the invention, the first reference pixel information can be obtained by averaging the pixel values of each pixel point in the white light interested pixel point set, the second reference pixel information can be obtained by averaging the pixel values of each pixel point in the non-white light interested pixel point set, the first pixel adjustment information is generated according to the absolute value of the difference between the first reference pixel information and the second reference pixel information, whether the distinction degree is enough or not is judged according to the size of the adjustment pixel information, and the larger the distinction degree of the adjustment pixel information is, the smaller the distinction degree of the adjustment pixel information is, the distinction degree is distinguished through the first pixel adjustment information, the definition is facilitated by a doctor, and the postoperative rehabilitation of a patient is facilitated.
In one possible implementation manner, the step S1304 specifically includes:
and generating first reference pixel information according to the pixel value of each pixel point in the white light interested pixel point set. According to the technical scheme provided by the invention, summation and average are carried out according to each pixel value in the white light interested pixel point set D, and the first reference pixel information is obtained through adjustment of the first pixel reference weight value, so that the pixel values are unified for facilitating comparison of the pixel values.
And generating second reference pixel information according to the pixel value of each pixel point in the non-white light interesting pixel point set. According to the technical scheme provided by the invention, summation and average are carried out according to each pixel value in the non-white light interested pixel point set E, and second reference pixel information is obtained through second pixel reference weight value adjustment, so that the pixel values are unified for convenience and comparison with the white light interested pixel point set D.
And calculating the first reference pixel information and the second reference pixel information to obtain first pixel adjustment information. According to the technical scheme provided by the invention, the absolute value is obtained by carrying out difference calculation on the first reference pixel information and the second reference pixel information, the first pixel adjustment information is obtained through the pixel deviation value weight, the adjustment information at the moment is equivalent to the degree of distinguishing between the white light region of interest and the non-white light region of interest, the larger the value of the corresponding adjustment information is, the larger the degree of distinguishing between the white light region of interest and the non-white light region of interest is, the smaller the value of the adjustment information is, the smaller the degree of distinguishing between the white light region of interest and the non-white light region of interest is, and whether the degree of distinguishing between the focus region and the non-focus region meets the requirements of doctors is judged.
The first reference pixel information, the second reference pixel information and the first pixel adjustment information are calculated by the following formulas,
wherein p is Base 1 For the first reference pixel information,r value of pixel value RGB of ith pixel point in white light interested pixel point set,/and/or>B value of pixel value RGB of ith pixel point in white light interested pixel point set,/and/or>Blue value, n, of pixel value RGB for the ith pixel in the set of pixel points of interest for white light 1 Reference weight value for first pixel, q Base 2 For the second reference pixel information +.>R value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>B value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>Blue value, n, of pixel value RGB for the ith pixel in the set of non-white light pixels of interest 2 For the reference weight value of the second pixel, i is the ith pixel point, n is the upper limit of the pixel point, and k 1 For the pixel deviation value weight, W is the first pixel adjustment information,/>is the sum of pixel values of the pixel points of interest for white light,/for>Is the sum of the pixel values of the non-white light pixel of interest.
According to the technical scheme provided by the invention, the average pixel point value in the white light interested pixel point set and the average pixel point value in the non-white light interested pixel point set are calculated and adjusted through the pixel reference weight value, so that the difference value |p between the first reference pixel information and the second reference pixel information is obtained by comparing the two reference information Base 1 -q Base 2 |*k 1 It can be appreciated that if |p Base 1 -q Base 2 |*k 1 The larger the difference value is, the larger the corresponding first pixel adjustment information W is, the larger the difference between the pixel value of the white light pixel point set of interest and the pixel value of the non-white light pixel point set of interest is, the smaller the corresponding pixel value of the white light pixel point set of interest and the pixel value of the non-white light pixel point set of interest are, and at the moment, the pixel value of the white light pixel point set of interest needs to be adjusted, so that the adjusted picture meets the requirement; by a first pixel reference weight value n 1 Weighting and normalizing the pixel difference value, and passing through a second pixel reference weight value n 2 And weighting and normalizing the pixel difference value.
And step S1305, processing the white light interested pixel points according to the first pixel adjustment information to obtain a replaced white light pixel point set. According to the technical scheme provided by the invention, the pixel value of the white light pixel of interest is processed according to the size of the first pixel adjustment information W, if the value of the first pixel adjustment information W is larger than the preset adjustment information value, the distinguishing degree at the moment is proved to meet the requirement, the pixel value of the white light pixel of interest is directly displayed, if the value of the first pixel adjustment information W is smaller than the preset adjustment information value, the distinguishing degree is proved to be unsatisfactory, the pixel value of the white light pixel of interest is required to be adjusted, the distinguishing degree is obvious, the doctor can observe conveniently, and the operation is better performed.
In one possible implementation manner, in the step of processing the white light interested pixel point according to the first pixel adjustment information to obtain the replaced white light pixel point set, the technical scheme provided by the invention specifically includes:
if the first pixel adjustment information is larger than the preset adjustment information, directly taking the pixel value of the pixel point of interest in white light as the pixel value of the corresponding pixel point in the set of white light pixel points after replacement. According to the technical scheme provided by the invention, if the first pixel adjustment information W is larger than the preset adjustment information value, for example: the preset adjustment information value is set to 60 according to the standard, and the corresponding pixel adjustment information W needs to be greater than 60, such as 61, 24, 65, 80, etc., without limitation, and the pixel value of the white light interested pixel point is not required to be operated to directly display if the differentiation requirement is met.
And if the first pixel adjustment information is smaller than the preset adjustment information, comparing the second reference pixel information with the preset standard information to obtain first conversion trend information. According to the technical scheme provided by the invention, if the first pixel adjustment information W is smaller than the preset adjustment information, for example: setting the preset adjustment information value to 60 according to the standard, if the corresponding pixel adjustment information W is smaller than 60, such as 59, 30, 28, etc., but not limited to this, it is explained that the degree of distinction between the pixel value of the white light pixel of interest and the pixel value of the non-white light pixel of interest is insufficient, the pixel value of the white light pixel of interest needs to be adjusted to raise the distinction degree, but the corresponding increase or decrease is not practical if the pixel value is already large, such as the pixel value RGB is (240,240,240), and further increase is not practical at this time, so an intermediate reference value is needed as a reference for adjustment, the second reference pixel information is compared with the preset standard information to obtain the first transformation trend information, the value of the preset standard information is the intermediate value of the pixel value, the intermediate value of the pixel value is (128,128,128), and the comparison of the second reference pixel information with the preset standard information can be known at this time If the pixel value is dark or light, the first conversion trend information is negative, if the pixel value is dark, the first conversion trend information is positive, for example, the value of the second reference pixel information is 0, the value of RGB corresponding to the value of RGB is (0, 0), and the RGB value (128,128,128) corresponding to the preset standard information value is compared withThe first transformation trend information is negative, and it can be known that the pixel value of the white light interested pixel point is close to the pixel value of the non-white light interested pixel point, and the pixel value of the image is dark, so that the pixel value in the set needs to be increased, if the value exceeds 255, the default is 255, and if the value of the second reference pixel information is 255, the corresponding pixel value of the image is bright, the pixel value in the set needs to be decreased, and if the value is less than 0, the default is 0.
And adjusting the white light interested pixel points according to the first conversion trend information and the first pixel adjustment information to obtain a replaced white light pixel point set. According to the technical scheme provided by the invention, the pixel value of the white light pixel point set of interest is adjusted according to the positive and negative values of the first conversion trend information and the first pixel adjustment information, so that the pixel value of the white light pixel point of interest and the pixel value of the non-white light pixel point of interest have obvious distinguishing degree, and the replaced white light pixel point set is obtained, for example: and if the first conversion trend information is a positive value, performing a reduction operation on the pixel value of the white light pixel of interest by using the first pixel adjustment information, and if the first conversion trend information is a negative value, performing an increase operation on the pixel value of the white light pixel of interest by using the first pixel adjustment information.
In one possible implementation manner, in the step of adjusting the white light interested pixel point according to the first conversion trend information and the first pixel adjustment information to obtain the replaced white light pixel point set, the method specifically includes:
if the first transformation trend information is positive, the pixel value of each pixel point in the white light interested pixel point set is reduced, the specific formula is as follows,
if the first transformation trend information is negative, the pixel value of each pixel point in the white light interested pixel point set is increased, the specific formula is as follows,
wherein I is 1 Reducing correction value for pixel value, I 2 Adding correction value, k, to pixel value r K is the correction value of R value of pixel value RGB g Correction value k for G value of pixel value RGB b A correction value for the B value of the pixel value RGB,for the R value of the pixel value RGB of the ith pixel point in the reduced white light interesting pixel point set,/-degree>G value of pixel value RGB of ith pixel point in reduced white light interest pixel point set,/-degree>For the reduced B value of the pixel value RGB of the ith pixel point in the white light pixel point set of interest,/->To increase the R value of the pixel value RGB of the i-th pixel in the white light pixel of interest set, G value of pixel value RGB of ith pixel point in increased white light interest pixel point set is +.>And B value of pixel value RGB of the ith pixel point in the increased white light interested pixel point set. According to the technical scheme provided by the invention, the pixel value of the white light interested pixel point is adjusted according to the first conversion trend information and the first pixel adjustment information, for example: under the condition that the degree of distinction is not obvious, the first conversion trend information is that the non-white light interested pixel point set area is too bright and the pixel values of the white light interested pixel point set are too bright and cannot be distinguished, the pixel values of the white light interested pixel point set are reduced, and the pixel values are reduced by-> Correcting by the correction value to obtain a corresponding reduced white light interested pixel point set, and obtaining a replaced white light pixel point set; under the condition that the degree of distinction is not obvious, the first conversion trend information is used for negatively indicating that the non-white light interested pixel point set area is dark, and the pixel value of the white light interested pixel point set is dark and cannot be distinguished, and the pixel value of the white light interested pixel point set is enlarged by> And correcting by the correction value to obtain a corresponding darkened white light interested pixel point set to obtain a replaced white light pixel point set.
In one possible implementation manner, the technical scheme provided by the invention further comprises:
and according to the second pixel adjustment information actively input by the doctor, adjusting the pixel value of each pixel in the white light interested pixel set to obtain an adjusted white light interested pixel set. According to the technical scheme provided by the invention, after the distinguishing degree of the white light interested pixel point set and the non-white light interested pixel point set is displayed according to the first pixel adjustment information, a doctor can check the fused image, so that the doctor can possibly perform heightening or lowering processing on the first pixel adjustment information according to actual conditions, the doctor can actively input second pixel adjustment information, the second pixel adjustment information can be set through an input device, the processor can change the adjustment information from the first pixel adjustment information to the second pixel adjustment information to perform distinguishing comparison, and the pixel deviation value weight is corrected, so that the corrected pixel deviation value weight is more suitable for personal habits of the doctor or more suitable for actual application scenes.
And obtaining a pixel adjustment trend according to the white light interesting pixel point set before adjustment and the white light interesting pixel point set before adjustment.
And correcting the pixel deviation value weight according to the pixel adjustment trend. According to the technical scheme provided by the invention, the pixel deviation value weight is corrected according to the pixel adjustment trend, so that the next time the doctor uses the equipment, the personal use habit of the doctor is met.
The corrected pixel deviation value weight is obtained by the following formula,
wherein W is 2 Adjusting information, k, for the second pixel 2 For corrected pixel deviation value weight, U 1 Increasing trend correction value for pixel adjustment information, U 2 The trend correction value is reduced for the pixel adjustment information. According to the technical scheme provided by the invention, when a doctor actively inputs second pixel adjustment information, the first pixel adjustment information which is output at the beginning is defaulted to have a certain difference from the ideal distinction degree required by the doctor, so that the first pixel adjustment information is compared with the second pixel adjustment information which is actively input by the doctor, and the pixel deviation value weight is corrected according to the comparison condition of the first pixel adjustment information and the second pixel adjustment information which is actively input by the doctor, so that the pixel deviation value weight is corrected after correctionThe pixel deviation value weight of the system is more suitable for personal habit of doctors, and is more convenient for the doctors to distinguish, and operation and diagnosis are more convenient to carry out.
The technical proposal provided by the invention is thatWhen the second pixel adjustment information is larger than the first pixel adjustment information, the first pixel adjustment information calculated and output by the method is proved to be smaller, so that the pixel deviation value weight k is needed 1 Carry out enlargement treatment->Obtaining the pixel deviation value weight k after increasing and correcting 2 . The invention is through->When the second pixel adjustment information is smaller than the first pixel adjustment information, the first pixel adjustment information calculated and output by the method is proved to be larger, so that the pixel deviation value weight k is needed 1 Reduced treatment->Obtaining a reduced corrected pixel deviation value weight k 2 Corresponding to the corrected pixel deviation value weight k 2 More accords with the actual diagnosis condition and the requirements and the use habits of the doctor. Through the mode, the formula for calculating the first pixel adjustment information provided by the invention can be continuously trained, so that the calculation accuracy is improved, and the use requirement of a current scene is met.
And step 140, generating a fusion video image according to the replaced white light pixel point set. According to the technical scheme provided by the invention, the replaced white light interested pixel point set is the area with the pixel value change in the final lesion area, and under the condition that the lesion area and the non-lesion area have a certain degree of identification, the white light interested pixel point set and the first white light image are fused, so that a doctor can observe more conveniently.
In one possible implementation manner, before the steps of obtaining the first fluorescence video image and the first white light video image of the focus under the irradiation of fluorescence and white light respectively, the technical scheme provided by the invention specifically comprises the following steps:
and (3) calibrating a preset origin of coordinates, randomly extracting N pixel points in the fluorescent pixel point set, and if the abscissa value and the ordinate value of the N pixel points are equal to the abscissa value and the ordinate value in the corresponding white pixel point set, completing the calibration. According to the technical scheme provided by the invention, because the imaging positions of the first fluorescent video image and the first white light video image are the same, after a coordinate system is established through the preset origin of coordinates, in order to prevent lens damage and the like caused by movement of equipment or overlong service time, the first fluorescent video image and the first white light video image also need to be calibrated after the coordinates are established.
If the abscissa and the ordinate value of a certain pixel point in the N pixel points are not equal to the abscissa and the ordinate value in the corresponding white light pixel point set, the calibration fails, and the equipment needs to be debugged. According to the technical scheme provided by the invention, if the coordinates of one pixel point in the randomly extracted pixel points are not corresponding, equipment debugging is needed, so that the problem that the final focus area cannot be accurately displayed due to the initial equipment factors can be prevented.
The embodiment of the invention also provides a visual field adjustable dual video fusion imaging system, as shown in a structural schematic diagram in fig. 5, comprising:
the acquisition module is used for acquiring a focus first fluorescent video image and a focus first white light video image under the irradiation of fluorescence and white light respectively, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set;
the extraction module is used for extracting pixel points positioned in a first preset pixel interval in the fluorescent pixel point set to generate a fluorescent pixel point set of interest;
the replacing module is used for comparing the fluorescence interest pixel point set with the white light pixel point set to determine the white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set;
and the fusion module is used for generating a fusion video image according to the replaced white light pixel point set.
Referring to fig. 5, an embodiment of the present invention further provides a hardware architecture diagram of a field-of-view adjustable dual video fusion imaging apparatus, where the field-of-view adjustable dual video fusion imaging apparatus 20 includes: a processor 21, a memory 22 and a computer program stored in the memory, the processor running the computer program to perform the first aspect of the invention and the methods to which the first aspect may relate; wherein the method comprises the steps of
A memory 22 for storing said computer program, which memory may also be a flash memory (flash). Such as application programs, functional modules, etc. implementing the methods described above.
And a processor 21 for executing the computer program stored in the memory to implement the steps executed by the apparatus in the above method. Reference may be made in particular to the description of the embodiments of the method described above.
Alternatively, the memory 22 may be separate or integrated with the processor 21.
When the memory 22 is a device separate from the processor 21, the apparatus may further include:
a bus 23 for connecting the memory 22 and the processor 21.
The present invention also provides a readable storage medium having stored therein a computer program for implementing the methods provided by the various embodiments described above when executed by a processor.
The readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media can be any available media that can be accessed by a general purpose or special purpose computer. For example, a readable storage medium is coupled to the processor such that the processor can read information from, and write information to, the readable storage medium. In the alternative, the readable storage medium may be integral to the processor. The processor and the readable storage medium may reside in an application specific integrated circuit (Application Specific Integrated Circuits, ASIC for short). In addition, the ASIC may reside in a user device. The processor and the readable storage medium may reside as discrete components in a communication device. The readable storage medium may be read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tape, floppy disk, optical data storage device, etc.
The present invention also provides a program product comprising execution instructions stored in a readable storage medium. The at least one processor of the device may read the execution instructions from the readable storage medium, the execution instructions being executed by the at least one processor to cause the device to implement the methods provided by the various embodiments described above.
In the above embodiment of the apparatus, it should be understood that the processor may be a central processing unit (english: central Processing Unit, abbreviated as CPU), or may be other general purpose processors, digital signal processors (english: digital Signal Processor, abbreviated as DSP), application specific integrated circuits (english: application Specific Integrated Circuit, abbreviated as ASIC), or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (6)

1. The visual field adjustable double-video fusion imaging method is characterized in that after a user adjusts the visual field of an image, double-video fusion imaging is performed through the following steps:
respectively obtaining a focus first fluorescent video image and a focus first white light video image under fluorescent light and white light irradiation, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set;
extracting pixel points in the first preset pixel interval in the fluorescence pixel point set to generate a fluorescence interested pixel point set;
comparing the fluorescence interest pixel point set with the white light pixel point set to determine a white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set;
generating a fusion video image according to the replaced white light pixel point set;
the method specifically comprises the steps of respectively obtaining a focus first fluorescent video image and a focus first white light video image under fluorescent light and white light irradiation, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set, wherein the method specifically comprises the following steps of:
Selecting a first pixel point with the same first fluorescent image and first white light image as a preset origin of coordinates;
decomposing the first fluorescent image according to a preset origin of coordinates to obtain each pixel pointCoordinates of nth fluorescent pixel point in pixel point set, < ->Is the abscissa of the nth fluorescent pixel,>is the ordinate of the nth fluorescent pixel point;
decomposing the first white light image according to a preset origin of coordinates to obtain each pixel pointN-th white light pixel point coordinates in pixel point set, < >>Is the abscissa of the nth white light pixel,>is the ordinate of the nth white light pixel point;
the step of extracting the pixel points in the first preset pixel interval in the fluorescence pixel point set and generating the fluorescence pixel point set of interest specifically comprises the following steps:
traversing a fluorescent pixel value of each fluorescent pixel point in a fluorescent pixel point set, and extracting the fluorescent pixel point if the fluorescent pixel value is positioned in the first preset pixel interval;
coordinates of nth fluorescence interest pixel point in interest pixel point set, +.>For the abscissa of the nth fluorescence pixel of interest, +.>Ordinate of pixel point of interest for nth fluorescence；
In the step of comparing the fluorescence interest pixel point set with the white light pixel point set to determine the white light interest pixel point set, replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain the replaced white light pixel point set, the method specifically comprises the following steps:
Coordinate of each pixel in the fluorescence interest pixel set and the white light pixel setN-th white light pixel point coordinate of interest in pixel point set of interest, +.>The abscissa of the pixel of interest for the nth white light, +.>An ordinate of an nth white light pixel of interest;
replacing the pixel value of the pixel point with the same coordinate in the white light pixel point set to obtain a replaced white light pixel point set;
coordinates of an nth fluorescence interest pixel point in the light interest pixel point set are +.>Is the abscissa of the nth non-white light pixel of interest,is the ordinate of the nth non-white light pixel of interest;
generating first pixel adjustment information according to the pixel value of each pixel in the replaced white light interested pixel set and the pixel value of each pixel in the non-white light interested pixel set;
processing the replaced white light interested pixel point set according to the first pixel adjustment information to obtain a replaced white light pixel point set;
the step of generating the first pixel adjustment information according to the pixel value of each pixel in the replaced white light interested pixel point set and the pixel value of each pixel in the non-white light interested pixel point set specifically includes:
Generating first reference pixel information according to the pixel value of each pixel point in the replaced white light interested pixel point set;
generating second reference pixel information according to the pixel value of each pixel point in the non-white light interesting pixel point set;
calculating the first reference pixel information and the second reference pixel information to obtain first pixel adjustment information;
the first reference pixel information, the second reference pixel information and the first pixel adjustment information are calculated by the following formulas,
wherein p is Base 1 For the first reference pixel information,r value of pixel value RGB of ith pixel point in the replaced white light interested pixel point set is +.>G value of pixel value RGB of ith pixel point in the replaced white light interested pixel point set is +.>For the B value, n of the pixel value RGB of the ith pixel point in the replaced white light interested pixel point set 1 Reference weight value for first pixel, q Base 2 For the second reference pixel information +.>R value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>G value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>B value, n of pixel value RGB of ith pixel point in non-white light interested pixel point set 2 For the reference weight value of the second pixel, i is the ith pixel point, n is the upper limit of the pixel point, and k 1 For the pixel deviation value weight, W is the first pixel adjustment information,is the sum of pixel values of the pixel points of interest for white light,/for>Is the sum of the pixel values of the non-white light pixel of interest.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the step of processing the replaced white light interested pixel point set according to the first pixel adjustment information to obtain a replaced white light pixel point set specifically includes:
if the first pixel adjustment information is larger than the preset adjustment information, directly taking the pixel value of the pixel point of interest of the white light as the pixel value of the corresponding pixel point in the set of the white light pixel points after replacement;
if the first pixel adjustment information is smaller than the preset adjustment information, comparing the second reference pixel information with the preset standard information to obtain first conversion trend information;
and adjusting the white light interested pixel points according to the first conversion trend information and the first pixel adjustment information to obtain a replaced white light pixel point set.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
The step of adjusting the white light interested pixel point according to the first conversion trend information and the first pixel adjustment information to obtain a replaced white light pixel point set specifically includes:
if the first transformation trend information is positive, the pixel value of each pixel point in the white light interested pixel point set is reduced, the specific formula is as follows,
if the first transformation trend information is negative, the pixel value of each pixel point in the white light interested pixel point set is increased, the specific formula is as follows,
wherein I is 1 Reducing correction value for pixel value, I 2 Adding correction value, k, to pixel value r K is the correction value of R value of pixel value RGB g Correction value k for G value of pixel value RGB b A correction value for the B value of the pixel value RGB,for the R value of the pixel value RGB of the ith pixel point in the reduced white light interesting pixel point set,/-degree>G value of pixel value RGB of ith pixel point in reduced white light interest pixel point set,/-degree>For the reduced B value of the pixel value RGB of the ith pixel point in the white light pixel point set of interest,/->To increase the R value of the pixel value RGB of the i-th pixel in the white light pixel of interest set, G value of pixel value RGB of ith pixel point in increased white light interest pixel point set is +.>And B value of pixel value RGB of the ith pixel point in the increased white light interested pixel point set.
4. A method according to claim 3, further comprising:
according to the second pixel adjustment information actively input by the doctor, adjusting the pixel value of each pixel in the white light interested pixel set to obtain an adjusted white light interested pixel set;
obtaining a pixel adjustment trend according to the white light interesting pixel point set before adjustment and the white light interesting pixel point set after adjustment;
correcting the pixel deviation value weight according to the pixel adjustment trend;
the corrected pixel deviation value weight is obtained by the following formula,
wherein W is 2 Adjusting information, k, for the second pixel 2 For corrected pixel deviation value weight, U 1 Increasing trend correction value for first pixel adjustment information, U 2 The trend correction value is reduced for the first pixel adjustment information.
5. The method of claim 1, wherein the step of determining the position of the substrate comprises,
before the step of obtaining a first fluorescence video image and a first white light video image of a focus under the irradiation of fluorescence and white light respectively, the method specifically comprises the following steps:
Randomly extracting N pixel points in a fluorescent pixel point set, and if the horizontal coordinate value and the vertical coordinate value of the N pixel points are equal to the horizontal coordinate value and the vertical coordinate value in the corresponding white pixel point set, finishing the calibration;
if the abscissa and the ordinate value of a certain pixel point in the N pixel points are not equal to the abscissa and the ordinate value in the corresponding white light pixel point set, the calibration fails, and the equipment needs to be debugged.
6. A field-of-view adjustable dual video fusion imaging system, comprising:
the acquisition module is used for acquiring a focus first fluorescent video image and a focus first white light video image under the irradiation of fluorescence and white light respectively, and decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set;
the extraction module is used for extracting pixel points positioned in a first preset pixel interval in the fluorescent pixel point set to generate a fluorescent pixel point set of interest;
the replacing module is used for comparing the fluorescence interest pixel point set with the white light pixel point set to determine the white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set;
The fusion module is used for generating a fusion video image according to the replaced white light pixel point set;
respectively obtaining a focus first fluorescent video image and a first white light video image under fluorescent light and white light irradiation, decomposing the first fluorescent image and the first white light image based on a preset origin of coordinates to obtain a fluorescent pixel point set and a white light pixel point set, and specifically comprising the following steps:
selecting a first pixel point with the same first fluorescent image and first white light image as a preset origin of coordinates;
decomposing the first fluorescent image according to a preset origin of coordinates to obtain each pixel pointCoordinates of nth fluorescent pixel point in pixel point set, < ->Is the abscissa of the nth fluorescent pixel,>is the ordinate of the nth fluorescent pixel point;
decomposing the first white light image according to a preset origin of coordinates to obtain each pixel pointN-th white light pixel point coordinates in pixel point set, < >>Is the abscissa of the nth white light pixel,>is the ordinate of the nth white light pixel point;
extracting pixel points in the first preset pixel interval from the fluorescence pixel point set to generate a fluorescence interested pixel point set, wherein the method specifically comprises the following steps of:
traversing a fluorescent pixel value of each fluorescent pixel point in a fluorescent pixel point set, and extracting the fluorescent pixel point if the fluorescent pixel value is positioned in the first preset pixel interval;
The nth fluorescence interest pixel point coordinate in the interest pixel point set,for the abscissa of the nth fluorescence pixel of interest, +.>Is the ordinate of the nth fluorescence interest pixel point;
comparing the fluorescence interest pixel point set with the white light pixel point set to determine a white light interest pixel point set, and replacing the pixel value of the pixel point in the white light interest pixel point set with the pixel value of the corresponding pixel point in the fluorescence interest pixel point set to obtain a replaced white light pixel point set, wherein the method specifically comprises the following steps:
coordinate of each pixel in the fluorescence interest pixel set and the white light pixel set
The nth white light pixel point of interest coordinates in the pixel point of interest set,the abscissa of the pixel of interest for the nth white light, +.>An ordinate of an nth white light pixel of interest;
replacing the pixel value of the pixel point with the same coordinate in the white light pixel point set to obtain a replaced white light pixel point set;
coordinates of an nth fluorescence interest pixel point in the light interest pixel point set are +.>Is the abscissa of the nth non-white light pixel of interest, Is the ordinate of the nth non-white light pixel of interest;
generating first pixel adjustment information according to the pixel value of each pixel in the replaced white light interested pixel set and the pixel value of each pixel in the non-white light interested pixel set;
processing the replaced white light interested pixel point set according to the first pixel adjustment information to obtain a replaced white light pixel point set;
generating first pixel adjustment information according to the pixel value of each pixel in the replaced white light interested pixel set and the pixel value of each pixel in the non-white light interested pixel set specifically includes:
generating first reference pixel information according to the pixel value of each pixel point in the replaced white light interested pixel point set;
generating second reference pixel information according to the pixel value of each pixel point in the non-white light interesting pixel point set;
calculating the first reference pixel information and the second reference pixel information to obtain first pixel adjustment information;
the first reference pixel information, the second reference pixel information and the first pixel adjustment information are calculated by the following formulas,
Wherein p is Base 1 For the first reference pixel information,r value of pixel value RGB of ith pixel point in the replaced white light interested pixel point set is +.>G value of pixel value RGB of ith pixel point in the replaced white light interested pixel point set is +.>For the B value, n of the pixel value RGB of the ith pixel point in the replaced white light interested pixel point set 1 Reference weight value for first pixel, q Base 2 For the second reference pixel information +.>R value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>G value of pixel value RGB of ith pixel point in non-white light interested pixel point set,/and/or>B value, n of pixel value RGB of ith pixel point in non-white light interested pixel point set 2 Is the firstThe sum of the pixel values of the pixel of interest.
CN202210313868.1A 2022-03-28 2022-03-28 Visual field adjustable double-video fusion imaging method and system Active CN115719415B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210313868.1A CN115719415B (en) 2022-03-28 2022-03-28 Visual field adjustable double-video fusion imaging method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210313868.1A CN115719415B (en) 2022-03-28 2022-03-28 Visual field adjustable double-video fusion imaging method and system

Publications (2)

Publication Number Publication Date
CN115719415A CN115719415A (en) 2023-02-28
CN115719415B true CN115719415B (en) 2023-11-10

Family

ID=85253749

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210313868.1A Active CN115719415B (en) 2022-03-28 2022-03-28 Visual field adjustable double-video fusion imaging method and system

Country Status (1)

Country Link
CN (1) CN115719415B (en)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2793055A1 (en) * 1999-04-29 2000-11-03 Ge Medical Syst Sa METHOD AND SYSTEM FOR FUSIONING TWO DIGITAL RADIOGRAPHIC IMAGES
WO2013035450A1 (en) * 2011-09-08 2013-03-14 オリンパス株式会社 Fluorescence observation device
CN106303279A (en) * 2016-08-31 2017-01-04 北京数字精准医疗科技有限公司 Multi-spectrum endoscopic automatic exposure formation method
WO2020052626A1 (en) * 2018-09-12 2020-03-19 上海逸思医学影像设备有限公司 Image exposure-based multi-spectrum imaging system and method
CN111563552A (en) * 2020-05-06 2020-08-21 浙江大华技术股份有限公司 Image fusion method and related equipment and device
CN112037216A (en) * 2020-09-09 2020-12-04 南京诺源医疗器械有限公司 Image fusion method for medical fluorescence imaging system
CN112089403A (en) * 2020-10-02 2020-12-18 深圳市中安视达科技有限公司 Multispectral medical multi-path imaging method and system thereof
CN113208567A (en) * 2021-06-07 2021-08-06 上海微创医疗机器人（集团）股份有限公司 Multispectral imaging system, imaging method and storage medium
CN113496489A (en) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 Training method of endoscope image classification model, image classification method and device
CN113610823A (en) * 2021-08-13 2021-11-05 南京诺源医疗器械有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN113693724A (en) * 2021-08-19 2021-11-26 南京诺源医疗器械有限公司 Irradiation method, device and storage medium suitable for fluorescence image navigation operation
CN113842212A (en) * 2021-10-09 2021-12-28 南京诺源医疗器械有限公司 Fluorescence scattering optical tomography processing method and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10706512B2 (en) * 2017-03-07 2020-07-07 Adobe Inc. Preserving color in image brightness adjustment for exposure fusion

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2793055A1 (en) * 1999-04-29 2000-11-03 Ge Medical Syst Sa METHOD AND SYSTEM FOR FUSIONING TWO DIGITAL RADIOGRAPHIC IMAGES
WO2013035450A1 (en) * 2011-09-08 2013-03-14 オリンパス株式会社 Fluorescence observation device
CN106303279A (en) * 2016-08-31 2017-01-04 北京数字精准医疗科技有限公司 Multi-spectrum endoscopic automatic exposure formation method
WO2020052626A1 (en) * 2018-09-12 2020-03-19 上海逸思医学影像设备有限公司 Image exposure-based multi-spectrum imaging system and method
CN111563552A (en) * 2020-05-06 2020-08-21 浙江大华技术股份有限公司 Image fusion method and related equipment and device
CN112037216A (en) * 2020-09-09 2020-12-04 南京诺源医疗器械有限公司 Image fusion method for medical fluorescence imaging system
CN112089403A (en) * 2020-10-02 2020-12-18 深圳市中安视达科技有限公司 Multispectral medical multi-path imaging method and system thereof
CN113208567A (en) * 2021-06-07 2021-08-06 上海微创医疗机器人（集团）股份有限公司 Multispectral imaging system, imaging method and storage medium
CN113610823A (en) * 2021-08-13 2021-11-05 南京诺源医疗器械有限公司 Image processing method, image processing device, electronic equipment and storage medium
CN113693724A (en) * 2021-08-19 2021-11-26 南京诺源医疗器械有限公司 Irradiation method, device and storage medium suitable for fluorescence image navigation operation
CN113496489A (en) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 Training method of endoscope image classification model, image classification method and device
CN113842212A (en) * 2021-10-09 2021-12-28 南京诺源医疗器械有限公司 Fluorescence scattering optical tomography processing method and system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Oral Cancer Detection in Fluorescent Image by Color Image Fusion;C. F. Jiang等;《Proceedings of the 26th Annual International Conference of the IEEE EMBS 》;第1260-1262页 *
基于双树复小波变换和频域U-Net 的多光谱图像融合算法;李建飞等;《生物医学工程研究》;第39卷(第2期);第145-150页 *
肿瘤分子影像学：进展及挑战;卢光明;《国际医学放射学杂志》;第44卷(第1期);第1-5页 *

Also Published As

Publication number Publication date
CN115719415A (en) 2023-02-28

Similar Documents

Publication Publication Date Title
US11276166B2 (en) Systems and methods for patient structure estimation during medical imaging
WO2021031458A1 (en) Method and device for image color correction applicable in endoscope, and storage medium
US20150145966A1 (en) 3d corrected imaging
US20110123074A1 (en) Systems and methods for suppressing artificial objects in medical images
US20220012915A1 (en) Apparatuses, systems, and methods for managing auto-exposure of image frames depicting signal content against a darkened background
Richa et al. Fundus image mosaicking for information augmentation in computer-assisted slit-lamp imaging
WO2014208287A1 (en) Detection device, learning device, detection method, learning method, and program
CN110782400A (en) Self-adaptive uniform illumination realization method and device
US20080165247A1 (en) Image processing apparatus and method
CN115719415B (en) Visual field adjustable double-video fusion imaging method and system
JP3931792B2 (en) Time-series processed image display device and display method
WO2015037316A1 (en) Organ-imaging device and organ-imaging method
CN109919882B (en) Image optimization method based on fundus color photograph image and related equipment
CN117314872A (en) Intelligent segmentation method and device for retina image
US11936989B2 (en) Apparatuses, systems, and methods for gaze-based auto-exposure management of image frames
US20230255443A1 (en) Apparatuses, systems, and methods for discounting an object while managing auto-exposure of image frames depicting the object
CN115861147A (en) Endoscope dark area enhancing method and device, electronic equipment and storage medium
CN116402735A (en) Endoscope image reconstruction method based on multidirectional visual angle calibration
CN108305252B (en) Image fusion method for portable electronic endoscope
CN116271386A (en) Visual double-cavity trachea cannula with imaging enhancement function and imaging enhancement method thereof
Zhang et al. A fundus image enhancer based on illumination-guided attention and optic disc perception GAN
CN114125225B (en) Endoscope brightness automatic adjustment method, device and system and electronic equipment
CN111652805B (en) Image preprocessing method for fundus image stitching
CN109949301B (en) Wireless endoscope camera shooting illumination system
Abel et al. Automatic glare removal in endoscopic imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant