WO2020215361A1 - 一种基于改进的线扫描成像***的稳像方法及*** - Google Patents
一种基于改进的线扫描成像***的稳像方法及*** Download PDFInfo
- Publication number
- WO2020215361A1 WO2020215361A1 PCT/CN2019/085503 CN2019085503W WO2020215361A1 WO 2020215361 A1 WO2020215361 A1 WO 2020215361A1 CN 2019085503 W CN2019085503 W CN 2019085503W WO 2020215361 A1 WO2020215361 A1 WO 2020215361A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mirror
- fundus
- scanning
- lso
- line scan
- Prior art date
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/12—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for looking at the eye fundus, e.g. ophthalmoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/14—Arrangements specially adapted for eye photography
Definitions
- the invention relates to laser fundus target tracking and imaging technology, in particular to an image stabilization method and system based on an improved line scan imaging system.
- the existing target tracking technology based on the Line Scan Ophthalmoscope (LSO) imaging system uses one frame of image as a unit to calculate the amount of movement of the fundus target.
- LSO Line Scan Ophthalmoscope
- the control system has a time delay of at least one frame, which results in a decrease in tracking accuracy.
- the target tracking signal method of the existing LSO is completely digital. When extracting the signal from the image, the lack of optical closed-loop control measures inside the LSO will also cause the calculation of the fundus movement signal to be unreliable.
- the main purpose of the present invention is to provide an image stabilization method and system based on an improved line scan imaging system, which aims to overcome the inherent optical and control defects of the existing LSO imaging system and greatly improve its clinical application Stability, accuracy, and imaging efficiency in the medium.
- An image stabilization system based on an improved line scan imaging system, including a line scan fundus camera LSO imaging system.
- the LSO imaging system also includes an orthogonal galvanometer scanning device for generating an orthogonal scan surface on the fundus.
- the scanning surface is adjusted to any position in the 360-degree space; and used to combine with the intelligent control algorithm to realize the LSO optical system to perform two-dimensional fundus scanning while simultaneously performing the fundus optical tracking inside the LSO.
- the orthogonal galvanometer scanning device is a dual-mirror structure composed of a first mirror SM11 and a second mirror SM12, or a MEMS scanning mirror of a micro-electromechanical system that vibrates orthogonally and bidirectionally, or other orthogonal bidirectional vibrating mirrors composition.
- the orthogonal galvanometer scanning device combined with an intelligent control algorithm realizes that the LSO optical system performs fundus optical tracking inside the LSO while scanning the two-dimensional fundus.
- the image stabilization system has the following relationship:
- (x t , y t ) represents the control commands on the first mirror SM11 and the second mirror SM12 at the current sampling moment, which are equivalent to their respective motion offsets;
- ( ⁇ x t , ⁇ y t ) represents the slave The relative motion of the image target frame and the reference frame recorded by the line scan camera;
- g represents the gain of the closed-loop control system;
- (x t+1 ,y t+1 ) represents the existing signal applied to the first mirror SM11 and the second reflection
- the next set of new commands of the mirror SM12 is equivalent to the movement offset.
- An image stabilization method based on an improved line scan imaging system includes the following steps:
- An orthogonal galvanometer scanning device is added to the online scanning fundus camera LSO imaging system.
- the orthogonal galvanometer scanning device is used to generate an orthogonal scanning surface on the fundus, so that the scanning surface can be adjusted to any position in a 360-degree space ;
- An image stabilization system based on an improved line scan imaging system, further comprising a rotating device for setting the cylindrical mirror L13 that generates the line light source and the line scan camera coupled with it on a 360-degree space controllable rotating bracket, Make the line controllable light source rotate at any position in 360-degree space.
- the sawtooth wave is used as the driving signal base of the first mirror SM11 and the second mirror SM12, and the rotation angle is set to ⁇ , and the amplitude due to each mirror is multiplied by the respective base signal, then Equation (1) can be updated as:
- ⁇ t is the angle imposed by the closed-loop control system on the rotating support;
- (x t ,y t ) is the translational amount imposed on the first mirror SM11 and the second mirror SM12, (x t ,y t ) It is still superimposed on the translation amount of each mirror used to generate the scanning signal;
- (x t , y t , ⁇ t ) is the current sampling time on the first mirror SM11 and the second mirror SM12 and the rotating bracket Control instructions, which are equivalent to their respective movement offsets and rotation angles;
- ( ⁇ x t , ⁇ y t , ⁇ t ) is the relative movement of the image target frame and the reference frame recorded from the line scan camera;
- g is the closed-loop control system
- (X t+1 ,y t+1 , ⁇ t+1 ) is the current signal applied to the first mirror SM11 and the second mirror SM12 as well as the cylindrical mirror L13 and the line scan camera coupled
- An image stabilization method of an image stabilization system based on an improved line scan imaging system includes the following steps:
- An orthogonal galvanometer scanning device is added to the online scanning fundus camera LSO imaging system.
- the orthogonal galvanometer scanning device is used to generate an orthogonal scanning surface on the fundus, so that the scanning surface can be adjusted to any position in a 360-degree space ;
- the present invention is based on an improved image stabilization method and system of a line scan imaging system, and has the following beneficial effects:
- the closed-loop tracking control system is used to control the line scan imaging system, so as to achieve the purpose of high-speed, stable and precise control.
- the LSO closed-loop control system can be used to obtain the fundus movement signal, and through the pre-calibrated spatial transformation relationship, to control another one or more optical systems to achieve the corresponding fundus target tracking purpose.
- each scan line arrives at the host system in each frame of the image
- a frame of image is divided into multiple sub-frame elements in chronological order, and each sub-frame element contains one or more scan lines .
- the fundus movement information contained in each sub-frame element is calculated in real time, and then immediately fed back to the tracking device, such as a high-speed tilt mirror and a rotating table. Using this frequency multiplication method can greatly improve the spatial accuracy and time bandwidth of target tracking.
- Figure 1 is a schematic diagram of the optical structure of a conventional line scan fundus camera
- Figure 2 is a schematic diagram of a sawtooth wave used to control the scanning mirror SM;
- FIG. 3 is a schematic diagram of a fundus image obtained according to the optical system of the line scan fundus camera shown in FIG. 1;
- Figure 4 is a schematic diagram of an existing line scan imaging system, including a main LSO imaging system without optical tracking integrated with a secondary OCT imaging system;
- Fig. 5 is a schematic diagram of calculating the amount of fundus movement from an image in a frame unit obtained by the image stabilization system based on the line scan imaging system of the present invention
- FIG. 6 is a schematic diagram of an improved LSO optical system with internal optical tracking according to an embodiment of the present invention.
- FIG. 7 is a schematic diagram of the working state of two inclined mirrors SM11 and SM12 in the improved LSO optical system shown in FIG. 6;
- FIG. 8 is a schematic diagram of adjusting the position of the imaging surface in a 360-degree space by changing the offset of the mirrors SM11 and SM12;
- FIG. 9 is a schematic diagram of the position of the line light source generated by the rotating cylindrical lens L13 and the line scan camera coupled with the rotating device in a 360-degree space;
- FIG. 10 is a schematic diagram of a state where a rotating cylindrical mirror generates a line light source with an arbitrary rotation angle and a scanning surface related to it;
- FIG. 11 is a schematic diagram of the principle of integrating a main LSO imaging system with closed-loop optical tracking into another auxiliary imaging system according to an embodiment of the present invention
- FIG. 12 is a schematic diagram of using frequency doubling technology to reduce the time delay of fundus calculation in an embodiment of the present invention
- FIG. 13 is a schematic diagram of the scanning (reflecting) mirror SM11 scanning signal and the division mode of sub-frame elements
- Figure 14 is a schematic diagram of scanning signals and synchronization signals of a line scanning system
- Figure 15 is a signal that combines a line reference clock and a frame synchronization signal to trigger a line scan camera.
- FIG. 1 is a schematic diagram of the optical structure of a conventional line scan fundus camera.
- the light emitted from the point light source L11 is collimated by the lens L12, the surface light source is converted into a line light source through a cylinder lens (Cylinder Lens) L13, and then it is relayed to the collimating lens L14.
- the installation direction of the cylindrical lens L13 determines the expansion direction of the line light source in space (see Figures 9 and 10 for details), and the lens L12, lens L13, and lens L14 determine to a certain extent the line light source's illumination (expansion) in the fundus size.
- Part of the light emitted by the lens L14 passes through the beam splitter (BS) and reaches the scanning mirror (Steering Mirror or Scanning Mirror, SM); the other part passes through the beam splitter (BS) to the collimating lens L17, and then passes through a Group filter (FILTER), reach line scan camera (Line Scan Camera).
- BS beam splitter
- FILTER Group filter
- the function of the scanning mirror SM is to generate periodic scanning in the orthogonal direction of the line light source, and the light passes through two collimating zoom lenses L15 and L16 to generate a two-dimensional scanning space at the bottom of the eye (Eye).
- the motion track of the scanning mirror (SM) generally presents a sawtooth wave as shown in FIG. 2.
- Figure 2 is a schematic diagram of a sawtooth wave used to control a scanning mirror (SM).
- the frequency of the sawtooth wave determines the image frame rate of the imaging system, and the amplitude of the sawtooth wave determines the size of the optical field of view in the scanning direction.
- the center of the sawtooth wave is not always at the zero position of the sawtooth wave.
- the center offset of the sawtooth wave actually determines the center position of the scanning field of view.
- users are supported to control the center position of the scanning field of view by adjusting the center offset of the sawtooth wave.
- the returned signal is reflected from the beam splitter BS to the collimating lens L17 through the same optical path, and then passes through a set of filters (FILTER) to reach the line scan camera ( Line Scan Camera).
- FILTER filters
- the signal returned from the fundus may be a reflected signal, a fluorescent signal, or other signals; it may also be a variety of other signals that reach the line scan camera at the same time.
- FIG. 3 is a schematic diagram of a fundus image obtained according to the optical system of the line scan fundus camera shown in FIG. 1. That is, the schematic diagram of the fundus image obtained by the line scan fundus camera shown in FIG. 1.
- Fig. 4 is a system schematic diagram of an existing line scan imaging system, including a main LSO imaging system without optical image stabilization (or tracking) and an integrated auxiliary optical coherence tomography (OCT) imaging system.
- OCT optical coherence tomography
- the main LSO imaging system is the imaging system shown in Figure 1.
- the main LSO imaging system is used clinically, it will be equipped with an auxiliary imaging system customized according to the embodiment of the present invention, such as the OCT product Cirrus of Carl Zeiss.
- the auxiliary imaging system shown in FIG. 4 is an OCT Device.
- the light emitted from the second point light source L21 reaches the orthogonal scanning mirrors SM21 and SM22 through the collimating system (including the collimating lenses L22 and L23), and then it is focused to the splitting mirror through the focusing lens L24.
- the Dichotic Mirror DM
- the DM is also located on the focal plane of the main LSO imaging system.
- the main and auxiliary (integrated optical) imaging systems shown in FIG. 4 support simultaneous implementation of two-dimensional fundus reflection (or fluorescence) imaging and three-dimensional OCT tomographic imaging.
- the main LSO imaging system One function of the main LSO imaging system is to provide fundus positioning and navigation for the auxiliary imaging system, and to display the current OCT tomogram in the corresponding position of the fundus two-dimensional space to the current user. Another function of the main LSO imaging system is to calculate fundus/eye movement information (x, y, ⁇ ) obtained from the LSO image by executing a preset algorithm (see below for details). Among them, (x, y) is the amount of translation of fundus movement, and ⁇ is the amount of rotation.
- (x, y, ⁇ ) is applied to the scanning mirrors SM21 and SM22 of the auxiliary imaging system, and the corresponding spatial positions of the scanning mirrors SM21 and SM22 are adjusted in real time to obtain the tomographic image of the required fundus position.
- the acquisition of (x, y, ⁇ ) here is completely digital, and is not achieved through the optical closed loop described in equation (1) or equation (2).
- the main LSO system simply obtains an image similar to Fig. 3 and uses cross-correlation or similar algorithms to calculate digital fundus motion information (x, y, ⁇ ).
- the acquisition of (x, y, ⁇ ) is entirely digital, not through the optical closed loop described in equation (1) or equation (2).
- the fundus tracking only occurs on the scanning mirrors SM21 and SM22 of the auxiliary imaging system.
- the main LSO system does not adjust its own optical parameters to lock the LSO scanning (imaging) position of the fundus accordingly.
- the accuracy and reliability of the numerical calculation results (x, y, ⁇ ) here are largely dependent on various parameters, including fundus image quality, normal fundus movements including blinking and saccade ) And micro saccade.
- fundus image quality the amount of movement to be calculated
- normal fundus movements including blinking and saccade
- micro saccade the amount of eye movement
- FIG. 5 is a schematic diagram of an image obtained by calculating the amount of fundus movement from an image in a frame unit.
- f 1 is the first frame image captured by the LSO, and f 1 is used and defined as a "reference frame” image.
- the images obtained after the system are f 2 , f 3 , ..., f n , f n+1 , which are defined as "target frame” images.
- the algorithm program obtains (x k , y k , ⁇ k )
- it is immediately converted to the scanning mirrors SM21 and SM22 of the auxiliary imaging system through the predetermined spatial mapping relationship, so that the scanning mirrors SM21 and SM22 are locked in the required The position of the fundus scan.
- the prerequisite for applying the cross correlation algorithm (Cross Correlation) to calculate the amount of eye movement from the image is that the image is needed. As mentioned above, it takes 33-40 milliseconds to acquire a frame of image, plus the time of the algorithm, the (x k , y k , ⁇ k ) obtained from the algorithm is converted into the electronic delay of the control signals of the scanning mirrors SM21 and SM22, and then The scanning mirrors SM21 and SM22 themselves respond to the mechanical delay of the control signal. A complete control cycle, from the start of the eye movement to the scanning mirror SM21 and SM22 tracking the movement, the delay time reaches 40-50 milliseconds is a very common phenomenon. From the above analysis process, it can be seen that among all the factors that can cause delay, the (image) sampling delay of 33-40 milliseconds is usually the dominant latency.
- the (image) sampling delay of 33-40 milliseconds is usually the dominant latency.
- one way to shorten the above-mentioned time delay is to greatly increase the frame rate of image output, for example, LSO outputs 200 frames per second, so that the delay of image sampling can be reduced to 5 milliseconds.
- the side effect of the increase in the frame rate of the imaging system is the rapid increase in the nonlinearity of the imaging laser dose. This is not clinically feasible because the use of laser doses is restricted by safety standards.
- the existing LSO imaging systems have deficiencies in optics, electronics, and control, so that the present invention is capable of optics, electronics, software, and control based on the systems of Figures 1 and 4 Further improvements and enhancements.
- Fig. 6 is a schematic diagram of an improved LSO optical system according to an embodiment of the present invention.
- the two one-dimensional galvanometers (SM11 and SM12) in FIG. 6 can also use a Microelectro Mechanical Systems (MEMS) scanning mirror with orthogonal bidirectional vibration, or other orthogonal scanning reflectors. Mirror replacement.
- MEMS Microelectro Mechanical Systems
- Fig. 6 the difference from Fig. 1 is that a tilting (reflecting) mirror is added.
- the mirror (SM) in Fig. 1 is named the first mirror SM11, and the newly added mirror is named the second mirror SM12 .
- the working process of the mirrors SM11 and SM12 is shown in FIG. 7.
- the dual-mirror structure composed of the first mirror SM11 and the second mirror SM12, or a bidirectionally vibrating micro-electromechanical system (MEMS) scanning mirror, or other forms of orthogonal scanning mirror structures can be collectively referred to as orthogonal Galvo scanning device.
- MEMS micro-electromechanical system
- FIG. 7 is a schematic diagram of the working state of two inclined mirrors SM11 and SM12 in the improved LSO optical system shown in FIG. 6.
- a spatial reference coordinate (x, y, z), as shown in Figure 7A.
- SM1 mirror SM11
- FIG. 7B a line light source A is incident on the mirror SM11.
- the rotation axis of the mirror SM11 is on the x-axis of the spatial coordinates, so that the mirror SM11 swings on the y-z plane, and then a two-dimensional scanning surface is generated at the position B.
- the position of B is directly conjugated to the imaging surface of the fundus.
- a second tilting (reflecting) mirror SM12 is inserted at the position B in FIG. 7C.
- the rotation axis of the mirror SM12 is on the z-axis and swings in the x-y plane.
- the reference coordinates (x, y, z) in FIG. 7A can be defined arbitrarily, as long as the motion axis of the mirror SM11 and the motion axis of the mirror SM12 are orthogonal.
- the working mode of this double mirror can be realized by the double mirror structure shown in Figure 6, for example, two Cambridge Technology one-dimensional 6210H galvanometers or 6220H galvanometers, or one with two independent orthogonal motions It can be realized by the tilt mirror of the axis, such as PI's S-335.2SH fast tilt mirror (Tip/Tilt Mirror).
- Fig. 8 is a schematic diagram of adjusting the position of the imaging surface in a 360-degree space by changing the offset of the mirrors SM11 and SM12.
- the parameter of controlling the mirror SM12 in simple cases is a translation amount to adjust the horizontal position of the imaging surface, which can be used to track the target Movement in the horizontal direction.
- the mirror SM11 is used for scanning, and on the other hand, for vertical translation of the imaging surface or target tracking (refer to FIG. 2).
- the LSO optical system can realize the fundus optical tracking inside the LSO while scanning the two-dimensional fundus.
- the functions of the mirrors SM11 and SM12 combined with the intelligent control algorithm, the LSO optical system can realize the fundus optical tracking inside the LSO while scanning the two-dimensional fundus.
- Figure 6 constitutes a complete closed-loop control system.
- the light from the point light source L11 reaches the fundus through the mirrors SM11 and SM12, which is a two-dimensional scanning space.
- the signal returned from the space where the fundus is scanned is scanned by the mirrors SM11 and SM12 again to reach the photodetector, here is a Line Scan Camera, which is used to record the image signal returned from the fundus.
- Figure 6 of the present invention can constitute a complete closed-loop control system.
- (x t , y t ) represents the control commands on the mirrors SM11 and SM12 (equivalent to their respective movement offsets) at the current sampling time
- ( ⁇ x t , ⁇ y t ) represents the slave
- g represents the gain of the closed-loop control system
- (x t+1 ,y t+1 ) represents the existing signal applied to the mirror SM11
- the next set of new commands of SM12 (equivalent to motion offset).
- the motion signal from the fundus has been optically compensated by mirrors SM11 and SM12, so the motion signal obtained from the photodetector is always a residual motion signal, which is equation (1) ⁇ ( ⁇ x t , ⁇ y t ).
- the closed-loop control described above can also compensate for the rotation signal of the eyeball.
- One method is to install the cylindrical mirror L13 that produces the line light source in Figure 6 and the coupled line scan camera on a 360-degree controllable rotating bracket, so that the line-extended light source can be rotated at any position in the 360-degree space (reference Figure 9).
- FIG. 9 is a schematic diagram of the position of the line light source generated by the rotating cylindrical mirror L13 in the 360-degree space by the rotating device/mechanism.
- the axis of the cylindrical mirror and the coupled line scan camera are installed in a controllable rotating mechanism at the coordinate origin O position (Shown by the thick dashed line), it can rotate freely within 360 degrees of the xy plane.
- the optical axis of the optical system is in the z direction shown in FIG. 9.
- the plane light source ABCD coming from the right side as shown in Figure 9 is focused into a linear light source A'B' through a cylindrical lens.
- Cylindrical mirror can also be installed on any rotating mechanism to generate line light source A'B' in any direction.
- FIG. 10 is a schematic diagram of the state of a linear light source with an arbitrary rotation angle and a scanning surface related to the rotating cylindrical mirror.
- the scanning surface abcd as shown in the figure is generated from the line light source A'B'.
- both the scanning (reflecting) mirrors SM11 and SM12 of Figure 6 must participate in the scanning, instead of just being shown in Figure 8 Scanning (reflection) mirror SM11 participates in scanning.
- Scanning (reflecting) mirrors SM11 and SM12 participate in the scanning technology at the same time.
- the sawtooth wave shown in Figure 2 is used as the driving signal basis of SM11 and SM12, and then each scanning is performed according to the rotation angle of Figure 10.
- the (reflective) mirror's due amplitude is multiplied by the respective base signal.
- the amplitude obtained by the scanning base of the scanning (reflection) mirror SM11 is (A'B'/2)sin( ⁇ )
- the scanning (reflection) mirror SM12 is obtained by the scanning base
- the amplitude is (A'B'/2)cos( ⁇ ). It should be pointed out that the definition of this scanning direction and rotation direction is arbitrary.
- ⁇ t is the angle imposed by the closed-loop control system on the rotating support;
- (x t , y t ) is the translation amount imposed on the scanning (reflection) mirrors SM11 and SM12, and at the same time, (x t , y t ) is still The amount of translation superimposed on each of the scanning (reflection) mirrors SM11 and SM12 for generating the scanning signal of FIG. 10.
- (x t , y t , ⁇ t ) are the control commands on the mirrors SM11, SM12 and the cylindrical mirror and the line scan camera rotating bracket (equivalent to their respective Movement offset and rotation angle);
- ( ⁇ x t , ⁇ y t , ⁇ t ) is the relative movement of the image (target frame) recorded from the line scan camera and the reference image (reference frame);
- g is the gain of the closed-loop control system ;
- (X t+1 ,y t+1 , ⁇ t+1 ) is the next set of new commands applied to the mirrors SM11, SM12, cylindrical mirrors and line scan camera rotation brackets (equivalent to motion Offset and rotation angle).
- Figures 6-10 in the above embodiments describe the main LSO imaging system of the present invention, which integrates an internal fundus optical tracking closed-loop control system, such as the control mode of formula (1) or formula (2).
- an auxiliary imaging system as shown in Figure 4 is added.
- the auxiliary imaging system can be an OCT system, or it can be used for other purposes, such as a fundus laser treatment system.
- the specific technical implementation of these two parts is described in detail in another patent application.
- FIG. 11 is a schematic diagram of the principle of integrating a main LSO imaging system with closed-loop optical tracking into another auxiliary imaging system according to an embodiment of the present invention.
- the main LSO imaging system on the left integrates another auxiliary imaging system in the upper part, and the main LSO imaging system on the left has its own closed-loop optical tracking function. Its working principle is as follows:
- control signals (x, y, ⁇ ) are applied to the scanning (reflection) mirrors SM11, SM12 and the cylindrical mirror L13 of the main LSO system and the rotating bracket of the line scanning camera.
- the parameters of the control signal are as shown by the dashed line with arrows, which come from the closed-loop control system inside the LSO, and are consistent with formula (1) and formula (2).
- this group of closed-loop control motion signals has the following advantages: 1) smooth; 2) stable; 3) strong anti-interference.
- control signals (x', y', ⁇ ') applied to the scanning (reflection) mirrors SM21 and SM22 of the auxiliary system completely inherit the advantages of the above-mentioned closed-loop control signals (x, y, ⁇ ), because (x',y', ⁇ ') is obtained by space transformation (x,y, ⁇ ), as shown in formula (3):
- the spatial transformation relation f (x', y', ⁇ '; x, y, ⁇ ) of formula (3) is completely determined by the parameters of the optical system.
- the spatial transformation relationship f(x',y', ⁇ ';x,y, ⁇ ) from the main LSO system to the auxiliary system is measured quickly, efficiently, accurately and fully automatically.
- the existing data processing technology calculates fundus movement from the LSO image in units of frames.
- frequency doubling technology is used for calculation.
- FIG. 12 is a schematic diagram of using frequency doubling technology to reduce the time delay of fundus calculation in an embodiment of the present invention.
- each frame of image is divided into a plurality of equally spaced sub-frame elements, such as S 1,1 , S 1,2 , S 1,3 ,..., according to the time sequence of the data reached by the scanning camera.
- S 1,M are all M subframes in the reference frame
- Sk,1 , Sk,2 , Sk,3 ,..., Sk,M are all M subframes in the kth target frame yuan.
- the method of the present invention is to scan any frame of image in the scanning direction of SM11 (as mentioned above, usually it is the combination of SM11 and SM12 shown in Figure 10.
- SM11 as mentioned above, usually it is the combination of SM11 and SM12 shown in Figure 10.
- SM11 shown in Figure 8 is used here as For reference, the following description is the same.
- Fig. 12A and Fig. 12B show sub-frame elements in the horizontal direction, indicating that the SM11 is scanned in the vertical direction.
- the combination of SM11 and SM12 allows the optical system to scan in any direction in a 360-degree space, so the division of sub-frame elements in Figure 12 needs to be adjusted to the corresponding orthogonal direction.
- the SM11 scanning signal in FIG. 2 and refer to FIG. 13 for the division method of the sub-frame elements.
- Fig. 13 is a schematic diagram of the scanning (reflecting) mirror SM11 scanning signal and the division of sub-frame elements.
- the vertical dotted line represents the time (equivalent space) position of each subframe element; the solid thick line represents the sawtooth wave that drives SM11 (or a combination of SM11 and SM12, with the same context) to scan.
- sawtooth wave has scan area and return area, as shown in Figure 13.
- the time in the return zone is 0, then the sawtooth wave becomes a triangle wave.
- a triangular wave can also be used instead of a sawtooth wave as the scanning signal of SM11, as long as the scanning mirrors SM11 and SM12 are not damaged.
- a Wasatch Photonics line scan camera (OCTOPLUS3) is used, and the camera receives a 16kHz trigger signal. That is, the camera is set to receive 16,000 line signals per second.
- the 16kHz trigger clock is generated from Xilinx FPGA (SP605), or it can be generated from other chips such as DSP.
- the SM11 scans each cycle includes 544 lines, of which 512 lines are in the scan area and 32 lines are in the return area. So the frame rate of the image is:
- the 512 lines of the scanning area are used for imaging, which is the image shown in Figure 12.
- the data in the backhaul area is automatically discarded by the system.
- the key point of the embodiment of the present invention is that once 16 lines arrive at the camera, that is, the data of a sub-frame element is ready, the data of the sub-frame element is immediately sent from the camera to the main PC or other computing units, such as CPU, GPU, DSP, FPGA, etc., the processing unit in the embodiment of the present invention uses nVidia graphics processor GTX1050.
- the algorithm immediately starts, for example, the Cross Correlation algorithm to calculate the position of the subframe element relative to the reference frame. Normally, it is to find the relative position of the target frame subframe element S k,m and the reference frame subframe element S 1,m . However, it may also be to find the relative position of the target frame subframe element S k,m and other reference frame subframe elements S 1,p (p ⁇ m) .
- the above-mentioned specific algorithm implementation process has been disclosed in US Patent No. 9406133.
- the best existing device adjusts the frequency of SM11 is the image frame rate of 29.4 Hz, and the device of the present invention adjusts the frequency of SM11 and SM12 to the sampling time of sub-frame element 1000 Hz.
- This is the frequency multiplication technique mentioned above.
- the specific number here is only an example in the invention. Different systems and different applications can use different parameters to achieve the above frequency multiplication technology.
- the present invention adopts the technology of transplanting the Cross Correlation algorithm from the CPU to the nVidia GPU (GTX1050), which brings the advantage of improving the spatial accuracy and 3dB time bandwidth of the tracking system by more than an order of magnitude.
- the data sampling of the frame element of the line scan system can be gradually realized by the following method (refer to Figure 14).
- Fig. 14 is a schematic diagram of scanning signals and synchronization signals of a line scanning system.
- the 16kHz line pulse is the system reference clock generated by the FPGA.
- the scan signal in Figure 13 (the upper part of Figure 14) and the 29.4Hz frame synchronization signal in the lower part of Figure 14 are all obtained from the 16kHz reference pulse phase-locked .
- the scan signal and the frame synchronization signal are also completely synchronized.
- the frame synchronization signal is at a low level; during the backhaul of the scan signal, the frame synchronization signal is at a high level.
- the generation of these signals can be implemented on FPGA or DSP or other electronic hardware.
- an FPGA development board SP605 (Spartan 6 chip) of Xilinx is used for implementation.
- controlling the data output mode of the line scan camera is realized by the user inputting a trigger signal of the line scan camera.
- This trigger signal must include both the 16kHz reference pulse of Figure 14 and the frame synchronization signal of Figure 14, which is a combination of the two. As shown in Figure 15, it is required by the Wasatch Photonics line scan camera OCTOPLUS3 described above. Synchronous trigger signal.
- Figure 15 shows a signal that combines a line reference clock and a frame synchronization signal to trigger a line scan camera, but the standard method shown in Figure 15 cannot trigger a line scan camera to send a sub-frame of 1000 Hz Metadata. Only using the 16kHz reference clock of Figure 14 can not guarantee the synchronization of the received image and the scan signal. In order to obtain a 1000 Hz sub-frame elementary image synchronized with the scanning signal, the existing trigger technology is also appropriately improved in the embodiment of the present invention.
- the trigger signal of the line scan camera only uses the 16kHz reference clock in Figure 14, and the buffer size is 16 lines. This means that the line scan camera regardless of the frame synchronization status, once the line scan camera receives 16 lines of data, it immediately sends it to the PC.
- the embodiment of the present invention makes an additional synchronization in hardware implementation.
- any camera has a state to start and end data sampling. Once the user clicks on the software interface to start sampling, the 16kHz reference clock transmitted to the line scan camera does not start immediately, but waits until the rising or falling edge of the frame synchronization signal to trigger the 16kHz reference clock of the line scan camera.
- Verilog code is used:
- v_sync is the 29.4Hz frame synchronization signal shown in Figure 14
- camera_start is the status register for the user to turn on and off the camera
- camera_trigger is the trigger clock sent to the line scan camera.
- the code example is the rising edge trigger of v_sync (posedge v_sync), and the other case is the falling edge trigger (negedge v_sync).
- the 16kHz reference clock is sent to the line scan camera, otherwise, the line scan camera always gets a low level and is in the sampling waiting state.
- Sampling here is defined as sending image data from the camera to the receiving device such as PC, GPU, DSP, or other devices.
- the trigger difference between rising edge and falling edge is as shown in Figure 14.
- the rising edge is triggered, the first and second units of every 34 subframe elements are the data in the return area and need to be eliminated.
- the falling edge is triggered, the 33rd and 34th units of every 34 subframe elements are the data in the return area and need to be eliminated.
- the frequency of the reference line clock can also be adjusted, from 16kHz to 20kHz or down to 15kHz, etc., which are all parameters that can be changed.
- the size of the subframe element can also be adjusted. For example, the above 1000 Hz can be changed to 500 Hz, each sub-frame element has 32 lines. It can also be other sub-frame element sampling frequencies.
Landscapes
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Ophthalmology & Optometry (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Eye Examination Apparatus (AREA)
Abstract
Description
Claims (7)
- 一种基于改进的线扫描成像***的稳像***,包括一个线扫描眼底相机LSO成像***,其特征在于,所述LSO成像***中还包括正交振镜扫描装置,用于在眼底产生正交扫描面,能够将所述扫描面调整到360度空间的任一位置;以及用于结合智能控制算法实现LSO光学***在二维眼底扫描的同时进行LSO内部的眼底光学跟踪。
- 根据权利要求1所述基于改进的线扫描成像***的稳像***,其特征在于,所述正交振镜扫描装置为由第一反射镜(SM11)和第二反射镜(SM12)组成的双镜结构,或正交双向振动的微机电***MEMS扫描镜,或者由其他正交双向振动反射镜组成的结构。
- 根据权利要求2所述基于改进的线扫描成像***的稳像***,其特征在于,所述正交振镜扫描装置结合智能控制算法实现LSO光学***在二维眼底扫描的同时进行LSO内部的眼底光学跟踪,所述稳像***存在如下关系:(x t+1,y t+1)=(x t,y t)+g(Δx t,Δy t) (1)其中,(x t,y t)代表当前采样时刻在第一反射镜(SM11)和第二反射镜(SM12)上的控制指令,其等效于各自的运动偏移量;(Δx t,Δy t)代表从线扫描相机记录下的图像目标帧和参考帧的相对运动量;g代表闭环控制***的增益;(x t+1,y t+1)代表现有信号施加给第一反射镜(SM11)和第二反射镜(SM12)的下一组新的指令,其等效于运动偏移量。
- 一种基于改进的线扫描成像***的稳像方法,其特征在于,包括如下步骤:A、在线扫描眼底相机LSO成像***中增设正交振镜扫描装置,利用所述正交振镜扫描装置在眼底产生正交扫描面,使所述扫描面能够调整到360度空间的任一位置;B、将所述正交振镜扫描装置结合智能控制算法实现LSO光学***在二 维眼底扫描的同时进行LSO内部的眼底光学跟踪。
- 一种包括权利要求1~3任一项所述基于改进的线扫描成像***的稳像***,其特征在于,还包括旋转装置,用于将产生线光源的柱面镜(L13)以及与之耦合的线扫描相机设置在360度空间可控旋转的支架上,使线可控光源在360度空间的任一位置旋转。
- 根据权利要求5所述基于改进的线扫描成像***的稳像***,其特征在于,将锯齿波作为第一反射镜(SM11)和第二反射镜(SM12)的驱动信号基,设旋转角为θ,将每个反射镜应得的幅度乘到各自的基信号上,则式(1)可更新为:(x t+1,y t+1,θ t+1)=(x t,y t,θ t)+g(Δx t,Δy t,Δθ t) (2)其中,θ t为该闭环控制***施加在旋转支架上的角度;(x t,y t)为施加在第一反射镜(SM11)和第二反射镜(SM12)上的平移量,(x t,y t)还是叠加在每个反射镜各自的用于产生扫描信号上的平移量;(x t,y t,θ t)为当前采样时刻在第一反射镜(SM11)和第二反射镜(SM12)以及柱面镜(L13)和与之耦合的线扫描相机的旋转支架上的控制指令,其等效于各自的运动偏移量和旋转角度;(Δx t,Δy t,Δθ t)为从线扫描相机记录下来的图像目标帧和参考帧的相对运动量;g是闭环控制***的增益;(x t+1,y t+1,θ t+1)为现有信号施加给第一反射镜(SM11),第二反射镜(SM12),以及柱面镜(L13)和与之耦合的线扫描相机的旋转支架的下一组新的指令,其等效于运动偏移量和旋转角度。
- 一种根据权利要求5所述基于改进的线扫描成像***的稳像***的稳像方法,其特征在于,包括如下步骤:A、在线扫描眼底相机LSO成像***中增设正交振镜扫描装置,利用所述正交振镜扫描装置在眼底产生正交扫描面,使所述扫描面能够调整到360度空间的任一位置;B、将产生线光源的柱面镜(L13)以及与之耦合的线扫描相机安装在一 个360度的可控旋转支架上,使得线扩展光源可以出现在360度空间的任何一个位置旋转;C、将所述正交振镜扫描装置结合智能控制算法实现LSO光学***在二维眼底扫描的同时进行LSO内部的眼底光学跟踪。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910339900.1A CN109924943B (zh) | 2019-04-25 | 一种基于改进的线扫描成像***的稳像方法及*** | |
CN201910339900.1 | 2019-04-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020215361A1 true WO2020215361A1 (zh) | 2020-10-29 |
Family
ID=66990991
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/085503 WO2020215361A1 (zh) | 2019-04-25 | 2019-05-05 | 一种基于改进的线扫描成像***的稳像方法及*** |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2020215361A1 (zh) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022175736A1 (en) * | 2021-02-22 | 2022-08-25 | Alcon Inc. | Tracking of retinal traction through digital image correlation |
US11880128B2 (en) | 2021-07-27 | 2024-01-23 | Vista Innotech Limited | Non-sensor type closed-loop stabilization control algorithm and application circuit thereof |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011234750A (ja) * | 2010-04-30 | 2011-11-24 | Canon Inc | 光干渉断層撮像装置及びその処理方法 |
CN102499634A (zh) * | 2011-10-26 | 2012-06-20 | 中国科学院光电技术研究所 | 一种带稳像功能的活体人眼视网膜动态成像装置及方法 |
CN102885612A (zh) * | 2012-02-14 | 2013-01-23 | 苏州微清医疗器械有限公司 | 一种用于临床诊断的眼底成像设备 |
CN103429142A (zh) * | 2011-03-30 | 2013-12-04 | 卡尔蔡司医疗技术股份公司 | 使用跟踪有效获得人眼测量的***和方法 |
CN103750814A (zh) * | 2013-12-31 | 2014-04-30 | 苏州微清医疗器械有限公司 | 一种眼底扫描成像装置 |
JP2016198214A (ja) * | 2015-04-08 | 2016-12-01 | 株式会社トプコン | 眼底撮影方法及び眼底撮影装置 |
-
2019
- 2019-05-05 WO PCT/CN2019/085503 patent/WO2020215361A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011234750A (ja) * | 2010-04-30 | 2011-11-24 | Canon Inc | 光干渉断層撮像装置及びその処理方法 |
CN103429142A (zh) * | 2011-03-30 | 2013-12-04 | 卡尔蔡司医疗技术股份公司 | 使用跟踪有效获得人眼测量的***和方法 |
CN102499634A (zh) * | 2011-10-26 | 2012-06-20 | 中国科学院光电技术研究所 | 一种带稳像功能的活体人眼视网膜动态成像装置及方法 |
CN102885612A (zh) * | 2012-02-14 | 2013-01-23 | 苏州微清医疗器械有限公司 | 一种用于临床诊断的眼底成像设备 |
CN103750814A (zh) * | 2013-12-31 | 2014-04-30 | 苏州微清医疗器械有限公司 | 一种眼底扫描成像装置 |
JP2016198214A (ja) * | 2015-04-08 | 2016-12-01 | 株式会社トプコン | 眼底撮影方法及び眼底撮影装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022175736A1 (en) * | 2021-02-22 | 2022-08-25 | Alcon Inc. | Tracking of retinal traction through digital image correlation |
US11950969B2 (en) | 2021-02-22 | 2024-04-09 | Alcon Inc. | Tracking of retinal traction through digital image correlation |
US11880128B2 (en) | 2021-07-27 | 2024-01-23 | Vista Innotech Limited | Non-sensor type closed-loop stabilization control algorithm and application circuit thereof |
Also Published As
Publication number | Publication date |
---|---|
CN109924943A (zh) | 2019-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020215360A1 (zh) | 一种基于线扫描成像***的光学稳像方法及*** | |
JP6541243B2 (ja) | 走査型レーザー・オフサルモスコープにおける改良または走査型レーザー・オフサルモスコープに関する改良 | |
US10117576B2 (en) | System, method and computer accessible medium for determining eye motion by imaging retina and providing feedback for acquisition of signals from the retina | |
JP2012525184A (ja) | 走査検眼鏡の改善又は走査検眼鏡に関する改善 | |
WO2020215361A1 (zh) | 一种基于改进的线扫描成像***的稳像方法及*** | |
US20220155599A1 (en) | Eye accommodation distance measuring device and method for head-mounted display, and head-mounted display | |
CN102824161A (zh) | 眼科oct***及眼科oct成像方法 | |
JP4055590B2 (ja) | 画像表示装置 | |
WO2020215358A1 (zh) | 一种线扫描成像***的眼底目标运动量的计算方法 | |
CN210228108U (zh) | 一种基于线扫描成像***的光学稳像*** | |
US20160213245A1 (en) | Tomographic image capturing apparatus, method for capturing tomographic image, program, and storage medium | |
CN109924943B (zh) | 一种基于改进的线扫描成像***的稳像方法及*** | |
CN110301886B (zh) | 一种实时闭环控制眼底相机的光学***及其实现方法 | |
CN110200582A (zh) | 一种基于眼底成像技术的激光束控制***及其方法 | |
JP2000098027A (ja) | レーザレーダ装置 | |
JP2002054989A (ja) | レーザビーム計測装置および制御装置 | |
JP2003344781A (ja) | サンプリングクロック発生装置 | |
CN112731345B (zh) | 具有主动光学防抖功能的抗振型面阵扫频测距/厚的装置和方法 | |
CN116919337A (zh) | 一种线扫描视网膜成像装置及其成像方法 | |
JP2004184699A (ja) | 共焦点顕微鏡装置 | |
JPS6152850A (ja) | 眼球屈折力測定装置 | |
EP1905348A1 (en) | Ophthalmologic observation apparatus | |
CN219475378U (zh) | 内置式同轴激光反射光学成像结构 | |
CN110215184B (zh) | 一种眼底相机的闭环控制***及其方法 | |
US20230157537A1 (en) | Imaging a target within an eye and calibrating imaging devices |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19926671 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19926671 Country of ref document: EP Kind code of ref document: A1 |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19926671 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 02/06/2022) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19926671 Country of ref document: EP Kind code of ref document: A1 |