CN112396667B

CN112396667B - Method for matching electrode positions of retina stimulator

Info

Publication number: CN112396667B
Application number: CN201910703118.3A
Authority: CN
Inventors: 陈志�; 钟灿武; 王追; 陈大伟
Original assignee: Shenzhen Sibionics Technology Co Ltd
Current assignee: Shenzhen Silicon Bionics Technology Co ltd
Priority date: 2019-07-31
Filing date: 2019-07-31
Publication date: 2022-05-03
Anticipated expiration: 2039-07-31
Also published as: CN114693645A; CN114820821A; CN112396667A

Abstract

The present disclosure relates to a method for matching electrode positions of a retinal stimulator, which includes an imaging device installed outside an eye and a stimulating electrode array implanted in the retina, and is characterized by comprising the following steps: (a) acquiring an initial image with a predetermined number of pixels by using a camera device; (b) acquiring a fundus picture containing a stimulation electrode array, and acquiring the configuration position of the stimulation electrode array on the retina based on the fundus picture; (c) selecting a target area corresponding to the stimulation electrode array on the initial image according to the configuration position; and (d) processing the target area to match a target number of pixels of the target area to a number of electrodes of the stimulation electrode array. Thereby enabling the position of the output image to be adjusted without changing hardware.

Description

Method for matching electrode positions of retina stimulator

Technical Field

The present disclosure relates specifically to a method of matching electrode positions of a retinal stimulator.

Background

Retinal diseases such as RP (retinitis pigmentosa), AMD (age-related macular degeneration), and the like are important blinding diseases, and patients suffer from visual deterioration or blindness due to obstruction of the light-sensing pathway. With the research and development of the technology, there has appeared a technical means for repairing the above-mentioned retinal diseases using a retinal stimulator or the like. The existing retinal stimulators generally include a camera device disposed outside the patient's body, an image processing device, and an intraocular implant (also referred to as an "implant device") placed in the patient's eyeball. In which an image pickup device outside the body captures an image of the outside world to obtain an image signal, and an image processing device processes the image signal and transmits the processed image signal (also referred to as a "visual signal") to the implant. The implant further converts these image signals into electrical stimulation signals to stimulate ganglion cells or bipolar cells on the retina, thereby producing light perception to the patient.

However, since the state of the living retinal cells of each patient is not consistent, the surgeon needs to select the most suitable position for placing the stimulation electrode array according to the actual situation, so that the position of the stimulation electrode array of the implant device disposed in the eyeball of the patient may be deviated, for example, the stimulation electrode array is inclined with respect to the visual plane of the eye. In this case, correction needs to be performed individually for each patient so that the patient can feel a normal image angle of view.

Existing calibration methods are typically implemented by adjusting the camera module. Specifically, the camera module of the imaging apparatus is not usually shipped to a factory. After clinical operation accomplished, shoot outside for example T word pattern at the in-process camera device of start-up adaptation, then medical personnel rotate the camera module according to actual patient's impression. Medical personnel glue again and fix the camera module when the patient can feel T word pattern. However, because the eyeball of the person needs to move continuously in daily use, the stimulation electrode array often generates slow displacement after being implanted, so that the output image is skewed, under the condition, the medical staff usually needs to adjust the hardware such as the camera module and the like again, and bad use experience is caused to the patient.

Disclosure of Invention

The present disclosure has been made in view of the above circumstances, and an object thereof is to provide a method for matching electrode positions of a retinal stimulator, which can adjust the output image position while suppressing repeated modification of hardware after displacement of an electrode.

To this end, the present disclosure provides a method for matching electrode positions of a retinal stimulator including an imaging device mounted outside an eye and a stimulating electrode array implanted in the retina, the method comprising the steps of: (a) acquiring an initial image with a predetermined number of pixels by using the camera device; (b) acquiring a fundus picture containing the stimulating electrode array, and acquiring the configuration position of the stimulating electrode array on the retina based on the fundus picture; (c) selecting a target area corresponding to the stimulation electrode array on the initial image according to the configuration position; and (d) processing the target area to match a target number of pixels of the target area to a number of electrodes of the stimulation electrode array.

In the disclosure, an initial image is acquired by using a camera device, a fundus picture containing a stimulation electrode array is acquired, the configuration position of the stimulation electrode array on the retina can be acquired according to the fundus picture, so that a corresponding target area on the initial image is selected, and the target area is processed to enable the target pixel number of the target area to be matched with the number of electrodes of the stimulation electrode array. Thereby enabling the position of the output image to be adjusted without adjusting the position of the stimulation electrode array.

In the method for matching electrode positions according to the present disclosure, optionally, before step (b), the method further includes: carrying out graying processing on the initial image to obtain a grayscale image, and carrying out binarization processing on the grayscale image to obtain a binary image. In this case, even when the number of electrodes is small and the information receiving capability is limited, it is possible to optimize the processing of the image and to retain useful information of the image such as the outline of the object or the obstacle as much as possible.

In the electrode position matching method according to the present disclosure, optionally, before the binarization processing, the grayscale image is further subjected to compression processing. In this case, it is possible to reduce redundant information of the image, reducing the number of pixels of the image, in order to extract useful information of the image in a subsequent step.

In the electrode position matching method according to the present disclosure, optionally, the number of pixels of the target region is not less than the number of electrodes of the stimulation electrode array. This can increase the range of visibility of the patient.

In the electrode position matching method according to the present disclosure, optionally, in step (d), the target region is compressed to match a target number of pixels of the target region with the number of electrodes of the stimulation electrode array. This can help the patient to recognize the image better.

In the electrode position matching method according to the present disclosure, optionally, in step (d), a plurality of sub-regions having a number of pixels matching the number of electrodes of the stimulation electrode array are selected from the target region. This can help the patient to recognize the image better.

In the electrode position matching method according to the present disclosure, optionally, the method further includes periodically acquiring a fundus picture including the stimulation electrode array, and acquiring the arrangement position of the stimulation electrode array on the retina again when the arrangement position of the stimulation electrode array on the retina changes. In this case, it is possible to make the patient see an image of a normal viewing angle without changing hardware after the position of the stimulation electrode array is changed.

In the electrode position matching method according to the present disclosure, optionally, the configuration position is an inclination angle of the stimulation electrode array with respect to a horizontal plane when the eye is in front of the front view. Therefore, the corresponding target area on the initial image can be selected according to the inclination angle of the stimulating electrode array.

In the electrode position matching method according to the present disclosure, optionally, the predetermined number of pixels is greater than the target number of pixels. This can increase the range of visibility of the patient.

In the electrode position matching method according to the present disclosure, optionally, the stimulating electrode array is disposed on the retina. Therefore, the retina can be stimulated to generate light sensation through the electrodes on the stimulating electrode array.

According to the matching method of the present disclosure, it is possible to make a patient see an image of a normal view angle without changing hardware.

Drawings

Fig. 1 is a schematic diagram showing a structure of a retinal stimulator according to an example of the present disclosure.

Fig. 2 is a flow chart diagram illustrating an electrode position matching method according to an example of the present disclosure.

Fig. 3 is a flow diagram illustrating the pre-processing of an initial image in accordance with an example of the present disclosure.

Fig. 4 is a schematic diagram showing a procedure in a compression process according to an example of the present disclosure.

Fig. 5 is a schematic diagram showing the arrangement position of the stimulation electrode array in the fundus photograph according to an example of the present disclosure.

Fig. 6 is a schematic diagram illustrating a target area of a stimulation electrode array according to an example of the present disclosure in a corresponding lattice of an initial image.

Detailed Description

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same components are denoted by the same reference numerals, and redundant description thereof is omitted. The drawings are schematic and the ratio of the dimensions of the components and the shapes of the components may be different from the actual ones.

In addition, the headings and the like referred to in the following description of the present disclosure are not intended to limit the content or scope of the present disclosure, but merely serve as a reminder for reading. Such a subtitle should neither be understood as a content for segmenting an article, nor should the content under the subtitle be limited to only the scope of the subtitle.

The present disclosure provides a method of matching electrode positions of a retinal stimulator. In the present disclosure, a target region corresponding to a stimulation electrode array can be corrected a plurality of times after a clinical operation, thereby improving the phenomenon in which image skew occurs due to relative displacement of electrodes. The present disclosure is described in detail below with reference to the attached drawings.

Fig. 1 is a schematic diagram showing a structure of a retinal stimulator according to an example of the present disclosure. The retinal stimulator 1 of the present disclosure may be particularly useful, for example, in patients who have retinopathy leading to blindness, but whose visual pathways remain intact, such as bipolar cells, ganglion cells, etc. In the present disclosure, the retinal stimulator 1 is also sometimes referred to as "artificial retina", "artificial retinal system", or the like.

In some examples, as shown in fig. 1, the retinal stimulator 1 may include an implant device 10, a camera device 20, and an image processing device 30. The implant device 10 may receive the visual signal and generate an electrical stimulation signal based on the visual signal to create a sensation of light in the patient. Wherein, the visual signal can be collected by the camera device 20 and processed by the image processing device 30.

In some examples, the implant device 10 may include a stimulation electrode array 11 (see fig. 5). The stimulation electrode array 11 may include a predetermined number of stimulation electrodes (also referred to as "electrodes" in some cases). (refer to fig. 5). The stimulation electrode may generate an electrical stimulation signal based on the visual signal. In particular, the implant device 10 may receive visual signals and the stimulation electrodes may convert the received visual signals into bi-directional pulsed current signals as electrical stimulation signals, thereby delivering bi-directional pulsed current signals to ganglion cells or bipolar cells of the retina to produce light sensation. Alternatively, the implant device 10 may be implanted in a human body, such as an eyeball.

In some examples, the visual signals received by the implant device 10 may be captured and processed by the camera device 20 and the image processing device 30.

In some examples, the camera 20 may be used to capture images and convert the captured images into visual signals. For example, the camera 20 may capture images of the environment in which the patient is located.

In some examples, the image capture device 20 may be an apparatus having an image capture function, such as a video camera, a still camera, or the like. For ease of use, a camera of smaller volume may be designed on (e.g., embedded in) the eyewear.

In other examples, the patient may also capture images by wearing lightweight camera-enabled glasses as the camera 20. The imaging device 20 may be implemented by *** glasses or the like. In addition, the camera device 20 may be mounted on smart wearable devices such as smart glasses, smart headsets, and smart bracelets.

In some examples, the image processing device 30 may receive visual signals generated by the camera device 20. The image processing device 30 may process the visual signal and send it to the implant device 10 via the transmitting antenna.

In some examples, the image processing device 30 may be connected with the camera device 20. The connection between the imaging device 20 and the image processing device 30 may be a wired connection or a wireless connection. The wired connection can be data line connection, the wireless connection can be Bluetooth connection, WiFi connection, infrared connection, NFC connection or radio frequency connection and the like.

In some examples, the camera device 20 and the image processing device 30 may be configured outside the patient's body. For example, the patient may wear the imaging device 20 on glasses. The patient may also wear the camera device 20 on a wearable accessory such as a headgear, hair band, or brooch. The patient can wear the image processing device 30 on the waist, and the patient can wear the image processing device 30 on the arm, leg, or the like. Examples of the present disclosure are not limited thereto, and for example, the patient may also place the image processing device 30 in, for example, a handbag or a backpack that is carried around.

Hereinafter, the procedure of the electrode position matching method of the retinal stimulator 1 will be described in detail with reference to the drawings. The method of matching the electrode positions of the retinal stimulator 1 related to the present disclosure may be simply referred to as a method of matching the electrode positions. Fig. 2 is a flow chart diagram illustrating an electrode position matching method according to an example of the present disclosure. Fig. 3 is a flow diagram illustrating the pre-processing of an initial image in accordance with an example of the present disclosure.

In the present embodiment, as shown in fig. 2, the method for matching the electrode positions of the retinal stimulator 1 includes the following steps: (a) acquiring an initial image having a predetermined number of pixels by the image pickup device 20 (step S10); (b) acquiring a fundus picture including the stimulating electrode array 11, and acquiring the arrangement position of the stimulating electrode array 11 on the retina based on the fundus picture (step S20); (c) selecting a target area M corresponding to the stimulation electrode array 11 on the initial image according to the configuration position (step S30); and (d) processing the target area M so that the target number of pixels of the target area M matches the number of electrodes of the stimulation electrode array 11 (step S40).

In the method of matching the electrode positions of the retinal stimulator 1 according to the present embodiment, if the stimulating electrode array 11 is distorted after a clinical operation without changing hardware (for example, readjusting a camera module), the target region M corresponding to the stimulating electrode array 11 can be corrected a plurality of times so that the patient can see an image at a normal viewing angle.

In step S10, an initial image having a predetermined number of pixels may be acquired by the image capture device 20. As described above, the image pickup device 20 may be a camera.

In some examples, the initial image is, for example, an external environment in which the patient is located, such as a life scene, a traffic scene, and so forth. By photographing the external environment by the camera device 20, a desired initial image can be captured. In other examples, the camera 20 may capture an initial image every preset time T.

In the present embodiment, the predetermined number of pixels of the initial image may be, for example, 30 ten thousand, 100 ten thousand, 200 ten thousand, 500 ten thousand, 1200 ten thousand, or the like, but the present embodiment is not limited thereto. In some examples, the number of electrodes of the stimulation electrode array 11 may be 16, 20, 32, 60, 128, 256, 1200, etc. In this case, the predetermined number of pixels of the initial image may be greater than the number of electrodes of the stimulation electrode array 11.

In some examples, the initial image may be an image captured by the camera 20 without any processing. In general, the initial image obtained by capturing the surrounding environment by the image capturing device 20 is a color image. That is, the initial image captured by the image capturing apparatus 20 without any processing may be a color image. In some examples, the color image may be an HSI image. The color image may also be an RGB image. However, examples of the present disclosure are not limited thereto, and the initial image captured by the image capture device 20 may be a grayscale image, a binary image, or the like.

Generally, the appearance of objects or obstacles in the initial image is the information of major interest to the patient, and in particular, the identification of the outline of objects or obstacles is beneficial for the blind or low-vision patient. On the other hand, since information such as color features in a color image is not always used to reflect morphological features of an object in an initial image, the information can relatively well retain the contour of the object or an obstacle even if a portion of the color image is removed. On the other hand, the number of electrodes of the stimulation electrode array 11 in the implant device 10 of the retinal stimulator 1 is still relatively small at present, and the number of the electrodes is, for example, 60, 100, 150 or 256.

In general, it is difficult for a relatively small number of electrodes to completely transmit all information of an initial image, and it is often difficult to transmit information such as the outline of an object or an obstacle in the initial image. In such a case, if information on a large number of pixels of the initial image is directly associated with the very limited number of stimulation electrodes in the implant device 10 of the retinal stimulator 1, the amount of information on the pixels of the initial image cannot be completely reflected by the number of stimulation electrodes, and thus the image is likely to be distorted significantly. Based on this, the present disclosure can optimize the processing on the initial image by performing the graying processing and the binarization processing on the initial image, and can retain useful information of the image, such as the contour of an object or an obstacle, as much as possible, even when the number of stimulation electrodes is small and the capability of receiving information is limited.

In some examples, in step S10, the initial image may be pre-processed. As shown in fig. 3, a graying process may be performed on the initial image to obtain a grayscale image (step S11). In addition, binarization processing may be performed on the grayscale image to obtain a binary image (step S12). Further, the compression processing may be performed on the gradation image (step S13). In some examples, the pre-processing of the initial image may be implemented by the image processing device 30.

In some examples, the grayscale image may be R, G, B a special color image with the same size of the three components (i.e., R-G-B value), which has less information than a normal color image. Each pixel of a grayscale image has a corresponding grayscale value. In some examples, each gray value may be represented using, for example, an 8-bit binary number, i.e., the gray value of the gray image ranges from 0-255. In other examples, each gray value may be represented by a 16-bit binary number, for example, or may be represented by a binary number of 24 bits or more, for example.

In some examples, the graying processing in step S11 mainly processes the color information of the initial image, and does not change the initial image information other than the color information. For example, the graying process may help to highlight useful information of the image in the initial image, such as morphological feature information of an object or an obstacle in the initial image, at the time of the subsequent process.

In some examples, the graying processing method may be a component method, i.e., a value of any one of the three components may be selected R, G, B as the grayscale value. For example, for a pixel in the initial image, if R is 70, G is 110, and B is 150, then 70 may be selected as the grayscale value of the pixel, that is, R is 70; for example, 110 may be selected as the gradation value of the pixel, and for example, 150 may be selected as the gradation value of the pixel. In this case, a grayscale image can be obtained by sequentially processing each pixel in the initial image.

In addition, in some examples, the graying processing method may also be a maximum value method, i.e., the maximum value of the three components may be selected R, G, B as the grayscale value. For example, if R is 70, G is 110, and B is 150 for one pixel in the initial image, 150 may be selected as the grayscale value of the pixel. In this case, a grayscale image can be obtained by sequentially processing each pixel in the initial image.

In addition, in some examples, the graying processing method may also be an average value method, that is, an average value of R, G, B three components may be selected as the grayscale value. For example, if R is 70, G is 110, and B is 150 for one pixel in the initial image, the average of the three values R, G, B is 110, and 110 may be selected as the grayscale value of the pixel. In this case, a grayscale image can be obtained by sequentially processing each pixel in the initial image.

In addition, in some examples, the graying processing method may also be a weighting method, that is, R, G, B three components may be weighted according to different weighting coefficients to obtain the grayscale value. For example, for a pixel in the initial image, if R70, G110, and B150, the weighting coefficient of R is 0.3, the weighting coefficient of G is 0.5, and the weighting coefficient of B is 0.2, the gray-scale value of the pixel is 0.3 +0.5 + 110+0.2 + 150 is 106. In this case, a grayscale image can be obtained by sequentially processing each pixel in the initial image.

In the above example, the graying processing can reduce the data amount (or information amount) of the initial image, facilitate the subsequent processing of the image, and contribute to highlighting useful information of the image in the initial image at the time of the subsequent processing. The useful information of the image may be, for example, contour information of an object or an obstacle.

In addition, since the implantation device 10 of the retinal stimulator 1 needs to be implanted into the eyeball, the size of the implantation device 10 is severely limited, and the number of stimulation electrodes of the stimulation electrode array 11 in the implantation device 10 is also small. Therefore, by performing binarization processing on the grayscale image, a binary image is obtained to efficiently transfer the information of the pixels to the respective stimulation electrodes. For example, the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 255. Examples of the present disclosure are not limited thereto, and for example, the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 255.

In some examples, the binarization process in step S12 may include comparing the magnitude of the gradation value of each pixel in the gradation image with a preset gradation value. The gray values in the gray image can be set into two types, namely a maximum gray value and a minimum gray value, and the binary image can be obtained after the gray values are changed. In some examples, the preset gray value may be set by the relevant person or determined by the relevant algorithm of the software used. In this case, the binary image can be obtained by performing binarization processing on the grayscale image.

In some examples, in the process of preprocessing the initial image, the gray scale processing is performed on the initial image to obtain a gray scale image, but the gray scale image still includes much redundant information relative to useful information of the image, such as the outline of an object or an obstacle, for example, spatial redundancy caused by correlation between adjacent pixels in the gray scale image. The number of pixels of the grayscale image can be reduced by the compression processing, and the complexity in the subsequent image processing (such as binarization processing) can be reduced, so that useful information of the image can be extracted in the subsequent steps.

In some examples, as shown in fig. 3, before the binarization process (step S12), the grayscale image may be further subjected to a compression process (step S13), that is, the grayscale image is subjected to a compression process to obtain a low-pixel grayscale image, so that the number of pixels of the grayscale image after the compression process is reduced, thereby being capable of facilitating extraction of useful information of the image in a subsequent step. In some examples, the number of steps of the compression process may be greater than or equal to two steps.

In some examples, the step of compression processing (step S13) may include: firstly, carrying out partition processing on a gray image to obtain a plurality of gray image areas Y; calculating an average gray value of pixels for any one gray image area Y among the plurality of gray image areas Y, and taking the average gray value as the gray value of the gray image area Y; each gray image area Y of the gray image is taken as a pixel having an average gray value to obtain a low pixel gray image. In this case, the grayscale image is compressed to extract useful information of the image in a subsequent step.

For example, the pixels of the grayscale image are 160 × 144, and in order to extract useful information of the image in the subsequent steps, the pixels of the grayscale image can be reduced to 20 × 18 by the compression process described above. Fig. 4 is a schematic diagram showing a procedure in a compression process according to an example of the present disclosure. As shown in fig. 4, the gray image having a pixel size of 160 × 144 may be divided into 20 × 18 gray image regions Y. Wherein each gray scale image region Y contains 8 x 8 pixels. Fig. 4 shows 8 × 8 pixels included in one gray image area Y in the gray image area Y and the gray values of the pixels, and the average gray value of the gray image area Y can be calculated and can be regarded as one pixel, and the average gray value can be regarded as the gray value of the pixel. The gray values of the other gray image regions Y can be obtained as the gray image region Y, and thus a low-pixel gray image can be obtained. The low pixel grayscale image may then be binarized. In some examples, the number of pixels of the compressed low-pixel gray image should be no less than the number of stimulation electrodes of the stimulation electrode array 11.

But examples of the present disclosure are not limited thereto, and other compression methods may be used in addition to the above-described compression method. For example, in some examples, the step of compression processing (step S13) may include: firstly, carrying out partition processing on a gray level image to obtain a plurality of gray level image areas Y, wherein each gray level image area Y comprises a plurality of pixels; calculating an average gray value of pixels for any one gray image area Y among the plurality of gray image areas Y, and taking the average gray value as the gray value of the gray image area Y; comparing the average gray value of each gray image area Y with a preset average gray value to determine an effective gray image area Y' in the gray image area Y; the effective gray image area Y' is taken as a pixel having an average gray value, and the respective pixels are combined in order to obtain a low-pixel gray image. In this case, the first compression process is performed on the grayscale image to extract useful information of the image in a subsequent step.

As described above, the low-pixel grayscale image after the compression processing has a reduced number of pixels as compared with the grayscale image before the compression processing, and it is possible to reduce redundancy of image data caused by correlation between adjacent pixels in the grayscale image accordingly. Thereby, useful information of the image can be extracted in the subsequent steps.

In some examples, the compression process and the binarization process described above may each be implemented using an image processing algorithm arranged in an FPGA (field programmable gate array). In the field of image processing, the FPGA has the advantages of high reliability, good flexibility, large throughput, short development period, small risk and the like, and can greatly reduce the volume and power consumption of a system and realize high-speed real-time compression of images. In addition, the above-described compression processing and binarization processing may also be realized using an Application Specific Integrated Circuit (ASIC), a software program disposed on a computer, or the like.

Fig. 5 is a schematic diagram showing the arrangement position of the stimulation electrode array 11 in a fundus photograph according to an example of the present disclosure.

In step S20, as shown in fig. 5, a fundus picture including the stimulation electrode array 11 may be taken, and the arrangement position of the stimulation electrode array 11 on the retina is acquired based on the fundus picture. In some examples, the stimulation electrode array 11 may be implanted into the retina and affixed to the retina by clinical surgery. In general, since each patient has different states of the viable retinal cells, the surgeon selects the most suitable position for placing the stimulation electrode array 11 according to the actual condition of the patient during the clinical operation, so that the configuration positions of the stimulation electrode arrays 11 of different patients are deviated. The arrangement position is the inclination angle of the stimulation electrode array 11 with respect to the horizontal plane when the eye is looking straight ahead. In this case, the inclination angle θ of the stimulation electrode array 11 implanted in the retina of the patient with respect to the horizontal plane L (also referred to as "horizontal plane L") when the eye is looking straight ahead may be different. For example, the tilt angle θ of the stimulation electrode array 11 may be 0 °, 10 °, 15 °, 30 °, 60 °, 90 °, 135 °, etc.

In some examples, after the stimulation electrode array 11 is implanted in the retinal fundus, a fundus picture is taken by a fundus camera (e.g., topokang TRC-NW 400). The acquired fundus picture includes the stimulation electrode array 11, and particularly includes the arrangement position of the stimulation electrode array 11 on the retina. In this case, the inclination angle θ of the stimulation electrode array 11 with respect to the horizontal plane L can be obtained by analyzing the fundus picture.

In some examples, a fundus picture including the stimulation electrode array 11 may also be periodically acquired to periodically know whether the arrangement position of the stimulation electrode array 11 on the retina has changed. Thus, the arrangement position of the stimulation electrode array on the retina in step S20 can be periodically updated. When the arrangement position of the stimulation electrode array 11 on the retina changes, the arrangement position of the stimulation electrode array 11 on the retina can be acquired again.

Additionally, in some examples, the stimulating electrode array 11 may be disposed on the retina. In other examples, the stimulation electrode array 11 may also be disposed subretinally.

In step S30, a target region corresponding to the stimulation electrode array 11 on the initial image (or the pre-processed initial image) may be selected according to the configuration position in step S20.

Fig. 6 is a schematic diagram showing a target area of the stimulation electrode array 11 according to an example of the present disclosure in a corresponding lattice of an initial image. In some examples, as shown in fig. 6, a dot matrix X corresponding to the initial image may be drawn by software in the image processing apparatus 30, and the dot matrix X may correspond to pixels of the initial image. For example, one square in the lattice X in fig. 6 may represent one pixel of the initial image. And selecting a target area M corresponding to the stimulation electrode array 11 from the lattice X of the initial image according to the inclination angle theta of the stimulation electrode array 11.

In some examples, as shown in fig. 6, the shape of the target area M may be the same as the shape of the stimulation electrode array 11, for example, in the case where the shape of the stimulation electrode array 11 is rectangular, the shape of the target area M may be rectangular. In some examples, the tilt angle Δ of the target region M in the lattice X with respect to the horizontal plane L may be made the same as the tilt angle θ of the stimulation electrode array 11. In this case, it may be achieved that the target area M corresponds to the stimulation electrode array 11, for example, the corresponding area a and the corresponding area b shown in fig. 6 may correspond to the stimulation electrode 111 and the stimulation electrode 112, respectively, in the stimulation electrode array 11 shown in fig. 5.

In some examples, the actual range occupied by the selected target area M in the dot matrix X may be set manually. In some examples, the actual range occupied by the selected target area M in the dot array X may be smaller than the number of electrodes of the stimulation electrode array 11, i.e., the number of pixels of the target area M is smaller than the number of electrodes of the stimulation electrode array 11 (not shown). In some examples, the actual range occupied by the selected target area M in the dot array X may be no less than the number of electrodes of the stimulation electrode array 11 (as shown in fig. 6), i.e., the number of pixels of the target area M is no less than the number of electrodes of the stimulation electrode array 11. For example, the number of the electrodes of the stimulation electrode array 11 in the implant device 10 of the retinal stimulator 1 may be 60, 100, 150, 256, or the like. The number of pixels of the target area M may be less or not less than 60, 100, 150, 256, or the like.

In some examples, since the stimulation electrode array in step S20 is updated periodically at the configured position of the retina (the configured position that is updated periodically may or may not be changed), the target area is also updated periodically.

In step S40, the target region M may be processed so that the target number of pixels of the target region M matches the number of electrodes of the stimulation electrode array 11. In some examples, the predetermined number of pixels of the initial image should be greater than the target number of pixels of the target area M.

In some examples, the number of pixels of the target area M may be less than or equal to the number of electrodes of the stimulation electrode array 11. In this case, according to the positional correspondence relationship (not shown) between the target region M and each electrode in the stimulation electrode array 11, each pixel of the target region M can be completely corresponded to the electrode in the stimulation electrode array 11, and information corresponding to each pixel can be transmitted to each stimulation electrode for stimulation. The information corresponding to each pixel may be determined by the gray scale value corresponding to each pixel. For example, the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 255. Examples of the present disclosure are not limited thereto, and for example, the electrical stimulation signal may be at a high level and may correspond to a pixel having a gray scale value of 0, and the electrical stimulation signal may be at a low level and may correspond to a pixel having a gray scale value of 255. In this case, the patient can be made to feel an image of a normal viewing angle.

In some examples, the number of pixels of the target area M may be greater than the number of electrodes of the stimulation electrode array 11. In this case, the target region M may be compressed so that the pixels of the target region M have the target number of pixels and match the number of electrodes of the stimulation electrode array 11, and information corresponding to each pixel of the compressed target region M may be transmitted to each stimulation electrode for stimulation.

In step S40, the compression method of the compression process in step S13 may also be used. For example, the compression process may include: firstly, the target area M is partitioned to obtain a plurality of corresponding areas M (not shown); calculating an average gradation value of pixels for any one of the plurality of corresponding regions m, taking the average gradation value as a gradation value of the corresponding region m; the processing of the target area M can be completed by taking each corresponding area M of the target area M as a pixel having an average gray value. In some examples, the number of divided corresponding regions m may be no greater than the number of electrodes of the stimulation electrode array 11. In this case, the number of pixels corresponding to the compressed target region M may be matched to the number of electrodes of the stimulation electrode array 11, while having the target number of pixels.

In some examples, the target area M may be divided into a plurality of corresponding areas M as in the compression process described above. A part of the corresponding area m shown in fig. 6 corresponds to, for example, the area a and the area b. The average gradation values of the pixels of the corresponding region a and the corresponding region b may be obtained, the average gradation values may be regarded as the gradation values of the corresponding region a and the corresponding region b, the corresponding region a and the corresponding region b may be regarded as one pixel having the average gradation value, the same processing may be performed on the other regions (not shown), and the processing on the target region M may be completed. For example, the compressed pixels of the corresponding region a may be matched with the stimulation electrodes 111 of the stimulation electrode array 11 in fig. 5, and the information corresponding to the compressed pixels of the corresponding region a (i.e. the average gray value of the corresponding region a) may be transmitted to the stimulation electrodes 111 for stimulation; the compressed pixels of the corresponding region b may be matched with the stimulation electrodes 112 of the stimulation electrode array 11 in fig. 5, for example, and the information corresponding to the compressed pixels of the corresponding region b (i.e., the average gray-scale value of the corresponding region b) may be transmitted to the stimulation electrodes 112 for stimulation. If each corresponding area M stimulates the patient with corresponding information through the corresponding stimulation electrode, the patient can feel useful information of the image corresponding to the target area M.

For example, in some examples, another compression method of the compression process in step S10 may be employed, and the step of the compression process may include: firstly, partitioning a target area M to obtain a plurality of corresponding areas M, wherein each corresponding area M comprises a plurality of pixels; calculating an average gradation value of pixels for any one of the plurality of corresponding regions m, taking the average gradation value as a gradation value of the corresponding region m; comparing the average gray value of each corresponding area m with a preset average gray value to determine an effective corresponding area m' of the corresponding area m; and taking the effective corresponding area M' as a pixel with an average gray value, and combining the pixels in sequence to finish the processing of the target area M. In this case, the pixels corresponding to the divided sub-regions may have the target number of pixels and be matched with the number of electrodes of the stimulation electrode array 11.

But examples of the present disclosure are not limited thereto, and other compression methods may be used in addition to the above-described compression method. By compressing the target region M by the compression method as described above, the patient can feel useful information of the image corresponding to the target region M.

In some examples, the number of pixels of the target area M may be greater than the number of electrodes of the stimulation electrode array 11, and a plurality of sub-areas having a number of pixels matching the number of electrodes of the stimulation electrode array 11 may be selected from the target area M. The number of the pixels corresponding to each sub-region may be matched with the number of the electrodes of the stimulation electrode array 11, and any one of the sub-regions may be sequentially selected from the plurality of sub-regions as a target corresponding region to respectively transmit corresponding information to each corresponding stimulation electrode for stimulation.

In some examples, as described above for the processing method of the target area M, as shown in fig. 6, the target area M may be divided into the sub-area Q and other areas except the sub-area Q in the target area M, so that the sub-area Q and the other areas may have pixels respectively matched with the number of the electrodes of the stimulation electrode array 11. For example, the number of pixels in the sub-region Q and other regions may be the same as the number of electrodes in the stimulation electrode array 11, so that the pixels in the sub-region Q or other regions may be in one-to-one correspondence with the electrodes in the stimulation electrode array 11. In this case, the sub-region Q and other regions may be sequentially selected as target corresponding regions, and information corresponding to each pixel in each region is transmitted to each stimulation electrode corresponding to one to stimulate, so that the patient may feel useful information of corresponding images in the sub-region Q and other regions, respectively.

In some examples, after the stimulation electrode array 11 is implanted in the retina, the continuous movement of the eyeball may cause the stimulation electrode array 11 to slowly displace, thereby skewing the image perceived by the patient. In the present disclosure, by periodically acquiring fundus photographs containing the stimulation electrode array 11, and when the configuration position of the stimulation electrode array 11 on the retina changes, acquiring the configuration position of the stimulation electrode array 11 on the retina again, and re-determining the target area M of the stimulation electrode array 11 in the dot matrix X, the position of the output image (i.e., the target area) can be dynamically adjusted, and the patient can see images at normal viewing angles without changing hardware.

While the present disclosure has been described in detail in connection with the drawings and examples, it should be understood that the above description is not intended to limit the disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from the true spirit and scope of the disclosure, which fall within the scope of the disclosure.

Claims

1. A method for matching electrode positions of a retinal stimulator including an image pickup device installed outside an eye and a stimulating electrode array implanted in a retina,

the method comprises the following steps:

(a) acquiring an initial image with a predetermined number of pixels by using the camera device;

(b) acquiring a fundus picture containing the stimulating electrode array by using a fundus camera, and acquiring the configuration position of the stimulating electrode array on the retina based on the fundus picture, wherein the configuration position is the inclination angle of the stimulating electrode array relative to the horizontal plane when the eye is in front of the front view;

(c) selecting a target area corresponding to the stimulation electrode array on the initial image according to the configuration position; and is

(d) Processing the target area to match a target number of pixels of the target area to a number of electrodes of the stimulation electrode array.

2. Matching method according to claim 1, characterized in that:

before the step (b), performing a graying process on the initial image to obtain a grayscale image, and performing a binarization process on the grayscale image to obtain a binary image.

3. Matching method according to claim 2, characterized in that:

before the binarization processing is performed, compression processing is also performed on the grayscale image.

4. Matching method according to claim 1, characterized in that:

the number of pixels of the target area is not less than the number of the electrodes of the stimulating electrode array.

5. Matching method according to claim 1, characterized in that:

in step (d), compressing the target area to match a target number of pixels of the target area to the number of electrodes of the stimulation electrode array.

6. Matching method according to claim 1, characterized in that:

in step (d), a plurality of sub-regions having a number of pixels matching the number of electrodes of the stimulation electrode array are selected from the target region.

7. Matching method according to claim 1, characterized in that:

the method also comprises the steps of periodically acquiring a fundus picture containing the stimulation electrode array, and acquiring the arrangement position of the stimulation electrode array on the retina again when the arrangement position of the stimulation electrode array on the retina is changed.

8. Matching method according to claim 1, characterized in that:

the predetermined number of pixels is greater than the target number of pixels.

9. Matching method according to claim 1, characterized in that:

the stimulating electrode array is disposed on the retina.