JP6117206B2 - 機械視覚のための網膜符号化器 - Google Patents
機械視覚のための網膜符号化器 Download PDFInfo
- Publication number
- JP6117206B2 JP6117206B2 JP2014527338A JP2014527338A JP6117206B2 JP 6117206 B2 JP6117206 B2 JP 6117206B2 JP 2014527338 A JP2014527338 A JP 2014527338A JP 2014527338 A JP2014527338 A JP 2014527338A JP 6117206 B2 JP6117206 B2 JP 6117206B2
- Authority
- JP
- Japan
- Prior art keywords
- algorithm
- retinal
- machine vision
- image
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000001525 retina Anatomy 0.000 title claims description 87
- 238000004422 calculation algorithm Methods 0.000 claims description 165
- 238000000034 method Methods 0.000 claims description 143
- 230000004256 retinal image Effects 0.000 claims description 115
- 230000002207 retinal effect Effects 0.000 claims description 86
- 238000012545 processing Methods 0.000 claims description 70
- 238000012549 training Methods 0.000 claims description 53
- 230000033001 locomotion Effects 0.000 claims description 31
- 238000010304 firing Methods 0.000 claims description 26
- 238000006243 chemical reaction Methods 0.000 claims description 25
- 230000003287 optical effect Effects 0.000 claims description 19
- 241000251539 Vertebrata <Metazoa> Species 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 230000036755 cellular response Effects 0.000 claims description 10
- 230000009466 transformation Effects 0.000 claims description 10
- 230000005055 memory storage Effects 0.000 claims description 8
- 230000004913 activation Effects 0.000 claims description 7
- 230000002829 reductive effect Effects 0.000 claims description 5
- 238000007635 classification algorithm Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000012015 optical character recognition Methods 0.000 claims description 3
- 210000004027 cell Anatomy 0.000 description 100
- 230000000875 corresponding effect Effects 0.000 description 51
- 230000006870 function Effects 0.000 description 50
- 230000000007 visual effect Effects 0.000 description 34
- 230000004044 response Effects 0.000 description 32
- 238000012360 testing method Methods 0.000 description 28
- 238000013459 approach Methods 0.000 description 27
- 208000003098 Ganglion Cysts Diseases 0.000 description 22
- 208000005400 Synovial Cyst Diseases 0.000 description 22
- 230000004438 eyesight Effects 0.000 description 21
- 230000008569 process Effects 0.000 description 18
- 210000002569 neuron Anatomy 0.000 description 16
- 230000001537 neural effect Effects 0.000 description 15
- 238000007781 pre-processing Methods 0.000 description 15
- 210000004556 brain Anatomy 0.000 description 12
- 238000003860 storage Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 210000003994 retinal ganglion cell Anatomy 0.000 description 10
- 210000000887 face Anatomy 0.000 description 9
- 230000009467 reduction Effects 0.000 description 9
- 238000010561 standard procedure Methods 0.000 description 9
- 241000282693 Cercopithecidae Species 0.000 description 8
- 241000699666 Mus <mouse, genus> Species 0.000 description 8
- 238000003708 edge detection Methods 0.000 description 8
- 230000036961 partial effect Effects 0.000 description 8
- 230000002123 temporal effect Effects 0.000 description 8
- 230000014509 gene expression Effects 0.000 description 7
- 230000003278 mimic effect Effects 0.000 description 7
- 108090000623 proteins and genes Proteins 0.000 description 7
- 230000003068 static effect Effects 0.000 description 7
- 241000282326 Felis catus Species 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 6
- 238000001727 in vivo Methods 0.000 description 6
- 239000000463 material Substances 0.000 description 6
- 238000012552 review Methods 0.000 description 6
- 241000894007 species Species 0.000 description 6
- 241001465754 Metazoa Species 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000006872 improvement Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 5
- 230000004298 light response Effects 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000003252 repetitive effect Effects 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 5
- 206010028980 Neoplasm Diseases 0.000 description 4
- 241000288906 Primates Species 0.000 description 4
- 241000519996 Teucrium chamaedrys Species 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 4
- FPIPGXGPPPQFEQ-OVSJKPMPSA-N all-trans-retinol Chemical compound OC\C=C(/C)\C=C\C=C(/C)\C=C\C1=C(C)CCCC1(C)C FPIPGXGPPPQFEQ-OVSJKPMPSA-N 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000001276 controlling effect Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 239000003814 drug Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000001415 gene therapy Methods 0.000 description 4
- 230000000670 limiting effect Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 239000013598 vector Substances 0.000 description 4
- 201000004569 Blindness Diseases 0.000 description 3
- 108091006146 Channels Proteins 0.000 description 3
- 241000702421 Dependoparvovirus Species 0.000 description 3
- 230000036982 action potential Effects 0.000 description 3
- 206010001902 amaurosis Diseases 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 201000011510 cancer Diseases 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 229940079593 drug Drugs 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 238000001476 gene delivery Methods 0.000 description 3
- 230000002068 genetic effect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000007935 neutral effect Effects 0.000 description 3
- 238000003909 pattern recognition Methods 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 230000035945 sensitivity Effects 0.000 description 3
- 230000011664 signaling Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- FPIPGXGPPPQFEQ-UHFFFAOYSA-N 13-cis retinol Natural products OCC=C(C)C=CC=C(C)C=CC1=C(C)CCCC1(C)C FPIPGXGPPPQFEQ-UHFFFAOYSA-N 0.000 description 2
- 241000702423 Adeno-associated virus - 2 Species 0.000 description 2
- 206010010356 Congenital anomaly Diseases 0.000 description 2
- 241000408659 Darpa Species 0.000 description 2
- 241000282412 Homo Species 0.000 description 2
- 241001529936 Murinae Species 0.000 description 2
- 241000699670 Mus sp. Species 0.000 description 2
- 241000700159 Rattus Species 0.000 description 2
- 241000405217 Viola <butterfly> Species 0.000 description 2
- 238000002679 ablation Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000002059 diagnostic imaging Methods 0.000 description 2
- 230000009365 direct transmission Effects 0.000 description 2
- 230000005670 electromagnetic radiation Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 235000013305 food Nutrition 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012886 linear function Methods 0.000 description 2
- 238000009126 molecular therapy Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000037361 pathway Effects 0.000 description 2
- 108091008695 photoreceptors Proteins 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 229960003471 retinol Drugs 0.000 description 2
- 235000020944 retinol Nutrition 0.000 description 2
- 239000011607 retinol Substances 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012421 spiking Methods 0.000 description 2
- 239000003826 tablet Substances 0.000 description 2
- 230000001225 therapeutic effect Effects 0.000 description 2
- 230000036962 time dependent Effects 0.000 description 2
- 238000010361 transduction Methods 0.000 description 2
- 230000026683 transduction Effects 0.000 description 2
- 238000000844 transformation Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 238000009966 trimming Methods 0.000 description 2
- 239000013603 viral vector Substances 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- NCYCYZXNIZJOKI-IOUUIBBYSA-N 11-cis-retinal Chemical compound O=C/C=C(\C)/C=C\C=C(/C)\C=C\C1=C(C)CCCC1(C)C NCYCYZXNIZJOKI-IOUUIBBYSA-N 0.000 description 1
- 206010001497 Agitation Diseases 0.000 description 1
- 241000271566 Aves Species 0.000 description 1
- 229930194845 Bahia Natural products 0.000 description 1
- 102100026189 Beta-galactosidase Human genes 0.000 description 1
- 208000006992 Color Vision Defects Diseases 0.000 description 1
- 206010013801 Duchenne Muscular Dystrophy Diseases 0.000 description 1
- 101000844510 Homo sapiens Transient receptor potential cation channel subfamily M member 1 Proteins 0.000 description 1
- 241000254158 Lampyridae Species 0.000 description 1
- 244000147568 Laurus nobilis Species 0.000 description 1
- 235000017858 Laurus nobilis Nutrition 0.000 description 1
- 241000282553 Macaca Species 0.000 description 1
- 241000124008 Mammalia Species 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 102100038300 Metabotropic glutamate receptor 6 Human genes 0.000 description 1
- 101100259948 Mus musculus Tbata gene Proteins 0.000 description 1
- 108010025020 Nerve Growth Factor Proteins 0.000 description 1
- 102000007072 Nerve Growth Factors Human genes 0.000 description 1
- 241000283973 Oryctolagus cuniculus Species 0.000 description 1
- 241000282376 Panthera tigris Species 0.000 description 1
- 229920002873 Polyethylenimine Polymers 0.000 description 1
- 206010038910 Retinitis Diseases 0.000 description 1
- 208000007014 Retinitis pigmentosa Diseases 0.000 description 1
- 108700025701 Retinoblastoma Genes Proteins 0.000 description 1
- 102000004330 Rhodopsin Human genes 0.000 description 1
- 108090000820 Rhodopsin Proteins 0.000 description 1
- 241000283984 Rodentia Species 0.000 description 1
- 240000004808 Saccharomyces cerevisiae Species 0.000 description 1
- 241000270295 Serpentes Species 0.000 description 1
- 102000003617 TRPM1 Human genes 0.000 description 1
- 108010031650 Thy-1 Antigens Proteins 0.000 description 1
- 102100033523 Thy-1 membrane glycoprotein Human genes 0.000 description 1
- 108700019146 Transgenes Proteins 0.000 description 1
- 150000001323 aldoses Chemical class 0.000 description 1
- VREFGVBLTWBCJP-UHFFFAOYSA-N alprazolam Chemical compound C12=CC(Cl)=CC=C2N2C(C)=NN=C2CN=C1C1=CC=CC=C1 VREFGVBLTWBCJP-UHFFFAOYSA-N 0.000 description 1
- 210000000411 amacrine cell Anatomy 0.000 description 1
- 238000010171 animal model Methods 0.000 description 1
- 230000008485 antagonism Effects 0.000 description 1
- 239000002249 anxiolytic agent Substances 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 108010005774 beta-Galactosidase Proteins 0.000 description 1
- 230000009172 bursting Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000020411 cell activation Effects 0.000 description 1
- 210000005056 cell body Anatomy 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 210000001787 dendrite Anatomy 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000002999 depolarising effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000011438 discrete method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000000763 evoking effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 208000030533 eye disease Diseases 0.000 description 1
- 238000005188 flotation Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000012010 growth Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 210000002287 horizontal cell Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 239000012678 infectious agent Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003834 intracellular effect Effects 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 210000001542 lens epithelial cell Anatomy 0.000 description 1
- 230000004301 light adaptation Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 230000036210 malignancy Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 230000001404 mediated effect Effects 0.000 description 1
- 108010038450 metabotropic glutamate receptor 6 Proteins 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000000394 mitotic effect Effects 0.000 description 1
- 238000010369 molecular cloning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000003562 morphometric effect Effects 0.000 description 1
- 238000013425 morphometry Methods 0.000 description 1
- 230000008904 neural response Effects 0.000 description 1
- 239000002858 neurotransmitter agent Substances 0.000 description 1
- 239000003900 neurotrophic factor Substances 0.000 description 1
- 210000001743 on-bipolar cell Anatomy 0.000 description 1
- 238000013041 optical simulation Methods 0.000 description 1
- 230000029553 photosynthesis Effects 0.000 description 1
- 238000010672 photosynthesis Methods 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 244000062645 predators Species 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 230000001566 pro-viral effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000004224 protection Effects 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
- 230000007420 reactivation Effects 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 201000000757 red-green color blindness Diseases 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000011347 resin Substances 0.000 description 1
- 229920005989 resin Polymers 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 210000000964 retinal cone photoreceptor cell Anatomy 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000005204 segregation Methods 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 230000019491 signal transduction Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000005549 size reduction Methods 0.000 description 1
- 230000005477 standard model Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 230000002739 subcortical effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 238000011269 treatment regimen Methods 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 238000004800 variational method Methods 0.000 description 1
- 210000000605 viral structure Anatomy 0.000 description 1
- 230000003612 virological effect Effects 0.000 description 1
- 238000001429 visible spectrum Methods 0.000 description 1
- 210000000857 visual cortex Anatomy 0.000 description 1
- 230000004382 visual function Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/007—Transform coding, e.g. discrete cosine transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/62—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding by frequency transforming in three dimensions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biodiversity & Conservation Biology (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Discrete Mathematics (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Medicines Containing Material From Animals Or Micro-Organisms (AREA)
Description
本出願は、米国特許仮出願第61/527493号(2011年8月25日出願)及び第61/657406号(2012年6月8日出願)の利益を主張するものである。上述の出願の各々の内容は、その全体が引用によって組み込まれている。
本発明は、米国国立衛生研究所(NIH)の国立眼学研究所によって委託された「R01 EY12978」の下に米国政府の支援によって行われたものである。米国政府は、本発明にある一定の権利を有する。
ここで、*は、時空畳み込みを表し、Lmは、m番目の細胞の時空カーネルに対応する線形フィルタであり、Nmは、m番目の細胞の非線形性を説明する関数であり、前節と同じくXは、予備処理段階の出力であり、jは、ピクセルの場所であり、tは、時間である。この場合に、上述したように、発火率網膜画像を生成するために発火率λmを使用することができる。
ここで、
は、外積を表し、Sk及びTkは、k番目の空間関数及び時間関数それぞれである(kは1からQの範囲にわたる)。
ここで、
は、外積を表し、S1とT1とは、空間関数と時間関数とで構成される第1の対であり、S2とT2とは、空間関数と時間関数とで構成される第2の対である。
機械視覚に対する1つの手法の有効性を評価する一実施例において、それが特に困難であるので(空間と時間の両方で処理することを必要とするので)、ナビゲーションタスクを用いた。この手法は、例えば、各々がその全体が引用によって本明細書に組み込まれているLeCun,Y.他(2010年)「Convolutional Networks and Applications in Vision(畳み込みネットワーク及びビジョンにおける適用)」、Proc.International Symposium on Circuits and Systems(回路及びシステムに関する国際シンポジウム会報)(ISCAS’10)、253〜256ページ、IEEE、Szarvas,M.他(2005年)「Pedestrian detection with convolutional neural networks(畳み込みニューラルネットワークを用いた歩行者検出)」、Proc.Intelligent Vehicles Symposium(インテリジェント車両シンポジウム会報)、224〜229ページ、IEEE、Jackel,L.D.他(2006年)「The DARPA LAGR program:Goals,challenges,methodology,and phase I results(DARPA LAGRプログラム:ターゲット、タスク、方法、及び段階Iの結果)」、Journal of Field Robotics,23(フィールドロボティクスジャーナル第23号)、945〜973ページに記載されている、ナビゲーションにおいて一般的に使用されるいくつかの学習アルゴリズムの態様を適用した。これらの技術を用いて、畳み込みニューラルネットワークという学習アルゴリズムを用いて自体の環境を学習するナビゲータを構成した。CNNは、Theanoと呼ばれるオープンソース数値処理及び自動微分パッケージ(http://deeplearning.net/software/theano/において一般的に公開されている)を用いて構成した。
この実施例は、機械視覚における別の積年の問題である映像内の顔の認識に対する本出願に説明する手法の有効性を評価する。顔認識及び歩行者検出において一般的に使用される学習アルゴリズム[Viola及びJones 2001年、Viola、Jones、及びSnow 2005年を参照されたい]を用いて、映像内の個人の顔を認識するためのシステム、すなわち、過去に見たことがない画像ストリームを「ターゲット顔」対別の顔又は「非ターゲット」顔として分類することができるものを構成した。同じ手法は、歩行者検出、物体認識、物体追跡、全身認識、虹彩検出等であるが、これらに限定されない多くの他の目的に使用することができる。このシステムは、Pythonプログラミング言語及びNumPy数値計算パッケージを用いて実行したものである。
B 網膜画像ストリーム
Claims (18)
- 一連の生画像に対応する生画像データを受け入れる段階と、
脊椎動物網膜の入力/出力変換を実質的に模倣する入力/出力変換によって特徴付けられる符号化器を用いて符号化されたデータを生成するために前記生画像データを処理する段階であって、
網膜出力細胞応答値を生成するために前記生画像データに時空変換を適用し、該時空変換の適用は、自然光景を含む刺激を用いて生成される実験データから直接決定される一連の重みを含む単一段階時空変換の適用を含む、段階と、
前記網膜出力細胞応答値に基づいて符号化されたデータを生成する段階と、
を含む段階と、
前記符号化されたデータに少なくとも部分的に基づいて生成されたデータに第1の機械視覚アルゴリズムを適用する段階と、
を含むことを特徴とする方法。 - 前記符号化されたデータに基づいて一連の網膜画像を生成する段階を更に含むことを特徴とする請求項1に記載の方法。
- 前記符号化されたデータに基づいて前記網膜画像内のピクセル値を決定する段階を含み、該符号化されたデータに基づいて前記網膜画像内のピクセル値を決定する段階が、網膜細胞応答を示す符号化されたデータに基づいてピクセル強度又は色を決定する段階を含み、網膜細胞応答を示す該データが、網膜細胞発火率、網膜細胞出力パルス列、及び起動電位から構成されるリストのうち少なくとも1つを示すことを特徴とする請求項2に記載の方法。
- 前記一連の網膜画像に前記第1の機械視覚アルゴリズムを適用する段階、
を更に含み、該機械視覚アルゴリズムは、物体認識アルゴリズム、画像分類アルゴリズム、顔認識アルゴリズム、光学文字認識アルゴリズム、コンテンツベースの画像取り出しアルゴリズム、姿勢推定アルゴリズム、運動解析アルゴリズム、自己運動決定アルゴリズム、移動追跡アルゴリズム、オプティカルフロー決定アルゴリズム、光景再現アルゴリズム、3D容積認識アルゴリズム、及びナビゲーションアルゴリズムから構成されるリストから選択された少なくとも1つを含むことを特徴とする請求項2又は請求項3に記載の方法。 - 前記機械視覚アルゴリズムは、前記一連の網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する生画像セットに適用された時よりも良好な性能を示すことを特徴とする請求項2から請求項4のいずれか1項に記載の方法。
- 前記機械視覚アルゴリズムは、自然光景を含む一連の網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する一連の生画像に適用された時よりも良好な性能を示すことを特徴とする請求項5に記載の方法。
- 前記機械視覚アルゴリズムは、一連の画像内での人間の検出又は識別のためのアルゴリズムを含み、
前記機械視覚アルゴリズムは、前記人間を含む様々な網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する生画像セットに適用された時よりも良好な検出精度又は識別精度を示し、
前記人間を含む前記一連の画像は自然光景に位置する人間の画像を含み、該人間を含む該一連の画像は、前記機械視覚アルゴリズムをトレーニングするのに使用された自然光景とは異なる自然光景に位置する人間の画像を含む、
ことを特徴とする請求項5又は請求項6に記載の方法。 - 前記機械視覚アルゴリズムは、実環境又は仮想環境を通じたナビゲーションのためのアルゴリズムを含み、
前記機械視覚アルゴリズムは、自然光景を含む一連の網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する生画像セットに適用された時よりも良好なナビゲーション性能を示し、
前記機械視覚アルゴリズムは、自然光景を含む一連の網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する生画像セットに適用された時よりも少ないナビゲーション中の望まない衝突イベントを示し、
前記一連の網膜画像は、前記機械視覚アルゴリズムをトレーニングするのに使用されなかった環境に対応する、
ことを特徴とする請求項5又は請求項6に記載の方法。 - 前記一連の網膜画像に機械画像処理アルゴリズムを適用して1つ又はそれよりも多くの着目する網膜画像を識別する段階と、
前記着目する網膜画像に対応する1つ又はそれよりも多くの着目する生画像を識別する段階と、
前記着目する生画像を処理する段階であって、該着目する生画像に第2の機械視覚アルゴリズムを適用する段階を含む段階と、
を更に含むことを特徴とする請求項2から請求項8のいずれか1項に記載の方法。 - 前記第1の機械視覚アルゴリズムは、網膜画像セットに対してトレーニングされたアルゴリズムを含み、
前記第2の機械視覚アルゴリズムは、生画像セットに対してトレーニングされたアルゴリズムを含む、
ことを特徴とする請求項9に記載の方法。 - 前記第1の機械視覚アルゴリズムを適用する段階は、ナビゲーションアルゴリズムを適用する段階を含み、
前記ナビゲーションアルゴリズムを適用する段階は、
前記一連の網膜画像を処理して、該一連の画像内の複数の画像位置での運動を示す運動情報を決定する段階と、
前記運動情報に基づいて前記一連の画像内の空間領域を分類する段階と、
前記空間領域の前記分類に基づいてナビゲーション決定を生成する段階と、
を含む、
ことを特徴とする請求項2から請求項10のいずれか1項に記載の方法。 - 運動情報が、前記一連の画像内のオプティカルフローを示し、
畳み込みニューラルネットワークを使用して前記空間領域を分類する段階と、
ナビゲーションアルゴリズムからの結果に基づいてロボット装置の運動を制御する段階と、
を更に含むことを特徴とする請求項11に記載の方法。 - ナビゲーションアルゴリズムからの結果に基づいて仮想空間内の仮想物体の運動を制御する段階を更に含み、該ナビゲーションアルゴリズムは、仮想空間を表す画像データに基づいてトレーニングされたものであることを特徴とする請求項9から請求項12のいずれか1項に記載の方法。
- 前記網膜画像に基づいて機械視覚アルゴリズムをトレーニングする段階を更に含み、
前記機械視覚アルゴリズムをトレーニングする段階は、
(i)前記機械視覚アルゴリズムを網膜画像セットに適用して出力を生成する段階と、 (ii)前記出力に基づいて前記機械視覚アルゴリズムの性能を示す性能情報を決定する段階と、
(iii)前記性能情報に基づいて前記機械視覚アルゴリズムの1つ又はそれよりも多くの特性を修正する段階と、
(iv)選択された性能基準に達するまで段階(i)から段階(iii)までを反復的に繰り返す段階と、
を含む、
ことを特徴とする請求項2から請求項13のいずれか1項に記載の方法。 - 前記トレーニングされた機械視覚アルゴリズムは、パラメータセットによって特徴付けられ、
前記パラメータは、前記網膜画像に対応する生画像を用いた前記機械視覚アルゴリズムの同等のトレーニングによって得られると考えられる対応するパラメータとは異なる、
ことを特徴とする請求項14に記載の方法。 - 符号化されたデータを生成するために符号化器を用いて前記生画像データを処理する段階は、対応する該生画像データと比較して低減された情報量を含む符号化されたデータを生成する段階を含み、
前記機械視覚アルゴリズムは、前記一連の網膜画像に適用された時に、前記符号化器を用いて処理されていない対応する生画像セットに適用された時よりも良好な性能を示し、前記符号化されたデータに含まれる前記情報量は、対応する前記生画像データと比較して少なくとも約2倍だけ圧縮される
ことを特徴とする請求項4から請求項15のいずれか1項に記載の方法。 - 生画像データを格納するように構成された少なくとも1つのメモリストレージデバイスと、
前記メモリと作動可能に結合され、かつ請求項1から請求項16のいずれか1項に記載の方法を実行するようにプログラムされた少なくとも1つのプロセッサと、
を含むことを特徴とする装置。 - 請求項1から請求項16のいずれか1項に記載の方法の段階を実施するためのコンピュータ実行可能命令を有する持続性コンピュータ可読媒体。
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161527493P | 2011-08-25 | 2011-08-25 | |
US61/527,493 | 2011-08-25 | ||
US201261657406P | 2012-06-08 | 2012-06-08 | |
US61/657,406 | 2012-06-08 | ||
PCT/US2012/052348 WO2013029008A1 (en) | 2011-08-25 | 2012-08-24 | Retinal encoder for machine vision |
Publications (3)
Publication Number | Publication Date |
---|---|
JP2014524630A JP2014524630A (ja) | 2014-09-22 |
JP2014524630A5 JP2014524630A5 (ja) | 2015-10-08 |
JP6117206B2 true JP6117206B2 (ja) | 2017-04-19 |
Family
ID=47746908
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2014527338A Active JP6117206B2 (ja) | 2011-08-25 | 2012-08-24 | 機械視覚のための網膜符号化器 |
Country Status (9)
Country | Link |
---|---|
US (4) | US9547804B2 (ja) |
EP (1) | EP2748765B1 (ja) |
JP (1) | JP6117206B2 (ja) |
KR (2) | KR102111000B1 (ja) |
CN (1) | CN103890781B (ja) |
CA (1) | CA2883091C (ja) |
HK (1) | HK1199768A1 (ja) |
IL (1) | IL231063A (ja) |
WO (1) | WO2013029008A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022075349A1 (ja) | 2020-10-08 | 2022-04-14 | 国立研究開発法人理化学研究所 | 画像処理装置、画像処理方法、及び画像処理プログラムを格納した非一時的なコンピュータ可読媒体 |
Families Citing this family (91)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DK2539015T3 (da) | 2010-02-26 | 2016-04-04 | Univ Cornell | Nethindeprotese |
US9302103B1 (en) | 2010-09-10 | 2016-04-05 | Cornell University | Neurological prosthesis |
US10140699B2 (en) | 2010-12-07 | 2018-11-27 | University Of Iowa Research Foundation | Optimal, user-friendly, object background separation |
CA2883091C (en) | 2011-08-25 | 2020-02-25 | Cornell University | Retinal encoder for machine vision |
US10250939B2 (en) * | 2011-11-30 | 2019-04-02 | Elwha Llc | Masking of deceptive indicia in a communications interaction |
US9965598B2 (en) | 2011-11-30 | 2018-05-08 | Elwha Llc | Deceptive indicia profile generation from communications interactions |
US20130139254A1 (en) | 2011-11-30 | 2013-05-30 | Elwha LLC, a limited liability corporation of the State of Delaware | Deceptive indicia notification in a communications interaction |
US9832510B2 (en) | 2011-11-30 | 2017-11-28 | Elwha, Llc | Deceptive indicia profile generation from communications interactions |
CN104170318B (zh) | 2012-04-09 | 2018-06-01 | 英特尔公司 | 使用交互化身的通信 |
US9195903B2 (en) * | 2014-04-29 | 2015-11-24 | International Business Machines Corporation | Extracting salient features from video using a neurosynaptic system |
KR101549178B1 (ko) * | 2014-07-31 | 2015-09-02 | 고려대학교 산학협력단 | 홀로그램 생성 장치 및 방법 |
FR3025344B1 (fr) * | 2014-08-28 | 2017-11-24 | Commissariat Energie Atomique | Reseau de neurones convolutionnels |
US10147024B2 (en) * | 2014-09-16 | 2018-12-04 | Qualcomm Incorporated | Interfacing an event based system with a frame based processing system |
US10366506B2 (en) * | 2014-11-07 | 2019-07-30 | Lamina Systems, Inc. | Hyperacuity system and methods for real time and analog detection and kinematic state tracking |
MX2017008086A (es) * | 2014-12-17 | 2017-10-31 | Nokia Technologies Oy | Deteccion de objetos con red neuronal. |
WO2016101131A1 (en) | 2014-12-23 | 2016-06-30 | Intel Corporation | Augmented facial animation |
US10048749B2 (en) | 2015-01-09 | 2018-08-14 | Microsoft Technology Licensing, Llc | Gaze detection offset for gaze tracking models |
US9864430B2 (en) | 2015-01-09 | 2018-01-09 | Microsoft Technology Licensing, Llc | Gaze tracking via eye gaze model |
US9524450B2 (en) * | 2015-03-04 | 2016-12-20 | Accenture Global Services Limited | Digital image processing using convolutional neural networks |
US10115194B2 (en) * | 2015-04-06 | 2018-10-30 | IDx, LLC | Systems and methods for feature detection in retinal images |
US10515269B2 (en) * | 2015-04-20 | 2019-12-24 | Cornell University | Machine vision with dimensional data reduction |
CN104811627A (zh) * | 2015-05-21 | 2015-07-29 | 广东欧珀移动通信有限公司 | 拍摄预览方法及装置 |
NL2015087B1 (en) * | 2015-06-05 | 2016-09-09 | Univ Amsterdam | Deep receptive field networks. |
US9844881B2 (en) * | 2015-06-22 | 2017-12-19 | GM Global Technology Operations LLC | Robotic device including machine vision |
BE1023147B1 (nl) * | 2015-07-03 | 2016-12-01 | Cnh Industrial Belgium Nv | Controller voor een werkvoertuig |
CN105227828B (zh) * | 2015-08-25 | 2017-03-15 | 努比亚技术有限公司 | 拍摄装置和方法 |
US9904874B2 (en) * | 2015-11-05 | 2018-02-27 | Microsoft Technology Licensing, Llc | Hardware-efficient deep convolutional neural networks |
WO2017101094A1 (en) | 2015-12-18 | 2017-06-22 | Intel Corporation | Avatar animation system |
US9846808B2 (en) * | 2015-12-31 | 2017-12-19 | Adaptive Computation, Llc | Image integration search based on human visual pathway model |
KR102444604B1 (ko) * | 2016-02-02 | 2022-09-21 | 에스케이하이닉스 주식회사 | 데이터 저장 장치 및 그것의 동작 방법 |
JP6609505B2 (ja) * | 2016-04-06 | 2019-11-20 | Kddi株式会社 | 画像合成装置及びプログラム |
CN107305635A (zh) * | 2016-04-15 | 2017-10-31 | 株式会社理光 | 对象识别方法、对象识别装置和分类器训练方法 |
US9984314B2 (en) * | 2016-05-06 | 2018-05-29 | Microsoft Technology Licensing, Llc | Dynamic classifier selection based on class skew |
RU2665273C2 (ru) * | 2016-06-03 | 2018-08-28 | Автономная некоммерческая образовательная организация высшего образования "Сколковский институт науки и технологий" | Обучаемые визуальные маркеры и способ их продуцирования |
KR102631381B1 (ko) | 2016-11-07 | 2024-01-31 | 삼성전자주식회사 | 컨볼루션 신경망 처리 방법 및 장치 |
US11423548B2 (en) | 2017-01-06 | 2022-08-23 | Board Of Regents, The University Of Texas System | Segmenting generic foreground objects in images and videos |
US20180247161A1 (en) * | 2017-01-23 | 2018-08-30 | Intaimate LLC | System, method and apparatus for machine learning-assisted image screening for disallowed content |
US10191539B2 (en) * | 2017-03-20 | 2019-01-29 | Intel Corporation | User aware odometry correction technology |
US10678244B2 (en) | 2017-03-23 | 2020-06-09 | Tesla, Inc. | Data synthesis for autonomous control systems |
CN107067407B (zh) * | 2017-04-11 | 2020-01-31 | 广西科技大学 | 基于非经典感受野和线性非线性调制的轮廓检测方法 |
WO2018224471A1 (en) * | 2017-06-05 | 2018-12-13 | Deepmind Technologies Limited | Selecting actions using multi-modal inputs |
US10726307B2 (en) | 2017-06-30 | 2020-07-28 | Ai Systems Co., Ltd. | Real-time identification of moving objects in video images |
CN107609463B (zh) * | 2017-07-20 | 2021-11-23 | 百度在线网络技术(北京)有限公司 | 活体检测方法、装置、设备及存储介质 |
US10671349B2 (en) | 2017-07-24 | 2020-06-02 | Tesla, Inc. | Accelerated mathematical engine |
US11409692B2 (en) | 2017-07-24 | 2022-08-09 | Tesla, Inc. | Vector computational unit |
US11893393B2 (en) | 2017-07-24 | 2024-02-06 | Tesla, Inc. | Computational array microprocessor system with hardware arbiter managing memory requests |
US11157441B2 (en) | 2017-07-24 | 2021-10-26 | Tesla, Inc. | Computational array microprocessor system using non-consecutive data formatting |
US10384346B2 (en) * | 2017-09-08 | 2019-08-20 | Niantic, Inc. | Collision detection, estimation, and avoidance |
GB2570433A (en) * | 2017-09-25 | 2019-07-31 | Nissan Motor Mfg Uk Ltd | Machine vision system |
US10482572B2 (en) * | 2017-10-06 | 2019-11-19 | Ford Global Technologies, Llc | Fusion of motion and appearance features for object detection and trajectory prediction |
US11126914B2 (en) * | 2017-10-11 | 2021-09-21 | General Electric Company | Image generation using machine learning |
US11182933B2 (en) | 2017-11-28 | 2021-11-23 | Hewlett-Packard Development Company, L.P. | Indication of extent to which color-blind person is unable to distinguish features within digital image |
WO2019108170A1 (en) * | 2017-11-29 | 2019-06-06 | Intel Corporation | Methods and apparatus to convert images for computer-vision systems |
JP6985121B2 (ja) | 2017-12-06 | 2021-12-22 | 国立大学法人 東京大学 | 物体間関係認識装置、学習済みモデル、認識方法及びプログラム |
US11561791B2 (en) | 2018-02-01 | 2023-01-24 | Tesla, Inc. | Vector computational unit receiving data elements in parallel from a last row of a computational array |
CN108520532B (zh) * | 2018-04-03 | 2020-12-22 | 北京京东尚科信息技术有限公司 | 识别视频中物体运动方向的方法及装置 |
US11823376B2 (en) | 2018-05-16 | 2023-11-21 | Benevis Informatics, Llc | Systems and methods for review of computer-aided detection of pathology in images |
US11215999B2 (en) | 2018-06-20 | 2022-01-04 | Tesla, Inc. | Data pipeline and deep learning system for autonomous driving |
CN109118460B (zh) * | 2018-06-27 | 2020-08-11 | 河海大学 | 一种分光偏振光谱信息同步处理方法及*** |
US11361457B2 (en) | 2018-07-20 | 2022-06-14 | Tesla, Inc. | Annotation cross-labeling for autonomous control systems |
US11636333B2 (en) | 2018-07-26 | 2023-04-25 | Tesla, Inc. | Optimizing neural network structures for embedded systems |
US11562231B2 (en) | 2018-09-03 | 2023-01-24 | Tesla, Inc. | Neural networks for embedded devices |
US10579924B1 (en) * | 2018-09-17 | 2020-03-03 | StradVision, Inc. | Learning method, learning device with multi-feeding layers and testing method, testing device using the same |
CN109299689B (zh) * | 2018-09-21 | 2024-03-08 | 石家庄铁道大学 | 基于机器视觉的环形生产线模台自动化识别装置及其识别方法 |
WO2020072427A1 (en) * | 2018-10-01 | 2020-04-09 | Cornell University | Kits and methods for performing optical dynamic clamp on excitable cells |
SG11202103493QA (en) | 2018-10-11 | 2021-05-28 | Tesla Inc | Systems and methods for training machine models with augmented data |
US11196678B2 (en) | 2018-10-25 | 2021-12-07 | Tesla, Inc. | QOS manager for system on a chip communications |
US11816585B2 (en) | 2018-12-03 | 2023-11-14 | Tesla, Inc. | Machine learning models operating at different frequencies for autonomous vehicles |
US11537811B2 (en) | 2018-12-04 | 2022-12-27 | Tesla, Inc. | Enhanced object detection for autonomous vehicles based on field view |
US11601644B2 (en) * | 2018-12-11 | 2023-03-07 | Google Llc | Image and video coding using machine learning prediction coding models |
CN109660297B (zh) * | 2018-12-19 | 2020-04-28 | 中国矿业大学 | 一种基于机器学习的物理层可见光通信方法 |
US11610117B2 (en) | 2018-12-27 | 2023-03-21 | Tesla, Inc. | System and method for adapting a neural network model on a hardware platform |
US11150664B2 (en) | 2019-02-01 | 2021-10-19 | Tesla, Inc. | Predicting three-dimensional features for autonomous driving |
US10997461B2 (en) | 2019-02-01 | 2021-05-04 | Tesla, Inc. | Generating ground truth for machine learning from time series elements |
US11567514B2 (en) | 2019-02-11 | 2023-01-31 | Tesla, Inc. | Autonomous and user controlled vehicle summon to a target |
US11062460B2 (en) * | 2019-02-13 | 2021-07-13 | Adobe Inc. | Representation learning using joint semantic vectors |
US10956755B2 (en) | 2019-02-19 | 2021-03-23 | Tesla, Inc. | Estimating object properties using visual image data |
US11631266B2 (en) | 2019-04-02 | 2023-04-18 | Wilco Source Inc | Automated document intake and processing system |
CN110119785B (zh) * | 2019-05-17 | 2020-12-01 | 电子科技大学 | 一种基于多层spiking卷积神经网络的图像分类方法 |
US11417096B2 (en) * | 2019-05-21 | 2022-08-16 | Vimeo.Com, Inc. | Video format classification and metadata injection using machine learning |
CN110598698B (zh) * | 2019-08-29 | 2022-02-15 | 华中科技大学 | 基于自适应区域建议网络的自然场景文本检测方法和*** |
CN111008550A (zh) * | 2019-09-06 | 2020-04-14 | 上海芯灵科技有限公司 | 基于Multiple loss损失函数的指静脉验证身份的识别方法 |
US11687778B2 (en) | 2020-01-06 | 2023-06-27 | The Research Foundation For The State University Of New York | Fakecatcher: detection of synthetic portrait videos using biological signals |
US11301977B2 (en) * | 2020-04-10 | 2022-04-12 | General Electric Company | Systems and methods for automatic defect recognition |
TWI791979B (zh) * | 2020-04-28 | 2023-02-11 | 長庚醫療財團法人林口長庚紀念醫院 | 三維醫學影像的建立方法 |
CN111844025B (zh) * | 2020-07-02 | 2024-04-16 | 上海发那科机器人有限公司 | 一种涂胶机器人视觉数据的记录装置及记录方法 |
CN112451857A (zh) * | 2020-12-10 | 2021-03-09 | 深圳先进技术研究院 | 视网膜神经节细胞激活状态的判别方法、存储介质和设备 |
CN113269235B (zh) * | 2021-05-10 | 2022-12-27 | 青岛理工大学 | 一种基于无监督学习的装配体变化检测方法及设备 |
CN114257817B (zh) * | 2022-03-01 | 2022-09-02 | 浙江智慧视频安防创新中心有限公司 | 一种多任务数字视网膜特征流的编码方法及解码方法 |
CN114449280B (zh) * | 2022-03-30 | 2022-10-04 | 浙江智慧视频安防创新中心有限公司 | 一种视频编解码方法、装置及设备 |
WO2024097128A1 (en) * | 2022-10-31 | 2024-05-10 | University Of Pittsburgh - Of The Commonwealth System Of Higher Education | Neuromorphic programmable multiple pathways event-based sensors |
Family Cites Families (89)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0357212B1 (en) * | 1988-08-23 | 1998-04-29 | John G. Sutherland | Artificial neural device and corresponding method |
US5103306A (en) * | 1990-03-28 | 1992-04-07 | Transitions Research Corporation | Digital image compression employing a resolution gradient |
JP2809954B2 (ja) | 1992-03-25 | 1998-10-15 | 三菱電機株式会社 | 画像感知および処理のための装置および方法 |
DE69429006T2 (de) * | 1993-08-26 | 2002-07-18 | Univ California | Neuronales netzwerk-topographische sensorische organe und verfahren |
US5856152A (en) | 1994-10-28 | 1999-01-05 | The Trustees Of The University Of Pennsylvania | Hybrid adenovirus-AAV vector and methods of use therefor |
US5974159A (en) | 1996-03-29 | 1999-10-26 | Sarnoff Corporation | Method and apparatus for assessing the visibility of differences between two image sequences |
US5836996A (en) | 1996-12-30 | 1998-11-17 | Doorish; John F. | Artificial retina |
US6530954B1 (en) | 1997-02-21 | 2003-03-11 | Iip Technologies Gmbh | Adaptive senso-motor encoder for neuroprostheses |
DE19707046A1 (de) * | 1997-02-21 | 1998-08-27 | Rolf Prof Dr Ing Eckmiller | Lernfähiger "Active Vision" Implant Encoder |
WO1998048027A2 (en) | 1997-04-21 | 1998-10-29 | University Of Florida | Materials and methods for treatment of retinal diseases |
US6458157B1 (en) * | 1997-08-04 | 2002-10-01 | Suaning Gregg Joergen | Retinal stimulator |
US6971066B2 (en) * | 1997-08-18 | 2005-11-29 | National Instruments Corporation | System and method for deploying a graphical program on an image acquisition device |
NZ511037A (en) | 1998-09-17 | 2005-02-25 | Univ Florida | Methods for treatment of degenerative retinal diseases |
US6165192A (en) | 1999-01-05 | 2000-12-26 | Second Sight, Llc | Method and apparatus for intraocular retinal tack inserter |
EP1864690A3 (en) | 1999-03-24 | 2008-01-02 | Second Sight Medical Products, Inc. | Logarithmic light intensifier for use with photoreceptorbased implanted retinal prosthetics and those prosthetics |
JP5021119B2 (ja) | 1999-03-24 | 2012-09-05 | セカンド サイト メディカル プロダクツ インコーポレイテッド | 色覚回復用の網膜の人工色補装具 |
US7186560B2 (en) | 1999-09-21 | 2007-03-06 | Rutgers, The State University Of New Jersey | High level expression of immunogenic proteins in the plastids of higher plants |
US7925354B2 (en) | 2000-05-26 | 2011-04-12 | Second Sight Medical Products, Inc. | Video processing methods for improving visual acuity and/or perceived image resolution |
AU2001267058A1 (en) | 2000-06-09 | 2001-12-17 | University Of Florida Research Foundation, Inc. | Aav vector-based compositions and therapeutic uses thereof |
US6970745B2 (en) | 2000-08-09 | 2005-11-29 | The United States Of America As Represented By The Secretary Of The Navy | Microelectronic stimulator array for stimulating nerve tissue |
US6647297B2 (en) | 2000-08-09 | 2003-11-11 | The United States Of America As Represented By The Secretary Of The Navy | Permanent retinal implant device |
US7149586B2 (en) | 2002-03-28 | 2006-12-12 | Second Sight Medical Products, Inc. | Variable pitch electrode array |
EP1381276A4 (en) | 2001-04-13 | 2005-02-02 | Univ Pennsylvania | METHOD FOR TREATMENT OR DEVELOPMENT SLUDGE DEGRADATION |
EP1381425A1 (en) | 2001-04-24 | 2004-01-21 | Neurodan A/S | Functional electrical therapy system (fets) |
WO2002089912A2 (en) | 2001-05-03 | 2002-11-14 | Universite Catholique De Louvain | Vision rehabilitation method and device |
US6801655B2 (en) * | 2001-05-10 | 2004-10-05 | The United States Of America As Represented By The Secretary Of The Navy | Spatial image processor |
US9143706B2 (en) | 2001-06-06 | 2015-09-22 | Andrew Zador | Imaging system utilizing spatial image oscillation |
US6675164B2 (en) | 2001-06-08 | 2004-01-06 | The Regents Of The University Of California | Parallel object-oriented data mining system |
US7209788B2 (en) | 2001-10-29 | 2007-04-24 | Duke University | Closed loop brain machine interface |
US20030105409A1 (en) | 2001-11-14 | 2003-06-05 | Donoghue John Philip | Neurological signal decoding |
AU2002346611A1 (en) | 2001-12-03 | 2003-06-17 | The Regents Of The University Of California | Expression of glial-derived neurotrophic factor for treatment of diseases of the eye |
US20060193830A1 (en) | 2002-03-20 | 2006-08-31 | Hauswirth William W | Raav vector compositions and methods for the treatment of choroidal neovascularization |
US7203356B2 (en) | 2002-04-11 | 2007-04-10 | Canesta, Inc. | Subject segmentation and tracking using 3D sensing technology for video compression in multimedia applications |
WO2003093479A1 (en) | 2002-05-01 | 2003-11-13 | University Of Florida Research Foundation, Inc. | Improved raav expression systems and methods for enhancing transduction of mammalian neural cells |
WO2003104413A2 (en) | 2002-06-05 | 2003-12-18 | University Of Florida | Production of pseudotyped recombinant aav virions |
AU2003277910A1 (en) | 2002-09-27 | 2004-04-19 | Novartis Ag | Ocular gene therapy |
KR100506533B1 (ko) * | 2003-01-11 | 2005-08-05 | 삼성전자주식회사 | 이동로봇 및 그에 따른 자율주행 시스템 및 방법 |
US7311723B2 (en) | 2003-07-11 | 2007-12-25 | University Of Washington | Scanning laser device and methods of use |
ITMI20031449A1 (it) | 2003-07-15 | 2005-01-16 | St Microelectronics Srl | Metodo per classificare una immagine digitale |
JP4695605B2 (ja) | 2004-02-05 | 2011-06-08 | モトリカ リミテッド | 神経筋刺激 |
WO2005080573A1 (en) | 2004-02-20 | 2005-09-01 | Universite De Montreal | Recombinant viral vectors to promote neuronal cell survival and uses thereof |
CN1770177A (zh) | 2004-06-28 | 2006-05-10 | 微软公司 | 编码高密度几何符号集的***和方法 |
US7751585B2 (en) | 2004-06-28 | 2010-07-06 | Microsoft Corporation | System and method for encoding high density geometric symbol set |
ATE431677T1 (de) | 2004-07-13 | 2009-05-15 | Koninkl Philips Electronics Nv | Verfahren zur räumlichen und snr- bildkomprimierung |
US8103352B2 (en) | 2004-12-03 | 2012-01-24 | Second Sight Medical Products, Inc. | Mimicking neural coding in retinal ganglion cells with short pulse electrical stimulation |
US8520944B2 (en) * | 2004-12-24 | 2013-08-27 | Mario Cimbalista, JR. | Method for improving visualization of infrared images |
JP2008529684A (ja) | 2005-02-16 | 2008-08-07 | セカンド サイト メディカル プロダクツ インコーポレイテッド | 持続性が増加した神経刺激 |
AU2006284425A1 (en) | 2005-07-22 | 2007-03-01 | The Board Of Trustees Of The Leland Stanford Junior University | Light-activated cation channel and uses thereof |
US20070050046A1 (en) | 2005-07-25 | 2007-03-01 | Georgopoulos Apostolos P | Methods for generating a signal indicative of an intended movement |
US8956396B1 (en) | 2005-10-24 | 2015-02-17 | Lockheed Martin Corporation | Eye-tracking visual prosthetic and method |
US20070198066A1 (en) | 2005-11-03 | 2007-08-23 | Greenberg Robert J | Method and apparatus for visual neural stimulation |
JP4572175B2 (ja) | 2006-04-25 | 2010-10-27 | 日本電信電話株式会社 | 非定常映像検出装置,非定常映像検出方法及びその方法を実装したプログラム |
WO2007127428A2 (en) | 2006-04-28 | 2007-11-08 | University Of Florida Research Foundation, Inc. | Double-stranded/self-complementary vectors with a truncated cba promoter and methods of gene delivery |
JP2009536219A (ja) | 2006-05-04 | 2009-10-08 | ウェイン・ステート・ユニバーシティー | ロドプシン拡散のインビボ送達による視覚応答の回復 |
US8197539B2 (en) | 2006-05-05 | 2012-06-12 | University Of Southern California | Intraocular camera for retinal prostheses |
US8311634B2 (en) | 2006-06-16 | 2012-11-13 | Second Sight Medical Products Inc. | Apparatus and method for electrical stimulation of human retina |
GB0612242D0 (en) | 2006-06-21 | 2006-08-02 | Imp Innovations Ltd | Retinal prosthetic devices |
EP1891976A1 (en) | 2006-08-23 | 2008-02-27 | Novartis Forschungsstiftung, Zweigniederlassung Friedrich Miescher Institute for Biomedical Research | Use of light sensitive genes |
CN100481123C (zh) | 2007-03-08 | 2009-04-22 | 上海交通大学 | 采用时空滤波器的视网膜编码器实现方法 |
EP2134415B1 (en) | 2007-03-08 | 2017-05-03 | Second Sight Medical Products, Inc. | Flexible circuit electrode array |
US9799098B2 (en) * | 2007-04-24 | 2017-10-24 | Massachusetts Institute Of Technology | Method and apparatus for image processing |
EP1995685A3 (en) | 2007-05-21 | 2012-08-01 | Biotronik CRM Patent AG | Medical device for monitoring biological signal |
US20090105786A1 (en) | 2007-10-22 | 2009-04-23 | University Of Washington | Method and device for strengthening synaptic connections |
US8195302B2 (en) | 2007-11-07 | 2012-06-05 | Second Sight Medical Products, Inc. | Video processing unit for a visual prosthetic apparatus |
US8195303B2 (en) | 2007-11-07 | 2012-06-05 | Second Sight Medical Products, Inc. | Video processing unit for a visual prosthetic apparatus |
EP2222372A2 (en) | 2007-12-06 | 2010-09-01 | Technion Research & Development Foundation Ltd. | Method and system for optical stimulation of neurons |
CN100586403C (zh) * | 2008-03-06 | 2010-02-03 | 上海交通大学 | 视觉假体图像处理装置及方法 |
CN102014731A (zh) | 2008-04-08 | 2011-04-13 | 新加坡国立大学 | 视网膜图像分析***和方法 |
EP2315833B8 (en) | 2008-05-20 | 2015-05-27 | Eos Neuroscience, Inc. | Vectors for delivery of light-sensitive proteins and methods of use |
US20100016732A1 (en) | 2008-07-17 | 2010-01-21 | Lockheed Martin Corporation | Apparatus and method for neural-signal capture to drive neuroprostheses or control bodily function |
EP2320830A4 (en) * | 2008-08-07 | 2012-02-01 | Massachusetts Inst Technology | CODING FOR SEA PROSTHESES |
CN101336856B (zh) | 2008-08-08 | 2010-06-02 | 西安电子科技大学 | 辅助视觉***的信息获取与传递方法 |
CN101393789A (zh) | 2008-11-04 | 2009-03-25 | 江苏圣安电缆有限公司 | 交联聚乙烯绝缘电力电缆内气体的排除方法 |
US8179452B2 (en) | 2008-12-31 | 2012-05-15 | Lg Electronics Inc. | Method and apparatus for generating compressed file, and terminal comprising the apparatus |
US8108147B1 (en) * | 2009-02-06 | 2012-01-31 | The United States Of America As Represented By The Secretary Of The Navy | Apparatus and method for automatic omni-directional visual motion-based collision avoidance |
US8744588B2 (en) | 2009-05-07 | 2014-06-03 | Hani Midani | Method and system for connecting an impaired nervous system to a muscle or a group of muscles based on template matching and intelligent end points |
DK2539015T3 (da) | 2010-02-26 | 2016-04-04 | Univ Cornell | Nethindeprotese |
US9180309B2 (en) | 2010-02-26 | 2015-11-10 | Cornell University | Retina prosthesis |
US20110213266A1 (en) | 2010-03-01 | 2011-09-01 | Williams Justin C | Closed Loop Neural Activity Triggered Rehabilitation Device And Method |
US9311593B2 (en) * | 2010-03-26 | 2016-04-12 | Brain Corporation | Apparatus and methods for polychronous encoding and multiplexing in neuronal prosthetic devices |
US8315305B2 (en) * | 2010-03-26 | 2012-11-20 | Brain Corporation | Systems and methods for invariant pulse latency coding |
US20110307079A1 (en) | 2010-04-29 | 2011-12-15 | Board Of Trustees Of Michigan State University, The | Multiscale intra-cortical neural interface system |
US8527056B2 (en) | 2010-04-30 | 2013-09-03 | Second Sight Medical Products, Inc. | Encoding of size and brightness of percepts in a visual prosthesis |
US9302103B1 (en) | 2010-09-10 | 2016-04-05 | Cornell University | Neurological prosthesis |
US20120123293A1 (en) | 2010-11-11 | 2012-05-17 | IINN, Inc. | Motor nerve root stimulation |
CA2883091C (en) | 2011-08-25 | 2020-02-25 | Cornell University | Retinal encoder for machine vision |
CN104168950A (zh) * | 2011-12-20 | 2014-11-26 | 约翰霍普金斯大学 | 用于神经假体的安全的直流激励器的人工控制和微型化 |
CZ304893B6 (cs) | 2012-06-20 | 2015-01-07 | Miloslav Hering | Kolečková brusle |
US9773155B2 (en) | 2014-10-14 | 2017-09-26 | Microsoft Technology Licensing, Llc | Depth from time of flight camera |
-
2012
- 2012-08-24 CA CA2883091A patent/CA2883091C/en active Active
- 2012-08-24 US US14/239,828 patent/US9547804B2/en active Active
- 2012-08-24 KR KR1020197012590A patent/KR102111000B1/ko active IP Right Grant
- 2012-08-24 EP EP12825425.7A patent/EP2748765B1/en active Active
- 2012-08-24 WO PCT/US2012/052348 patent/WO2013029008A1/en active Application Filing
- 2012-08-24 JP JP2014527338A patent/JP6117206B2/ja active Active
- 2012-08-24 CN CN201280052177.5A patent/CN103890781B/zh active Active
- 2012-08-24 KR KR1020147007453A patent/KR101976048B1/ko active IP Right Grant
-
2014
- 2014-02-20 IL IL231063A patent/IL231063A/en active IP Right Grant
- 2014-12-24 HK HK14112927.7A patent/HK1199768A1/xx unknown
-
2017
- 2017-01-17 US US15/408,178 patent/US10303970B2/en active Active
-
2019
- 2019-05-23 US US16/421,147 patent/US10769483B2/en active Active
-
2020
- 2020-09-03 US US17/011,651 patent/US11640681B2/en active Active
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2022075349A1 (ja) | 2020-10-08 | 2022-04-14 | 国立研究開発法人理化学研究所 | 画像処理装置、画像処理方法、及び画像処理プログラムを格納した非一時的なコンピュータ可読媒体 |
Also Published As
Publication number | Publication date |
---|---|
IL231063A0 (en) | 2014-03-31 |
CN103890781A (zh) | 2014-06-25 |
EP2748765B1 (en) | 2022-12-14 |
JP2014524630A (ja) | 2014-09-22 |
KR20190051076A (ko) | 2019-05-14 |
US20190279021A1 (en) | 2019-09-12 |
KR101976048B1 (ko) | 2019-05-09 |
CN103890781B (zh) | 2017-11-21 |
US11640681B2 (en) | 2023-05-02 |
WO2013029008A1 (en) | 2013-02-28 |
US20140355861A1 (en) | 2014-12-04 |
US10303970B2 (en) | 2019-05-28 |
US10769483B2 (en) | 2020-09-08 |
US20170255837A1 (en) | 2017-09-07 |
KR20140050741A (ko) | 2014-04-29 |
US9547804B2 (en) | 2017-01-17 |
CA2883091C (en) | 2020-02-25 |
EP2748765A4 (en) | 2016-06-22 |
US20200401837A1 (en) | 2020-12-24 |
IL231063A (en) | 2016-10-31 |
HK1199768A1 (en) | 2015-07-17 |
EP2748765A1 (en) | 2014-07-02 |
CA2883091A1 (en) | 2013-02-28 |
KR102111000B1 (ko) | 2020-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP6117206B2 (ja) | 機械視覚のための網膜符号化器 | |
CN107624061B (zh) | 具有维度数据缩减的机器视觉 | |
Trang et al. | Mango diseases identification by a deep residual network with contrast enhancement and transfer learning | |
CN112464844A (zh) | 一种基于深度学习与运动目标检测的人体行为动作识别方法 | |
CN112464930A (zh) | 目标检测网络构建方法、目标检测方法、装置和存储介质 | |
Jain et al. | An automated hyperparameter tuned deep learning model enabled facial emotion recognition for autonomous vehicle drivers | |
Gao et al. | Image Segmentation of Retinal Blood Vessels Based on Dual‐Attention Multiscale Feature Fusion | |
Putro et al. | Fast person detector with efficient multi-level contextual block for supporting assistive robot | |
Noor et al. | Multilevel thresholding of gel electrophoresis images using firefly algorithm | |
CN111553202B (zh) | 进行活体检测的神经网络的训练方法、检测方法及装置 | |
Hassan et al. | Optimising deep learning by hyper-heuristic approach for classifying good quality images | |
Nejatian et al. | Implementation real-time gender recognition based on facial features using a hybrid neural network Imperialist Competitive Algorithm | |
Huang et al. | Hierarchical Learning-Guided human motion quality assessment in big data environment | |
Khan et al. | Surrogate Gradient-Based Medical Image Classification Using Spike Neural Network | |
Xu | Deep Learning for Image Classification and Segmentation with Scarce Labelled Data | |
Greco | OPTIMIZING A CONVOLUTIONALNETWORK USING GENETICALGORITHM FOR PEDESTRIANINTENTION ESTIMATION | |
Bondalapati et al. | Intelligent Video Surveillance Systems Using Deep Learning Methods | |
Keil | From neuronal models to neuronal dynamics and image processing | |
Garbade | Semantic Segmentation and Completion of 2D and 3D Scenes | |
Malalur | Interpretable neural networks via alignment and dpstribution Propagation | |
Rueckauer et al. | Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning | |
CN114743134A (zh) | 一种基于多线索融合的动物行为识别方法 | |
Petkov et al. | Generalized Net Model of Face Recognition using ART2 Neural Network and Sobel Filter |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20150821 |
|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20150821 |
|
A977 | Report on retrieval |
Free format text: JAPANESE INTERMEDIATE CODE: A971007 Effective date: 20160726 |
|
A131 | Notification of reasons for refusal |
Free format text: JAPANESE INTERMEDIATE CODE: A131 Effective date: 20160801 |
|
A521 | Request for written amendment filed |
Free format text: JAPANESE INTERMEDIATE CODE: A523 Effective date: 20161031 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20170222 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20170322 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 6117206 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |
|
R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |