CN107063260A - A kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map - Google Patents
A kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map Download PDFInfo
- Publication number
- CN107063260A CN107063260A CN201710180995.8A CN201710180995A CN107063260A CN 107063260 A CN107063260 A CN 107063260A CN 201710180995 A CN201710180995 A CN 201710180995A CN 107063260 A CN107063260 A CN 107063260A
- Authority
- CN
- China
- Prior art keywords
- cell
- mrow
- rat
- msub
- gitter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Automation & Control Theory (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
A kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map of the present invention, belongs to bionics technical field.It is primarily based on striped cell to build the wild model of gitter cell grid, build the wild model in single Place cell position secondly based on gitter cell, finally builds in rat brain " cognitive map ".On the basis of constructed cognitive map, build one and include the BP network model of input layer, Place cell layer, action cellular layer and output layer, and realize using Q learning algorithms a certain target navigation task of rat space-oriented environment.The inventive method can be widely applied to the numerous areas such as bio-robot navigation, artificial intelligence.
Description
Technical field
The present invention relates to a kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map, belong to bionics technology neck
Domain.
Background technology
Autonomous positioning and towards a certain target navigation ability of residing space environment for animal and autonomous mobile robot extremely
Close important.Although robot can determine that it is presently in space by some prior informations in specific sensor or environment
Positional information, but in the case of not any priori, animals and humans are by coming from the imperfect space of sense organ
Information always can promptly position oneself and be presently in positional information.With in rat brain with Context aware related streak cell,
The discovery successively of gitter cell, Place cell, rapid self the determining in residing space environment of animals and humans is understood to deep
Position process provides possibility.
The three-way navigation description of test rat that Tolman in 1948 is done can be carried out by inherent " cognitive map "
Navigation, the cognitive map is that the global table of its residing space environment is reached in rat brain." cognitive map " is led in rat space
Vital effect is play in boat.Therefore, how " cognitive map " is formed, space how is carried out based on " cognitive map " leads
Boat is still an important experiment and theoretical question.
1971, O'keefe and Dostrovesky had found to live with space orientation first in rat hippocampus structure hippocampus
Dynamic Place cell neuron.Rat is when two-dimensional space is movable, when it passes through a certain specific narrow regions in environment, correspondence
Place cell will be discharged, and this region is referred to as Place cell by single Place cell to that should have single Place cell activation domain
Position is wild.With further research, the single position open country of Place cell is proved to be the basic composition unit of " cognitive map ", single
One Place cell position is wild to have accurate corresponding relation, multiple Place cell combined coding rats with locus residing for rat
Local environment, " cognitive map " is formed in rat brain.2005, Hafting et al. was found that separately in rat entorhinal cortex
A kind of outer gitter cell neuron with strong spatial flash-over characteristic, the neuron can be showed to a certain specific in space
The characteristic of repeated rule electric discharge occurs for region, and it is wild that this region is referred to as gitter cell grid.From unlike Place cell,
Gitter cell can discharge in multiple positions, and multiple gitter cell grid open countries mutually overlap into grid node one by one, connect grid
The equilateral triangle that node is formed is throughout the whole space environment residing for rat.Hippocampal formation truck shows that entorhinal cortex is arrived
Hippocampus Fiber Projections indicate the main input source that entorhinal cortex is hippocampus in hippocampal formation, i.e., gitter cell is as defeated
Enter to obtain the single Place cell position of hippocampus wild.Krupic in 2012, which is equal to, is being close to the cell of subiculum and the shallow cortex of entorhinal cortex
Electric discharge sampling display, has the wild cell of periodic stripe cell discharge, is referred to as striped cell, striped cell, which is mainly, to be passed through
Rat autokinesis information (speed and directional information) is integrated to obtain striped cell discharge characteristic, i.e. striped cell bar ripple.Fu
In leaf analysis show that the interaction of multiple striped cells can form periodic gitter cell grid open country.
Single Place cell position open country is the basic component unit of " cognitive map ", how to obtain single Place cell position
Open country is an important theoretical question.Since 1971 find Place cell, the wild model of many diverse location cell positions
Proposed in succession, since being found from Place cell, many wild models in different Place cell positions are proposed in succession, including
There are Gaussian function model, competition learning model, independent component analysis model, self organizing maps model and Kalman filter model
Deng.But model above is that they are all individually modeled just for Place cell the problem of exist, will have therewith
Striped cell, the gitter cell of information association are taken into account.
When rat enters a certain space environment, with rat freely activity, Place cell " position is wild " is quickly generated
And cover whole space environment residing for rat.Continuous exploration i.e. with rat to environment, Place cell " position is wild " is formd
Characterize the cognitive map of space environment residing for rat.Some experimental studies enter to the object-oriented navigation learning of rodent
Correlative study is gone, the object-oriented navigation based on hippocampus " cognitive map " is mainly what is completed by intensified learning, biological
Learn research find, only with cognitive map (Place cell discharge activities) can not correctly predicted rat Future movement direction, greatly
Brain ventral tegmental area (ventral tegmental area, VTA) is mainly the dopamine related to reward predictive error signal
Serotonergic neuron (dopaminergic neurons), information is entered to be projected to nucleus accumbens septi by dopaminergic neuron neuron
(nucleus accumbens, NA), nucleus accumbens septi input mostlys come from hippocampus, there are between prefrontal cortex and nucleus accumbens septi
Two-way Fiber Projections.I.e. nucleus accumbens septi receives space environment information residing for rat from hippocampus, receives related from brain ventral tegmental area
Prediction error information is rewarded, and comes correctly predicted rat Future movement direction with prefrontal cortex interaction.Brain forehead
Neuron-action cell mainly related to motion in leaves layer.And it is mainly Place cell in hippocampus.It is biological based on more than
Related close is learned to find, the object-oriented navigation task neural basal of rat be probably hippocampus Place cell and nucleus accumbens septi neuron it
Between related to prize signal cynapse regulation, information projection further realized rat just by nucleus accumbens septi to rat prefrontal cortex
Really predict Future movement direction.Realize that rat is object-oriented by intensified learning between continuous state space and motion space
Navigation task, continuous state space herein is referred in Place cell discharge activities, i.e. rat brain " cognitive map ".It is based on
This, builds a BP network model for including input layer, Place cell layer, action cellular layer and output layer, using Q
Learning algorithm realizes the object-oriented navigation task of rat.
The content of the invention
In summary discuss, the purpose of the present invention is to build rat based on striped cell, gitter cell, Place cell first
Brain " cognitive map ".Next to that on the basis of cognitive map, building one and including input layer, Place cell layer, action cell
The BP network model of layer and output layer, the object-oriented navigation task of rat is realized using Q learning algorithms.
To achieve the above object, the technical solution adopted by the present invention is a kind of imitative based on mouse cerebral hippocampal structure cognitive map
Raw air navigation aid, model overall schematic is as shown in Figure 1.From model overall schematic, model is made up of two big modules.The
One module is to be based on rat autokinesis information, how to obtain the expression for its residing space environment inside rat brain, that is, recognizes
Know map, its structural representation is as shown in Figure 2.Second module is on the basis of cognitive map, to be realized by Q study big
The a certain target navigation task of mouse space-oriented environment, its structural representation is as shown in Figure 3.
Two models adopt the following technical scheme that realization：
S1 builds a virtual rat random search two-dimensional space environment and obtains its two-dimensional space movement locus figure, wherein
The autokinesis information of rat is made up of speed and directional information, using trunnion axis as reference direction, αtRat is represented when front direction,
vtRepresent rat present speed size.
S2 is that speed and directional information obtain striped cell spatial characterization, i.e. striped cell bar based on rat autokinesis information
Ripple.Striped cell discharge activitiesFor：
Wherein, r=(x, y) represents rat and is presently in environment position coordinate, kiRepresent wave vector, i=1,2,3, wave vector
What amount direction was represented is the direction that ripple equiphase is advanced, and size is referred to as wave number ki, kiFor：
Wherein, λ cos represent cos ripple wavelength.
It is wild that S3 obtains gitter cell spatial characterization, i.e. gitter cell grid based on striped cell spatial characterization.Three directions
The striped cell superposition of 60 ° of difference obtains gitter cell grid open country, and gitter cell grid open country spatial characterization ψ (r) is：
Wherein, r=(x, y) represents rat and is presently in environment position coordinate.
S4 obtains Place cell spatial characterization based on gitter cell spatial characterization, i.e., single Place cell position is wild, position
Cell spaces characterize P (x, y)：
Wherein, WnThe connection weight between n-th of gitter cell and Place cell is represented, gn (x, y) represents n-th of grid
Cell is located at the activity ratio at space environment (x, y) location point, and N represents gitter cell quantity, N=4,10,20.
S5 rats form the Place cell discharge characteristic in each location point, finally during constantly environment is explored
Form the expression to its residing space environment, i.e. cognitive map in rat brain.
S6 has built one and has been made up of input layer, Place cell, action cell and output layer on the basis of cognitive map
BP network model simultaneously realizes the object-oriented navigation task of rat by Q learning algorithms.
Brief description of the drawings
Fig. 1 model overall schematics of the present invention.
Fig. 2 mouse cerebral hippocampals cognitive map builds structural representation.
Fig. 3 is based on mouse cerebral hippocampal cognitive map and learns object-oriented navigational structure schematic diagram with Q.
Fig. 4 rat space motion path schematic diagrames.
Fig. 5 is in the wild schematic diagram of striped cell bar ripple gitter cell grid
Graph of a relation between Fig. 6 gitter cells grid open country spacing λ and two dimension cos ripple wavelength Xs cos.
Fig. 7 gitter cells linear superposition handles experimental result schematic diagram without Sigmoid functions
Fig. 8 Sigmoid functional arrangements
Fig. 9 gitter cells linear superposition handles experimental result schematic diagram through Sigmoid functions
Figure 10 cognitive map forming process schematic diagrams
Figure 11 experimental situation schematic diagrames
Figure 12 BP network model schematic diagrames
Figure 13 input layers are to position cellular layer feedforward network schematic diagram.
Figure 14 rat space navigation schematic diagrames.
Figure 15 is by input layer (Place cell), the feed-forward network model schematic diagram of action cell construction.
40 running orbit experimental result schematic diagrames of Figure 16 rats
Figure 17 rats reach step number schematic diagram needed for target location
Embodiment
The present invention is further explained below in conjunction with drawings and examples.
S1 rat autokinesis information is made up of head direction and velocity information.Using trunnion axis as reference direction, αtRepresent rat
When front direction.vtRepresent rat present speed size.Δ t represents the time cycle.Based on the current autokinesis information of rat and upper
One moment rat positional information (xt-1,yt-1) calculate rat current location information (xt,yt), current autokinesis information is referred to
Head is towards αtWith speed vt, as shown in formula (5).
Rat starting position coordinates are (x0,y0)=(0,0), rat space motion path figure is as shown in Figure 4.
S2 striped cells are found to be present in the 3rd layer of entorhinal cortex, and electric discharge of the cell in two-dimensional space environment is lived
Dynamic is cluster cluster bar ripple, and striped cell is passed the information on into the 2nd layer of entorhinal cortex after integrating rat autokinesis information, its institute
The cluster cluster bar ripple of generation is based on by being superimposed gitter cell grid open country of the formation with different spaces position phase, orientation, spacing
Striped cell bar ripple gitter cell grid open country schematic diagram is as shown in Figure 5.
For striped cell discharge activities, represented with two-dimentional cos ripples, as shown in formula (6).
Wherein, r=(x, y) represents rat and is presently in environment position coordinate, and what wave vector direction was represented is ripple equiphase
The direction of traveling, size is referred to as wave number ki, as shown in formula (7).
Wherein, λ cos represent cos ripple wavelength.
S3 biological studies confirm that the whole environment that gitter cell grid open country is dispersed throughout residing for rat in equilateral triangle is worked as
In.Based on this, gitter cell activity ratio function can be represented by three striped cell discharge activities superpositions, three striped cells
Wave vector is towards differing 60 °, as shown in formula (8).
As shown in formula (8), during r=(0,0), Ψ (r) has maximum to be 1.If selecting any one space bit in space environment
Phase r0=(x0, y0) is changed into such as formula (9) as the wild a certain peak point of gitter cell grid, then gitter cell activity ratio function
It is shown.
ψ (r)=ψ (r-r0) (9)
Wave vector is the function of wave number, and gitter cell grid open country spacing is used as the 1 of sign gitter cell space discharge characteristic
Individual parameter.As shown in fig. 6, circle represents the wild node of gitter cell grid, that travers are represented is striped cell two dimension cos
Ripple, shown in the relation such as formula (10) between gitter cell grid open country spacing λ and the wavelength X cos of two dimension cos ripples.
Again from formula (7), shown in the relation such as formula (11) between wave number and the wild spacing λ of gitter cell grid.
S4 is wild in order to obtain the equilateral triangle grid cell grid consistent with biologically finding, chooses wave vector direction
Gitter cell grid needed for being obtained after three striped cells superposition of 60 ° of difference is wild.60 °, 120 °, 180 °, choose k1, k2, k3
As shown in formula (12).
Wherein, θ represents gitter cell grid open country orientation.
From formula (8), Ψ (r) values are between [- 1/2,1], in order that gitter cell activity ratio value is arrived between 0
Between 1, gitter cell activity ratio function changes as shown in formula (13).
Formula (8) and formula (9) are substituted into formula (13) to obtain shown in gitter cell activity ratio function such as formula (14).
It is sea that entorhinal cortex indicates entorhinal cortex to hippocampus Fiber Projections in S5 biological studies discovery hippocampal formation
The main input source of hippocampus in horse structure.Gitter cell and Place cell are nerve cell, they be by cell body and
Cell process is constituted, and cell process is the elongated portion that cell body itself extends out, and elongated portion is divided into dendron and axle again
It is prominent.Signal, is delivered to other tissues or another neuron by each neuron only one of which aixs cylinder, and each neuron has multiple
Dendron, receiving stimulates and by excited incoming cell body, and information transmission is also such between gitter cell and Place cell.Ascend the throne
Cell is put from the gitter cell receive information with different spaces feature, then the weights between coupled gitter cell enter
Place cell discharge characteristic is obtained after row weighted sum, the connection weight function such as formula (15) between gitter cell and Place cell
It is shown.
Wherein, Wn represents the connection weight between n-th of gitter cell and Place cell, and λ n represent n-th gitter cell
Grid open country spacing, σ (σ=8cm) represents Place cell electric discharge activation domain standard deviation.
S6 is from formula (14) and formula (15), shown in Place cell activity ratio function such as formula (16).
Wherein, Wn represents the connection weight between n-th of gitter cell and Place cell, and gn (x, y) represents n-th of grid
Cell is located at the activity ratio at space environment (x, y) location point, and N represents gitter cell quantity, N=4,10,20.
S7 biological studies find that simple inputs the connection weight between gitter cell and Place cell to gitter cell
It is that the position with multiple activation domains is wild to carry out the output that frequently results in of linear superposition, its result schematic diagram as shown in fig. 7, this with
Existing correlative study confirms single Place cell to that should have the wild conclusion in single position inconsistent.Herein in gitter cell grid
Between open country input and gitter cell and Place cell on the basis of connection weight weighted sum, Sigmoid functions are introduced,
Sigmoid functional arrangements are as shown in figure 8, to the output after linear superposition handle obtaining finding phase one with biological study
The single Place cell position caused is wild, realizes mapping relations of the gitter cell to single Place cell position between wild, it is tested
Result schematic diagram is as shown in figure 9, the Place cell activity ratio such as formula (17) after being handled through Sigmoid functions is shown.
P'(x, y)=1/ (1+e-(P-b)/a) (17)
Wherein, what P was represented is Place cell activity ratio, and what a was represented is Sigmoid function inclination factors, and what b was represented is
Sigmoid functions center.
S8 rats form the Place cell discharge characteristic in each location point, finally during constantly environment is explored
Form the expression to its residing space environment, i.e. cognitive map, cognitive map forming process schematic diagram such as Figure 10 institutes in rat brain
Show.
Some experimental studies of S9 have carried out correlative study to the object-oriented navigation learning of rodent, based on hippocampus
The object-oriented navigation of " cognitive map " is mainly what is completed by intensified learning, and biological study is found, only with cognitive ground
Scheming (Place cell discharge activities) can not correctly predicted rat Future movement direction, brain ventral tegmental area
(ventraltegmentalarea, VTA) is mainly the dopaminergic neuron related to reward predictive error signal
(dopaminergicneurons), information is entered to be projected to nucleus accumbens septi (nucleus by dopaminergic neuron neuron
Accumbens, NA), nucleus accumbens septi input mostlys come from hippocampus, and two-way fiber is there are between prefrontal cortex and nucleus accumbens septi and is thrown
Penetrate.I.e. nucleus accumbens septi receives space environment information residing for rat from hippocampus, and receiving resultant awards prediction from brain ventral tegmental area misses
Poor information, and come correctly predicted rat Future movement direction with prefrontal cortex interaction.It is main in Prefrontal Cortex cortex
If the neuron related to motion-action cell.And it is mainly Place cell in hippocampus.Hair is closed based on above biology correlation
Existing, the object-oriented navigation task neural basal of rat is probably with rewarding letter between hippocampus Place cell and nucleus accumbens septi neuron
Information projection is further realized rat correctly predicted future by number related cynapse regulation, nucleus accumbens septi to rat prefrontal cortex
The direction of motion.The object-oriented navigation task of rat is realized by intensified learning between continuous state space and motion space,
Continuous state space herein is referred in Place cell discharge activities, i.e. rat brain " cognitive map ".Based on this, one is built
Include input layer, Place cell layer, action cellular layer and output layer BP network model, using Q learning algorithms come
Realize the object-oriented navigation task of rat.
The square cartridge that S10 experimental situations, which are sizes, to be made up of 10000 × 10000 points is (such as Figure 11 institutes
Show).As rat constantly explores its residing space environment, the Place cell position for gradually forming each location point of space is wild, most
Formd eventually in rat brain and-cognitive map is levied to the inherently chart of environment.Therefore in a model, it will work as herein residing for rat
Front position point information (xt,yt) it is used as input information.
S11 constructs a feedforward neural network mould being made up of input layer, Place cell, action cell and output layer
Type realizes the object-oriented navigation task of rat, and BP network model is as shown in figure 12.
S12 input layers are as shown in figure 13 to position cellular layer feedforward network.In input layer, X is inputted：(xt,yt) work as rat
Preceding present position inputs information.The feedforward network is a network connected entirely, and each neuron of input layer passes through connection
Weight Wi=[wi,1,wi,2···wi,n] be sequentially connected with all neurons of feedforward network output layer.Here, i=1 ... Q, Q
=500 be Place cell sum.Weight is by function fuCarry out random initializtion, function fuDescribed by below equation (18).
In formula, u comes from obeying [0；1] an equally distributed random value, v=0.5 and σ=0.2 between.Utilize
Information and weight is inputted to calculate Place cell discharge rate (see formula (19)), first random initializtion weights.Learned using competition
Algorithm is practised, Place cell can be encouraged by some specific input, so that the Place cell has choosing for locus
Selecting property.
I-th of Place cell discharge rate is described with below equation (19)：
In formula, σf=0.07 defines the wild width in Place cell position, and n is the space dimensionality for inputting information, and norm is represented
Be Euclidean distance.According to the victor is a king, mechanism is adjusted weights in constructed BP network model, that is,
Say the Place cell neuron χ won using Competitive Learning AlgorithmtWeights between input information can change, remaining
Do not change, triumph Place cell neuron χtDescribed with below equation (20)：
χt=argmini||Xt-Wt i|| (20)
The weights of triumph neuron change according to below equation (21)：
In formula, what 0 ＜ α ＜ ＜ 1 were represented is the learning efficiency factor.
S13 is rat current location information as input and produces influence to the activity of position cell discharge first, next to that position
Cell is put with motor neuron to be connected by the certain action of Q study generations.Rat can be from any by the continuous theory of study
Beginning, position was to the navigation between target location, and its space navigation signal is as shown in figure 14.Using experimental situation as shown in figure 11,
The original position of rat local environment is located at experimental situation lower left (as shown in Figure 14 orbicular spots mark).Target point is located at real
Test environment upper right side (as shown in square in Figure 14).During beginning, the exploration environment of rat at random, in the mistake of random search environment
Find source location (random search path is as shown in phantom in Figure 14) in journey, and when rat after learning after a while,
It can just find the shortest path from original position to target location quickly.In experimentation, whenever rat finds target
After location point, rat is reapposed over the experiment that original position restarts a new round.In the model of structure, most of
In the case of (80%), rat first test in can find source location in 200 steps, that is to say, that even if rat is
First during random search experimental situation herein, 200 steps are so that rat finds source location.
S14 based on hippocampus navigate research institute it is conventional be nitrification enhancement.It is used herein to be and Reynolds classes
As Q learning algorithms.The algorithm is applied in the feedforward neural network (as shown in figure 15) of two layers, wherein Place cell work
For the input of the network.Each Place cell successively with representing 8 different directions (northern (N), northeast (NE), eastern (E), east respectively
Southern (SE), southern (S), southwestern (SE), western (W), northwest (NW)) motor neuron be connected.The actual direction of motion is by eight
Maximum Q values are determined in possible direction.In horizontal direction the motion of rat westwards eastwards by below equation (22) and (23) Lai
Description：
Δ x=± (Δ s+c ψx) (22)
Δ y=c ψy (23)
In formula, what Δ s=500 was represented is that rat often walks step size, ψxAnd ψyCome from obedience [- 1；1] it is uniformly distributed
Random value, c=100 is noise amplitude.Negative sign represents that rat westwards moves, and positive sign represents that rat moves eastwards.Likewise, right
Described in the motion of rat on southwest and northeastward with below equation (24) and (25)：
S15 rat motors to when to calculate gained Q values be 0 for current location, rat be no longer only limitted in current location (north (N),
Northeast (NE), eastern (E), the southeast (SE), southern (S), southwestern (SE), western (W), northwest (NW)) 8 directions, but visited at random
Suo Yundong, the direction of motion is not known.But in this case, rat keeps the constant possibility in direction to be 1-pk, and it with
Machine selects the probability of a new direction to be pk=0.25.When Q values are not 0, with continuous exploration of the rat for environment, mostly
Number time rat can all determine the direction of motion of current location subsequent time according to Q values.From Place cell to action cell
Study mechanism is Q learning algorithms.For simplicity i-th of Place cell discharge rate of t is retouched with below equation (26)
State：
In formula, i=1 ... Q, Q=500 are Place cell sums.
S16 defines action value function by below equation (27)：
In formula, Γi,aWhat is represented is the connection weight between i-th of Place cell and motor neuron a.According to
The average Q learning rules of use mentioned by Reynolds.Namely really produced in moment t according to below equation (28) to update
Act atWeights
In formula, what β=0.7 was represented is learning rate, and what δ=0.7 was represented is reduction coefficient, and what R was represented is reward.Will prize
Encourage function RtDescribed to minor function (29)：
The present invention is mainly based upon striped cell, gitter cell, Place cell and builds rat brain " cognitive map ".Recognizing
Know on the basis of map, build a feedforward neural network for including input layer, Place cell layer, action cellular layer and output layer
Model, the object-oriented navigation task of rat is realized using Q learning algorithms.40 running orbit experimental results of rat and rat arrive
Step number schematic diagram difference is as shown in figure 17 needed for up to target location.
Claims (1)
1. a kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map, the model of this method is made up of two big modules；
First module is to be based on rat autokinesis information, how to obtain the expression for its residing space environment inside rat brain, i.e.,
Cognitive map；Second module is that on the basis of cognitive map, a certain target of rat space-oriented environment is realized by Q study
Navigation task；
Two models adopt the following technical scheme that realization：
S1 builds a virtual rat random search two-dimensional space environment and obtains its two-dimensional space movement locus figure, wherein rat
Autokinesis information be made up of speed and directional information, using trunnion axis as reference direction, αtRat is represented when front direction, vtGeneration
Table rat present speed size；
S2 is that speed and directional information obtain striped cell spatial characterization, i.e. striped cell striped based on rat autokinesis information
Ripple；Striped cell discharge activitiesFor：
Wherein, r=(x, y) represents rat and is presently in environment position coordinate, kiRepresent wave vector, i=1,2,3, wave vector direction
What is represented is the direction that ripple equiphase is advanced, and size is referred to as wave number ki, kiFor：
<mrow>
<msub>
<mi>k</mi>
<mi>i</mi>
</msub>
<mo>=</mo>
<mfrac>
<mrow>
<mn>2</mn>
<mi>&pi;</mi>
</mrow>
<msub>
<mi>&lambda;</mi>
<mi>cos</mi>
</msub>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, λ cos represent cos ripple wavelength；
It is wild that S3 obtains gitter cell spatial characterization, i.e. gitter cell grid based on striped cell spatial characterization；Three direction differences
60 ° of striped cell superposition obtains gitter cell grid open country, and gitter cell grid open country spatial characterization ψ (r) is：
<mrow>
<mi>&psi;</mi>
<mrow>
<mo>(</mo>
<mi>r</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfrac>
<mrow>
<mi>c</mi>
<mi>o</mi>
<mi>s</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>k</mi>
<mn>1</mn>
</msub>
<mi>r</mi>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>c</mi>
<mi>o</mi>
<mi>s</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>k</mi>
<mn>2</mn>
</msub>
<mi>r</mi>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>c</mi>
<mi>o</mi>
<mi>s</mi>
<mrow>
<mo>(</mo>
<msub>
<mi>k</mi>
<mn>3</mn>
</msub>
<mi>r</mi>
<mo>)</mo>
</mrow>
</mrow>
<mn>3</mn>
</mfrac>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, r=(x, y) represents rat and is presently in environment position coordinate；
S4 obtains Place cell spatial characterization based on gitter cell spatial characterization, i.e., single Place cell position is wild, Place cell
Spatial characterization P (x, y) is：
<mrow>
<mi>P</mi>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<munderover>
<mo>&Sigma;</mo>
<mn>1</mn>
<mi>N</mi>
</munderover>
<msub>
<mi>W</mi>
<mi>n</mi>
</msub>
<msub>
<mi>g</mi>
<mi>n</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>4</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, WnRepresent the connection weight between n-th of gitter cell and Place cell, gn(x, y) represents n-th of gitter cell
Activity ratio at space environment (x, y) location point, N represents gitter cell quantity, N=4,10,20；
S5 rats form the Place cell discharge characteristic in each location point, ultimately formed during constantly environment is explored
Expression in rat brain to its residing space environment, i.e. cognitive map；
S6 has built a feedforward being made up of input layer, Place cell, action cell and output layer on the basis of cognitive map
Neural network model simultaneously realizes the object-oriented navigation task of rat by Q learning algorithms.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710180995.8A CN107063260B (en) | 2017-03-24 | 2017-03-24 | Bionic navigation method based on rat brain hippocampus structure cognitive map |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710180995.8A CN107063260B (en) | 2017-03-24 | 2017-03-24 | Bionic navigation method based on rat brain hippocampus structure cognitive map |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107063260A true CN107063260A (en) | 2017-08-18 |
| CN107063260B CN107063260B (en) | 2021-03-30 |
Family
ID=59618250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710180995.8A Active CN107063260B (en) | 2017-03-24 | 2017-03-24 | Bionic navigation method based on rat brain hippocampus structure cognitive map |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107063260B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108362284A (en) * | 2018-01-22 | 2018-08-03 | 北京工业大学 | A kind of air navigation aid based on bionical hippocampus cognitive map |
| CN109000655A (en) * | 2018-06-11 | 2018-12-14 | 东北师范大学 | Robot bionic indoor positioning air navigation aid |
| CN109425340A (en) * | 2017-08-28 | 2019-03-05 | 中国科学院沈阳自动化研究所 | Class brain robot navigation method based on the fusion of Bayes's multimodel perceptions |
| CN109668566A (en) * | 2018-12-05 | 2019-04-23 | 大连理工大学 | Robot scene cognition map construction and navigation method based on mouse brain positioning cells |
| CN109886384A (en) * | 2019-02-15 | 2019-06-14 | 北京工业大学 | A kind of bionic navigation method based on the reconstruct of mouse cerebral hippocampal gitter cell |
| CN110210462A (en) * | 2019-07-02 | 2019-09-06 | 北京工业大学 | A kind of bionical hippocampus cognitive map construction method based on convolutional neural networks |
| CN112525194A (en) * | 2020-10-28 | 2021-03-19 | 北京工业大学 | Cognitive navigation method based on endogenous and exogenous information of hippocampus-striatum |
| CN113643748A (en) * | 2021-07-28 | 2021-11-12 | 哈尔滨工业大学 | Grid cell spatial information decoding method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102401656A (en) * | 2011-11-08 | 2012-04-04 | 中国人民解放军第四军医大学 | Place cell bionic robot navigation algorithm |
| CN102799179A (en) * | 2012-07-06 | 2012-11-28 | 山东大学 | Mobile robot path planning algorithm based on single-chain sequential backtracking Q-learning |
| CN103699125A (en) * | 2013-12-09 | 2014-04-02 | 北京工业大学 | Robot simulated navigation method based on rat brain-hippocampal navigation |
| CN103778332A (en) * | 2014-01-16 | 2014-05-07 | 北京工业大学 | Method for analyzing function of mouse hippocampus structure in environment cognition process |
| CN104571113A (en) * | 2015-01-20 | 2015-04-29 | 博康智能网络科技股份有限公司 | Route planning method for mobile robot |
| CN106125730A (en) * | 2016-07-10 | 2016-11-16 | 北京工业大学 | A kind of robot navigation's map constructing method based on Mus cerebral hippocampal spatial cell |
-
2017
- 2017-03-24 CN CN201710180995.8A patent/CN107063260B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102401656A (en) * | 2011-11-08 | 2012-04-04 | 中国人民解放军第四军医大学 | Place cell bionic robot navigation algorithm |
| CN102799179A (en) * | 2012-07-06 | 2012-11-28 | 山东大学 | Mobile robot path planning algorithm based on single-chain sequential backtracking Q-learning |
| CN103699125A (en) * | 2013-12-09 | 2014-04-02 | 北京工业大学 | Robot simulated navigation method based on rat brain-hippocampal navigation |
| CN103778332A (en) * | 2014-01-16 | 2014-05-07 | 北京工业大学 | Method for analyzing function of mouse hippocampus structure in environment cognition process |
| CN104571113A (en) * | 2015-01-20 | 2015-04-29 | 博康智能网络科技股份有限公司 | Route planning method for mobile robot |
| CN106125730A (en) * | 2016-07-10 | 2016-11-16 | 北京工业大学 | A kind of robot navigation's map constructing method based on Mus cerebral hippocampal spatial cell |
Non-Patent Citations (2)
| Title |
|---|
| GEORGE DRAGOI ET.AL: "Preplay of future place cell sequences by hippocampal cellular assemblies", 《NATURE》 * |
| 于乃功等: "基于直接强化学习的面向目标的仿生导航模型", 《中国科学》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109425340A (en) * | 2017-08-28 | 2019-03-05 | 中国科学院沈阳自动化研究所 | Class brain robot navigation method based on the fusion of Bayes's multimodel perceptions |
| CN109425340B (en) * | 2017-08-28 | 2022-03-22 | 中国科学院沈阳自动化研究所 | Brain-like robot navigation method based on Bayes multi-mode perception fusion |
| CN108362284A (en) * | 2018-01-22 | 2018-08-03 | 北京工业大学 | A kind of air navigation aid based on bionical hippocampus cognitive map |
| CN109000655B (en) * | 2018-06-11 | 2021-11-26 | 东北师范大学 | Bionic indoor positioning and navigation method for robot |
| CN109000655A (en) * | 2018-06-11 | 2018-12-14 | 东北师范大学 | Robot bionic indoor positioning air navigation aid |
| CN109668566A (en) * | 2018-12-05 | 2019-04-23 | 大连理工大学 | Robot scene cognition map construction and navigation method based on mouse brain positioning cells |
| CN109668566B (en) * | 2018-12-05 | 2022-05-13 | 大连理工大学 | Robot scene cognition map construction and navigation method based on mouse brain positioning cells |
| CN109886384B (en) * | 2019-02-15 | 2021-01-05 | 北京工业大学 | Bionic navigation method based on mouse brain hippocampus grid cell reconstruction |
| CN109886384A (en) * | 2019-02-15 | 2019-06-14 | 北京工业大学 | A kind of bionic navigation method based on the reconstruct of mouse cerebral hippocampal gitter cell |
| CN110210462A (en) * | 2019-07-02 | 2019-09-06 | 北京工业大学 | A kind of bionical hippocampus cognitive map construction method based on convolutional neural networks |
| CN112525194A (en) * | 2020-10-28 | 2021-03-19 | 北京工业大学 | Cognitive navigation method based on endogenous and exogenous information of hippocampus-striatum |
| CN112525194B (en) * | 2020-10-28 | 2023-11-03 | 北京工业大学 | Cognitive navigation method based on in vivo source information and exogenous information of sea horse-striatum |
| CN113643748A (en) * | 2021-07-28 | 2021-11-12 | 哈尔滨工业大学 | Grid cell spatial information decoding method and system |
| CN113643748B (en) * | 2021-07-28 | 2024-03-12 | 哈尔滨工业大学 | Grid cell space information decoding method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107063260B (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107063260A (en) | A kind of bionic navigation method based on mouse cerebral hippocampal structure cognitive map | |
| Ni et al. | Bioinspired intelligent algorithm and its applications for mobile robot control: a survey | |
| CN106949896B (en) | Scene cognition map construction and navigation method based on mouse brain hippocampus | |
| CN109886384B (en) | Bionic navigation method based on mouse brain hippocampus grid cell reconstruction | |
| Lechner et al. | Designing worm-inspired neural networks for interpretable robotic control | |
| Cao et al. | Concentrated coverage path planning algorithm of UAV formation for aerial photography | |
| CN111578940A (en) | Indoor monocular navigation method and system based on cross-sensor transfer learning | |
| CN113741449B (en) | Multi-agent control method for sea-air collaborative observation task | |
| Llofriu et al. | Goal-oriented robot navigation learning using a multi-scale space representation | |
| Gershenson | Computing networks: A general framework to contrast neural and swarm cognitions | |
| Hasani et al. | A natural lottery ticket winner: Reinforcement learning with ordinary neural circuits | |
| CN106096911A (en) | A kind of based role have people/unmanned collaborative command and control system and method thereof | |
| CN110210462A (en) | A kind of bionical hippocampus cognitive map construction method based on convolutional neural networks | |
| Taniguchi et al. | Hippocampal formation-inspired probabilistic generative model | |
| Ma et al. | Multi-AUV collaborative operation based on time-varying navigation map and dynamic grid model | |
| Kurdi et al. | Proposed system of artificial Neural Network for positioning and navigation of UAV-UGV | |
| Nolfi | D.: Evolutionary Robotics | |
| Wang et al. | A fuzzy logic based bio-inspired system for mobile robot navigation | |
| CN108459614B (en) | UUV real-time collision avoidance planning method based on CW-RNN network | |
| CN108981712A (en) | Robot goes on patrol method and robot | |
| Zhang et al. | Biomimetic slam algorithm based on growing self-organizing map | |
| Gao et al. | Weighted area coverage of maritime joint search and rescue based on multi-agent reinforcement learning | |
| Obayashi et al. | An emotional model embedded reinforcement learning system | |
| Huang et al. | Automatic Path Planning for Spraying Drones Based on Deep Q-Learning | |
| Tanaka et al. | Action acquisition method for constructing cognitive development system through instructed learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |