CN112509051A - Bionic-based autonomous mobile platform environment sensing and mapping method - Google Patents
Bionic-based autonomous mobile platform environment sensing and mapping method Download PDFInfo
- Publication number
- CN112509051A CN112509051A CN202011521855.0A CN202011521855A CN112509051A CN 112509051 A CN112509051 A CN 112509051A CN 202011521855 A CN202011521855 A CN 202011521855A CN 112509051 A CN112509051 A CN 112509051A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- cell
- map
- information
- imu
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000013507 mapping Methods 0.000 title claims abstract description 24
- 239000011664 nicotinic acid Substances 0.000 title claims abstract description 8
- 230000001149 cognitive effect Effects 0.000 claims abstract description 51
- 230000011218 segmentation Effects 0.000 claims abstract description 31
- 235000001968 nicotinic acid Nutrition 0.000 claims abstract description 18
- 238000001514 detection method Methods 0.000 claims abstract description 12
- 238000012549 training Methods 0.000 claims abstract description 12
- 238000000605 extraction Methods 0.000 claims abstract description 9
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 7
- 230000008569 process Effects 0.000 claims abstract description 4
- 210000004027 cell Anatomy 0.000 claims description 178
- 239000011159 matrix material Substances 0.000 claims description 64
- 239000013598 vector Substances 0.000 claims description 33
- 210000002569 neuron Anatomy 0.000 claims description 32
- 230000001133 acceleration Effects 0.000 claims description 26
- 238000012546 transfer Methods 0.000 claims description 19
- 230000000694 effects Effects 0.000 claims description 17
- 238000005259 measurement Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 16
- 238000006073 displacement reaction Methods 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 12
- 230000005484 gravity Effects 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 10
- 238000013519 translation Methods 0.000 claims description 10
- 230000005764 inhibitory process Effects 0.000 claims description 9
- 230000010354 integration Effects 0.000 claims description 8
- 238000003384 imaging method Methods 0.000 claims description 7
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 210000004556 brain Anatomy 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 6
- 230000002964 excitative effect Effects 0.000 claims description 6
- 230000002401 inhibitory effect Effects 0.000 claims description 6
- 210000001020 neural plate Anatomy 0.000 claims description 6
- 230000000946 synaptic effect Effects 0.000 claims description 6
- 238000012360 testing method Methods 0.000 claims description 5
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 206010001497 Agitation Diseases 0.000 claims description 3
- 241000283984 Rodentia Species 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 210000001320 hippocampus Anatomy 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000001419 dependent effect Effects 0.000 claims description 2
- 238000004364 calculation method Methods 0.000 abstract description 4
- 230000008447 perception Effects 0.000 abstract description 4
- 210000003128 head Anatomy 0.000 description 15
- 238000010276 construction Methods 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 241000124008 Mammalia Species 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000037361 pathway Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/002—Biomolecular computers, i.e. using biomolecules, proteins, cells
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Chemical & Material Sciences (AREA)
- Organic Chemistry (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an autonomous mobile platform environment sensing and mapping method based on bionics, which comprises the following steps: s1, collecting an automatic driving data set, sending the pictures into an improved semantic segmentation network, and training the improved semantic segmentation network; s2, initializing binocular vision and IMU combination; s3, fusing binocular vision and IMU; s4, constructing a bionic cell model; and S5, constructing the cognitive map. The method uses the bionics principle to establish the empirical map, greatly reduces the parameter quantity in the map, and can store the scene map information with a larger range. Meanwhile, the robustness of detection can be improved by semantically segmenting and modeling the place information and fusing the place information with the position cells. The powerful feature extraction and learning capabilities of the convolutional neural network are fully utilized, the problems of excessive map building calculation amount and inaccurate recognition of the traditional SLAM method can be improved, and the accuracy of the bionic environment perception and the place judgment in the map building process is better improved.
Description
Technical Field
The invention belongs to the technical field of autonomous mobile platform positioning navigation, and particularly relates to an autonomous mobile platform environment sensing and mapping method based on bionics.
Background
The instant positioning and Mapping (SLAM) refers to a technology that a robot actively creates a map in an unknown environment and performs self-positioning on the map according to estimation of the state of the robot and the map. The accurate positioning information is beneficial to the autonomous mobile platform to complete tasks such as path planning and map drawing. A traditional immediate positioning and map construction algorithm adopts a Bayesian probability estimation-based method, and solves speed and pose by acquiring external input information. And complex nonlinear optimization knowledge is used to solve for prior information in the environment. A huge database is constructed as a retrieval source. However, when a model is constructed in a real environment with complex environmental characteristics, huge computing power is consumed. And the resulting modeling results cannot be used for deeper level navigation tasks such as semantic analysis and cognitive understanding.
Aiming at the requirements of environment modeling and intelligent navigation on unknown terrain in an autonomous environment, the bionic environment sensing and mapping method is developed by simulating the biological characteristics of animal recognition environment and navigation. The method plays an important role in the tasks of environment perception and map construction and navigation, and is a hot problem for intelligent navigation and cognitive scientific research in the future.
Disclosure of Invention
The invention aims to provide an autonomous mobile platform environment sensing and mapping method based on bionics. The problems set forth in the background above are solved. The following presents a simplified summary of various aspects in order to provide a basic understanding of such aspects. This summary does not detail all contemplated aspects, however, its sole purpose is to present concepts of the aspects in a simplified form to facilitate a detailed description of the overall innovation.
Studies have found that the brain of lower mammals bears many similarities to the human brain, where environmental information is presented in the form of cognitive maps. Cognitive maps are the reproduction of internal nerves and cells to the external environment. With respect to external environmental information, mammals respond to the brain's where pathway, and specific cells accumulate electrical discharges to obtain current pathway information. The mammal recognizes the image seen by the eyeball through the what visual path to obtain the object attribute information of the current position, and activates the specific position cell through the discharge of the position information. And the information of the position cell is combined with the object attribute information obtained from the visual access to judge whether the current memory unit is a place which does not appear on the cognitive map or not, and if so, the information is sent to a node of the cognitive map link expansion cognitive map. And obtaining a complete cognitive map after traversing the whole area.
The method aims at the conditions of much random noise, uneven illumination and unobvious environment boundary in a natural scene. According to the scheme, a method of obtaining information by combining binocular vision and IMU is adopted, and pose information obtained by an IMU sensor is sent into head orientation cells and stripe cells. Obtaining category information, characteristic information and depth information of the binocular vision image through semantic segmentation, and combining the obtained semantic characteristic vector and the semantic category information. And judging whether the current position is stored or not, and if not, mapping to a position node to construct a cognitive map. The position node contains the depth information and semantic feature information of the current position object. And comparing the performances of the SURF, ORB and SIFT feature points in an object perception module, selecting an ORB feature descriptor to extract matching points of the binocular view, and recovering the depth of the image through the matching points. And simultaneously, respectively extracting semantic features from the binocular images, and establishing an offline dictionary of the semantic features and a query image database.
An autonomous mobile platform environment sensing and mapping method based on bionics comprises the following steps:
s1, collecting an automatic driving data set, sending the pictures into an improved semantic segmentation network, and training the improved semantic segmentation network;
s2, initializing binocular vision and IMU combination;
s3, fusing binocular vision and IMU;
s4, constructing a bionic cell model;
and S5, constructing the cognitive map.
Preferably, the improved convolutional neural network is used for semantic modeling, and comprises a feature extraction layer, a feature fusion layer, a feature coding layer and a feature decoding layer; in the feature extraction stage, an improved deep Lab network (encoder-decoder network with separable convolution) is used for fully extracting the features of an input image through a plurality of convolution layers, the features are extracted by operating with cavity convolution kernels with the sizes of c × c and b × b respectively, and the result is sent to a feature decoding layer through the convolution of c × c to carry out pixel prediction; in the decoding stage, firstly, 4 times of upsampling is carried out on the feature map, then, the feature map is connected with the conv3 bottom-layer features, 4 times of convolution kernels with the size of 3 are expanded to obtain a semantic segmentation result, and semantic category vectors are extracted from the semantic segmentation result; and adding a convolution kernel with the size of 1 to the middle convolution layer, converting the feature map into a one-dimensional object feature vector, and merging the semantic segmentation vector and the object feature vector to obtain a semantic location vector.
Preferably, step S1 specifically includes:
s1-1, collecting pictures of roads in the automatic driving data set, dividing the pictures into rectangular areas, and dividing a training set, a test set and a verification set according to a proportion;
and S1-2, sending the pictures in the training set into an improved semantic segmentation network, performing semantic segmentation test on the training set, and storing a group of network parameters with better effect obtained by performing semantic segmentation on the data set.
Preferably, initializing binocular vision comprises the steps of:
s2-1, carrying out combined calibration on the binocular camera modules, and specifically representing the corresponding relation from a camera coordinate system to a world coordinate system by using a matrix:
where Xc, Yc, Zc are points of the camera coordinate system, Xw, Yw, Zw are world coordinate systems, R is a rotation matrix of the camera coordinate system relative to the world coordinate system, t is a translation matrix of the camera coordinate system relative to the world coordinate system, M is a rotation matrix of the camera coordinate system relative to the world coordinate system1Is a matrix of internal parameters about the camera;
the translation vector of the imaging point to the origin of the pixel coordinate system is [ d ]x，dy]TIf the focal length is f and the unit is pixel, then pixel point [ u, v]TTransformation to image coordinate system:
wherein x, y are the coordinates of the image coordinate system, u0And v0Half the length and width of the imaging plane;
the transformation matrix of the points on the plane of the camera coordinate system and the image coordinate system is:
the above formula is combined to obtain a conversion formula from an image coordinate system to a world coordinate system, the formula is used for calibrating the binocular camera,
where M1 is a matrix of internal parameters about the camera,m2 is a matrix of external parameters of the camera, the two matrices being a simplified representation of a homogeneous transformation matrix as described by the equation, M representing a normalized internal and external parameter matrix, PwThree-dimensional coordinates of the spatial points; giving coordinates of a plurality of points of the calibration plate, and substituting the coordinates of the image of the corresponding points of the calibration plate and the coordinates of the world system to obtain a parameter matrix M of the camera, wherein the matrix M is used for calculating the spatial position of the matching points of the binocular camera;
s2-2, operating the binocular camera module and the IMU module, initializing an Inertial Measurement Unit (IMU) by using the pose of each key frame measured by binocular vision simultaneous positioning and mapping (SLAM), calculating the following parameters including the dimension, the gravity direction, the speed, the acceleration and the offset of a gyroscope, wherein an IMU coordinate system is represented by B, a world coordinate system is represented by W, sampling IMU output at a certain time interval delta t, and measuring the acceleration of the IMUWa (t) and angular velocityBωWB(t), the IMU measurement model is:
whereinBω′WB(t)∈R3Representing the true value of the instantaneous angular velocity ω of the gyroscope in a coordinate system B relative to the world coordinate system, where Bg(t) represents the IMU internal error, η, which varies slowly with time tg(t) is the measurement of white noise,wa(t)∈R3representing the instantaneous acceleration, g, of the IMU in W coordinatesWRepresenting the gravity acceleration vector under the world coordinate system W,Ba' (t) represents the true value of the instantaneous acceleration a of the gyroscope in the coordinate system B relative to the world coordinate system, RWB T(t) represents the transpose of the rotation matrix of the gyroscope relative to the world coordinate system W under the coordinate system B, Ba(t) represents the acceleration error, η, of the IMU slowly varying over time ta(t) represents measurement white noise of the acceleration.
Preferably, the binocular camera module is a vision module and is used for collecting image vision information and fixing the image vision information on the front side of the autonomous mobile platform, a binocular system is automatically initialized, a homography matrix is selected for a planar scene to calculate, a basic matrix is selected for a non-planar scene to calculate, two image frames are selected as initial frames through matching, and the pose between the two frames is calculated according to matching; the disparity of the binocular view at corresponding pixel points is calculated to form a disparity map, a global energy function associated with the disparity map is set, the optimal disparity of each pixel is solved by minimizing the global energy function, depth information is obtained through the disparity between the left image and the right image, and then the depth map is obtained.
Preferably, the IMU module is an encoder module, is configured to acquire displacement information of the autonomous mobile platform, and is installed on a base of the autonomous mobile platform.
Preferably, initializing the IMU includes the steps of:
s2-2-1, using several continuous key frames to estimate the bias of gyroscope, and assuming a constant bgOn the basis of (2) plus the variation
The amount of update of the gyroscope bias is calculated by minimizing the rotation error:
where N is the number of key frames,
indicating rotation by camera
Rotation transformation matrix R from rigid body to cameraCB；ΔRi，i+1Is the detection quantity of the pre-integral rotation of two continuous image frames;
representing the rotation matrix, Δ R, from the IMU sensor coordinate system to the world coordinate systemi，i+1Is a pre-integral rotation measurement of two consecutive key frames, given a zeroThe initial value is obtained, and the initial bias of the gyroscope corresponding to the minimum target value of the function is solved through Gauss-Newton iteration;
s2-2-2, solving by calculating a linear equation between continuous frames to obtain an acceleration offset estimation value, a scale and a gravity direction accurate value;
s2-2-3, and substituting the formula (5) to calculate R under the condition of knowing the dimension and the gravity acceleration directionWBPosition and velocity measurements, completing the IMU initialization.
Preferably, step S3 includes the steps of:
s3-1-1, two images shot at the same time are called as a frame, a group of left and right images shot by a binocular camera with stable IMU data are selected as initial frames, the IMU data started at the moment are stored, and the image frames are preprocessed through distortion correction;
s3-1-2, respectively extracting key points from a left image and a right image of the binocular camera, detecting and describing point features by adopting an ORB operator, wherein the ORB descriptor is a 256-bit binary descriptor, and an offline dictionary of the key point features is established by normalizing the feature descriptor;
s3-1-3, carrying out point feature matching from frame to frame, and eliminating mismatching of ORB feature points by adopting Random Sample Consensus (Random Sample Consensus) algorithm Random Sample Consensus;
s3-1-4, selecting matched feature points in the feature map to calculate the parallax, forming a parallax map, setting a global energy function related to the parallax map, minimizing the energy function, and solving the parallax optimal result of each pixel.
Preferably, step S4 includes the steps of:
s4-1, describing a heading angle theta of the autonomous mobile platform gyroscope sensor through a head heading cell model, wherein the head heading cell is a head heading dependent neuron and is expressed as a Gaussian model, and obtaining a head heading transfer matrix by inputting an angle and an angle offset of the gyroscope sensor; modeling yields the head-oriented cell transfer equation for the velocity signal as:
wherein the signal generated by the i-th head towards the cell at time t is oriented predominantly in the direction θbAnd the angular offset thetaiDetermining that D (t) is the head orientation transfer matrix, typically setting the predominant orientation of the head towards the cell to 0 DEG θiIs the i-th head-to-cell offset, θ0～θmRepresents the respective orientation angles of the m heads towards the cells, and the value range is more than 0 DEG thetai< 360 DEG, v (t) is the moving speed of the robot along the theta (t) direction at the time t;
s4-2, describing the linear velocity obtained by the accelerometer sensor by using a stripe cell model, wherein stripe cells have a discharge field similar to a parallel stripe shape, and the stripe cell model obtains the current path of the autonomous mobile platform through integration; establishing a transfer model of the streak cells to the displacement signals as follows:
the stripe cells are coded into displacement in a specific direction, the discharge activity of the stripe cells is a periodic reset coding process and is responsible for path integration of linear velocity, an initial direction delta is defined, a discharge phase beta is defined, a discharge period is f, the moving speed of the robot in the theta (t) direction at the time t is v (t), and the speed in the delta direction is:
vθ(t)＝cos(δ-θ(t))v(t) (9)；
s4-3, expressing the movement of the current autonomous mobile platform in a two-dimensional space by using a grid cell model, wherein the grid cells are cells which are regularly discharged by rodents in the two-dimensional space; the discharge field represents the expression of partial environment of a certain position, direction information and displacement information are sent into a grid cell model, the grid cell model is modeled according to a continuous attractor network to generate grid cell group activity driven by speed, in an attractor network, neurons are arranged on a two-dimensional neural plate, in order to solve the problem of attractor network boundary, the neurons on the upper, lower, left and right boundaries of the neuron plate are connected to form a two-dimensional annular attractor model, the neurons are connected through local excitability and global inhibitability to form an activity packet on the neuron plate, the activity packet moves on the neuron plate along with the input of speed signals to generate the grid cell group, and the kinetic equation of a grid cell i based on the speed is expressed as follows:
wherein SiThe current active state of the lattice cell, TcIs the time constant of the response of the neuron,
inhibitory input of peripheral ring neurons to neuron i, FiIs the forward excitatory projection of the grid cells by the head-facing cells, determined primarily by velocity and direction, and is expressed for the above model as a function of f (x), where the independent variable x is expressed as the sum of excitatory and inhibitory projections, as in equation (10)
Is solved as follows
Connection weight
The connection weight of the grid cells i to j, g the gain for adjusting the activity of the grid cell group, the initialization gamma is 1.05 beta,
wherein λgridActive space of lattice cell wherein is active space of lattice cell, xiIs the location of the grid cell i on the neuron plate,
is the reset distance of the striped cell model;
the head-oriented cell model and the stripe cell model integrate information to form regular discharge and send signals to grid cells; after receiving the information, the grid cells form specific grid cell grid fields with fixed intervals through regular repeated discharge, and the grid fields correspond to a plurality of activation regions of an external space;
s4-4, describing the current position through a position cell model: the position cells are cells which are discharged at specific positions in a map space, have space specificity, map a position field in a brain hippocampus body with external specific space positions through selective discharge, discharge maximally to form position activation when an autonomous mobile platform reaches the specific positions in the space, establish new position points, and model a grid cell model through a Hebbian neural network (Hebbian neural network)
Wherein: a is the positional cell gain constant; mlIs the number of layers of neural plates, where r represents the position of the autonomous mobile platform;
the connection weight of grid cell i to position cell j; cplaceIs the inhibition constant of the site cell, sjAnd expressing the discharge rate of the grid cells, and obtaining a relation between the position cells and the grid cells:
where k is the learning rate, piAs a model of the location of cell discharge, sjEquation (13) represents the relationship between the activation rate of the grid cells, the inhibition rate of the grid cells, and the strength of the synaptic connections, and the synaptic connections become stronger if the activation is greater than the inhibition, and weaker otherwise.
Preferably, step S5 includes the steps of:
s5-1, obtaining the semantic segmentation result of the current frame: sending the image frame to the improved semantic segmentation network in the step S1 to obtain a place semantic vector;
s5-2, if the place semantic feature vector does not exist in the existing cognitive map nodes and the depth information of the image frame is different from the known place, judging that a new position is reached and generating a new position cell;
s5-3, if a new position cell is generated, nodularizing the current position cell, adding the nodularized current position cell into an existing cognitive map, storing semantic vector information of a current image into a position library, adding depth information of a view object into the current position cell, expressing cognitive map network nodes by e, wherein each cognitive node comprises position cell discharge p, environment image characteristics V, topological relation L between cognitive nodes and depth information D of an environment image:
ei＝{pi，Vi，Li，D} (14)
judging position cell state p through environment image characteristic ViEnvironment image feature ViAnd when the information is not matched with the information in the cognitive nodes, a new cognitive node is created:
ei＝{pi，Vi，D，Li+ Δ L } (equation 15)
Wherein, the delta L is the distance between the current cognitive node and the previous cognitive node, and the displacement information comes from the odometer system;
updating the pose information p of each node through the transfer matrix t of the experience map, and updating the current node eiPose information and successful closed loop detection matching node ejThe pose information is fused to obtain a transfer matrix t updating formula of the experience map, wherein the transfer matrix t updating formula is as follows:
where α is the correction rate, NfFor the current node eiWith other nodesNumber of connections, NtIs the number of connections, p, of the successful closed-loop detection matching nodeiAnd Δ piRespectively representing the current value and the variation of the pose, pjAnd pkRespectively representing the pose values, Δ p, of the j and k positionsijAnd Δ pkiTranslation information updated for the location;
s5-4, along with the movement of the autonomous mobile platform, the position cells activate the whole cognitive map, the cognitive node semantic information and the depth information of the image frame are obtained according to the cognitive map, and navigation is performed according to the environment information and the map information;
s5-5, finally, starting a new thread to execute global optimization to optimize the optimal structure of the whole cognitive map
Compared with the prior art, the invention has the beneficial effects that:
the method aims at solving the problem that the traditional identification navigation method can not be accurately identified under the condition of overlarge map range or uneven scene illumination. The method uses the bionics principle to establish the empirical map, greatly reduces the parameter quantity in the map, and can store the scene map information with a larger range. Meanwhile, the robustness of detection can be improved by semantically segmenting and modeling the place information and fusing the place information with the position cells. The powerful feature extraction and learning capabilities of the convolutional neural network are fully utilized, and the problems of excessive map building calculation amount and inaccurate identification of the traditional SLAM method can be improved.
Drawings
FIG. 1 is a flowchart of an environment sensing and mapping method for an autonomous mobile platform based on bionics according to an embodiment of the present invention;
fig. 2 is a structural diagram of an improved semantic segmentation network in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 and fig. 2, an environment sensing and mapping method for an autonomous mobile platform based on bionics includes the following steps:
s1, collecting an automatic driving data set, sending the pictures into an improved semantic segmentation network, and training the improved semantic segmentation network;
the method aims at solving the problem that the common identification navigation method cannot accurately detect the map in the condition of overlarge map range. The method uses a cognitive map to greatly reduce the number of parameters in the map. So that a wide range of scene map information can be saved. Meanwhile, the robustness of detection can be improved by semantically segmenting modeling location information and comparing the location information with the location cells. The powerful feature extraction and learning capacity of the convolutional neural network is fully utilized, and the problems of excessive map building calculation amount and inaccurate identification of the traditional SLAM method are solved.
Specifically, a picture of a road in the automatic driving data set is collected, the road scene picture is divided into rectangular areas of 128 × 128, and the ratio of the area to the scene is 6: 3: 1, dividing a training set, a test set and a verification set, and sending pictures in the training set into a convolutional neural network.
The improved convolutional neural network is used for semantic place modeling, and as shown in fig. 2, the network is divided into a feature extraction layer, a feature fusion layer, a feature coding layer and a feature decoding layer. In the feature extraction stage, the network uses a modified deep lab network (encoder-decoder network with separable convolutions), which adequately extracts features of the image through a plurality of convolutional layers of different sizes, then operating with the hollow convolution kernels with the sizes of 1 × 1 and 3 × 3 respectively to further extract features, sending the obtained convolution result to a feature decoding layer through a convolution layer with the size of 1 × 1 to carry out pixel prediction, carrying out 4 times of upsampling on a feature graph in a decoding stage, then, performing connection operation with the conv3 bottom layer characteristics, expanding by 4 times through a convolution kernel with the size of 3 to obtain a semantic segmentation result, extracting a semantic category vector from the semantic segmentation result, and adding a convolution kernel with the size of 1 in the middle convolution layer, converting the feature map into a one-dimensional object feature vector, and merging the semantic segmentation vector and the object feature vector to obtain a location semantic vector.
Step S2, binocular vision and IMU initialization.
S2-1, initializing binocular stereo vision, wherein the camera adopts a pinhole imaging model to jointly calibrate the existing binocular camera modules, and a matrix is used for specifically representing the corresponding relation from a camera coordinate system to a world coordinate system, wherein Xc, Yc and Zc are points of the camera coordinate system, Xw, Yw and ZW are the world coordinate system, R is a rotation matrix of the camera coordinate system relative to the world coordinate system, t is a translation matrix of the camera coordinate system relative to the world coordinate system, and M is a translation matrix of the camera coordinate system relative to the world coordinate system1Is a simplified form of the homogeneous transformation matrix described by the preceding equation, representing the intrinsic parameter matrix of the camera;
the translation vector of the imaging point to the origin of the pixel coordinate system is [ d ]x，dy]TIf the focal length is f and the unit is pixel, then pixel point [ u, v]TTransformation to image coordinate system:
wherein the coordinates of the image coordinate system are xy. u. of0And v0Half the length and width of the imaging plane.
The transformation matrix of the points on the plane of the camera coordinate system and the image coordinate system is:
the above formula is combined to obtain a conversion formula from an image coordinate system to a world coordinate system, the formula is used for calibrating the binocular camera,
wherein M is1Is a matrix of internal parameters, M, with respect to the camera2Are matrices for camera extrinsic parameters, both matrices being a simplified representation of the homogeneous transformation matrix described in the preceding equation, matrix M representing a normalized extrinsic parameter matrix, PwIs the three-dimensional coordinates of a spatial point. Coordinates of a plurality of points of the calibration plate are given and are substituted into image coordinates and world system coordinates of corresponding points of the calibration plate, and a parameter matrix M of the camera can be obtained and used for calculating the space position of the matching points of the binocular camera.
And S2-2, wherein the IMU module is a posture input module of the autonomous mobile platform environment perception system and is used for acquiring displacement information of the autonomous mobile platform. Operating a binocular module and an IMU module, initializing the IMU by using the pose of each key frame measured by binocular vision simultaneous localization and mapping (SLAM), wherein the calculation parameters comprise scale, gravity direction, speed, acceleration and gyroscope bias, an IMU coordinate system is represented by B, a world coordinate system is represented by W, IMU output is sampled at a certain time interval delta t, and the acceleration of the IMU is measuredWa (t) and angular velocityBωWB(t), the IMU measurement model is:
whereinBω′WB(t)∈R3Representing the true value of the instantaneous angular velocity ω of the gyroscope in a coordinate system B relative to the world coordinate system, where Bg(t) represents the IMU internal error, η, which varies slowly with time tg(t) is the measurement of white noise,Wa(t)∈R3representing the instantaneous acceleration, g, of the IMU in W coordinatesWRepresenting the gravity acceleration vector under the world coordinate system W.Ba' (t) represents the true value of the instantaneous acceleration a of the gyroscope in the coordinate system B relative to the world coordinate system, RWB T(t) represents the transpose of the rotation matrix of the gyroscope relative to the world coordinate system W under coordinate system B. ba(t) represents the acceleration error, η, of the IMU slowly varying over time ta(t) represents measurement white noise of the acceleration.
S2-2-1, using several continuous key frames to estimate the bias of gyroscope, and assuming a constant bgOn the basis of (2) adding
The amount of change of the gyro bias is calculated by minimizing the rotation error:
where N is the number of key frames,
indicating rotation by camera
Rotation transformation matrix R from rigid body to cameraCB；ΔRi，i+1Is the detection quantity of the pre-integral rotation of two continuous image frames;
representing the rotation matrix, Δ R, from the IMU sensor coordinate system to the world coordinate systemi，i+1The method comprises the steps of performing pre-integration rotation measurement on two continuous key frames, giving an initial value of zero, and solving initial bias of a gyroscope corresponding to the minimum target value of the function through Gauss-Newton iteration;
s2-2-2, solving an acceleration offset estimation, a scale and a gravity direction accurate value by calculating a linear equation between continuous frames;
s2-2-3, and substituting the formula (5) to calculate R under the condition of knowing the dimension and the gravity acceleration directionWBPosition and velocity measurements, completing the IMU initialization.
And step S3, fusing binocular vision and IMU.
S3-1-1, two images shot at the same time are called as a frame, a group of left and right images shot by a binocular camera with stable IMU data are selected as initial frames, the IMU data started at the moment are stored, and the image frames are preprocessed through distortion correction; and selecting a group of binocular views with stable IMU data as a starting frame, and recording the corresponding IMU data.
And S3-1-2, respectively extracting key points from the left image and the right image of the binocular camera according to the pose between the two frames calculated by matching, and detecting and describing the point characteristics by adopting an ORB operator. The ORB descriptor is a 256-bit binary characteristic vector, and an offline dictionary of key point characteristics is established by normalizing the characteristic vector;
s3-1-3, carrying out point feature matching from frame to frame, and eliminating mismatching of ORB feature points by adopting a RANSAC algorithm.
S3-1-4, forming a disparity map by calculating the disparity of the binocular view at corresponding pixel points, setting a global energy function associated with the disparity map, solving the optimal disparity of each pixel by minimizing the global energy function, and acquiring depth information by the disparity between the left image and the right image so as to acquire the depth map.
And S4, constructing a bionic cell model.
S4-1, constructing a head orientation cell model to describe the orientation angle theta of the autonomous mobile platform gyroscope sensor. Head-oriented cells are represented as a gaussian model. By inputting the angle and angular offset of the gyro sensor. A head orientation transfer matrix is obtained. Modeling yields the head-oriented cell transfer equation for the velocity signal as:
wherein the signal generated by the i-th head towards the cell at time t is oriented predominantly in the direction θbAnd the angular offset thetaiDetermining that D (t) is the head orientation transfer matrix, typically setting the predominant orientation of the head towards the cell to 0 DEG θiIs the i-th head-to-cell offset, θ0～θmRepresents the respective orientation angles of the m heads towards the cells, and the value range is more than 0 DEG thetai< 360 DEG, v (t) is t time along theta (t)t) speed of movement in the direction;
s4-2, constructing a stripe cell model to describe the linear velocity obtained by the accelerometer sensor. The striped cells have a discharge field resembling a parallel stripe shape. The cell model can obtain the path of the current autonomous mobile platform through integration. Establishing a transfer model of the streak cells to the displacement signals as follows:
the stripe cells are coded into displacement in a specific direction, the discharge activity of the stripe cells is a periodic reset coding process and is responsible for path integration of linear velocity, an initial direction delta is defined, a discharge phase beta is defined, a discharge period is f, the moving speed of the robot in the theta (t) direction at the time t is v (t), and the speed in the delta direction is:
vθ(t)＝cos(δ-θ(t))v(t) (9)；
s4-3, expressing the movement of the current autonomous mobile platform in a two-dimensional space by using a grid cell model, wherein the grid cells are cells which are regularly discharged by rodents in the two-dimensional space; the discharge field represents the expression of partial environment of a certain position, direction information and displacement information are sent into a grid cell model, the grid cell model is modeled according to a continuous attractor network to generate grid cell group activity driven by speed, in an attractor network, neurons are arranged on a two-dimensional neural plate, in order to solve the problem of attractor network boundary, the neurons on the upper, lower, left and right boundaries of the neuron plate are connected to form a two-dimensional annular attractor model, the neurons are connected through local excitability and global inhibitability to form an activity packet on the neuron plate, the activity packet moves on the neuron plate along with the input of speed signals to generate the grid cell group, and the kinetic equation of a grid cell i based on the speed is expressed as follows:
wherein SiThe current active state of the lattice cell, TcIs the time constant of the response of the neuron,
inhibitory input of peripheral ring neurons to neuron i, FiIs the forward excitatory projection of the grid cells by the head-facing cells, determined primarily by velocity and direction, and is expressed for the above model as a function of f (x), where the independent variable x is expressed as the sum of excitatory and inhibitory projections, as in equation (10)
Is solved as follows
Connection weight
The connection weight of the grid cells i to j, g the gain for adjusting the activity of the grid cell group, the initialization gamma is 1.05 beta,
wherein λgridActive space of lattice cell wherein is active space of lattice cell, xiIs the location of the grid cell i on the neuron plate,
is the reset distance of the striped cell model;
the head-oriented cell model and the stripe cell model integrate information to form regular discharge and send signals to grid cells; after receiving the information, the grid cells form specific grid cell grid fields with fixed intervals through regular repeated discharge, and the grid fields correspond to a plurality of activation regions of an external space;
s4-4, describing the current position through a position cell model: the position cells are cells which are discharged at specific positions in a map space, have space specificity, map a position field in a brain hippocampus body with external specific space positions through selective discharge, discharge maximally to form position activation when an autonomous mobile platform reaches the specific positions in the space, establish new position points, and model a grid cell model through a Hebbian neural network (Hebbian neural network)
Wherein: a is the positional cell gain constant; mlThe number of layers of the neural plate is shown, and r is the position of the autonomous mobile platform;
the connection weight of grid cell i to position cell j; cplaceIs the inhibition constant of the site cell, sjAnd expressing the discharge rate of the grid cells, and obtaining a relation between the position cells and the grid cells:
where k is the learning rate, piAs a model of the location of cell discharge, sjEquation (13) represents the relationship between the activation rate of the grid cells, the inhibition rate of the grid cells, and the strength of the synaptic connections, and the synaptic connections become stronger if the activation is greater than the inhibition, and weaker otherwise.
Step S5, cognitive map construction
S5-1, calculating the semantic segmentation result of the current frame: and sending the initial pose to the improved semantic segmentation network in the step S1 to obtain a place semantic vector.
And S5-2, if the place semantic feature vector does not exist in the existing place library node and the depth information of the object is different from the known place, judging that a new position is reached and generating a new position cell.
S5-3, if a new position cell is generated, nodularizing the current position cell, adding the nodularized current position cell into an existing cognitive map, storing semantic vector information of a current image into a position library, adding depth information of a view object into the current position cell, expressing cognitive map network nodes by e, wherein each cognitive node comprises position cell discharge p, environment image characteristics V, topological relation L between cognitive nodes and depth information D of an environment image:
ei＝{pi，Vi，Li，D} (14)
judging position cell state p through environment image characteristic ViEnvironment image feature ViAnd when the information is not matched with the information in the cognitive nodes, a new cognitive node is created:
ei＝{pi，Vi，D，Li+ΔL} (15)
wherein, the delta L is the distance between the current cognitive node and the previous cognitive node, and the displacement information comes from the odometer system;
updating the pose information p of each node through the transfer matrix t of the experience map, and updating the current node eiPose information and successful closed loop detection matching node ejThe pose information is fused to obtain a transfer matrix t updating formula of the experience map, wherein the transfer matrix t updating formula is as follows:
where α is the correction rate, NfFor the current node eiNumber of connections to other nodes, NtIs the number of connections, p, of the successful closed-loop detection matching nodeiAnd Δ piRespectively representing the current value and the variation of the pose, pjAnd pkRespectively representing the pose values, Δ p, of the j and k positionsijAnd Δ pkiTranslation information updated for the location;
s5-4, along with the movement of the autonomous mobile platform, the position cells activate the whole cognitive map, the cognitive node semantic information and the depth information of the image frame are obtained according to the cognitive map, and navigation is performed according to the environment information and the map information;
and S5-5, finally, starting a new thread to execute global optimization, and calculating the optimal structure of the whole cognitive map.
While, for purposes of simplicity of explanation, the above-described text has been described as a series of steps, it is to be understood and appreciated that the methodologies are not limited by the order of operations, as some acts may occur in different orders and apart from as performed in one or more steps, as one skilled in the art will appreciate the principles of how such acts occur.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. An autonomous mobile platform environment sensing and mapping method based on bionics is characterized by comprising the following steps:
s1, collecting an automatic driving data set, sending the pictures into an improved semantic segmentation network, and training the improved semantic segmentation network;
s2, initializing binocular vision and IMU combination;
s3, fusing binocular vision and IMU;
s4, constructing a bionic cell model;
and S5, constructing the cognitive map.
2. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 1, wherein: the improved convolutional neural network is used for semantic modeling and comprises a feature extraction layer, a feature fusion layer, a feature coding layer and a feature decoding layer; in the characteristic extraction stage, an improved DeepLab network with a coder-decoder network capable of separating convolution is used, the characteristics of an input image are fully extracted through a plurality of convolution layers, operation is carried out on the characteristics and cavity convolution kernels with the sizes of c × c and b × b respectively to extract the characteristics, and the result is sent to a characteristic decoding layer through the convolution of c × c to carry out pixel prediction; in the decoding stage, firstly, 4 times of upsampling is carried out on the feature map, then, the feature map is connected with the conv3 bottom-layer features, 4 times of convolution kernels with the size of 3 are expanded to obtain a semantic segmentation result, and semantic category vectors are extracted from the semantic segmentation result; and adding a convolution kernel with the size of 1 to the middle convolution layer, converting the feature map into a one-dimensional object feature vector, and merging the semantic segmentation vector and the object feature vector to obtain a semantic location vector.
3. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 2, wherein: step S1 specifically includes:
s1-1, collecting pictures of roads in the automatic driving data set, dividing the pictures into rectangular areas, and dividing a training set, a test set and a verification set according to a proportion;
and S1-2, sending the pictures in the training set into an improved semantic segmentation network, performing semantic segmentation test on the training set, and storing a group of network parameters with better effect obtained by performing semantic segmentation on the data set.
4. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 3, wherein: initializing binocular vision comprises the steps of:
s2-1, carrying out combined calibration on the binocular camera modules, and specifically representing the corresponding relation from a camera coordinate system to a world coordinate system by using a matrix:
where Xc, Yc, Zc are points of the camera coordinate system, Xw, Yw, Zw are world coordinate systems, R is a rotation matrix of the camera coordinate system relative to the world coordinate system, t is a translation matrix of the camera coordinate system relative to the world coordinate system, M is a rotation matrix of the camera coordinate system relative to the world coordinate system1Is a matrix of internal parameters about the camera:
the translation vector of the imaging point to the origin of the pixel coordinate system is [ d ]x,dy]TIf the focal length is f and the unit is pixel, then pixel point [ u, v]TTransformation to image coordinate system:
wherein x, y are the coordinates of the image coordinate system, u0And v0Half the length and width of the imaging plane;
the transformation matrix of the points on the plane of the camera coordinate system and the image coordinate system is:
the above formula is combined to obtain a conversion formula from an image coordinate system to a world coordinate system, the formula is used for calibrating the binocular camera,
wherein M is1Is a matrix of internal parameters, M, with respect to the camera2Is a matrix relating to the external parameters of the camera, the two matrices being simplified representations of a homogeneous transformation matrix as described by the equation, the matrix M representing a normalized internal and external parameter matrix, PwThree-dimensional coordinates of the spatial points; giving coordinates of a plurality of points of the calibration plate, and substituting the coordinates of the image of the corresponding points of the calibration plate and the coordinates of the world system to obtain a parameter matrix M of the camera, wherein the matrix M is used for calculating the spatial position of the matching points of the binocular camera;
s2-2, operating the binocular camera module and the IMU module, initializing an Inertial Measurement Unit (IMU) by using the pose of each key frame measured by binocular vision simultaneous positioning and mapping (SLAM), and calculating the following parameters including the dimension, the gravity direction, the speed, the acceleration and the offset of a gyroscope, wherein an IMU coordinate system is represented by B, a world coordinate system is represented by W, and the IMU coordinate system is represented by a certain time intervalSampling IMU output at intervals of delta t, measuring IMU accelerationWa (t) and angular velocityBωWB(t), the IMU measurement model is:
whereinBω′WB(t)∈R3Representing the true value of the instantaneous angular velocity ω of the gyroscope in a coordinate system B relative to the world coordinate system, where Bg(t) represents the IMU internal error, η, which varies slowly with time tg(t) is the measurement of white noise,Wa(t)∈R3representing the instantaneous acceleration, g, of the IMU in W coordinatesWRepresenting the gravity acceleration vector under the world coordinate system W,Ba' (t) represents the true value of the instantaneous acceleration a of the gyroscope in the coordinate system B relative to the world coordinate system, RWB T(t) represents the transpose of the rotation matrix of the gyroscope relative to the world coordinate system W under the coordinate system B, Ba(t) represents the acceleration error, η, of the IMU slowly varying over time ta(t) represents measurement white noise of the acceleration.
5. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 4, wherein: the binocular camera module is a vision module and is used for collecting image vision information and fixing the image vision information on the front side of the autonomous mobile platform, a binocular system is automatically initialized, a homography matrix is selected for a planar scene to calculate, a basic matrix is selected for a non-planar scene to calculate, two image frames are selected as initial frames through matching, and the pose between the two frames is calculated according to matching; the disparity of the binocular view at corresponding pixel points is calculated to form a disparity map, a global energy function associated with the disparity map is set, the optimal disparity of each pixel is solved by minimizing the global energy function, depth information is obtained through the disparity between the left image and the right image, and then the depth map is obtained.
6. The context awareness and mapping method for the autonomous mobile platform based on bionics of claim 5, wherein: the IMU module is an encoder module and is used for acquiring displacement information of the autonomous mobile platform and is installed on a base of the autonomous mobile platform.
7. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 6, wherein: initializing the IMU includes the steps of:
s2-2-1, using several continuous key frames to estimate the bias of gyroscope, and assuming a constant bgOn the basis of (2) plus the variation
The amount of update of the gyroscope bias is calculated by minimizing the rotation error:
where N is the number of key frames,
indicating rotation by camera
Rotation transformation matrix R from rigid body to cameraCB；ΔRi,i+1Is the detection quantity of the pre-integral rotation of two continuous image frames;
representing the rotation matrix, Δ R, from the IMU sensor coordinate system to the world coordinate systemi,i+1The method comprises the steps of performing pre-integration rotation measurement on two continuous key frames, giving an initial value of zero, and solving initial bias of a gyroscope corresponding to the minimum target value of the function through Gauss-Newton iteration;
s2-2-2, solving by calculating a linear equation between continuous frames to obtain an acceleration offset estimation value, a scale and a gravity direction accurate value;
s2-2-3, and substituting the formula (5) to calculate R under the condition of knowing the dimension and the gravity acceleration directionWBPosition and velocity measurements, completing the IMU initialization.
8. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 7, wherein: step S3 includes the following steps:
s3-1-1, two images shot at the same time are called as a frame, a group of left and right images shot by a binocular camera with stable IMU data are selected as initial frames, the IMU data started at the moment are stored, and the image frames are preprocessed through distortion correction;
s3-1-2, respectively extracting key points from a left image and a right image of the binocular camera, detecting and describing point features by adopting an ORB operator, wherein the ORB descriptor is a 256-bit binary descriptor, and an offline dictionary of the key point features is established by normalizing the feature descriptor;
s3-1-3, carrying out point feature matching from frame to frame, and eliminating mismatching of ORB feature points by adopting Random Sample Consensus (Random Sample Consensus) algorithm Random Sample Consensus;
s3-1-4, selecting matched feature points in the feature map to calculate the parallax, forming a parallax map, setting a global energy function related to the parallax map, minimizing the energy function, and solving the parallax optimal result of each pixel.
9. The context awareness and mapping method for the autonomous mobile platform based on bionics as claimed in claim 8, wherein: step S4 includes the following steps:
s4-1, describing a heading angle theta of the autonomous mobile platform gyroscope sensor through a head heading cell model, wherein the head heading cell is a head heading dependent neuron and is expressed as a Gaussian model, and obtaining a head heading transfer matrix by inputting an angle and an angle offset of the gyroscope sensor; modeling yields the head-oriented cell transfer equation for the velocity signal as:
wherein the signal generated by the i-th head towards the cell at time t is oriented predominantly in the direction θbAnd the angular offset thetaiDetermining that D (t) is the head orientation transfer matrix, typically setting the predominant orientation of the head towards the cell to 0 DEG θiIs the i-th head-to-cell offset, θ0～θmRepresenting the respective orientation angles of the m heads towards the cells, in the range of 0 °<θi<360 degrees, v (t) is the moving speed of the robot along the theta (t) direction at the moment t;
s4-2, describing the linear velocity obtained by the accelerometer sensor by using a stripe cell model, wherein stripe cells have a discharge field similar to a parallel stripe shape, and the stripe cell model obtains the current path of the autonomous mobile platform through integration; establishing a transfer model of the streak cells to the displacement signals as follows:
the stripe cells are coded into displacement in a specific direction, the discharge activity of the stripe cells is a periodic reset coding process and is responsible for path integration of linear velocity, an initial direction delta is defined, a discharge phase beta is defined, a discharge period is f, the moving speed of the robot in the theta (t) direction at the time t is v (t), and the speed in the delta direction is:
vθ(t)＝cos(δ-θ(t))v(t) (9)；
s4-3, expressing the movement of the current autonomous mobile platform in a two-dimensional space by using a grid cell model, wherein the grid cells are cells which are regularly discharged by rodents in the two-dimensional space; the discharge field represents the expression of partial environment of a certain position, direction information and displacement information are sent into a grid cell model, the grid cell model is modeled according to a continuous attractor network to generate grid cell group activity driven by speed, in an attractor network, neurons are arranged on a two-dimensional neural plate, in order to solve the problem of attractor network boundary, the neurons on the upper, lower, left and right boundaries of the neuron plate are connected to form a two-dimensional annular attractor model, the neurons are connected through local excitability and global inhibitability to form an activity packet on the neuron plate, the activity packet moves on the neuron plate along with the input of speed signals to generate the grid cell group, and the kinetic equation of a grid cell i based on the speed is expressed as follows:
wherein SiThe current active state of the lattice cell, TcIs the time constant of the response of the neuron,
inhibitory input of peripheral ring neurons to neuron i, FiIs the forward excitatory projection of the grid cells by the head-facing cells, determined primarily by velocity and direction, and is expressed for the above model as a function of f (x), where the independent variable x is expressed as the sum of excitatory and inhibitory projections, as in equation (10)
Is solved as follows
Connection weight
The connection weight of the grid cells i to j, g the gain for adjusting the activity of the grid cell group, the initialization gamma is 1.05 beta,
wherein λgridActive space of lattice cell wherein is active space of lattice cell, xiIs the location of the grid cell i on the neuron plate,
is the reset distance of the striped cell model;
the head-oriented cell model and the stripe cell model integrate information to form regular discharge and send signals to grid cells; after receiving the information, the grid cells form specific grid cell grid fields with fixed intervals through regular repeated discharge, and the grid fields correspond to a plurality of activation regions of an external space;
s4-4, describing the current position through a position cell model: the position cells are cells which are discharged at specific positions in a map space, have space specificity, map a position field in a brain hippocampus body with external specific space positions through selective discharge, discharge maximally to form position activation when an autonomous mobile platform reaches the specific positions in the space, establish new position points, and model a grid cell model through a Hebbian neural network (Hebbian neural network)
Wherein A is a positional cell gain constant; mlThe number of layers of the neural plate is shown, and r represents the position of the autonomous mobile platform;
the connection weight of grid cell i to position cell j; cplaceIs the inhibition constant of the site cell, sjAnd expressing the discharge rate of the grid cells, and obtaining a relation between the position cells and the grid cells:
where k is the learning rate, piAs a model of the location of cell discharge, sjEquation (13) represents the relationship between the activation rate of the grid cells, the inhibition rate of the grid cells, and the strength of the synaptic connections, and the synaptic connections become stronger if the activation is greater than the inhibition, and weaker otherwise.
10. The context awareness and mapping method for the autonomous mobile platform based on bionics of claim 9, wherein: step S5 includes the following steps:
s5-1, obtaining the semantic segmentation result of the current frame: sending the image frame to the improved semantic segmentation network in the step S1 to obtain a place semantic vector;
s5-2, if the place semantic feature vector does not exist in the existing cognitive map nodes and the depth information of the image frame is different from the known place, judging that a new position is reached and generating a new position cell;
s5-3, if a new position cell is generated, nodularizing the current position cell, adding the nodularized current position cell into an existing cognitive map, storing semantic vector information of a current image into a position library, adding depth information of a view object into the current position cell, expressing cognitive map network nodes by e, wherein each cognitive node comprises position cell discharge p, environment image characteristics V, topological relation L between cognitive nodes and depth information D of an environment image:
ei＝{pi,Vi,Li,D} (14)
judging position cell state p through environment image characteristic ViEnvironment image feature ViAnd when the information is not matched with the information in the cognitive nodes, a new cognitive node is created:
ei＝{pi,Vi,D,Li+ΔL} (15)
wherein, the delta L is the distance between the current cognitive node and the previous cognitive node, and the displacement information comes from the odometer system;
updating the pose information p of each node through the transfer matrix t of the experience map, and updating the current node eiPose information and successful closed loop detection matching node ejThe pose information of the map is fused to obtain the conversion of the experience mapThe updating formula of the motion matrix t is as follows:
where α is the correction rate, NfFor the current node eiNumber of connections to other nodes, NtIs the number of connections, p, of the successful closed-loop detection matching nodeiAnd Δ piRespectively representing the current value and the variation of the pose, pjAnd pkRespectively representing the pose values, Δ p, of the j and k positionsijAnd Δ pkiTranslation information updated for the location;
s5-4, obtaining cognitive node semantic information and depth information of the image frame according to the cognitive map, and navigating according to the environment information and the map information;
and S5-5, starting a new thread to execute global optimization, and optimizing the optimal structure of the whole cognitive map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521855.0A CN112509051A (en) | 2020-12-21 | 2020-12-21 | Bionic-based autonomous mobile platform environment sensing and mapping method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011521855.0A CN112509051A (en) | 2020-12-21 | 2020-12-21 | Bionic-based autonomous mobile platform environment sensing and mapping method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112509051A true CN112509051A (en) | 2021-03-16 |
Family
ID=74921888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011521855.0A Pending CN112509051A (en) | 2020-12-21 | 2020-12-21 | Bionic-based autonomous mobile platform environment sensing and mapping method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112509051A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114612767A (en) * | 2022-03-11 | 2022-06-10 | 电子科技大学 | Scene graph-based image understanding and expressing method, system and storage medium |
| CN114812565A (en) * | 2022-06-23 | 2022-07-29 | 北京航空航天大学 | Dynamic navigation method based on artificial intelligence network |
| CN116542859A (en) * | 2023-07-06 | 2023-08-04 | 武汉船舶职业技术学院 | Intelligent generation method of building structure column image thumbnail for intelligent construction |
| CN117723048A (en) * | 2023-12-18 | 2024-03-19 | 哈尔滨工业大学 | Multi-robot compressed communication collaborative mapping method and system under communication limitation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180046153A1 (en) * | 2016-07-10 | 2018-02-15 | Beijing University Of Technology | Method of Constructing Navigation Map by Robot using Mouse Hippocampal Place Cell Model |
| CN109668566A (en) * | 2018-12-05 | 2019-04-23 | 大连理工大学 | Robot scene cognition map construction and navigation method based on mouse brain positioning cells |
| CN111376273A (en) * | 2020-04-23 | 2020-07-07 | 大连理工大学 | Brain-like inspired robot cognitive map construction method |
| CN111462135A (en) * | 2020-03-31 | 2020-07-28 | 华东理工大学 | Semantic mapping method based on visual S L AM and two-dimensional semantic segmentation |
| CN111583136A (en) * | 2020-04-25 | 2020-08-25 | 华南理工大学 | Method for simultaneously positioning and establishing image of autonomous mobile platform in rescue scene |
-
2020
- 2020-12-21 CN CN202011521855.0A patent/CN112509051A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180046153A1 (en) * | 2016-07-10 | 2018-02-15 | Beijing University Of Technology | Method of Constructing Navigation Map by Robot using Mouse Hippocampal Place Cell Model |
| CN109668566A (en) * | 2018-12-05 | 2019-04-23 | 大连理工大学 | Robot scene cognition map construction and navigation method based on mouse brain positioning cells |
| CN111462135A (en) * | 2020-03-31 | 2020-07-28 | 华东理工大学 | Semantic mapping method based on visual S L AM and two-dimensional semantic segmentation |
| CN111376273A (en) * | 2020-04-23 | 2020-07-07 | 大连理工大学 | Brain-like inspired robot cognitive map construction method |
| CN111583136A (en) * | 2020-04-25 | 2020-08-25 | 华南理工大学 | Method for simultaneously positioning and establishing image of autonomous mobile platform in rescue scene |
Non-Patent Citations (1)
| Title |
|---|
| WU XINSHENG 等: "A Semantic Segmentation Algorithm Based on Improved Attention Mechanism", 《IEEE》, pages 244 - 248 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114612767A (en) * | 2022-03-11 | 2022-06-10 | 电子科技大学 | Scene graph-based image understanding and expressing method, system and storage medium |
| CN114812565A (en) * | 2022-06-23 | 2022-07-29 | 北京航空航天大学 | Dynamic navigation method based on artificial intelligence network |
| CN116542859A (en) * | 2023-07-06 | 2023-08-04 | 武汉船舶职业技术学院 | Intelligent generation method of building structure column image thumbnail for intelligent construction |
| CN116542859B (en) * | 2023-07-06 | 2023-09-01 | 武汉船舶职业技术学院 | Intelligent generation method of building structure column image thumbnail for intelligent construction |
| CN117723048A (en) * | 2023-12-18 | 2024-03-19 | 哈尔滨工业大学 | Multi-robot compressed communication collaborative mapping method and system under communication limitation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhou et al. | To learn or not to learn: Visual localization from essential matrices | |
| Dong et al. | 4D crop monitoring: Spatio-temporal reconstruction for agriculture | |
| CN110827415B (en) | All-weather unknown environment unmanned autonomous working platform | |
| US10372968B2 (en) | Object-focused active three-dimensional reconstruction | |
| CN113345018B (en) | Laser monocular vision fusion positioning mapping method in dynamic scene | |
| CN112509051A (en) | Bionic-based autonomous mobile platform environment sensing and mapping method | |
| Paz et al. | Probabilistic semantic mapping for urban autonomous driving applications | |
| CN113674416A (en) | Three-dimensional map construction method and device, electronic equipment and storage medium | |
| CN111998862A (en) | Dense binocular SLAM method based on BNN | |
| Yin et al. | Automerge: A framework for map assembling and smoothing in city-scale environments | |
| Saleem et al. | Neural network-based recent research developments in SLAM for autonomous ground vehicles: A review | |
| Chiu et al. | Sub-meter vehicle navigation using efficient pre-mapped visual landmarks | |
| Zhuang et al. | A biologically-inspired simultaneous localization and mapping system based on LiDAR sensor | |
| CN115690343A (en) | Robot laser radar scanning and mapping method based on visual following | |
| Billy et al. | Adaptive SLAM with synthetic stereo dataset generation for real-time dense 3D reconstruction | |
| Sujiwo et al. | Robust and accurate monocular vision-based localization in outdoor environments of real-world robot challenge | |
| Liu et al. | Laser 3D tightly coupled mapping method based on visual information | |
| Yang et al. | ALeader-following Method Based on Binocular Stereo Vision For Quadruped Robots | |
| Li et al. | RF-LOAM: Robust and Fast LiDAR Odometry and Mapping in Urban Dynamic Environment | |
| Le Barz et al. | Absolute geo-localization thanks to Hidden Markov Model and exemplar-based metric learning | |
| Lu | Image-based localization for self-driving vehicles based on online network adjustment in a dynamic scope | |
| Mo | Towards a fast, robust and accurate visual-inertial simultaneous localization and mapping system | |
| Lee et al. | Autonomous Vehicle Localization Without Prior High-Definition Map | |
| CN117557599B (en) | 3D moving object tracking method and system and storage medium | |
| Song et al. | Recalling direct 2d-3d matches for large-scale visual localization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20240507 |