CN114549852A - Pulse neural network training method based on color antagonism and attention mechanism - Google Patents

Pulse neural network training method based on color antagonism and attention mechanism Download PDF

Info

Publication number
CN114549852A
CN114549852A CN202210174117.6A CN202210174117A CN114549852A CN 114549852 A CN114549852 A CN 114549852A CN 202210174117 A CN202210174117 A CN 202210174117A CN 114549852 A CN114549852 A CN 114549852A
Authority
CN
China
Prior art keywords
pulse
characteristic diagram
weight
feature map
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210174117.6A
Other languages
Chinese (zh)
Other versions
CN114549852B (en
Inventor
高绍兵
姚智伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN202210174117.6A priority Critical patent/CN114549852B/en
Publication of CN114549852A publication Critical patent/CN114549852A/en
Application granted granted Critical
Publication of CN114549852B publication Critical patent/CN114549852B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pulse neural network training method based on color antagonism and attention mechanism, which comprises the following steps: s1, performing visual path processing based on color information and contour path processing based on a Gabor operator to obtain a pulse characteristic diagram A and a pulse characteristic diagram B; s2, obtaining a fusion pulse characteristic diagram C; s3, introducing an attention mechanism to obtain the weight of each feature map in the C; adjusting the weights using a color antagonism mechanism and pulse timing-dependent plasticity; and (4) assigning different weights to different feature maps by using an attention mechanism, and multiplying the obtained weight of each feature map in the C by the corresponding feature map in the C to obtain a new feature map. The invention can effectively combine the color information and the outline information in the color input image to obtain the pulse characteristic diagram with more abundant information and provide the pulse characteristic diagram for the pulse neural network to learn.

Description

Pulse neural network training method based on color antagonism and attention mechanism
Technical Field
The invention belongs to the technical field of computer vision and image processing, relates to unsupervised training of an impulse neural network, and particularly relates to an impulse neural network training method based on a color antagonism and attention mechanism.
Background
The impulse neural network belongs to a third generation neural network, and compared with an artificial neural network, the neurons of the impulse neural network are impulse neurons. The impulse neuron records its own voltage value and once the voltage value is above a certain threshold, it releases an impulse that is passed along the connection to the next layer of impulse neurons, but the general back propagation cannot be used to train the impulse neural network due to the inconductivity of the impulse release function. In recent years, the field of impulse neural networks has become abnormal fire heat, and training methods thereof can be mainly classified into three categories: an impulse neural network is trained based on methods of transition from artificial neural networks, back-propagation methods based on gradient substitution, and using unsupervised learning algorithms. The STDP algorithm commonly used in the method for training the impulse neural network by using the unsupervised learning algorithm is the pulse timing sequence dependence plasticity in the method.
For most training methods based on the STDP and its variants, they mostly use grayscale images as the input of the network, abandon the color information of the images, and even if there are a few methods to consider the extraction and learning of color information, they do not start from biological vision, do not simulate biological vision path, and are difficult to provide better biological interpretability.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a pulse neural network training method based on color antagonism and attention mechanism, which can effectively combine color information and contour information in a color input image to obtain a pulse characteristic diagram with more abundant information and provide the pulse characteristic diagram for pulse neural network learning.
The purpose of the invention is realized by the following technical scheme: the impulse neural network training method based on the color antagonism and attention mechanism comprises the following steps:
s1, performing visual path processing based on color information and contour path processing based on a Gabor operator in parallel to respectively obtain a pulse characteristic diagram A and a pulse characteristic diagram B;
s2, obtaining a fusion pulse characteristic diagram C: splicing the pulse characteristic diagram A and the pulse characteristic diagram B end to end, and then performing pooling operation to obtain a pulse characteristic diagram C;
s3, introducing an attention mechanism to obtain the weight of each feature map in the C; adjusting the weights using a color antagonism mechanism and pulse timing-dependent plasticity; allocating different weights to different feature maps by using an attention mechanism, multiplying the weight of each feature map in the C with the corresponding feature map in the C to serve as a new feature map, and then performing convolution operation on the new feature map and a convolution kernel; the color profile in C uses color antagonism and pulse timing dependent plasticity adjusted weights, and the profile in C uses only pulse timing dependent plasticity adjusted weights.
Further, the specific implementation method of step S1 is as follows: converting the RGB color image into an LMS color space based on a visual access of color information, and determining the time for releasing the pulse according to the pixel value to obtain a pulse characteristic diagram A; the specific treatment method comprises the following steps:
Figure BDA0003518396830000021
Figure BDA0003518396830000022
wherein the content of the first and second substances,
Figure BDA0003518396830000023
and
Figure BDA0003518396830000024
representing matrix representations of the original image in an RGB color space, an XYZ color space and an LMS color space respectively; equations (1) and (2) represent the process of converting the original RGB image into the LMS space;
contour path processing based on Gabor operators uses a plurality of Gabor operators to extract image contours, and determines the time for releasing pulses according to the pixel values to obtain a pulse characteristic diagram B; the specific method comprises the following steps: the contour information in the image is extracted by convolving 4 Gabor operators with different directions (pi/8, pi/4 + pi/8, pi/2 + pi/8, 3 pi/4 + pi/8) with the gray scale image.
Further, the specific implementation method of step S3 is as follows:
the attention mechanism processing is carried out, and the realization method comprises the following steps: firstly, performing convolution operation on the pulse characteristic diagram C to obtain a pulse characteristic diagram D after convolution, and calculating the weight of each characteristic diagram in the C according to the C and the D:
Figure BDA0003518396830000025
wherein, afRepresenting the weight, p, of the f-th feature map+And p-Representing the parameters for enhancement or suppression of the feature map respectively,
Figure BDA0003518396830000026
and
Figure BDA0003518396830000027
respectively, representing the mean time of the release pulse for all points on the profile before and after convolution, the Sigmoid () function being used to correct for
Figure BDA0003518396830000028
Zooming;
multiplying the obtained weight of each feature map in the C by the corresponding feature map in the C to obtain a new feature map;
the color antagonism mechanism is realized by the following steps:
Figure BDA0003518396830000029
Figure BDA00035183968300000210
Figure BDA00035183968300000211
Figure BDA0003518396830000031
Figure BDA0003518396830000032
Figure BDA0003518396830000033
DSLthe values of the weight change amounts corresponding to the S feature map and the L feature map corresponding to the LMS space, DSMThe numerical value of the corresponding weight variation of the S characteristic diagram and the M characteristic diagram corresponding to the LMS space is shown, and T represents the maximum time step; the parameter rho is used to scale the index part parameter and is [0, 3 ]]Any real number within; | Δ tSL| represents the absolute value of the difference between the S profile and the time point of the release pulse at each position on the L profile; | Δ tSM| represents the absolute value of the difference between the S profile and the time point of the release pulse at each position on the M profile;
equations (6), (7), (8) and (9) represent the amount of change in the weights corresponding to the characteristic diagram finally applied to S, L, M, respectively; Δ WS＝ΔWS1+ΔWS2Represents the amount of change, Δ W, of the weight ultimately applied to the S-profileLRepresents the amount of change, Δ W, of the weight ultimately applied to the L profileMThe variable quantity of the weight corresponding to the final action on the M characteristic diagram is shown, and alpha + and alpha-respectively show a positive parameter and a negative parameter which are used for controlling the increase and the decrease of the weight; Δ tpre-postRepresenting the time difference between the release of the pulse by the pre-convolution neuron and the post-convolution neuron;
the specific implementation of pulse timing-dependent plasticity is:
Figure BDA0003518396830000034
Δ W represents the calculated amount of change in the weight of a connection, W represents the current value of the weight of the connection; p is a radical of+And p-Respectively representing a positive and a negative parameter, for controlling the increase or decrease of the weight, where p+And p-The values of (a) are 0.05 and-0.015, respectively; t is tpreAnd tpostRespectively representing the time when the connected front and back neurons release the pulse;
the method for training the impulse neural network based on the color antagonism and the impulse timing sequence dependence plasticity adjustment weight comprises the following steps:
Figure BDA0003518396830000035
Wfrepresenting the convolution kernel corresponding to the f-th feature map.
The invention has the beneficial effects that: the method can effectively train the impulse neural network, can effectively combine the color information and the contour information in the color input image, and obtains the impulse characteristic diagram with more abundant information to be provided for the impulse neural network to learn. Meanwhile, the accuracy of the classification of the impulse neural network can be effectively improved by simulating the processing mode of the biological visual pathway on the color information and the attention mechanism, and a reliable thought and method are provided for the subsequent training of exploring the biological visual pathway and the impulse neural network.
Drawings
FIG. 1 is a flow chart of a method for spiking neural network training based on color antagonism and attention mechanism according to the present invention;
FIG. 2 is a data set image employed in the present embodiment;
FIG. 3 is a diagram illustrating the result of processing using a visual path based on color information and a contour path based on Gabor operator according to the present embodiment;
FIG. 4 shows the result of pooling after stitching two pulse profiles according to this example.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
As shown in fig. 1, the method for training the impulse neural network based on the color antagonism and attention mechanism of the present invention includes the following steps:
s1, performing visual path processing based on color information and contour path processing based on a Gabor operator in parallel to respectively obtain a pulse characteristic diagram A and a pulse characteristic diagram B;
the specific implementation method comprises the following steps: converting the RGB color image into an LMS color space based on a visual access of color information, and determining the time for releasing the pulse according to the pixel value to obtain a pulse characteristic diagram A; the specific treatment method comprises the following steps:
Figure BDA0003518396830000041
Figure BDA0003518396830000042
wherein the content of the first and second substances,
Figure BDA0003518396830000043
and
Figure BDA0003518396830000044
representing matrix representations of the original image in an RGB color space, an XYZ color space and an LMS color space respectively; equations (1) and (2) represent the process of converting the original RGB image into the LMS space;
contour path processing based on Gabor operators uses a plurality of Gabor operators to extract image contours, and determines the time for releasing pulses according to the pixel values to obtain a pulse characteristic diagram B; the specific method comprises the following steps: extracting contour information in the image by using 4 Gabor operators with different directions (pi/8, pi/4 + pi/8, pi/2 + pi/8 and 3 pi/4 + pi/8) to be convolved with the gray image;
the method for determining the time for releasing the pulse according to the pixel value is as follows: using delay coding, each point calculates its pulse time. The larger the pixel value of a dot, the earlier the dot releases the pulse. The time interval here ranges from 0, 15, i.e. closer to 0 and vice versa. The specific method comprises the following steps: and sequencing the pixel values of the points, equally dividing the values into T time step lengths, generating a pulse characteristic graph which has the same size as the original graph and T times, and releasing pulses from the points with the maximum pixel values on the first graph. In this example, for an image with size (62, 62), the number of pixels is 62 × 62 — 3844, and assuming that the length of the time step is set to 15, there are new 3844/15-256 pixel points that release pulses at each time step (the larger the pixel value, the earlier the pulse is released), that is, the value is set to 1, and the rest of the points are set to 0.
The size of the finally generated pulse characteristic graph is (15, 62, 62), the size of the first graph is (62, 62), 256 points on the graph are 1, and the rest points are 0; in the second (62, 61), not only 256 points on the first graph are 1, but also new 256 points (the pixel values of which are smaller than those of the 256 points on the first graph) are set to 1, and the rest of the points are 0; by analogy …
This embodiment uses an RGB image with an image size of (3, 64, 64), as shown in fig. 2, which is derived from an ETH-80 image dataset, showing only two of the eight categories; in step S1 (formulas (1) to (2)), a RGB image of "horse" is processed using a visual path based on color information, that is, the image is first cut into RGB images of size (3, 62, 62), and then converted into LMS space, where a total time step length is set to 15, and a pulse feature map a of size (15, 3, 62, 62) is generated according to the pixel value size, as shown in fig. 3 (b); and the RGB image with the image size of (3, 64, 64) is converted into a grayscale image with the size of (64, 64) based on the contour path of the Gabor operator, then the contours in different directions are extracted by using 4 Gabor operators in different directions to generate a feature map with the size of (4, 62, 62), and then the feature map is converted into a pulse feature map B with the size of (15, 4, 62, 62), as shown in fig. 3 (B).
S2, obtaining a fusion pulse characteristic diagram C: splicing the pulse characteristic diagram A and the pulse characteristic diagram B end to obtain a pulse characteristic diagram with richer information, and then performing pooling operation on the pulse characteristic diagram to reduce the size of the characteristic diagram and obtain translation deformation-free to obtain a pulse characteristic diagram C; the pulse feature map a and the pulse feature map B are merged into a fused pulse feature map with the size of (15, 7, 62, 62), and pooling is performed to obtain a fused pulse feature map C with the size of (15, 7, 30, 30), as shown in fig. 4.
S3, introducing an attention mechanism to obtain the weight of each feature map in the C; adjusting the weights using a color antagonism mechanism and pulse timing-dependent plasticity; allocating different weights to different feature maps by using an attention mechanism, multiplying the weight of each feature map in the C with the corresponding feature map in the C to serve as a new feature map, and then performing convolution operation on the new feature map and a convolution kernel; the color feature graph in the graph C uses a color antagonism mechanism and pulse timing-dependent plasticity to adjust the weight, and the contour feature graph in the graph C only uses the pulse timing-dependent plasticity to adjust the weight;
the specific implementation method comprises the following steps:
the method for realizing attention mechanism processing comprises the following steps: firstly, performing convolution operation on the pulse characteristic diagram C to obtain a pulse characteristic diagram D after convolution, and calculating the weight of each characteristic diagram in the C according to the C and the D:
Figure BDA0003518396830000061
wherein, afRepresenting the weight, p, of the f-th feature map+And p-Respectively, the parameters for enhancing or suppressing the characteristic diagram, p in this embodiment+And p-The values of (A) were taken to be 0.005 and-0.001, respectively.
Figure BDA0003518396830000062
And
Figure BDA0003518396830000063
respectively representing the release of all points on the characteristic map before and after convolutionMean time of pulse, Sigmoid () function is used for
Figure BDA0003518396830000064
Zooming;
multiplying the obtained weight of each feature map in the C by the corresponding feature map in the C to serve as a new feature map, and then performing convolution operation on the new feature map and a convolution kernel;
the color antagonism mechanism is realized by the following steps:
Figure BDA0003518396830000065
Figure BDA0003518396830000066
Figure BDA0003518396830000067
Figure BDA0003518396830000068
Figure BDA0003518396830000069
Figure BDA00035183968300000610
DSLthe values of the weight change amounts corresponding to the S feature map and the L feature map corresponding to the LMS space, DSMThe numerical value of the corresponding weight variation of the S characteristic diagram and the M characteristic diagram corresponding to the LMS space is shown, and T represents the maximum time step; the parameter rho is used to scale the index part parameter and is [0, 3 ]]Any real number within; | Δ tSLI represents the time of the release pulse at each position on the S and L profilesThe absolute value of the difference between the points; | Δ tSM| represents the absolute value of the difference between the S profile and the time point of the release pulse at each position on the M profile;
equations (6), (7), (8) and (9) represent the amount of change in the weights corresponding to the characteristic diagram finally applied to S, L, M, respectively; Δ WS＝ΔWS1+ΔWS2Represents the amount of change, Δ W, of the weight ultimately applied to the S-profileLRepresents the amount of change, Δ W, of the weight ultimately applied to the L profileMRepresenting the variation of the weight finally acted on the M characteristic diagram, wherein alpha + and alpha-respectively represent a positive parameter and a negative parameter and are used for controlling the increase and decrease of the weight; Δ tpre-postRepresenting the time difference between the release of the pulse by the pre-convolution neuron and the post-convolution neuron; in this embodiment, ρ is taken as
Figure BDA0003518396830000071
α+And alpha-The values of (A) are 0.1 and-0.1, respectively.
ΔWS、ΔWL、ΔWMThe three variable quantities respectively correspond to the variable quantities on the three convolution kernels of the LMS characteristic diagram and are respectively added back to the corresponding convolution kernels; that is, Δ WSAnd added back to the convolution kernel corresponding to the L-feature map (i.e., the one convolved with the L-feature map in two dimensions during the convolution stage), for two other similar reasons.
The specific implementation of pulse timing-dependent plasticity is:
Figure BDA0003518396830000072
Δ W represents the calculated amount of change in the weight of a connection, W represents the current value of the weight of the connection; p is a radical of+And p-Respectively representing a positive and a negative parameter, for controlling the increase or decrease of the weight, where p+And p-The values of (a) are 0.05 and-0.015, respectively; t is tpreAnd tpostRespectively representing the time when the connected front and back neurons release the pulse;
the method for training the impulse neural network based on the color antagonism and the impulse timing sequence dependence plasticity adjustment weight comprises the following steps:
Figure BDA0003518396830000073
Wfwhat is meant is the value of the weight (i.e., the convolution kernel). W on the rightfIs the original weight value, and W on the leftfIs the weight value after the picture training. WfRepresenting the convolution kernel corresponding to the f-th feature map. Namely: a convolution kernel corresponding to the feature map of the extracted contour, trained using only STDP (adjusted once using STDP); the convolution kernels corresponding to the three characteristic maps of the LMS respectively need to use two adjusting modes of STDP and color antagonism, and the two adjusting modes do not have a sequence.
And adjusting the weight of the impulse neural network by using the method until the preset adjusting times are reached and finishing the training to obtain the trained impulse neural network.
S4, outputting the convolution result to a classifier for classification: and transforming the convolution result into a pulse sequence, and enabling a classifier to classify. First, a feature weight (each value corresponds to a feature map) with the size of (7, 1, 1) is calculated by using an attention mechanism (formula (3)), then, a fused pulse feature map C with the size of (15, 7, 30, 30) is multiplied by the feature weight with the size of (7, 1, 1), then, the multiplication result is convolved with N weights with the size of (7, 30, 30), a pulse feature map with the size of (15, N, 1, 1) after convolution is obtained, and the weights are updated by using a weight change method in S3.
After the network training is completed, step S4 converts the convolution result with the size of S3 being (15, N, 1, 1) into a pulse sequence tensor with the length of 15 × N, and outputs the pulse sequence tensor to the support vector machine for classification.
The above simple example is mainly illustrated and shown by taking an experiment of a single image on a trained impulse neural network as an example, during actual calculation, a training set (one image is input and processed in the above manner, and the next image is input after the weight is updated) is regarded as one training, then the training is stopped by setting the training for 30 times, after the training is finished, a test set (one image is input to the trained network, and the same is performed), and then the output of the network is transmitted to a classifier.
It will be appreciated by those of ordinary skill in the art that the embodiments described herein are intended to assist the reader in understanding the principles of the invention and are to be construed as being without limitation to such specifically recited embodiments and examples. Those skilled in the art can make various other specific changes and combinations based on the teachings of the present invention without departing from the spirit of the invention, and these changes and combinations are within the scope of the invention.

Claims (3)

1. The pulse neural network training method based on the color antagonism and attention mechanism is characterized by comprising the following steps of:
s1, performing visual path processing based on color information and contour path processing based on a Gabor operator in parallel to respectively obtain a pulse characteristic diagram A and a pulse characteristic diagram B;
s2, obtaining a fusion pulse characteristic diagram C: splicing the pulse characteristic diagram A and the pulse characteristic diagram B end to end, and then performing pooling operation to obtain a pulse characteristic diagram C;
s3, introducing an attention mechanism to obtain the weight of each feature map in the C; adjusting the weights using a color antagonism mechanism and pulse timing-dependent plasticity; allocating different weights to different feature maps by using an attention mechanism, multiplying the weight of each feature map in the C with the corresponding feature map in the C to serve as a new feature map, and then performing convolution operation on the new feature map and a convolution kernel; the color profile in C uses color antagonism and pulse timing dependent plasticity adjusted weights, and the profile in C uses only pulse timing dependent plasticity adjusted weights.
2. The method for training the spiking neural network based on the color antagonism and attention mechanism as claimed in claim 1, wherein the step S1 is implemented by: converting the RGB color image into an LMS color space based on a visual access of color information, and determining the time for releasing the pulse according to the pixel value to obtain a pulse characteristic diagram A; the specific treatment method comprises the following steps:
Figure FDA0003518396820000011
Figure FDA0003518396820000012
wherein the content of the first and second substances,
Figure FDA0003518396820000013
representing matrix representations of the original image in an RGB color space, an XYZ color space and an LMS color space respectively; equations (1) and (2) represent the process of converting the original RGB image into the LMS space;
contour path processing based on Gabor operators uses a plurality of Gabor operators to extract image contours, and determines the time for releasing pulses according to the pixel values to obtain a pulse characteristic diagram B; the specific method comprises the following steps: the contour information in the image is extracted by convolving 4 Gabor operators with different directions (pi/8, pi/4 + pi/8, pi/2 + pi/8, 3 pi/4 + pi/8) with the gray scale image.
3. The method for training the spiking neural network based on the color antagonism and attention mechanism as claimed in claim 1, wherein the step S3 is implemented by:
the attention mechanism processing is carried out, and the realization method comprises the following steps: firstly, performing convolution operation on the pulse characteristic diagram C to obtain a pulse characteristic diagram D after convolution, and calculating the weight of each characteristic diagram in the C according to the C and the D:
Figure FDA0003518396820000014
wherein, afRepresenting the weight, p, of the f-th feature map+And p-Representing the parameters for enhancement or suppression of the feature map respectively,
Figure FDA0003518396820000021
and
Figure FDA0003518396820000022
respectively, representing the mean time of the release pulse for all points on the profile before and after convolution, the Sigmoid () function being used to correct for
Figure FDA0003518396820000023
Zooming;
multiplying the weight of each feature map in the obtained C by the corresponding feature map in the C to obtain a new feature map;
the color antagonism mechanism is realized by the following steps:
Figure FDA0003518396820000024
Figure FDA0003518396820000025
Figure FDA0003518396820000026
Figure FDA0003518396820000027
Figure FDA0003518396820000028
Figure FDA0003518396820000029
DSLthe values of the weight change amounts corresponding to the S profile and the L profile corresponding to the LMS space are shown, DSMThe numerical value of the corresponding weight variation of the S characteristic diagram and the M characteristic diagram corresponding to the LMS space is shown, and T represents the maximum time step; the parameter rho is used to scale the index part parameter and is [0, 3 ]]Any real number within; | Δ tSL| represents the absolute value of the difference between the S profile and the time point of the release pulse at each position on the L profile; | Δ tSM| represents the absolute value of the difference between the S profile and the time point of the release pulse at each position on the M profile;
equations (6), (7), (8) and (9) represent the amount of change in the weights corresponding to the characteristic diagram finally applied to S, L, M, respectively; Δ WS＝ΔWS1+ΔWS2Represents the amount of change, Δ W, of the weight ultimately applied to the S-profileLRepresents the amount of change, Δ W, of the weight ultimately applied to the L profileMThe variable quantity of the weight corresponding to the final action on the M characteristic diagram is shown, and alpha + and alpha-respectively show a positive parameter and a negative parameter which are used for controlling the increase and the decrease of the weight; Δ tpre-postRepresenting the time difference between the release of the pulse by the pre-convolution neuron and the post-convolution neuron;
the specific implementation of pulse timing-dependent plasticity is:
Figure FDA0003518396820000031
Δ W represents the calculated amount of change in the weight of a connection, W represents the current value of the weight of the connection; p is a radical of+And p-Respectively representing a positive and a negative parameter, for controlling the increase or decrease of the weight, where p+And p-The values of (a) are 0.05 and-0.015, respectively; t is tpreAnd tpostRespectively representing the time when the connected front and back neurons release the pulse;
the method for training the impulse neural network based on the color antagonism and the impulse timing sequence dependence plasticity adjustment weight comprises the following steps:
Figure FDA0003518396820000032
Wfrepresenting the convolution kernel corresponding to the f-th feature map.
CN202210174117.6A 2022-02-24 2022-02-24 Impulse neural network training method based on color antagonism and attention mechanism Active CN114549852B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210174117.6A CN114549852B (en) 2022-02-24 2022-02-24 Impulse neural network training method based on color antagonism and attention mechanism

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210174117.6A CN114549852B (en) 2022-02-24 2022-02-24 Impulse neural network training method based on color antagonism and attention mechanism

Publications (2)

Publication Number Publication Date
CN114549852A true CN114549852A (en) 2022-05-27
CN114549852B CN114549852B (en) 2023-04-18

Family

ID=81677813

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210174117.6A Active CN114549852B (en) 2022-02-24 2022-02-24 Impulse neural network training method based on color antagonism and attention mechanism

Country Status (1)

Country Link
CN (1) CN114549852B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140081895A1 (en) * 2012-09-20 2014-03-20 Oliver Coenen Spiking neuron network adaptive control apparatus and methods
CN110555523A (en) * 2019-07-23 2019-12-10 中建三局智能技术有限公司 short-range tracking method and system based on impulse neural network
CN112633497A (en) * 2020-12-21 2021-04-09 中山大学 Convolutional pulse neural network training method based on reweighted membrane voltage
CN113111758A (en) * 2021-04-06 2021-07-13 中山大学 SAR image ship target identification method based on pulse neural network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140081895A1 (en) * 2012-09-20 2014-03-20 Oliver Coenen Spiking neuron network adaptive control apparatus and methods
CN110555523A (en) * 2019-07-23 2019-12-10 中建三局智能技术有限公司 short-range tracking method and system based on impulse neural network
CN112633497A (en) * 2020-12-21 2021-04-09 中山大学 Convolutional pulse neural network training method based on reweighted membrane voltage
CN113111758A (en) * 2021-04-06 2021-07-13 中山大学 SAR image ship target identification method based on pulse neural network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
MANTAS LUKOŠEVIČIUS 等: "Reservoir computing approaches to recurrent neural network training" *
NA GUO 等: "A Neurally Inspired Pattern Recognition Approach with Latency-Phase Encoding and Precise-Spike-Driven Rule in Spiking Neural Network" *
胡伟泰: "颜色视觉通路的脉冲神经网络模拟及特征提取" *

Also Published As

Publication number Publication date
CN114549852B (en) 2023-04-18

Similar Documents

Publication Publication Date Title
WO2022252272A1 (en) Transfer learning-based method for improved vgg16 network pig identity recognition
CN108510485B (en) Non-reference image quality evaluation method based on convolutional neural network
CN110414377B (en) Remote sensing image scene classification method based on scale attention network
CN112614077B (en) Unsupervised low-illumination image enhancement method based on generation countermeasure network
CN108520206B (en) Fungus microscopic image identification method based on full convolution neural network
Kong et al. Hexpo: A vanishing-proof activation function
CN112307714A (en) Character style migration method based on double-stage deep network
CN112818764A (en) Low-resolution image facial expression recognition method based on feature reconstruction model
CN113554599B (en) Video quality evaluation method based on human visual effect
CN111695590B (en) Deep neural network feature visualization method for constraint optimization class activation mapping
CN113420794B (en) Binaryzation Faster R-CNN citrus disease and pest identification method based on deep learning
CN114581560A (en) Attention mechanism-based multi-scale neural network infrared image colorizing method
CN109190666B (en) Flower image classification method based on improved deep neural network
CN114186672A (en) Efficient high-precision training algorithm for impulse neural network
CN110598737B (en) Online learning method, device, equipment and medium of deep learning model
CN111126155B (en) Pedestrian re-identification method for generating countermeasure network based on semantic constraint
CN114492634B (en) Fine granularity equipment picture classification and identification method and system
CN111832517A (en) Low-definition face key point detection method based on gated convolution
CN109934835B (en) Contour detection method based on deep strengthening network adjacent connection
CN114170657A (en) Facial emotion recognition method integrating attention mechanism and high-order feature representation
CN112270404A (en) Detection structure and method for bulge defect of fastener product based on ResNet64 network
Wu et al. Remote sensing image colorization based on multiscale SEnet GAN
CN114549852B (en) Impulse neural network training method based on color antagonism and attention mechanism
CN115526891B (en) Training method and related device for defect data set generation model
CN116993975A (en) Panoramic camera semantic segmentation method based on deep learning unsupervised field adaptation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant