US10915815B1 - Information processing method, system and device based on contextual signals and prefrontal cortex-like network - Google Patents
Information processing method, system and device based on contextual signals and prefrontal cortex-like network Download PDFInfo
- Publication number
- US10915815B1 US10915815B1 US16/971,691 US201916971691A US10915815B1 US 10915815 B1 US10915815 B1 US 10915815B1 US 201916971691 A US201916971691 A US 201916971691A US 10915815 B1 US10915815 B1 US 10915815B1
- Authority
- US
- United States
- Prior art keywords
- feature vector
- contextual
- network
- information
- denotes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000002360 prefrontal effect Effects 0.000 title claims abstract description 62
- 230000010365 information processing Effects 0.000 title claims abstract description 34
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 239000013598 vector Substances 0.000 claims abstract description 207
- 238000000605 extraction Methods 0.000 claims abstract description 30
- 239000011159 matrix material Substances 0.000 claims description 31
- 238000000034 method Methods 0.000 claims description 23
- 238000013528 artificial neural network Methods 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 15
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000015654 memory Effects 0.000 description 3
- 210000002442 prefrontal cortex Anatomy 0.000 description 3
- 241000288906 Primates Species 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000001747 exhibiting effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24143—Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]
-
- G06K9/6201—
-
- G06K9/6228—
-
- G06K9/6232—
-
- G06K9/6268—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/768—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using context analysis, e.g. recognition aided by known co-occurring patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Definitions
- the present invention belongs to the field of pattern recognition and machine learning, and more particularly, relates to an information processing method, system and device based on contextual signals and a prefrontal cortex-like network.
- the prefrontal cortex which is highly elaborated in primates, is pivotal for such an ability.
- the prefrontal cortex can quickly learn the “rules of the game” and dynamically apply them to map the sensory inputs to context-dependent tasks based on different actions.
- This process named cognitive control, allows primates to behave appropriately in an unlimited number of situations.
- the current artificial neural networks lack enough flexibility to work in complex situations in which the mapping rules may change according to the context and these rules need to be learned “on the go” from a small number of training samples. This constitutes a significant ability gap between artificial neural networks and human brains.
- the present invention provides an information processing method based on contextual signals and a prefrontal cortex-like network, including:
- step S 10 selecting a feature vector extractor based on obtained information to perform feature extraction to obtain an information feature vector
- step S 20 inputting the information feature vector into the prefrontal cortex-like network, and performing dimensional matching between the information feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set;
- step S 30 classifying each feature vector in the contextual feature vector set by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and the classification information.
- step S 10 includes:
- a method for constructing the feature vector extractor includes:
- step S 20 includes:
- step S 201 constructing a weight matrix based on the contextual signal and the prefrontal cortex-like network, and normalizing each column of the weight matrix;
- step S 202 based on the weight matrix, performing the dimensional matching between the contextual signal and the information feature vector to obtain the contextual feature vectors;
- Y out g ([ c 1 cos ⁇ 1 ,c 2 cos ⁇ 2 , . . . ,c m cos ⁇ m ] T ) ⁇ F ⁇ ,
- ⁇ denotes element-wise multiplication of the vector
- step S 203 constructing the contextual feature vector set by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the information feature vector.
- Y Lable denotes the classification information
- W out denotes a classification weight of the classifier
- Y out denotes an output feature of the prefrontal cortex-like network
- n denotes a dimension of an output weight of the prefrontal cortex-like network
- F denotes the information feature vector.
- configuration parameters of the Adam algorithm include:
- a learning rate of 0.1 a weight decay rate of 0.0001
- a batch size of 256 a batch size of 256.
- the contextual signal is a multi-dimensional word vector corresponding to a classification attribute, and a dimension of the word vector is 200.
- the weight matrix W in is:
- a matrix constructed based on a dimension of a word vector and a dimension of the prefrontal cortex-like network, wherein the matrix has a size of the dimension of the word vector x the dimension of the prefrontal cortex-like network.
- an information processing system based on contextual signals and a prefrontal cortex-like network includes an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module.
- the acquisition module is configured to obtain input information and an input contextual signal set and input.
- the feature extraction module is configured to extract features of the input information by using a feature vector extractor corresponding to the input information to obtain an information feature vector.
- the dimensional matching module is configured to input the information feature vector into the prefrontal cortex-like network, and perform dimensional matching between the information feature vector and each contextual signal in the input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set.
- the classification module is configured to classify each feature vector in the contextual feature vector set by a pre-constructed feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set.
- the output module is configured to output the acquired classification information set.
- a storage device wherein a plurality of programs are stored in the storage device, and the programs are configured to be loaded and executed by a processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
- a processing device includes a processor and a storage device.
- the processor is configured to execute a plurality of programs.
- the storage device is configured to store the plurality of programs.
- the programs are configured to be loaded and executed by the processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
- the present invention has the following advantages.
- the multi-task information processing method of the present invention uses a prefrontal cortex-like module to realize the multi-task learning oriented to the scene information.
- the mapping depending on the contextual information can be gradually learned step by step.
- the data processed by the method of the present invention can be applied to multi-task learning or sequential multi-task learning with higher requirements, whereby the network structure can be simplified, thus reducing the difficulty of multi-task learning and increasing the flexibility of the system.
- the deep neural network is used as the feature extractor, and the optimization method is designed on the linear layer. In this way, the deep neural network exerts its full role, and the design difficulty is reduced.
- the contextual signals are designed and can be changed according to the change of the current working environment, which solves the disadvantage that the neural network cannot respond differently to the same stimulus according to different goals, environments, internal states and other situations.
- FIG. 1 is a schematic flowchart of the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention
- FIG. 2 is a schematic diagram showing the network structure of the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention
- FIG. 3 is a schematic diagram showing the three-dimensional space of an embodiment of the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention
- FIG. 4 is a schematic diagram showing the network architectures of the traditional multi-task learning network and the sequential multi-task learning network according to the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention.
- FIG. 5 is a schematic diagram showing the accuracy of the face recognition task of the multi-task training and the sequential multi-scene training according to the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention.
- An information processing method based on contextual signals and a prefrontal cortex-like network includes:
- step S 10 a feature vector extractor is selected based on obtained information to perform feature extraction to obtain an information feature vector;
- step S 20 the image feature vector is input into a prefrontal cortex-like network, and dimensional matching is performed between the image feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set;
- step S 30 each feature vector in the contextual feature vector set is classified by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and the classification information.
- An information processing method based on contextual signals and a prefrontal cortex-like network includes steps S 10 -step S 30 , and each step is described in detail as follows.
- Step S 10 a feature vector extractor is selected based on obtained information to perform feature extraction to obtain an information feature vector.
- the feature vector extractor corresponding to the category of the obtained information is selected.
- the feature vector extractor library includes at least one selected from the group consisting of an image feature vector extractor, a speech feature vector extractor and a text feature vector extractor.
- the feature vector extractor library can also include feature vector extractors of other common information categories, which are not enumerated in detail herein.
- the feature vector extractor can be constructed based on the deep neural network.
- a deep neural network such as a residual neural network (ResNet) can be selected for the input image information.
- the deep neural network such as a convolutional neural network (CNN), long short-term memory (LSTM) neural network and gated recurrent unit (GRU) neural network can be selected for the input speech information.
- the deep neural network such as fastText, Text-convolutional neural network (TextCNN) and Text-recurrent neural network (TextRNN) can be used for the input text information.
- information is generally multimodal, and features thereof can be extracted by simultaneously using several feature vector processors in combination, which can not only enrich the expression of information, but also significantly reduce the dimension of the original information to facilitate processing the downstream information.
- a method for constructing each feature vector extractor in the feature vector extractor library includes:
- a weight of a parameter in a feature vector extraction network is iteratively updated by the Adam algorithm, wherein the feature vector extraction network is constructed based on a deep neural network;
- a network obtained after removing the last classification layer of the trained feature vector extraction network is used as the feature vector extractor.
- the Adam algorithm has a learning rate of 0.1, a weight decay rate of 0.0001, and a batch size of 256.
- the training data set used in the present invention is the face data set CelebFaces Attributes (CelebA) (CelebA is the open data of the Chinese University of Hong Kong, containing 202599 face images of 10177 celebrities, each with 40 corresponding attributes. Each attribute is used as a kind of scene, and different scenes correspond to different contextual signals.)
- the deep neural network ResNet50 is trained using the Adam algorithm, and the last classification layer of the ResNet50 network is removed.
- the output of the penultimate layer of the ResNet50 network is used as the feature of face data and has 2048 dimensions.
- Step S 20 the information feature vector is input into the prefrontal cortex-like network, and dimensional matching is performed between the information feature vector and each contextual signal in the input contextual signal set to obtain the contextual feature vectors to constitute the contextual feature vector set.
- FIG. 2 is a schematic diagram of the prefrontal cortex-like network structure based on contextual signals according to an embodiment of the present invention.
- the English word vectors trained by the default parameters of the gensim toolkit are used as the contextual signals
- the contextual signals of 40 attribute classification tasks are the 200-dimensional word vectors of the corresponding attribute labels.
- Step S 201 a weight matrix is constructed based on the contextual signals and the prefrontal cortex-like network, and each column of the weight matrix is normalized.
- the dimension of the contextual signals is 200, and the dimension of the contextual feature vector is 5000.
- the weight matrix with a size of 200 ⁇ 5000 is constructed.
- FIG. 3 schematically shows the three-dimensional space of the preferred embodiment of the present invention.
- k denotes the dimension of the input feature
- m denotes the dimension of the hidden layer
- i denotes the dimension index of the input feature.
- Step S 202 based on the weight matrix, dimensional matching is performed between the contextual signal and the information feature vector to obtain the contextual feature vectors, as shown in equation (3):
- ⁇ denotes element-wise multiplication of the vector;
- Step S 203 the contextual feature vector set is constructed by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the input information feature vector.
- Step S 30 each feature vector in the contextual feature vector set is classified by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and classification information.
- Y Lable denotes the classification information
- W out denotes the classification weight of the classifier
- Y out denotes the output feature of the prefrontal cortex-like network
- n denotes the dimension of the output weight of the prefrontal cortex-like network
- F denotes the information feature vector.
- FIG. 4 is a schematic diagram showing the network architectures of the traditional multi-task learning network and the sequential multi-task learning network according to the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention, wherein C represents the classifier.
- C represents the classifier.
- a switch module and n classifiers are required in multitask training, wherein n represents the number of contextual signals.
- FIG. 5 is a schematic diagram showing the accuracy of the face recognition task of the multi-task training and the sequential multi-scene training according to the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention.
- Each point represents one face attribute, and the number of points is 40 in total.
- Each attribute is associated with one contextual signal, so that sequential multi-task learning based on contextual signals can be performed to obtain the results of multi-task training.
- An information processing system based on contextual signals and a prefrontal cortex-like network includes an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module.
- the acquisition module is configured to obtain input information and an input contextual signal set and input.
- the feature extraction module is configured to extract the features of the input information by using a feature vector extractor corresponding to the input information to obtain an information feature vector.
- the dimensional matching module is configured to input the information feature vector into the prefrontal cortex-like network, and perform dimensional matching between the information feature vector and each contextual signal in the input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set.
- the classification module is configured to classify each feature vector in the contextual feature vector set by a pre-constructed feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set.
- the output module is configured to output the acquired classification information set.
- the information processing system based on the contextual signals and the prefrontal cortex-like network provided in the above embodiment is only exemplified by the division of the above functional modules.
- the above functions may be allocated to be completed by different functional modules as needed, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined.
- the modules in the above embodiments can be combined into one module, or can be further split into a plurality of sub-modules to complete all or a part of the functions of the above description.
- the designations of the modules and steps involved in the embodiments of the present invention are only intended to distinguish these modules or steps, and should not be construed as an improper limitation of the present invention.
- the third embodiment of the present invention provides a storage device, wherein a plurality of programs are stored in the storage device.
- the programs are configured to be loaded and executed by a processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
- a processing device includes a processor and a storage device.
- the processor is configured to execute a plurality of programs
- the storage device is configured to store the plurality of programs.
- the programs are configured to be loaded and executed by the processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
An information processing method based on contextual signals and a prefrontal cortex-like network includes: selecting a feature vector extractor based on obtained information to perform feature extraction to obtain an information feature vector; inputting the information feature vector into the prefrontal cortex-like network, and performing dimensional matching between the information feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set; and classifying each feature vector in the contextual feature vector set by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set. An information processing system based on contextual signals and a prefrontal cortex-like network includes an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module.
Description
This application is the national phase entry of International Application No. PCT/CN2019/083356, filed on Apr. 19, 2019, which is based upon and claims priority to Chinese Patent Application No. 201910058284.2, filed on Jan. 22, 2019, the entire contents of which are incorporated herein by reference.
The present invention belongs to the field of pattern recognition and machine learning, and more particularly, relates to an information processing method, system and device based on contextual signals and a prefrontal cortex-like network.
One of the hallmarks of advanced intelligence is flexibility. Humans can respond differentially to the same stimulus according to different goals, environments, internal states and other situations. The prefrontal cortex, which is highly elaborated in primates, is pivotal for such an ability. The prefrontal cortex can quickly learn the “rules of the game” and dynamically apply them to map the sensory inputs to context-dependent tasks based on different actions. This process, named cognitive control, allows primates to behave appropriately in an unlimited number of situations.
Current artificial neural networks are powerful in extracting advanced features from the original data to perform pattern classification and learning sophisticated mapping rules. Their responses, however, are largely dictated by network inputs, exhibiting stereotyped input-output mappings. These mappings are usually fixed once the network training is completed.
Therefore, the current artificial neural networks lack enough flexibility to work in complex situations in which the mapping rules may change according to the context and these rules need to be learned “on the go” from a small number of training samples. This constitutes a significant ability gap between artificial neural networks and human brains.
In order to solve the above-mentioned problems in the prior art, that is, to solve the problem that the system has a complex structure, poor flexibility, and requires massive training samples when performing complex multi-tasking, inspired by the function of the prefrontal cortex, the present invention provides an information processing method based on contextual signals and a prefrontal cortex-like network, including:
step S10: selecting a feature vector extractor based on obtained information to perform feature extraction to obtain an information feature vector;
step S20: inputting the information feature vector into the prefrontal cortex-like network, and performing dimensional matching between the information feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set; and
step S30: classifying each feature vector in the contextual feature vector set by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and the classification information.
In some preferred embodiments, the step of “selecting the feature vector extractor” in step S10 includes:
based on a preset feature vector extractor library, selecting the feature vector extractor corresponding to a category of the obtained information.
In some preferred embodiments, a method for constructing the feature vector extractor includes:
based on a training data set, iteratively updating a weight of a parameter in a feature vector extraction network by an Adam algorithm, wherein the feature vector extraction network is constructed based on a deep neural network; and
using a network obtained after removing the last classification layer of the trained feature vector extraction network as the feature vector extractor.
In some preferred embodiments, the step of “inputting the information feature vector into the prefrontal cortex-like network and performing the dimensional matching between the information feature vector and the each contextual signal in the input contextual signal set to obtain the contextual feature vectors” in step S20 includes:
step S201: constructing a weight matrix based on the contextual signal and the prefrontal cortex-like network, and normalizing each column of the weight matrix;
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
-
- where, Win denotes the weight matrix, ∥Wi in∥ denotes a normalized module of Wi in, i denotes a dimension index of a hidden layer, k denotes a dimension of an input feature, and m denotes a dimension of the hidden layer;
step S202: based on the weight matrix, performing the dimensional matching between the contextual signal and the information feature vector to obtain the contextual feature vectors;
Y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
Y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
where, Yout denotes the contextual feature vectors, and Yout=[y1, y2, . . . , ym]TϵRm; F denotes the information feature vector, and F=[f1, f2, . . . , fk]T ϵRk; C denotes the each contextual signal, and C=[c1, c2, . . . , cm]T ϵm; ⊙ denotes element-wise multiplication of the vector; θm denotes an angle between wm in and F, and g=max(0, x); Win denotes the weight matrix, and Win=[w1 in, w2 in, . . . , wm in]ϵRk×m; and
step S203: constructing the contextual feature vector set by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the information feature vector.
In some preferred embodiments, the feature vector classifier in step S30 is constructed based on the following equations:
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
where, YLable denotes the classification information, Wout denotes a classification weight of the classifier, Yout denotes an output feature of the prefrontal cortex-like network, n denotes a dimension of an output weight of the prefrontal cortex-like network, and F denotes the information feature vector.
In some preferred embodiments, configuration parameters of the Adam algorithm include:
a learning rate of 0.1, a weight decay rate of 0.0001, and a batch size of 256.
In some preferred embodiments, the contextual signal is a multi-dimensional word vector corresponding to a classification attribute, and a dimension of the word vector is 200.
In some preferred embodiments, the weight matrix Win is:
a matrix constructed based on a dimension of a word vector and a dimension of the prefrontal cortex-like network, wherein the matrix has a size of the dimension of the word vector x the dimension of the prefrontal cortex-like network.
According to the second aspect of the present invention, an information processing system based on contextual signals and a prefrontal cortex-like network includes an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module.
The acquisition module is configured to obtain input information and an input contextual signal set and input.
The feature extraction module is configured to extract features of the input information by using a feature vector extractor corresponding to the input information to obtain an information feature vector.
The dimensional matching module is configured to input the information feature vector into the prefrontal cortex-like network, and perform dimensional matching between the information feature vector and each contextual signal in the input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set.
The classification module is configured to classify each feature vector in the contextual feature vector set by a pre-constructed feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set.
The output module is configured to output the acquired classification information set.
According to the third aspect of the present invention, a storage device is provided, wherein a plurality of programs are stored in the storage device, and the programs are configured to be loaded and executed by a processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
According to the fourth aspect of the present invention, a processing device includes a processor and a storage device. The processor is configured to execute a plurality of programs. The storage device is configured to store the plurality of programs. The programs are configured to be loaded and executed by the processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
The present invention has the following advantages.
(1) The multi-task information processing method of the present invention based on the contextual signals uses a prefrontal cortex-like module to realize the multi-task learning oriented to the scene information. When the contextual information cannot be determined in advance, the mapping depending on the contextual information can be gradually learned step by step. The data processed by the method of the present invention can be applied to multi-task learning or sequential multi-task learning with higher requirements, whereby the network structure can be simplified, thus reducing the difficulty of multi-task learning and increasing the flexibility of the system.
(2) In the present invention, the deep neural network is used as the feature extractor, and the optimization method is designed on the linear layer. In this way, the deep neural network exerts its full role, and the design difficulty is reduced.
(3) In the method of the present invention, the contextual signals are designed and can be changed according to the change of the current working environment, which solves the disadvantage that the neural network cannot respond differently to the same stimulus according to different goals, environments, internal states and other situations.
By reading the detailed description of the non-restrictive embodiment with reference to the following drawings, other features, objectives and advantages of the present invention will become better understood.
The present invention is further described in detail hereinafter with reference to the drawings and embodiments. Understandably, the specific embodiments described herein are only used to explain the present invention rather than to limit the present invention. In addition, it should be noted that for the convenience of description, only the parts related to the present invention are shown in the drawings.
It should be noted that without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other. The present invention will be explained in detail with reference to the drawings and embodiments below.
An information processing method based on contextual signals and a prefrontal cortex-like network includes:
step S10: a feature vector extractor is selected based on obtained information to perform feature extraction to obtain an information feature vector;
step S20: the image feature vector is input into a prefrontal cortex-like network, and dimensional matching is performed between the image feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set; and
step S30: each feature vector in the contextual feature vector set is classified by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and the classification information.
In order to more clearly explain the information processing method based on the contextual signals and the prefrontal cortex-like network of the present invention, the steps in the embodiment of the method of the present invention are described in detail below with reference to FIG. 1 .
An information processing method based on contextual signals and a prefrontal cortex-like network according to the first embodiment of the present invention includes steps S10-step S30, and each step is described in detail as follows.
Step S10: a feature vector extractor is selected based on obtained information to perform feature extraction to obtain an information feature vector.
Based on a preset feature vector extractor library, the feature vector extractor corresponding to the category of the obtained information is selected.
The feature vector extractor library includes at least one selected from the group consisting of an image feature vector extractor, a speech feature vector extractor and a text feature vector extractor. Alternatively, the feature vector extractor library can also include feature vector extractors of other common information categories, which are not enumerated in detail herein. In the present invention, the feature vector extractor can be constructed based on the deep neural network. For example, a deep neural network such as a residual neural network (ResNet) can be selected for the input image information. The deep neural network such as a convolutional neural network (CNN), long short-term memory (LSTM) neural network and gated recurrent unit (GRU) neural network can be selected for the input speech information. The deep neural network such as fastText, Text-convolutional neural network (TextCNN) and Text-recurrent neural network (TextRNN) can be used for the input text information.
In the real environment, information is generally multimodal, and features thereof can be extracted by simultaneously using several feature vector processors in combination, which can not only enrich the expression of information, but also significantly reduce the dimension of the original information to facilitate processing the downstream information.
A method for constructing each feature vector extractor in the feature vector extractor library includes:
based on the training data set, a weight of a parameter in a feature vector extraction network is iteratively updated by the Adam algorithm, wherein the feature vector extraction network is constructed based on a deep neural network; and
a network obtained after removing the last classification layer of the trained feature vector extraction network is used as the feature vector extractor.
In the preferred embodiment of the present invention, the Adam algorithm has a learning rate of 0.1, a weight decay rate of 0.0001, and a batch size of 256. The training data set used in the present invention is the face data set CelebFaces Attributes (CelebA) (CelebA is the open data of the Chinese University of Hong Kong, containing 202599 face images of 10177 celebrities, each with 40 corresponding attributes. Each attribute is used as a kind of scene, and different scenes correspond to different contextual signals.)
The deep neural network ResNet50 is trained using the Adam algorithm, and the last classification layer of the ResNet50 network is removed. The output of the penultimate layer of the ResNet50 network is used as the feature of face data and has 2048 dimensions.
Step S20: the information feature vector is input into the prefrontal cortex-like network, and dimensional matching is performed between the information feature vector and each contextual signal in the input contextual signal set to obtain the contextual feature vectors to constitute the contextual feature vector set. FIG. 2 is a schematic diagram of the prefrontal cortex-like network structure based on contextual signals according to an embodiment of the present invention.
In the preferred embodiment of the present invention, the English word vectors trained by the default parameters of the gensim toolkit are used as the contextual signals, and the contextual signals of 40 attribute classification tasks are the 200-dimensional word vectors of the corresponding attribute labels.
Step S201: a weight matrix is constructed based on the contextual signals and the prefrontal cortex-like network, and each column of the weight matrix is normalized.
The dimension of the contextual signals is 200, and the dimension of the contextual feature vector is 5000. The weight matrix with a size of 200×5000 is constructed. FIG. 3 schematically shows the three-dimensional space of the preferred embodiment of the present invention.
The weight matrix Will is constructed by equation (1):
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m equation (1),
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m equation (1),
where, k denotes the dimension of the input feature, and m denotes the dimension of the hidden layer.
Each column of the weight matrix is normalized by equation (2):
∥w i in∥=1,i=1,2, . . . ,m equation (2),
∥w i in∥=1,i=1,2, . . . ,m equation (2),
where, i denotes the dimension index of the input feature.
Step S202: based on the weight matrix, dimensional matching is performed between the contextual signal and the information feature vector to obtain the contextual feature vectors, as shown in equation (3):
Y out =g((w in)T F)⊙C
=g([w 1 in ,w 2 in , . . . ,w m in]T F)⊙C
=g([c 1 ∥F∥∥w 1 in∥cos θ1 ,c 2 ∥F∥∥w 2 in∥cos θ2 , . . . ,c m ∥F∥∥w m in∥cos θm]T)
=g([c 1 ∥w 1 in∥cos θ1 ,c 2 ∥w 2 in∥cos θ2 , . . . ,c m ∥w m in∥cos θm]T)∥F∥
=g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥ equation (3),
where, Yout denotes the contextual feature vectors, and Yout=[y1, y2, . . . , ym]TϵRm; F denotes the information feature vector, and F=[f1, f2, . . . , fk]TϵRk; C denotes the contextual signal, and C=[c1, c2, . . . , cm]TϵRm; ⊙ denotes element-wise multiplication of the vector; θm denotes the angle between wm in and F, and g=max(0,x); Win denotes the weight matrix, and Win=[w1 in, w2 in, . . . , wm in]ϵRk×m.
Y out =g((w in)T F)⊙C
=g([w 1 in ,w 2 in , . . . ,w m in]T F)⊙C
=g([c 1 ∥F∥∥w 1 in∥cos θ1 ,c 2 ∥F∥∥w 2 in∥cos θ2 , . . . ,c m ∥F∥∥w m in∥cos θm]T)
=g([c 1 ∥w 1 in∥cos θ1 ,c 2 ∥w 2 in∥cos θ2 , . . . ,c m ∥w m in∥cos θm]T)∥F∥
=g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥ equation (3),
where, Yout denotes the contextual feature vectors, and Yout=[y1, y2, . . . , ym]TϵRm; F denotes the information feature vector, and F=[f1, f2, . . . , fk]TϵRk; C denotes the contextual signal, and C=[c1, c2, . . . , cm]TϵRm; ⊙ denotes element-wise multiplication of the vector; θm denotes the angle between wm in and F, and g=max(0,x); Win denotes the weight matrix, and Win=[w1 in, w2 in, . . . , wm in]ϵRk×m.
Step S203: the contextual feature vector set is constructed by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the input information feature vector.
Step S30: each feature vector in the contextual feature vector set is classified by a feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set, wherein the feature vector classifier is a mapping network of the contextual feature vectors and classification information.
The feature vector classifier is constructed based on equation (4) and equation (5):
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ equation (4),
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ equation (4),
where, YLable denotes the classification information, Wout denotes the classification weight of the classifier, Yout denotes the output feature of the prefrontal cortex-like network, n denotes the dimension of the output weight of the prefrontal cortex-like network, and F denotes the information feature vector.
An information processing system based on contextual signals and a prefrontal cortex-like network according to the second embodiment of the present invention includes an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module.
The acquisition module is configured to obtain input information and an input contextual signal set and input.
The feature extraction module is configured to extract the features of the input information by using a feature vector extractor corresponding to the input information to obtain an information feature vector.
The dimensional matching module is configured to input the information feature vector into the prefrontal cortex-like network, and perform dimensional matching between the information feature vector and each contextual signal in the input contextual signal set to obtain contextual feature vectors to constitute a contextual feature vector set.
The classification module is configured to classify each feature vector in the contextual feature vector set by a pre-constructed feature vector classifier to obtain classification information of the each feature vector to constitute a classification information set.
The output module is configured to output the acquired classification information set.
It can be clearly understood by those skilled in the art that for the convenience and brevity of the description, the specific working process and related description of the system described above can refer to the corresponding processes in the foregoing embodiments of the method of the present invention, which will not be repeatedly described here.
It should be noted that the information processing system based on the contextual signals and the prefrontal cortex-like network provided in the above embodiment is only exemplified by the division of the above functional modules. In practical applications, the above functions may be allocated to be completed by different functional modules as needed, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or can be further split into a plurality of sub-modules to complete all or a part of the functions of the above description. The designations of the modules and steps involved in the embodiments of the present invention are only intended to distinguish these modules or steps, and should not be construed as an improper limitation of the present invention.
The third embodiment of the present invention provides a storage device, wherein a plurality of programs are stored in the storage device. The programs are configured to be loaded and executed by a processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
A processing device according to the fourth embodiment of the present invention includes a processor and a storage device. The processor is configured to execute a plurality of programs, and the storage device is configured to store the plurality of programs. The programs are configured to be loaded and executed by the processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network described above.
It can be clearly understood by those skilled in the art that for the convenience and brevity of the description, the specific working process and related description of the storage device and the processing device described above can refer to the corresponding processes in the foregoing embodiments of the method of the present invention, which will not be repeatedly described here.
Those skilled in the art can realize that the exemplary modules and steps of methods described with reference to the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the electronic hardware and the computer software. The programs corresponding to modules of software and/or steps of methods can be stored in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of storage mediums well-known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, in the above description, the compositions and steps of each embodiment have been generally described functionally. Whether these functions are performed by electronic hardware or software depends on specific applications and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.
The terminology “include/comprise” or any other similar terminologies are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus/device including a series of elements not only includes those elements but also includes other elements that are not explicitly listed, or further includes elements inherent in the process, method, article or apparatus/device.
Hereto, the technical solutions of the present invention have been described in combination with the preferred implementations with reference to the drawings. However, it is easily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent modifications or replacements to related technical features, and the technical solutions obtained through these modifications or replacements shall fall within the scope of protection of the present invention.
Claims (20)
1. An information processing method based on contextual signals and a prefrontal cortex-like network, comprising:
selecting a feature vector extractor based on obtained information to perform feature extraction to obtain an information feature vector;
inputting the information feature vector into the prefrontal cortex-like network, and performing dimensional matching between the information feature vector and each contextual signal in an input contextual signal set to obtain contextual feature vectors, and constituting a contextual feature vector set according to the contextual feature vectors; and
classifying each contextual feature vector of the contextual feature vectors in the contextual feature vector set by a feature vector classifier to obtain classification information of the each contextual feature vector, and constituting a classification information set according to the classification information of the each contextual feature vector, wherein the feature vector classifier is a mapping network of the contextual feature vectors and the classification information.
2. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 1 , wherein, the step of selecting the feature vector extractor comprises:
based on a preset feature vector extractor library, selecting the feature vector extractor corresponding to a category of the obtained information.
3. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 2 , wherein, a method for constructing the feature vector extractor comprises:
based on a training data set, iteratively updating a weight of a parameter in a feature vector extraction network by an Adam algorithm to obtain a trained feature vector extraction network, wherein the feature vector extraction network is constructed based on a deep neural network; and
using a network obtained after removing a last classification layer of the trained feature vector extraction network as the feature vector extractor.
4. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 1 , wherein, the step of inputting the information feature vector into the prefrontal cortex-like network and performing the dimensional matching between the information feature vector and the each contextual signal in the input contextual signal set to obtain the contextual feature vectors step S20 comprises:
constructing a weight matrix based on the contextual signals and the prefrontal cortex-like network, and normalizing each column of the weight matrix;
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
where, Win denotes the weight matrix, ∥Wi in∥ denotes a normalized module of Wi in, i denotes a dimension index of a hidden layer, k denotes a dimension of an input feature, and m denotes a dimension of the hidden layer;
based on the weight matrix, performing the dimensional matching between the each contextual signal and the information feature vector to obtain the contextual feature vectors;
y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
where, Yout denotes the contextual feature vectors, and Yout=[y1, y2, . . . , ym]TϵRm; F denotes the information feature vector, and F=[f1, f2, . . . , fk]T ϵRk; C denotes the each contextual signal, and C=[c1, c2, . . . , cm]T ϵm; θm denotes an angle between wm in and F, and g=max(0,x); Win denotes the weight matrix, and Win=[w1 in, w2 in, . . . , wm in]ϵRk×m; and
constructing the contextual feature vector set by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the information feature vector.
5. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 1 , wherein, the feature vector classifier in is constructed based on the following equations:
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
ϕ=arccos(Σi=1 m W i out c i g(cos θi)/∥W out∥),
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
ϕ=arccos(Σi=1 m W i out c i g(cos θi)/∥W out∥),
where, YLable denotes the classification information, Wout denotes a classification weight of the classifier, Yout denotes an output feature of the prefrontal cortex-like network and F denotes the information feature vector.
6. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 3 , wherein, configuration parameters of the Adam algorithm comprise:
a learning rate of 0.1, a weight decay rate of 0.0001, and a batch size of 256.
7. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 3 , wherein, the each contextual signal is a multi-dimensional word vector corresponding to a classification attribute, and a dimension of the multi-dimensional word vector is 200.
8. The information processing method based on the contextual signals and the prefrontal cortex-like network of claim 4 , wherein, the weight matrix Win is:
a matrix constructed based on a dimension of a word vector and a dimension of the prefrontal cortex-like network, wherein the matrix has a size of multiplying the dimension of the word vector by the dimension of the prefrontal cortex-like network.
9. An information processing system based on contextual signals and a prefrontal cortex-like network, comprising an acquisition module, a feature extraction module, a dimensional matching module, a classification module and an output module; wherein,
the acquisition module is configured to obtain input information and an input contextual signal set and input;
the feature extraction module is configured to extract features of the input information by using a feature vector extractor corresponding to the input information to obtain an information feature vector;
the dimensional matching module is configured to input the information feature vector into the prefrontal cortex-like network, perform dimensional matching between the information feature vector and each contextual signal in the input contextual signal set to obtain a-contextual feature vectors, and constitute a contextual feature vector set according to the contextual feature vectors;
the classification module is configured to classify each contextual feature vector of the contextual feature vectors in the contextual feature vector set by a pre-constructed feature vector classifier to obtain classification information of the each contextual feature vector, and to constitute a classification information set according to the classification information of the each contextual feature vector; and
the output module is configured to output the classification information set.
10. A storage device, wherein a plurality of programs are stored in the storage device, and the plurality of programs are configured to be loaded and executed by a processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network of claim 1 .
11. A processing device, comprising:
a processor, and
a storage device, wherein
the processor is configured to execute a plurality of programs, and the storage device is configured to store the plurality of programs;
the plurality of programs are configured to be loaded and executed by the processor to implement the information processing method based on the contextual signals and the prefrontal cortex-like network of claim 1 .
12. The storage device of claim 10 , wherein, the step of selecting the feature vector extractor comprises:
based on a preset feature vector extractor library, selecting the feature vector extractor corresponding to a category of the obtained information.
13. The storage device of claim 12 , wherein, a method for constructing the feature vector extractor comprises:
based on a training data set, iteratively updating a weight of a parameter in a feature vector extraction network by an Adam algorithm to obtain a trained feature vector extraction network, wherein the feature vector extraction network is constructed based on a deep neural network; and
using a network obtained after removing a last classification layer of the trained feature vector extraction network as the feature vector extractor.
14. The storage device of claim 10 , wherein, the step of inputting the information feature vector into the prefrontal cortex-like network and performing the dimensional matching between the information feature vector and the each contextual signal in the input contextual signal set to obtain the contextual feature vectors comprises:
constructing a weight matrix based on the contextual signals and the prefrontal cortex-like network, and normalizing each column of the weight matrix;
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
W in=[w 1 in ,w 2 in , . . . ,w m in]ϵR k×m
∥w i in∥=1,i=1,2, . . . ,m,
where, Win denotes the weight matrix, ∥Wi in∥ denotes a normalized module of Wi in, i denotes a dimension index of a hidden layer, k denotes a dimension of an input feature, and m denotes a dimension of the hidden layer;
based on the weight matrix, performing the dimensional matching between the each contextual signal and the information feature vector to obtain the contextual feature vectors;
Y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
Y out =g([c 1 cos θ1 ,c 2 cos θ2 , . . . ,c m cos θm]T)∥F∥,
where, Yout denotes the contextual feature vectors, and Yout=[y1, y2, . . . , ym]TϵRm; F denotes the information feature vector, and F=[f1, f2, . . . , fk]T ϵRk; C denotes the each contextual signal, and C=[c1, c2, . . . , cm]T ϵm; θm denotes an angle between wm in and F, and g=max(0,x); Win denotes the weight matrix, and win=[w1 in, w2 in, . . . , Wm in]ϵRk+m; and
constructing the contextual feature vector set by the contextual feature vectors obtained after performing the dimensional matching between each contextual signal in the contextual signal set and the information feature vector.
15. The storage device of claim 10 , wherein, the feature vector classifier is constructed based on the following equations:
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
ϕ=arccos(Σi=1 m w i out c i g(cos θi)/∥W out∥),
Y Lable=(W out)T Y out =∥W out ∥∥F∥cos ϕ,
ϕ=arccos(Σi=1 m w i out c i g(cos θi)/∥W out∥),
where, YLable denotes the classification information, Wout denotes a classification weight of the classifier, Yout denotes an output feature of the prefrontal cortex-like network, and F denotes the information feature vector.
16. The storage device of claim 13 , wherein, configuration parameters of the Adam algorithm comprise:
a learning rate of 0.1, a weight decay rate of 0.0001, and a batch size of 256.
17. The storage device of claim 13 , wherein, the each contextual signal is a multi-dimensional word vector corresponding to a classification attribute, and a dimension of the multi-dimensional word vector is 200.
18. The storage device of claim 14 , wherein, the weight matrix Win is:
a matrix constructed based on a dimension of a word vector and a dimension of the prefrontal cortex-like network, wherein the matrix has a size of multiplying the dimension of the word vector by the dimension of the prefrontal cortex-like network.
19. The processing device of claim 11 , wherein, the step of selecting the feature vector extractor comprises:
based on a preset feature vector extractor library, selecting the feature vector extractor corresponding to a category of the obtained information.
20. The processing device of claim 19 , wherein, a method for constructing the feature vector extractor comprises:
based on a training data set, iteratively updating a weight of a parameter in a feature vector extraction network by an Adam algorithm to obtain a trained feature vector extraction network, wherein the feature vector extraction network is constructed based on a deep neural network; and
using a network obtained after removing a last classification layer of the trained feature vector extraction network as the feature vector extractor.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910058284.2 | 2019-01-22 | ||
CN201910058284.2A CN109784287A (en) | 2019-01-22 | 2019-01-22 | Information processing method, system, device based on scene class signal forehead leaf network |
CN201910058284 | 2019-01-22 | ||
PCT/CN2019/083356 WO2020151108A1 (en) | 2019-01-22 | 2019-04-19 | Information processing method, system and apparatus based on context signal prefrontal-lobe-like network |
Publications (2)
Publication Number | Publication Date |
---|---|
US10915815B1 true US10915815B1 (en) | 2021-02-09 |
US20210056415A1 US20210056415A1 (en) | 2021-02-25 |
Family
ID=66501994
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/971,691 Active US10915815B1 (en) | 2019-01-22 | 2019-04-19 | Information processing method, system and device based on contextual signals and prefrontal cortex-like network |
Country Status (3)
Country | Link |
---|---|
US (1) | US10915815B1 (en) |
CN (1) | CN109784287A (en) |
WO (1) | WO2020151108A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220036125A1 (en) * | 2020-07-31 | 2022-02-03 | Xiamen Sigmastar Technology Ltd. | Target data feature extraction method and device |
CN116244517A (en) * | 2023-03-03 | 2023-06-09 | 北京航空航天大学 | Multi-scene multi-task recommendation method based on hierarchical information extraction network |
Citations (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050036676A1 (en) | 2003-06-30 | 2005-02-17 | Bernd Heisele | Systems and methods for training component-based object identification systems |
CN101021849A (en) | 2006-09-14 | 2007-08-22 | 浙江大学 | Transmedia searching method based on content correlation |
US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
US20090157481A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for specifying a cohort-linked avatar attribute |
US20090157660A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems employing a cohort-linked avatar |
US20100249573A1 (en) * | 2009-03-30 | 2010-09-30 | Marks Donald H | Brain function decoding process and system |
CN101853426A (en) | 2010-05-21 | 2010-10-06 | 南京邮电大学 | Context fusing method based on neural network in pervasive computing environment |
US20120093381A1 (en) * | 2010-06-25 | 2012-04-19 | Institute Of Automation, Chinese Academy Of Sciences | Method and system for brain tumor segmentation in multi-parameter 3d mr images via robust statistic information propagation |
US8442321B1 (en) | 2011-09-14 | 2013-05-14 | Google Inc. | Object recognition in images |
CN105678150A (en) | 2016-01-11 | 2016-06-15 | 成都布林特信息技术有限公司 | User authority managing method |
CN106650796A (en) | 2016-12-06 | 2017-05-10 | 国家纳米科学中心 | Artificial intelligence based cell fluorescence image classification method and system |
CN106934068A (en) | 2017-04-10 | 2017-07-07 | 江苏东方金钰智能机器人有限公司 | The method that robot is based on the semantic understanding of environmental context |
CN107690034A (en) | 2017-10-27 | 2018-02-13 | 中国科学技术大学苏州研究院 | Intelligent scene mode switching system and method based on environmental background sound |
CN107766324A (en) * | 2017-09-25 | 2018-03-06 | 浙江大学 | A kind of text coherence analysis method based on deep neural network |
US9946344B2 (en) * | 2011-03-25 | 2018-04-17 | Drexel University | Functional near infrared spectroscopy based brain computer interface |
US20180184964A1 (en) * | 2014-06-30 | 2018-07-05 | Cerora, Inc. | System and signatures for a multi-modal physiological periodic biomarker assessment |
WO2018175698A1 (en) * | 2017-03-22 | 2018-09-27 | Larsx | Continuously learning and optimizing artificial intelligence (ai) adaptive neural network (ann) computer modeling methods and systems |
US20180314879A1 (en) * | 2017-05-01 | 2018-11-01 | Samsung Electronics Company, Ltd. | Determining Emotions Using Camera-Based Sensing |
CN108937968A (en) | 2018-06-04 | 2018-12-07 | 安徽大学 | lead selection method of emotion electroencephalogram signal based on independent component analysis |
CN109171769A (en) | 2018-07-12 | 2019-01-11 | 西北师范大学 | It is a kind of applied to depression detection voice, facial feature extraction method and system |
US20190082990A1 (en) * | 2017-09-19 | 2019-03-21 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement |
US20190107888A1 (en) * | 2017-10-06 | 2019-04-11 | Holland Bloorview Kids Rehabilitation Hospital | Brain-computer interface platform and process for classification of covert speech |
CN109726389A (en) * | 2018-11-13 | 2019-05-07 | 北京邮电大学 | A kind of Chinese missing pronoun complementing method based on common sense and reasoning |
US20190159712A1 (en) * | 2009-03-30 | 2019-05-30 | Donald H. Marks | Brain function decoding process and system |
US20200060566A1 (en) * | 2018-08-24 | 2020-02-27 | Newton Howard | Automated detection of brain disorders |
US20200129110A1 (en) * | 2018-10-24 | 2020-04-30 | Translational Research Institute Pty Ltd As Trustee For Translational Research Institute Trust | Functional analysis of human brain using functional magnetic resonance imaging (fmri) for acute stress and post traumatic stress disorder (ptsd) monitoring neuroplasticity |
US20200387603A1 (en) * | 2019-06-04 | 2020-12-10 | International Business Machines Corporation | Device protection based on prediction and contextual analysis |
-
2019
- 2019-01-22 CN CN201910058284.2A patent/CN109784287A/en active Pending
- 2019-04-19 US US16/971,691 patent/US10915815B1/en active Active
- 2019-04-19 WO PCT/CN2019/083356 patent/WO2020151108A1/en active Application Filing
Patent Citations (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050036676A1 (en) | 2003-06-30 | 2005-02-17 | Bernd Heisele | Systems and methods for training component-based object identification systems |
CN101021849A (en) | 2006-09-14 | 2007-08-22 | 浙江大学 | Transmedia searching method based on content correlation |
US20090157625A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for identifying an avatar-linked population cohort |
US20090157481A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems for specifying a cohort-linked avatar attribute |
US20090157660A1 (en) * | 2007-12-13 | 2009-06-18 | Searete Llc, A Limited Liability Corporation Of The State Of Delaware | Methods and systems employing a cohort-linked avatar |
US20100249573A1 (en) * | 2009-03-30 | 2010-09-30 | Marks Donald H | Brain function decoding process and system |
US20190159712A1 (en) * | 2009-03-30 | 2019-05-30 | Donald H. Marks | Brain function decoding process and system |
CN101853426A (en) | 2010-05-21 | 2010-10-06 | 南京邮电大学 | Context fusing method based on neural network in pervasive computing environment |
US20120093381A1 (en) * | 2010-06-25 | 2012-04-19 | Institute Of Automation, Chinese Academy Of Sciences | Method and system for brain tumor segmentation in multi-parameter 3d mr images via robust statistic information propagation |
US9946344B2 (en) * | 2011-03-25 | 2018-04-17 | Drexel University | Functional near infrared spectroscopy based brain computer interface |
US8442321B1 (en) | 2011-09-14 | 2013-05-14 | Google Inc. | Object recognition in images |
US20180184964A1 (en) * | 2014-06-30 | 2018-07-05 | Cerora, Inc. | System and signatures for a multi-modal physiological periodic biomarker assessment |
CN105678150A (en) | 2016-01-11 | 2016-06-15 | 成都布林特信息技术有限公司 | User authority managing method |
CN106650796A (en) | 2016-12-06 | 2017-05-10 | 国家纳米科学中心 | Artificial intelligence based cell fluorescence image classification method and system |
WO2018175698A1 (en) * | 2017-03-22 | 2018-09-27 | Larsx | Continuously learning and optimizing artificial intelligence (ai) adaptive neural network (ann) computer modeling methods and systems |
CN106934068A (en) | 2017-04-10 | 2017-07-07 | 江苏东方金钰智能机器人有限公司 | The method that robot is based on the semantic understanding of environmental context |
US20180314879A1 (en) * | 2017-05-01 | 2018-11-01 | Samsung Electronics Company, Ltd. | Determining Emotions Using Camera-Based Sensing |
US20190082990A1 (en) * | 2017-09-19 | 2019-03-21 | Neuroenhancement Lab, LLC | Method and apparatus for neuroenhancement |
CN107766324A (en) * | 2017-09-25 | 2018-03-06 | 浙江大学 | A kind of text coherence analysis method based on deep neural network |
US20190107888A1 (en) * | 2017-10-06 | 2019-04-11 | Holland Bloorview Kids Rehabilitation Hospital | Brain-computer interface platform and process for classification of covert speech |
CN107690034A (en) | 2017-10-27 | 2018-02-13 | 中国科学技术大学苏州研究院 | Intelligent scene mode switching system and method based on environmental background sound |
CN108937968A (en) | 2018-06-04 | 2018-12-07 | 安徽大学 | lead selection method of emotion electroencephalogram signal based on independent component analysis |
CN109171769A (en) | 2018-07-12 | 2019-01-11 | 西北师范大学 | It is a kind of applied to depression detection voice, facial feature extraction method and system |
US20200060566A1 (en) * | 2018-08-24 | 2020-02-27 | Newton Howard | Automated detection of brain disorders |
US20200129110A1 (en) * | 2018-10-24 | 2020-04-30 | Translational Research Institute Pty Ltd As Trustee For Translational Research Institute Trust | Functional analysis of human brain using functional magnetic resonance imaging (fmri) for acute stress and post traumatic stress disorder (ptsd) monitoring neuroplasticity |
CN109726389A (en) * | 2018-11-13 | 2019-05-07 | 北京邮电大学 | A kind of Chinese missing pronoun complementing method based on common sense and reasoning |
US20200387603A1 (en) * | 2019-06-04 | 2020-12-10 | International Business Machines Corporation | Device protection based on prediction and contextual analysis |
Non-Patent Citations (2)
Title |
---|
Guaxiong Zang et al., Continuous Learning of Context-dependent Processing in Neural Networks, Oct. 5, 2018, pp. 1-16. |
Wei-Dong Xu et al., Potato surface defects detection based on convolution neural networks and support vector machine algorithm, Jiangsu J. of Agr. Sci., 2018, pp. 1378-1385, 34(6). |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220036125A1 (en) * | 2020-07-31 | 2022-02-03 | Xiamen Sigmastar Technology Ltd. | Target data feature extraction method and device |
CN116244517A (en) * | 2023-03-03 | 2023-06-09 | 北京航空航天大学 | Multi-scene multi-task recommendation method based on hierarchical information extraction network |
CN116244517B (en) * | 2023-03-03 | 2023-11-28 | 北京航空航天大学 | Multi-scene multi-task model training method based on hierarchical information extraction network |
Also Published As
Publication number | Publication date |
---|---|
WO2020151108A1 (en) | 2020-07-30 |
CN109784287A (en) | 2019-05-21 |
US20210056415A1 (en) | 2021-02-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102591961B1 (en) | Model training method and device, and terminal and storage medium for the same | |
Spurr et al. | Guiding infogan with semi-supervision | |
WO2021129181A1 (en) | Portrait segmentation method, model training method and electronic device | |
Pietrow et al. | Objects detection and recognition system using artificial neural networks and drones | |
WO2019232772A1 (en) | Systems and methods for content identification | |
Ben Salah et al. | A novel approach for human skin detection using convolutional neural network | |
KR101914208B1 (en) | Apparatus and method for recognizing object using machine learning based on pyramid structure | |
US10915815B1 (en) | Information processing method, system and device based on contextual signals and prefrontal cortex-like network | |
Kumar et al. | A novel approach for ISL alphabet recognition using Extreme Learning Machine | |
Neto et al. | Sign language recognition based on 3d convolutional neural networks | |
Jeong et al. | Adaptive object recognition model using incremental feature representation and hierarchical classification | |
CN112989843A (en) | Intention recognition method and device, computing equipment and storage medium | |
Zhu et al. | Butterfly classification with machine learning methodologies for an android application | |
CN111639537A (en) | Face action unit identification method and device, electronic equipment and storage medium | |
Amaral et al. | Evaluating deep models for dynamic brazilian sign language recognition | |
Sönmez et al. | Convolutional neural networks with balanced batches for facial expressions recognition | |
Hegde et al. | Facial expression recognition using entire gabor filter matching score level fusion approach based on subspace methods | |
Brooks et al. | Second-order networks in PyTorch | |
CN111767710B (en) | Indonesia emotion classification method, device, equipment and medium | |
CN116266394A (en) | Multi-modal emotion recognition method, device and storage medium | |
Suzuki et al. | Support vector machine histogram: New analysis and architecture design method of deep convolutional neural network | |
Soji et al. | Indian Sign Language Recognition Using Surf Feature Extraction and MDAE for Patient Disability Discussion | |
Hoang et al. | Active vision: on the relevance of a bio-inspired approach for object detection | |
Rosu et al. | Asteroid image classification using convolutional neural networks | |
Suzuki et al. | An architecture design method of deep convolutional neural network |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES, CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZENG, GUANXIONG;CHEN, YANG;YU, SHAN;REEL/FRAME:054236/0254 Effective date: 20200701 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |