CN114037024A - Multitask neural network based data identification system and method - Google Patents
Multitask neural network based data identification system and method Download PDFInfo
- Publication number
- CN114037024A CN114037024A CN202210012305.9A CN202210012305A CN114037024A CN 114037024 A CN114037024 A CN 114037024A CN 202210012305 A CN202210012305 A CN 202210012305A CN 114037024 A CN114037024 A CN 114037024A
- Authority
- CN
- China
- Prior art keywords
- neural network
- master
- main branch
- task
- new
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a data identification system and method based on a multitask neural network, wherein the system comprises the following components: the data recognition model is constructed on the basis of a neural network and comprises a first main branch neural network and a plurality of second main branch neural networks which are added according to tasks and take the first main branch neural network as a main line; the task flow manager is used for adding a plurality of second main branch neural networks with unique identification characteristics by taking the first main branch neural network as a main line according to the task progress; the identifying feature is also used for task flow communication between the second main neural network and the first main neural network. The first main branch neural network and the second main branch neural network have corresponding information circulation, the first main branch neural network mainly extracts general characteristics for each task, and the second main branch neural network mainly provides a different attention mechanism for different tasks, so that corresponding task functions can be better completed.
Description
Technical Field
The invention relates to the technical field of data identification and data identification model construction, in particular to a multitask neural network data identification system and method based on a new fast learning task.
Background
The traditional data identification model carries out model training through sample fields, searches content characteristics of target fields and generates corresponding rule models, and the searching of the target fields has huge task amount and can reduce the speed of data model optimization.
An Artificial Neural Network (ANN), referred to as Neural Network (NN), is a mathematical model or computational model that mimics the structure and function of a biological neural network. The data recognition model can be constructed according to an artificial neural network to reduce the problem of difficult optimization in the traditional data recognition model, and the neural network is an operation model and is formed by a large number of nodes (or called neurons) and mutual connection. Each node represents a particular output function called the stimulus function, the activation function. The connection between every two nodes represents a weighted value, called weight, for the signal passing through the connection, which is equivalent to the memory of the artificial neural network. The output of the network is different according to the connection mode of the network, the weight value and the excitation function. The network itself is usually an approximation to some algorithm or function in nature, and may also be an expression of a logic strategy.
Adding a new task to a trained neural network can cause the problem of 'catastrophic forgetting', that is, the neural network forgets or loses some previously learned abilities when learning the new task, so that the performance of the original task is greatly reduced.
The current common solution is multi-task learning, which puts training data of all tasks together at the same time and uses a multi-task neural network for joint training. However, in this way, as the number of tasks increases, the number of samples of a new task is diluted, the efficiency of learning the new task by the neural network is not high, and the data of the previous task needs to be reviewed, so that the data demand is huge.
The performance of an original task is reduced when a new task is learned by a neural network at present, a new task sample is diluted in multi-task learning, the efficiency of learning the new task by the neural network is not high, data of a previous task is required to be reviewed, and the data demand is huge.
Disclosure of Invention
In view of the above, the present invention provides a system and method for identifying data based on a multitasking neural network.
The technical scheme adopted by the invention is as follows:
a multitasking neural network based data recognition system comprising:
the data recognition model is constructed on the basis of a neural network and comprises a first main branch neural network and a plurality of second main branch neural networks which are added according to tasks and take the first main branch neural network as a main line;
the task flow manager is used for adding a plurality of second main branch neural networks with unique identification characteristics by taking the first main branch neural network as a main line according to the task progress;
the identifying feature is also used for task flow communication between the second main neural network and the first main neural network.
Preferably, the first master neural network is used to extract identifying features for each task.
Preferably, the second master neural network provides corresponding attention mechanisms for different tasks to complete corresponding task functions.
Preferably, the task flow manager constructs the second master principal neural network using common ones of the identified features of the first master principal neural network selectively to new tasks to reduce the weights that the second master principal neural network needs to train.
Preferably, the task flow manager constructs the second principal neural network based on the extracted identifying features of the trained first principal neural network.
Preferably, when the first master neural network is trained, the training weight of the first master neural network may use random initialization or preset training weight, and the training weight of the first master neural network is smaller than the training weight of the second master neural network.
Preferably, when a new task flow enters, the task flow manager adds one or more new second main branch neural networks with unique identification features, the new second main branch neural networks are used for learning of the new task, and the task flow manager selectively freezes the first main branch neural network and other second main branch neural networks for the new task.
The invention also provides a data identification method based on the multitask neural network, which comprises the following steps: obtaining a current data recognition model, wherein the data recognition model is constructed based on a neural network and is provided with a first main branch neural network and a plurality of second main branch neural networks which are added by taking the first main branch neural network as a main line according to tasks,
when the first master neural network is trained, the training weight of the first master neural network can use random initialization or preset training weight, and the training weight of the first master neural network is smaller than that of the second master neural network;
when a new task flow enters, the task flow manager adds one or more new second main branch neural networks with unique identification characteristics, the new second main branch neural networks are used for learning of the new task, and at the moment, the task flow manager selectively freezes the first main branch neural network and other second main branch neural networks for the new task.
Preferably, the task flow manager constructs the second master principal neural network using common ones of the identified features of the first master principal neural network selectively to new tasks to reduce the weights that the second master principal neural network needs to train.
Preferably, the task flow manager constructs the second principal neural network based on the extracted identifying features of the trained first principal neural network.
The method and the device solve the problem that the performance of the original task is reduced when the neural network learns the new task by keeping the integrity of the weight used by the original task in the neural network structure, and achieve the purpose that the neural network keeps the performance of the original task when the new task is learned.
According to the method, the new weak main branch neural network is constructed by using the general characteristics with strong universality in the trained first main branch neural network, the weight of the new second main branch neural network to be trained is reduced, the network parameter is lightened, and the learning efficiency of a new task is improved.
According to the method, the data recognition model is constructed by the method of saving the structural integrity of the neural network of the original task and only training the newly-added second main neural network weight, so that huge data in the original task can be omitted when the newly-added task is trained, the problems that a new task sample is diluted and the data demand is huge in multi-task learning are solved, and the new task learning can be completed only by using a small amount of data of the new task.
Drawings
The invention is illustrated and described only by way of example and not by way of limitation in the scope of the invention as set forth in the following drawings, in which:
FIG. 1 is a schematic block diagram of the system of the present invention;
FIG. 2 is a flow chart of the present invention;
FIG. 3 is a block diagram of the present invention during initial training of a data stream;
FIG. 4 is a block diagram of the present invention in which training of new data streams is added.
Detailed Description
In order to make the objects, technical solutions, design methods, and advantages of the present invention more apparent, the present invention will be further described in detail by specific embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The data identification structure of the invention mainly constructs corresponding neural networks according to tasks, the constructed neural networks comprise a first main neural network and a second main neural network added according to the number of the tasks, the first main neural network and the second main neural network have corresponding information circulation, the first main neural network mainly extracts general characteristics for each task, and the second main neural network mainly provides a different attention mechanism for different tasks, thereby better completing corresponding task functions.
When a new task is added, the general features of the shallow layer, the middle layer and the deep layer on the first main branch neural network can be selectively used according to the features of the new task to construct a new second main branch neural network, or the general features similar to the shallow layer, the middle layer and the deep layer can be used for reducing the weight of the new second main branch neural network needing training and adapting to the new task more quickly.
The following technical scheme is specifically included:
referring to fig. 1 and 2, the present invention provides a multitask neural network based data recognition system, including:
the data recognition model is constructed on the basis of a neural network and comprises a first main branch neural network and a plurality of second main branch neural networks which are added according to tasks and take the first main branch neural network as a main line;
the task flow manager is used for adding a plurality of second main branch neural networks with unique identification characteristics by taking the first main branch neural network as a main line according to the task progress;
the identifying feature is also used for task flow communication between the second main neural network and the first main neural network. The first main branch neural network is used for extracting identification features for each task; the second main branch neural network provides corresponding attention mechanisms for different tasks so as to complete corresponding task functions.
In the above, the task flow manager constructs the second master principal neural network using common ones of the identified features of the first master principal neural network selectively to new tasks to reduce the weights that the second master principal neural network needs to train. Or the task flow manager constructs the second main branch neural network according to the general features in the extracted identification features of the trained first main branch neural network.
When the first master neural network is trained, the training weight of the first master neural network can use random initialization or preset training weight, and the training weight of the first master neural network is smaller than the training weight of the second master neural network.
When a new task flow enters, the task flow manager adds one or more new second main branch neural networks with unique identification characteristics, the new second main branch neural networks are used for learning of the new task, and at the moment, the task flow manager selectively freezes the first main branch neural network and other second main branch neural networks for the new task.
The invention also provides a data identification method based on the multitask neural network, which comprises the following steps: obtaining a current data recognition model, wherein the data recognition model is constructed based on a neural network and is provided with a first main branch neural network and a plurality of second main branch neural networks which are added by taking the first main branch neural network as a main line according to tasks,
when the first master neural network is trained, the training weight of the first master neural network can use random initialization or preset training weight, and the training weight of the first master neural network is smaller than that of the second master neural network;
referring to fig. 3, when a new task flow enters, the task flow manager adds one or more new second principal neural networks with uniquely identifying features, which are used for new task learning, at which time the task flow manager selectively freezes the first and other second principal neural networks for the new task. The task flow manager selectively uses common ones of the identified features of the first master principal neural network to construct the second master principal neural network for new tasks to reduce the weights that the second master principal neural network needs to train. Or the task flow manager constructs the second main branch neural network according to the general features in the extracted identification features of the trained first main branch neural network.
During initial training, the weight of the first master neural network can be initialized randomly or according to a preset training model weight, wherein the first master neural network uses a small learning rate, the second master neural network uses a learning rate which is larger than that of the strong master neural network according to task characteristics, and training is finished after a specified number of training iterations. The first main branch neural network obtains the general characteristic of strong universality of satisfying multiple tasks. When a new task is added, a new second main branch neural network can be constructed by using the general features of the shallow layer, the middle layer and the deep layer on the first main branch neural network according to the selective use of the features of the new task, or the general features similar to the shallow layer, the middle layer and the deep layer are used for reducing the weight of the new second main branch neural network needing to be trained, so that the new task can be adapted more quickly.
According to the method, the new weak main branch neural network is constructed by using the general characteristics with strong universality in the trained first main branch neural network, the weight of the new second main branch neural network to be trained is reduced, the network parameter is lightened, and the learning efficiency of a new task is improved.
In the above, the learning rate of the first main branch neural network is lower than 10%, and 90% of the resources used for the second main branch neural network are used according to the task characteristics, and the training is finished after the training iteration is performed for a specified number of times. The strong main neural network obtains the general characteristic of strong universality of satisfying multiple tasks.
Referring to fig. 4, during the second training, a second main branch neural network is needed to be added for learning of a new task, the weights of the first main branch neural network and other second main branch neural networks can be frozen, and only the newly added weak main branch neural network weight is trained. At the moment, the training of the new weak main neural network can be completed only by a small amount of data and training iteration times, so that the overall multi-task neural network can quickly adapt to the new task without influencing the performance of the original task, and the performance required by the new task is achieved.
Therefore, the data recognition model is constructed by the method of saving the structural integrity of the neural network of the original task and only training the newly added second main branch neural network weight, huge data in the original task can not be used when the newly added task is trained, the problems that a new task sample is diluted and the data demand is huge in multi-task learning are solved, and the new task learning can be completed only by using a small amount of data of the new task.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A multitask-based neural network data recognition system, comprising:
the data recognition model is constructed on the basis of a neural network and comprises a first main branch neural network and a plurality of second main branch neural networks which are added according to tasks and take the first main branch neural network as a main line;
the task flow manager is used for adding a plurality of second main branch neural networks with unique identification characteristics by taking the first main branch neural network as a main line according to the task progress;
the identifying feature is also used for task flow communication between the second main neural network and the first main neural network.
2. The multitask neural network-based data recognition system of claim 1, wherein the first master neural network is configured to extract recognition features for each task.
3. The multitask neural network-based data recognition system of claim 1, wherein the secondary master neural network provides a corresponding attention mechanism for different tasks for completion of corresponding task functions.
4. The multitask neural network-based data recognition system of claim 1, wherein the task flow manager constructs the second master neural network using common ones of the recognition features of the first master neural network selectively for new tasks to reduce the weights that the second master neural network needs to train.
5. The multitask neural network-based data recognition system of claim 4, wherein the task flow manager builds the second master principal neural network from the extracted recognition features of the already trained first master principal neural network.
6. The multitask neural network data recognition system according to claim 4 or 5, wherein the training weights of the first master neural network can use random initialization or preset training weights when the first master neural network is trained, and the training weights of the first master neural network are smaller than the training weights of the second master neural network.
7. The multitask neural network-based data recognition system of claims 4 or 5, wherein when a new task flow enters, the task flow manager adds one or more new second primary neural networks with unique identifying characteristics, the new second primary neural networks being used for new task learning, when the task flow manager selectively freezes the first primary neural network and the other second primary neural networks for the new task.
8. A data identification method based on a multitask neural network is characterized by comprising the following steps:
obtaining a current data recognition model, wherein the data recognition model is constructed based on a neural network and is provided with a first main branch neural network and a plurality of second main branch neural networks which are added by taking the first main branch neural network as a main line according to tasks,
when the first master neural network is trained, the training weight of the first master neural network can use random initialization or preset training weight, and the training weight of the first master neural network is smaller than that of the second master neural network;
when a new task flow enters, the task flow manager adds one or more new second main branch neural networks with unique identification characteristics, the new second main branch neural networks are used for learning of the new task, and at the moment, the task flow manager selectively freezes the first main branch neural network and other second main branch neural networks for the new task.
9. The multitask neural network-based data recognition method of claim 8, wherein the task flow manager selectively uses common features of the recognition features of the first master neural network to construct the second master neural network for new tasks to reduce the weights that the second master neural network needs to train.
10. The multitask neural network-based data recognition method of claim 8, wherein the task flow manager constructs the second master principal neural network from the extracted recognition features of the already trained first master principal neural network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210012305.9A CN114037024A (en) | 2022-01-07 | 2022-01-07 | Multitask neural network based data identification system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210012305.9A CN114037024A (en) | 2022-01-07 | 2022-01-07 | Multitask neural network based data identification system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114037024A true CN114037024A (en) | 2022-02-11 |
Family
ID=80141366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210012305.9A Pending CN114037024A (en) | 2022-01-07 | 2022-01-07 | Multitask neural network based data identification system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114037024A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565064A (en) * | 2022-04-26 | 2022-05-31 | 心鉴智控(深圳)科技有限公司 | Method, system and equipment for identifying multitask learning deep network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845549A (en) * | 2017-01-22 | 2017-06-13 | 珠海习悦信息技术有限公司 | A kind of method and device of the scene based on multi-task learning and target identification |
| CN109726812A (en) * | 2017-10-31 | 2019-05-07 | 通用电气公司 | Feature ordering neural network and method generate the method for simplifying feature set model |
| CN109961105A (en) * | 2019-04-08 | 2019-07-02 | 上海市测绘院 | A kind of Classification of High Resolution Satellite Images method based on multitask deep learning |
| CN111046973A (en) * | 2019-12-26 | 2020-04-21 | 北京市商汤科技开发有限公司 | Multitask detection method and device and storage medium |
| CN111328400A (en) * | 2017-11-14 | 2020-06-23 | 奇跃公司 | Meta-learning for multi-task learning of neural networks |
| CN113554156A (en) * | 2021-09-22 | 2021-10-26 | 中国海洋大学 | Multi-task learning model construction method based on attention mechanism and deformable convolution |
-
2022
- 2022-01-07 CN CN202210012305.9A patent/CN114037024A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845549A (en) * | 2017-01-22 | 2017-06-13 | 珠海习悦信息技术有限公司 | A kind of method and device of the scene based on multi-task learning and target identification |
| CN109726812A (en) * | 2017-10-31 | 2019-05-07 | 通用电气公司 | Feature ordering neural network and method generate the method for simplifying feature set model |
| CN111328400A (en) * | 2017-11-14 | 2020-06-23 | 奇跃公司 | Meta-learning for multi-task learning of neural networks |
| CN109961105A (en) * | 2019-04-08 | 2019-07-02 | 上海市测绘院 | A kind of Classification of High Resolution Satellite Images method based on multitask deep learning |
| CN111046973A (en) * | 2019-12-26 | 2020-04-21 | 北京市商汤科技开发有限公司 | Multitask detection method and device and storage medium |
| CN113554156A (en) * | 2021-09-22 | 2021-10-26 | 中国海洋大学 | Multi-task learning model construction method based on attention mechanism and deformable convolution |
Non-Patent Citations (1)
| Title |
|---|
| SHIKUN LIU ET AL.: "End-to-End Multi-Task Learning with Attention", 《2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114565064A (en) * | 2022-04-26 | 2022-05-31 | 心鉴智控(深圳)科技有限公司 | Method, system and equipment for identifying multitask learning deep network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902183B (en) | Knowledge graph embedding method based on diverse graph attention machine mechanism | |
| CN112699247B (en) | Knowledge representation learning method based on multi-class cross entropy contrast complement coding | |
| CN111260064A (en) | Knowledge inference method, system and medium based on knowledge graph of meta knowledge | |
| CN113535972B (en) | Knowledge graph link prediction model method and device fusing context semantics | |
| CN115391553B (en) | Method for automatically searching time sequence knowledge graph completion model | |
| CN114037024A (en) | Multitask neural network based data identification system and method | |
| CN115099606A (en) | Training method and terminal for power grid dispatching model | |
| CN113779988A (en) | Method for extracting process knowledge events in communication field | |
| CN115328782A (en) | Semi-supervised software defect prediction method based on graph representation learning and knowledge distillation | |
| Xu et al. | Living with artificial intelligence: A paradigm shift toward future network traffic control | |
| Fan et al. | Fast model update for iot traffic anomaly detection with machine unlearning | |
| CN118035448A (en) | Method, device and medium for classifying paper fields in citation network based on pseudo tag depolarization | |
| CN111767949B (en) | Feature and sample countersymbiotic-based multi-task learning method and system | |
| CN111368060B (en) | Self-learning method, device and system for conversation robot, electronic equipment and medium | |
| CN115587318A (en) | Source code classification method based on neural network | |
| WO2022186808A1 (en) | Method for solving virtual network embedding problem in 5g and beyond networks with deep information maximization using multiple physical network structure | |
| Xue et al. | An improved extreme learning machine based on variable-length particle swarm optimization | |
| Kristianto et al. | Autonomous sub-domain modeling for dialogue policy with hierarchical deep reinforcement learning | |
| CN114169449A (en) | Cross-social network user identity matching method | |
| CN112906868A (en) | Behavior clone-oriented demonstration active sampling method | |
| CN109409226A (en) | A kind of finger vena plot quality appraisal procedure and its device based on cascade optimization CNN | |
| CN109995756B (en) | Online single-classification active machine learning method for information system intrusion detection | |
| Liu et al. | Vanilla Graph Neural Networks | |
| CN116756343A (en) | Knowledge graph relation prediction method based on graph attention mechanism | |
| CN111460314A (en) | Social network subgraph embedding method and system based on triangular subgraph division |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |