CN113537365B - Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method - Google Patents
Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method Download PDFInfo
- Publication number
- CN113537365B CN113537365B CN202110820646.4A CN202110820646A CN113537365B CN 113537365 B CN113537365 B CN 113537365B CN 202110820646 A CN202110820646 A CN 202110820646A CN 113537365 B CN113537365 B CN 113537365B
- Authority
- CN
- China
- Prior art keywords
- task
- depth
- learning model
- task learning
- loss function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 230000006870 function Effects 0.000 claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 37
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000010606 normalization Methods 0.000 claims abstract description 8
- 230000011218 segmentation Effects 0.000 claims description 21
- 238000003708 edge detection Methods 0.000 claims description 15
- 230000003044 adaptive effect Effects 0.000 claims description 6
- 238000010586 diagram Methods 0.000 claims description 5
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 abstract description 4
- 238000005457 optimization Methods 0.000 abstract description 4
- 238000011160 research Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-task learning method based on information entropy dynamic weighting, and belongs to the technical field of machine learning. Firstly, an initial multi-task learning model M is built, model inference is carried out on an input image to obtain a plurality of task output graphs, normalization processing is carried out on the input image to obtain a corresponding normalized probability graph; then, calculating a fixed weight multi-task loss function by utilizing each normalized probability map, and performing preliminary training on the multi-task learning model M; and finally, constructing a final self-adaptive multi-task loss function through an information entropy dynamic weighting algorithm on the basis of the preliminarily trained multi-task learning model M, performing iterative optimization training on the preliminarily trained multi-task learning model until the multi-task learning model reaches convergence, and stopping training to obtain an optimized multi-task learning model M1. The method can effectively cope with different types of tasks, adaptively balance the relative importance of each task, and has strong algorithm applicability, simplicity and high efficiency.
Description
Technical Field
The invention belongs to the technical field of machine learning, and particularly relates to a multi-task learning self-adaptive balancing method based on information entropy dynamic weighting.
Background
Machine learning improves the performance of computer algorithms through empirical knowledge to achieve intelligent autonomous learning work, and is one of core technologies of artificial intelligence. Machine learning techniques typically require a large number of learning samples, and more recently, popular deep learning models typically require a large number of labeled samples to train the network. However, in many applications, certain task labels of training samples are difficult to collect or are labor and time consuming to manually annotate. In this case, multitasking learning may be utilized to maximize the utilization of the limited training samples in each task.
The multi-task learning aims to jointly learn a plurality of related tasks to improve the generalization performance of each task, and is widely applied to the fields of natural language processing, computer vision and the like. Each of which may be a general learning task such as a supervised task (e.g., classification or regression problem), an unsupervised task (e.g., clustering problem), a reinforcement learning task, or a multi-view learning task, etc.
In recent years, deep learning greatly improves the performance of various computer vision tasks, while multi-task learning jointly learns a plurality of tasks in one model so as to obtain better generalization performance and lower memory occupation, and the combination of the two, namely the deep multi-task learning research, has greatly progressed. However, the following problems still exist in deep multitasking learning: (1) The information exchange between different subtasks is insufficient and the advantage of multi-task learning is difficult to fully develop; (2) The loss function of most existing MTL studies is typically derived from the loss linear weighting of subtasks, which rely on human experience and lack adaptability.
Current deep multitasking studies focus mainly on the design of network architecture and optimization strategies:
in network structure research, there are two main ways of performing a multi-task learning mechanism in a deep neural network, namely hard parameter sharing and soft parameter sharing. Where hard parameter sharing typically shares a hidden layer among all tasks while preserving multiple task-specific output layers. Because the more tasks that are learned at the same time, the more the model needs to find expressions that are applicable to all tasks, hard parameter sharing greatly reduces the risk of overfitting. On the other hand, in soft parameter sharing, each task has its own model and corresponding parameters, and then regularized adjustment is performed on the model parameter distance to increase the phase degree of the parameters.
In the research of optimization strategies, most multi-task learning related works simply set the weights of the tasks to be fixed proportion, but the mode is seriously dependent on human experience, and in some cases, improper weights can lead to that some subtasks cannot work normally. Therefore, unlike the structure of designing a multi-tasking model, another part of research is focused on balancing the impact of different tasks on the network, including the researches of uncertainty weights, gradient normalization algorithms, dynamic weight averaging strategies, and the like.
In summary, since the multitasking model includes multiple learning tasks, how to adaptively balance the importance among different tasks has important research significance.
Disclosure of Invention
In order to improve generalization of a multi-task learning model, the invention designs a multi-task learning self-adaptive balance method based on information entropy dynamic weighting on a model optimization strategy through the characteristics of different tasks and the demand analysis of multi-task model application, namely, the relative weight of each task loss function is dynamically adjusted in the model training process, so that self-adaptive training and accurate prediction of the multi-task learning model are realized.
The information entropy dynamic weighting-based multi-task learning self-adaptive balancing method comprises the following specific steps:
firstly, constructing a multi-task learning model M, and carrying out model inference and normalization processing on an input image through the current multi-task learning model M to obtain different types of normalized probability diagrams;
the initial multi-task learning model M contains one shared encoder and three task-specific decoders.
The multi-task learning model M carries out model inference on an input image to generate three pixel-level task outputs which are respectively semantic segmentation output graphs P s Depth estimation output map P d And an edge detection output map P b Respectively carrying out normalization processing on each task output graph to obtain different types of normalized probability graphs, wherein the normalization probability graphs specifically comprise:
1) Semantic segmentation output map P s By usingProcessing the softmax function to obtain a normalized semantic segmentation probability map:
wherein M is the total category number of semantic segmentation, i represents the i-th layer semantic category in the predictive picture, and P s，i Outputting a map P for the model s Is divided into numerical graphs by the i-th layer of the semanteme, and P' s，i Then represents normalized i-th layer semantic segmentation probability map P' s 。
2) Edge detection output map P b Processing by adopting a sigmoid function to obtain a normalized edge detection probability map P' b ：
3) Depth estimation output map P d Converting the depth regression task into a classification task by utilizing a logarithmic space discretization strategy, and obtaining a normalized depth classification probability map by adopting a softmax function;
firstly, discrete division of depth values of a continuous space into K categories corresponding to K subintervals is performed by adopting a logarithmic space discretization strategy;
the method comprises the following steps: depth value interval [ D ] 1 ，D 2 ]Mapping to [ D ] 1 +1，D 2 +1]Is marked as [ D ]' 1 ，D′ 2 ]And according to the discretized depth threshold d k Dividing to obtain K sub-intervals { [ d ] 0 ，d 1 ]，[d 1 ，d 2 ]，...，[d K-1 ，d K ]}。
Discretized depth threshold d k The definition is as follows:
then discretizing the depth estimation truth value into a depth classification truth value according to the strategy;
i.e. when the depth estimation truth value is at [ d ] k-1 ，d k ]Class k is assigned to it and the deep task branches are trained with deep classification truth values.
Finally, obtaining a depth classification predictive graph in a training stage, and processing by adopting a softmax function to obtain a normalized depth classification probability graph P' d，k ；
The depth classification probability map is:
wherein K is the total category number of depth classification, K represents the kth depth category, and P d，k Representing a k-th layer depth classification prediction graph, P' d，k And (5) representing a normalized k-th layer depth classification probability map.
Calculating a multi-task loss function by using the normalized probability map, and performing preliminary training on the current multi-task learning model M;
the method comprises the following steps:
firstly, calculating the loss corresponding to each type of normalized probability map by adopting a cross entropy function;
cross entropy loss function L t The method comprises the following steps:
wherein y is t A supervision category label is provided for the one-hot form corresponding to each task; t is s, d or b, i.e. P' t Is a normalized probability map of semantic segmentation, edge detection or depth estimation tasks; c is the total category number corresponding to each task, and i represents the category of the ith layer in the predictive diagram.
Then, an equal weight addition and multitasking loss function L is constructed according to the fixed weight of each task mtl The method comprises the following steps:
finally, the multi-task loss function L is utilized mtl And carrying out gradient back transmission and parameter updating of the network model, and carrying out iterative training to obtain the preliminarily trained multi-task learning model.
Thirdly, constructing a final self-adaptive multi-task loss function L 'by utilizing an information entropy dynamic weighting algorithm on the basis of a multi-task learning model M of preliminary training' mtl 。
The method comprises the following steps:
first, the information entropy value E of each task is calculated by using each category multi-layer probability map t ：
Wherein W and H are the probability map row and column coordinates, respectively, and W and H are the maximum values of the probability map row and column lengths, respectively; and C is the channel value number of the probability graph, and C is the total number of categories corresponding to each task.
Then, the relative weight w of each task is distributed by utilizing the information entropy value t ；
Relative weight w t The method comprises the following steps:
when the prediction result of the task is worse, the uncertainty of the output probability map is higher, and the entropy value of the corresponding information is also larger. Therefore, a large weight is assigned to the task with poor prediction performance, and the model can be used for training the corresponding task in a weight mode.
Finally, according to the relative weight of each task and the cross entropy loss function L t And constructing a final self-adaptive multi-task loss function in a weighted summation mode.
Final adaptive multitasking loss function L' mtl The method comprises the following steps:
step four, utilizing a final self-adaptive multitasking loss function L' mtl Performing back propagation to obtain a parameter gradient of the current multi-task learning model M, and updating the parameter of the current multi-task learning model M by using a gradient descent algorithm to complete one-time iterative training;
and step five, after the iterative training is completed, a new multi-task learning model M1 is obtained, the next iteration is carried out in the step three until the multi-task learning model M1 reaches convergence, and the training is terminated.
The invention has the advantages that:
(1) The multi-task learning self-adaptive balancing method based on the information entropy dynamic weighting adopts the discretization strategy to convert the regression task into the classification task, can effectively cope with different types of tasks, and has strong algorithm applicability;
(2) According to the multi-task learning self-adaptive balance method based on the information entropy dynamic weighting, the information entropy is calculated by utilizing the predictive graph output by the task, the model structural design or the parameter updating process is not required to be changed, and the method is concise, efficient and plug-and-play;
(3) According to the multi-task learning self-adaptive balancing method based on the information entropy dynamic weighting, the weight of the task loss function is dynamically adjusted based on the information entropy value, the relative importance of each task can be self-adaptively balanced, and therefore overall task performance is improved.
(4) The self-adaptive balance method for multi-task learning based on information entropy dynamic weighting can effectively extract general shared features and task specific features of a model and quickly and uniformly complete training of the multi-task learning model.
Drawings
FIG. 1 is an overall flow chart of the method for adaptive balancing of multi-task learning based on dynamic weighting of information entropy of the present invention;
FIG. 2 is a schematic diagram of a multi-task learning model in accordance with the present invention;
FIG. 3 is a schematic representation of the discretization of the regression task in the present invention.
Detailed Description
The following describes the implementation method of the present invention in further detail by taking a multi-task learning network for realizing semantic segmentation, depth estimation and edge detection in combination in computer vision as an example in combination with the accompanying drawings.
The invention provides a multi-task learning self-adaptive balance method based on information entropy dynamic weighting, which adopts staged training, firstly uses a multi-task loss function with fixed weight to perform pre-training, and then uses a self-adaptive multi-task loss function with dynamic weighting to perform dynamic training. In the model training process, the information entropy algorithm can effectively evaluate the prediction result of each task, and the relative weight of the task is adjusted through a dynamic weighting strategy, so that the multi-task prediction model is more concerned with the task with relatively poor performance, and the self-adaptive balance learning of the performance of different tasks is realized.
The invention relates to a multi-task learning self-adaptive balancing method based on information entropy dynamic weighting, which is shown in figure 1 and comprises the following steps:
initializing network parameters, and training to obtain an initial multi-task learning model.
A multi-task learning network model based on a single encoder-multiple decoder is constructed, as shown in fig. 2, specifically:
the encoder contains network parameters shared by all tasks and is initialized with a skeletal network (e.g., resNet) pre-trained on ImageNet. The decoders contain task-specific network parameters, each task corresponds to one decoder, and a random parameter initialization mode is adopted. In this embodiment, three tasks are set to be solved: semantic segmentation, depth estimation and edge detection, the multi-task learning model contains one shared encoder and three task-specific decoders.
After the three tasks are output by the decoder, three cross entropy losses L are obtained 1 、L 2 And L 3 The relative weight w corresponding to each task 1 、w 2 And w 3 The cross entropy loss is summed by weighting to obtain a multi-task loss function L mtl ：
L mtl ＝w 1 L 1 +w 2 L 2 +w 3 L 3
Secondly, carrying out model inference and normalization processing on the input image through a multi-task learning model to obtain different types of normalized probability maps;
the multi-task learning model carries out model inference on an input image to generate three pixel-level task outputs, which are respectively used for outputting a graph P for semantic segmentation s Depth estimation output map P d And an edge detection output map P b Normalizing each task output graph to obtain different types of normalized probability graphs, wherein the normalized probability graphs specifically comprise:
1) Semantic segmentation output map P s Processing by adopting a softmax function to obtain a normalized multi-classification semantic segmentation probability map:
s is the total category number of semantic segmentation, i represents the ith semantic category in the predictive picture, and P s，i Outputting a map P for the model s Is divided into numerical graphs by the i-th layer of the semanteme, and P' s，i The normalized i-th layer semantic segmentation probability map is represented.
2) Edge detection output map P b Processing by adopting a sigmoid function (equivalent to a binary class softmax function) to obtain a normalized edge detection probability map P' b ：
3) Depth estimation output map P d Converting the depth regression task into a classification task by utilizing a logarithmic space discretization strategy, and obtaining a normalized depth classification probability map by adopting a softmax function;
first, as shown in fig. 3, a logarithmic space discretization strategy is adopted to discretely divide depth values of a continuous space into K sub-intervals corresponding to K categories, specifically:
depth value interval [ D ] 1 ，D 2 ]Mapping to [ D ] 1 +1，D 2 +1]Is marked as [ D ]' 1 ，D′ 2 ]And according to the discretized depth threshold d k Dividing to obtain K sub-intervals { [ d ] 0 ，d 1 ]，[d 1 ，d 2 ]，...，[d K-1 ，d K ]}。
Discretized depth threshold d k The definition is as follows:
then discretizing the depth estimation truth value into depth classification truth value according to the strategy, namely when the depth estimation truth value is in [ d ] k-1 ，d k ]Class k is assigned to it and the deep task branches are trained with deep classification truth values.
Finally, obtaining a depth classification predictive graph in a training stage, and processing by adopting a softmax function to obtain a normalized depth classification probability graph P' d，k ；
The depth classification probability map is:
wherein K is the total category number of depth classification, K represents the kth depth category, and P d，k Representing a k-th layer depth classification prediction graph, P' d，k And (5) representing a normalized k-th layer depth classification probability map.
In an embodiment of the present invention, discretization of depth estimation is performed taking k=80. The supervised truth of the depth branches is in the form of classification, so the depth estimation task is here also trained directly in the form of depth classification.
Step three, performing preliminary training on the multi-task learning model;
since the error of each task result predicted by the initialization model is larger and unstable, a multitasking network model needs to be initially trained, specifically:
firstly, calculating the loss corresponding to each type of normalized probability map by adopting a cross entropy function:
wherein y is t A supervision category label is provided for the one-hot form corresponding to each task; t corresponds to each task in step one and can be expressed as s, d or b, i.e. P' t Is a normalized probability map of semantic segmentation, edge detection or depth estimation tasks; c is the total category number corresponding to each task, and i represents the category of the ith layer in the predictive diagram.
Second, construct an equal weight addition and multitask loss function L mtl The method comprises the following steps:
during the preliminary training process, the loss function of each task is given an equal fixed weight.
Then, the multi-tasking loss function L is utilized mtl And carrying out gradient back transmission and parameter updating of the network model, and carrying out preliminary task prediction on the multi-task learning model obtained after training for a certain number of iterations.
And step four, constructing a self-adaptive multi-task loss function by utilizing an information entropy dynamic weighting algorithm on the basis of the multi-task learning model obtained through preliminary training, and further optimizing and training the multi-task learning model.
The method comprises the following steps:
first, the information entropy value E of each task is calculated by using each type of multi-layer probability map t ：
Wherein W and H are the probability map row and column coordinates, respectively, and W and H are the maximum values of the probability map row and column lengths, respectively; c is the channel number of the probability graph, and C is the total number of the corresponding classes of each task;
then, the relative weight w of each task is distributed by utilizing the information entropy value t ：
The information entropy can reflect the uncertainty of the predictive probability map, so the information entropy of the task output probability map can be utilized to assign relative weights:
when the prediction result of the task is worse, the uncertainty of the output probability map is higher, and the entropy value of the corresponding information is also larger. Therefore, tasks with poor prediction performance are assigned a larger weight so that the model is stressed to train the corresponding tasks.
Finally, according to the relative weight of each task and the cross entropy loss function L t And constructing an overall self-adaptive multitask loss function in a weighted summation mode.
Overall adaptive multitasking loss function L' mtl The method comprises the following steps:
step five, utilizing the integral self-adaptive multitasking loss function L' mtl Performing back propagation to obtain a model parameter gradient, and then updating model parameters by using a gradient descent algorithm to complete one-time iterative training;
and step six, after the model parameters are updated, a new multi-task learning model is obtained. And returning to the fourth step for the next iteration until the multi-task learning model reaches convergence, and ending training.
Because the prediction performance of each task can be changed after each network parameter is updated, the corresponding relative weight can be changed dynamically, and thus the self-adaptive adjustment of the loss function in the network model training is realized.
The above embodiment only describes three specific tasks of semantic segmentation, depth estimation and edge detection, but the application of the method is not limited to the three specific tasks, and the method can be applied to other tasks, can also be applied to more than three task conditions, and can be used for adjusting a multi-task learning model according to actual conditions. The situation that other tasks or three or more tasks are included belongs to the technical problem solved by the invention.
Claims (3)
1. A multi-task learning self-adaptive balancing method based on information entropy dynamic weighting is characterized by comprising the following steps:
firstly, an initial multi-task learning model M is built, an input image is inferred through the multi-task learning model M, different types of output of different tasks are obtained, normalization processing is carried out respectively, and a normalization probability map corresponding to the different tasks is obtained;
the multi-task learning model M carries out model inference on an input image to generate three pixel-level task outputs which are respectively semantic segmentation output graphs P s Depth estimation output map P d And an edge detection output map P b The corresponding normalized probability map is:
1) Semantic segmentation output map P s Processing by adopting a softmax function to obtain a normalized semantic segmentation probability map:
s is the total category number of semantic segmentation, i represents the i-th layer semantic category in the predictive picture, and P s,i Outputting a map P for the model s Is divided into numerical graphs by the i-th layer of the semanteme, and P' s,i Then represents normalized i-th layer semantic segmentation probability map P' s ；
2) Classified edge detection output map P b Processing by adopting a sigmoid function to obtain a normalized edge detection probability map P' b ：
3) Depth estimation output map P d Converting the depth regression task into a classification task by utilizing a logarithmic space discretization strategy, and obtaining a normalized depth classification probability map by adopting a softmax function;
firstly, discrete division of depth values of a continuous space into K categories corresponding to K sub-intervals is performed by adopting a logarithmic space discretization strategy, and the method specifically comprises the following steps:
depth value interval [ D ] 1 ,D 2 ]Mapping to [ D ] 1 +1,D 2 +1]Is marked as [ D ]' 1 ,D′ 2 ]And according to the discretized depth threshold d k Dividing to obtain K sub-intervals { [ d ] 0 ,d 1 ],[d 1 ,d 2 ],...,[d K-1 ,d K ]}；
Discretized depth threshold d k The definition is as follows:
then discretizing the depth estimation truth value into depth classification truth value according to the strategy, namely when the depth estimation truth value is in [ d ] k-1 ,d k ]The class is assigned as k, and the deep task branches are trained by depth classification truth values;
finally, obtaining a depth classification predictive graph in a training stage, and processing by adopting a softmax function to obtain a normalized depth classification probability graph P' d,k ；
The depth classification probability map is:
wherein K is the total category number of depth classification, K represents the kth depth category, and P d,k Representing a k-th layer depth classification prediction graph, P' d,k Representing a normalized k-th layer depth classification probability map;
then, calculating a multi-task loss function by utilizing each normalized probability map, and performing preliminary training on a multi-task learning model M through the multi-task loss function;
finally, on the basis of the preliminarily trained multi-task learning model M, constructing a final self-adaptive multi-task loss function through an information entropy dynamic weighting algorithm, obtaining a parameter gradient of the current multi-task learning model M by utilizing a back propagation algorithm, updating parameters, and completing one-time iterative training;
the specific process for constructing the final self-adaptive multitasking loss function is as follows:
step 501, calculating the information entropy value E of each task by using each class multi-layer probability map t ：
Wherein W and H are the probability map row and column coordinates, respectively, and W and H are the maximum values of the probability map row and column lengths, respectively; c is the channel number of the probability graph, and C is the total number of the corresponding classes of each task;
step 502, assigning relative weights w of the tasks by using the information entropy values t ；
Relative weight w t The method comprises the following steps:
step 503, according to the relative weight of each task and the cross entropy loss function L t Constructing a final self-adaptive multitask loss function in a weighted summation mode;
final adaptive multitasking loss function L' mtl The method comprises the following steps:
after iterative training, a new multi-task learning model M1 is obtained, the input image is inferred and normalized again, the next iteration is carried out by utilizing the self-adaptive multi-task loss function until the multi-task learning model M1 achieves convergence, and training is terminated.
2. The adaptive balancing method for multi-task learning based on dynamic weighting of information entropy according to claim 1, wherein the multi-task learning model comprises a shared encoder and decoders corresponding to specific tasks.
3. The adaptive balance method for multi-task learning based on information entropy dynamic weighting according to claim 1, wherein the specific process of calculating the multi-task loss function and performing preliminary training by the multi-task learning model is as follows:
firstly, calculating the loss corresponding to each type of normalized probability map by adopting a cross entropy function;
cross entropy loss function L t The method comprises the following steps:
wherein y is t A supervision category label is provided for the one-hot form corresponding to each task; t is s, d or b, i.e. P t ' is a normalized probability map for semantic segmentation, edge detection, or depth estimation tasks; c is the total category number corresponding to each task, i represents the category of the ith layer in the predictive diagram;
then, an equal weight addition and multitasking loss function L is constructed according to the fixed weight of each task mtl The method comprises the following steps:
finally, the multi-task loss function L is utilized mtl And carrying out gradient back transmission and parameter updating of the network model, and carrying out iterative training to obtain the preliminarily trained multi-task learning model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820646.4A CN113537365B (en) | 2021-07-20 | 2021-07-20 | Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820646.4A CN113537365B (en) | 2021-07-20 | 2021-07-20 | Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113537365A CN113537365A (en) | 2021-10-22 |
| CN113537365B true CN113537365B (en) | 2024-02-06 |
Family
ID=78100520
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110820646.4A Active CN113537365B (en) | 2021-07-20 | 2021-07-20 | Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113537365B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023097616A1 (en) * | 2021-12-02 | 2023-06-08 | Intel Corporation | Apparatus, method, device and medium for loss balancing in multi-task learning |
| CN114714146B (en) * | 2022-04-08 | 2023-04-07 | 北京理工大学 | Method for simultaneously predicting surface roughness and cutter abrasion |
| CN117273068B (en) * | 2023-09-28 | 2024-04-16 | 东南大学 | Model initialization method based on linearly expandable learning genes |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451620A (en) * | 2017-08-11 | 2017-12-08 | 深圳市唯特视科技有限公司 | A kind of scene understanding method based on multi-task learning |
| CN110837836A (en) * | 2019-11-05 | 2020-02-25 | 中国科学技术大学 | Semi-supervised semantic segmentation method based on maximized confidence |
-
2021
- 2021-07-20 CN CN202110820646.4A patent/CN113537365B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107451620A (en) * | 2017-08-11 | 2017-12-08 | 深圳市唯特视科技有限公司 | A kind of scene understanding method based on multi-task learning |
| CN110837836A (en) * | 2019-11-05 | 2020-02-25 | 中国科学技术大学 | Semi-supervised semantic segmentation method based on maximized confidence |
Non-Patent Citations (2)
| Title |
|---|
| Y Wang,et al..Boundary-aware multitask learning for remote sensing imagery.《IEEE》.2020,全文. * |
| 基于组合赋权法的舰船动力***使用效能评估研究;张磊;曹跃云;李彬;崔佳林;;舰船科学技术(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113537365A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113537365B (en) | Information entropy dynamic weighting-based multi-task learning self-adaptive balancing method | |
| WO2020019236A1 (en) | Loss-error-aware quantization of a low-bit neural network | |
| CN112115998B (en) | Method for overcoming catastrophic forgetting based on anti-incremental clustering dynamic routing network | |
| CN112699247A (en) | Knowledge representation learning framework based on multi-class cross entropy contrast completion coding | |
| CN112465120A (en) | Fast attention neural network architecture searching method based on evolution method | |
| WO2022252455A1 (en) | Methods and systems for training graph neural network using supervised contrastive learning | |
| CN110866113B (en) | Text classification method based on sparse self-attention mechanism fine-tuning burt model | |
| CN111104831B (en) | Visual tracking method, device, computer equipment and medium | |
| CN109558898B (en) | Multi-choice learning method with high confidence based on deep neural network | |
| CN116306686B (en) | Method for generating multi-emotion-guided co-emotion dialogue | |
| Ni et al. | Hdpg: Hyperdimensional policy-based reinforcement learning for continuous control | |
| CN114004383A (en) | Training method of time series prediction model, time series prediction method and device | |
| CN111353534B (en) | Graph data category prediction method based on adaptive fractional order gradient | |
| Dong et al. | Lambo: Large language model empowered edge intelligence | |
| CN114202021A (en) | Knowledge distillation-based efficient image classification method and system | |
| CN115599918B (en) | Graph enhancement-based mutual learning text classification method and system | |
| CN116467941A (en) | 4-degree-of-freedom ship motion online forecasting method considering different wind levels | |
| CN110717402A (en) | Pedestrian re-identification method based on hierarchical optimization metric learning | |
| CN113807005A (en) | Bearing residual life prediction method based on improved FPA-DBN | |
| CN114220127B (en) | Image recognition method based on gradient guided evolutionary algorithm | |
| US20230289563A1 (en) | Multi-node neural network constructed from pre-trained small networks | |
| CN113033495B (en) | Weak supervision behavior identification method based on k-means algorithm | |
| CN114118369B (en) | Image classification convolutional neural network design method based on group intelligent optimization | |
| CN111597814B (en) | Man-machine interaction named entity recognition method, device, equipment and storage medium | |
| CN117315397B (en) | Classification method for noise data containing labels based on class curvature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |