CN112688809A - Diffusion adaptive network learning method, system, terminal and storage medium - Google Patents

Diffusion adaptive network learning method, system, terminal and storage medium Download PDF

Info

Publication number
CN112688809A
CN112688809A CN202011521741.6A CN202011521741A CN112688809A CN 112688809 A CN112688809 A CN 112688809A CN 202011521741 A CN202011521741 A CN 202011521741A CN 112688809 A CN112688809 A CN 112688809A
Authority
CN
China
Prior art keywords
diffusion
variance
gradient descent
strategy
random
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011521741.6A
Other languages
Chinese (zh)
Other versions
CN112688809B (en
Inventor
张萌飞
靳丹琦
陈捷
雷攀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenggeng Intelligent Technology Xi'an Research Institute Co ltd
Original Assignee
Shenggeng Intelligent Technology Xi'an Research Institute Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenggeng Intelligent Technology Xi'an Research Institute Co ltd filed Critical Shenggeng Intelligent Technology Xi'an Research Institute Co ltd
Priority to CN202011521741.6A priority Critical patent/CN112688809B/en
Publication of CN112688809A publication Critical patent/CN112688809A/en
Application granted granted Critical
Publication of CN112688809B publication Critical patent/CN112688809B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Complex Calculations (AREA)

Abstract

A diffusion self-adaptive network learning method, system, terminal and storage medium, the method includes a random gradient descent process and a time-averaged variance reduction random gradient descent process, in which each node of a distributed network runs PkSub-least mean square strategy and collect this PkInput data received in a secondary run; in the time-averaged reduction variance stochastic gradient descent process, the length of the data pair is P by using the previously collected datakAveraging the random gradients in the time window of (a) to obtain an estimate of the mean gradient, and using the estimate to update the weight equation for variance reduction in the next m iterations. Meanwhile, a diffusion self-adaptive network learning system, a terminal and a storage medium are provided. The invention overcomes the defect that the traditional variance-reducing random gradient descent algorithm cannot be used in an online learning environment, and is applied to the self-adaptive diffusionIn the network algorithm, the online estimation performance of the distributed diffusion network is improved.

Description

Diffusion adaptive network learning method, system, terminal and storage medium
Technical Field
The invention belongs to the field of adaptive signal processing, and relates to a diffusion adaptive network learning method, a diffusion adaptive network learning system, a diffusion adaptive network learning terminal and a storage medium.
Background
In a multi-node network, due to the dispersion of the physical positions of nodes and the consideration of the limitation of the communication capacity among the nodes and the requirements of safety, robustness and the like, the network cannot transmit data in a large scale and collect all data in a central node for analysis by adopting a centralized strategy, so that the requirement of distributed processing is reflected; in the context of big data, data is often collected in a streaming manner over time, and a system model or parameters need to be re-estimated at each moment.
The advent of adaptive algorithms in distributed networks just meets these needs. In the last decade, a great deal of research is carried out on adaptive algorithms in a distributed network in the field, and the application of the adaptive algorithms is explored. According to the cooperation mode and the information flow mode among the nodes, the cooperation strategy in the distributed network mainly comprises three types: incremental strategies, consensus strategies, and diffusion strategies. The increment strategy forms a Hamiltonian loop in the network and sequentially accesses each node to carry out information interaction. Although the traffic required for the incremental strategy is theoretically small, building a Hamiltonian loop in any network is itself an NP-hard problem. Furthermore, such loops are very sensitive to failure of either the node or the link, and thus the incremental strategy is not well suited for distributed online adaptive signal processing. In the consensus strategy and the diffusion strategy, each node needs to communicate with the neighbor nodes in real time, and the global target parameters in the network are cooperatively estimated by utilizing the information exchange between the node and the neighbor nodes. Because each node needs to acquire the information of all the neighbor nodes at every moment, the two strategies need more communication resources than an increment strategy, but can fully utilize the cooperation of the nodes in the distributed network structure. In addition, the diffusion strategy provides the nodes with continuous adaptation and learning capabilities, and is a key research strategy in distributed adaptive signal processing because the scalability has great advantages and has proved to have better stability and dynamic range than the consensus strategy. The distributed adaptive diffusion least mean square algorithm is a random gradient descent algorithm, and the gradient noise of the random gradient greatly hinders the rapid convergence of the algorithm. Therefore, the research on how to reduce the influence of the gradient noise is of great significance to the improvement of the diffusion-based distributed online learning algorithm. Among the many alternatives, it is straightforward to apply a variance reducing random gradient algorithm to a distributed adaptive network. The variance reduction stochastic gradient algorithm is designed to minimize the loss function defined on all data samples of the batch. Typical algorithms include the random variance reduction gradient (SVRG) algorithm and the SAGA algorithm. The SVRG algorithm takes two cycles: the true gradient is calculated in the outer loop, and the variance random gradient is reduced and the inner loop is calculated. Whereas SAGA algorithms perform only one cycle, but require more memory to estimate true gradients, they are a great improvement in performance over the original stochastic gradient descent algorithms, however, their design is based on samples collected in batches rather than by learning the flow data in the problem online.
Disclosure of Invention
The invention aims to provide a diffusion adaptive network learning method, a diffusion adaptive network learning system, a terminal and a storage medium aiming at the problem that the variance reducing random gradient descent algorithm in the prior art cannot be used in an online learning environment, and the variance reducing random gradient descent algorithm can be applied to the online learning of the diffusion adaptive network to improve the online estimation performance of the distributed diffusion network.
In order to achieve the purpose, the invention has the following technical scheme:
a diffusion self-adaptive network learning method comprises a random gradient descent process and a time-averaged variance reduction random gradient descent process, wherein in the random gradient descent process, each node of a distributed network runs PkThe sub-least mean square strategy and collects this PkInput received in secondary operationData; using previously collected data during the time-averaged decreasing variance stochastic gradient descent for a length of PkAveraging the random gradients in the time window to obtain an estimated value of an average gradient, and updating a weight equation for reducing the variance by using the estimated value in the next m-time iterative computations; at the very beginning of PkAt each moment, executing a random gradient descent process, and when the iteration number is more than PkAnd then, calculating the average gradient under the window function, and further realizing the reduction of the variance of the random gradient, thereby accelerating the convergence speed of the self-adaptive algorithm of the whole diffusion network.
As a preferred scheme of the diffusion adaptive network learning method of the invention:
executing a random gradient descent strategy through a distributed network, and obtaining an estimation result w of a diffusion strategy by a node k at the moment ik,iAnd collecting input signal stream data x at time ik,iRepeating this strategy until i is greater than the length P of the window functionkAnd then stop.
As a preferred scheme of the diffusion adaptive network learning method of the invention:
the network of the random gradient descent strategy has a global cost function of the form
Figure BDA0002849218740000031
Wherein
Figure BDA0002849218740000032
N represents the total number of nodes in the network, and the symbol E (-) represents the data xk,iThe distribution of (a) is desired.
As a preferred scheme of the diffusion adaptive network learning method of the invention:
the first and second convergence step sizes of the node k of the random gradient descent strategy satisfy
Figure BDA0002849218740000033
Wherein, deltakRepresenting a cost function JkThe gradient vector of (a) satisfies δk-Lipschitz continuous stripAnd (3) a component.
As a preferred scheme of the diffusion adaptive network learning method of the invention:
from i > PkInitially, the distributed network implements a reduced variance stochastic gradient descent strategy, first wk,i-1Is assigned to the inner loop variable
Figure BDA0002849218740000034
Namely, it is
Figure BDA0002849218740000035
The average gradient is then estimated using a window function
Figure BDA0002849218740000036
Wherein
Figure BDA0002849218740000037
Representing a cost function JkAt input as signal xk,iWhen, to wk,i-1Of the gradient of (c).
As a preferred scheme of the diffusion adaptive network learning method of the invention:
the number m of the inner loop and the length P of the window functionkThere is a set relationship:
Figure BDA0002849218740000038
as a preferred scheme of the diffusion adaptive network learning method of the invention: in the next m times of iterative calculation, the obtained inner loop variable is utilized
Figure BDA0002849218740000039
And average gradient
Figure BDA00028492187400000310
Calculating a reduced variance random gradient
Figure BDA00028492187400000311
Then the node k obtains an estimation result w of the diffusion strategy at the moment ik,i(ii) a Within m times of executionAfter loop iteration, the inner loop variables are updated again
Figure BDA00028492187400000312
Until the algorithm converges;
in the random gradient of decreasing variance, the first convergence step of node k is satisfied
Figure BDA00028492187400000313
Second order convergence step size satisfy
Figure BDA00028492187400000314
Wherein v iskRepresenting a cost function
Figure BDA00028492187400000315
Is vkStrongly convex.
The invention also provides a diffusion self-adaptive network learning system, which comprises:
a random gradient descent execution module for operating each node of the distributed network by PkThe sub-least mean square strategy and collects this PkInput data received in a secondary run;
a time-averaged variance-reducing stochastic gradient descent execution module for executing data with length P collected by the stochastic gradient descent execution modulekAveraging the random gradients in the time window to obtain an estimated value of an average gradient, and updating a weight equation for reducing the variance by using the estimated value in the next m-time iterative computations;
a timing control module for controlling the timing of the first PkAt each moment, controlling to execute a random gradient descent process, and when the iteration number is more than PkAnd controlling to calculate the average gradient under the window function, thereby reducing the variance of the random gradient.
The invention also provides a terminal device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor realizes the steps of the diffusion adaptive network learning method when executing the computer program.
The present invention also provides a computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the steps of the diffusion adaptive network learning method.
Compared with the prior art, the invention has the following beneficial effects: the diffusion self-adaptive network learning method is a method for accelerating random gradient convergence in a streaming data processing environment, overcomes the defect that the traditional variance-reducing random gradient descent algorithm cannot be used in an online learning environment, and is applied to a self-adaptive diffusion network algorithm, so that the online estimation performance of a distributed diffusion network is improved. The invention effectively reduces the gradient noise in the online estimation of the distributed diffusion network, thereby accelerating the convergence speed of the algorithm and improving the performance of the algorithm. The method has certain expansibility, is not limited to a diffusion strategy, and can also be applied to other distributed strategies, such as an increment strategy, a consistency strategy and the like.
Drawings
FIG. 1 is a schematic diagram of an implementation of the diffusion adaptive network learning method of the present invention;
FIG. 2 is a flow chart of the design of the diffusion adaptive network learning method of the present invention;
fig. 3 shows the inventive system with a loss function model J for L50, N16 and N16 network nodesk(w；xk,i)＝(dk,i-wTxk,i)2The reduced variance diffusion strategy of (1).
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
The signal model and the related quantities of the problem studied by the invention are introduced as follows:
consider a distributed network of N nodes. At each node k, an unknown parameter vector of length lx 1 needs to be estimated
Figure BDA0002849218740000051
An input vector of length Lx 1 can be observed at node kxk,i。
The invention provides a diffusion self-adaptive network learning method, which comprises the following steps:
s1: executing a random gradient descent strategy by a distributed network, and obtaining an estimation result w of the diffusion strategy by a node k at the moment ik,iAnd collecting input signal stream data x at time ik,iRepeating this strategy until i is greater than the length P of the window functionkStopping the operation;
in the random gradient descent strategy, the global cost function in the network is in the form of
Figure BDA0002849218740000052
Wherein
Figure BDA0002849218740000053
N represents the total number of nodes in the network, and the symbol E (-) represents the data xk,iThe distribution of (a) is desired;
in the random gradient descent strategy, the first-order and second-order convergence step length of the node k meet
Figure BDA0002849218740000054
Wherein deltakRepresenting a cost function JkThe gradient vector of (a) satisfies δkLipschitz continuous conditions.
S2: from i > PkInitially, the distributed network implements a reduced variance stochastic gradient descent strategy.
Firstly, w isk,i-1Is assigned to the inner loop variable
Figure BDA0002849218740000055
Namely, it is
Figure BDA0002849218740000056
The number m of inner loops and the length P of the window function in step S1kThe sizes are close to each other and are generally set as
Figure BDA0002849218740000057
S3: estimating average gradient using window function
Figure BDA0002849218740000058
Wherein
Figure BDA0002849218740000059
Representing a cost function JkAt input as signal xk,iWhen, to wk,i-1Of the gradient of (c).
S4: in the next m iterations (i.e., inner loop), the inner loop variables obtained in step S2 are used
Figure BDA00028492187400000510
Average gradient obtained in step S3
Figure BDA00028492187400000511
A random gradient of decreasing variance is calculated:
Figure BDA0002849218740000061
then the node k obtains an estimation result w of the diffusion strategy at the moment ik,i；
The first-order convergence step length of the descending strategy node k for reducing the random gradient variance satisfies
Figure BDA0002849218740000062
Second order convergence step size satisfy
Figure BDA0002849218740000063
Wherein v iskRepresenting a cost function
Figure BDA0002849218740000064
Is vkStrongly convex.
S5, after m times of inner loop iteration, re-executing the step S2, updating the inner loop variable
Figure BDA0002849218740000065
S6: steps S2-S6 are repeatedly executed until the algorithm converges.
Examples
The experimental setup was as follows: wherein d isk,iFrom a linear model
Figure BDA0002849218740000066
To obtain zk,iIs a variance of
Figure BDA0002849218740000067
White gaussian noise, embodiments of the present invention assume all node optimization quantities w for convenience1＝…＝wN＝w＊，w＊Sampling in standard normal distribution, and setting fusion matrix C-I in diffusion strategy16For a non-cooperative policy setting a ═ I16Where matrix I represents the identity matrix and for the cooperation strategy is set as the standard union matrix A, its elements
Figure BDA0002849218740000068
Figure BDA0002849218740000069
Representing the number of neighbor nodes of the node k; in control experiments, the step size of the non-cooperative diffusion strategy was set to μ1＝…＝μN0.0012, the step size of the reduced variance diffusion strategy and the least mean square diffusion strategy is set to μ1＝…＝μN＝0.0015。
As shown in fig. 1 and fig. 2, a diffusion adaptive network learning method includes the following steps:
s1: executing a random gradient descent strategy by a distributed network, and obtaining an estimation result w of the diffusion strategy by a node k at the moment ik,iAnd collecting input signal stream data x at time ik,iRepeating this strategy until i is greater than the length P of the window functionkIs stopped, wherein xk,iIs a Gaussian random vector, P in the comparative experimentkRespectively 50 and 150, step size set to μ1＝…＝μN＝0.0015，Initialization wk,0Is an arbitrary value; the global cost function in the stochastic gradient descent strategy network is in the form of
Figure BDA00028492187400000610
Wherein the content of the first and second substances,
Figure BDA00028492187400000611
n represents the total number of nodes in the network, and the symbol E (-) represents the data xk,iThe distribution of (a) is desired. The first and second convergence step sizes of the node k of the stochastic gradient descent strategy satisfy
Figure BDA00028492187400000612
Wherein, deltakRepresenting a cost function JkThe gradient vector of (a) satisfies δkLipschitz continuous conditions.
S2: from i > PkInitially, the distributed network implements a reduced variance stochastic gradient descent strategy, first wk,i-1Is assigned to the inner loop variable
Figure BDA0002849218740000071
Namely, it is
Figure BDA0002849218740000072
Setting the number of internal cycles
Figure BDA0002849218740000073
S3: estimating average gradient using window function
Figure BDA0002849218740000074
Wherein
Figure BDA0002849218740000075
Representing a cost function JkAt input as signal xk,iWhen, to wk,i-1A gradient of (a);
s4: in the next m iterations (i.e. inner loop), the obtained inner loop variables are utilized
Figure BDA0002849218740000076
And the resulting average gradient
Figure BDA0002849218740000077
Calculating a reduced variance random gradient
Figure BDA0002849218740000078
Then the node k obtains an estimation result w of the diffusion strategy at the moment ik,i(ii) a The first-order convergence step length of the descending strategy node k for reducing the random gradient variance satisfies
Figure BDA0002849218740000079
Second order convergence step size satisfy
Figure BDA00028492187400000710
Wherein v iskRepresenting a cost function
Figure BDA00028492187400000711
Is vkStrongly convex.
S5, after m times of inner loop iteration, re-executing S2 and updating inner loop variables
Figure BDA00028492187400000712
S6: the steps S2-S6 are repeatedly executed until the algorithm converges.
As can be seen from FIG. 3, the online learning method of the diffusion adaptive network for reducing the random gradient variance provided by the invention has better performance compared with the standard least mean square diffusion strategy, and the effectiveness of the variance reduction technology is verified. In addition, the larger window PkThe convergence speed of the algorithm can be increased compared to a smaller window because of the large PkThe average gradient can be estimated more accurately.
The invention also provides a diffusion self-adaptive network learning system, which comprises:
the random gradient descent execution module enables each node of the distributed network to operate a sub-minimum mean square strategy and collects input data received in the operation;
the time average variance reduction random gradient descent execution module is used for averaging the random gradients in a time window with the length of the random gradients to obtain an average gradient estimation value by using the data collected by the random gradient descent execution module, and updating a weight equation for reducing the variance by using the estimation value in the next m-time iterative calculation;
and the time sequence control module is used for controlling and executing a random gradient descending process at the initial moment, and controlling and calculating the average gradient under the window function when the iteration number is greater than the initial iteration number so as to reduce the variance of the random gradient.
The invention further provides a terminal device, which includes a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the diffusion adaptive network learning method when executing the computer program.
The present invention also proposes a computer readable storage medium, in which a computer program is stored, which, when being executed by a processor, implements the steps of the above-mentioned diffusion adaptive network learning method according to the present invention.
The computer program may be divided into one or more modules/units, which are stored in the memory and executed by the processor to perform the method of the invention.
The terminal can be a desktop computer, a notebook, a palm computer, a cloud server and other computing equipment, and can also be a processor and a memory. The processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The memory may be used to store computer programs and/or modules, and the processor may implement the various functions of the diffusion adaptive web learning system by running or executing the computer programs and/or modules stored in the memory, as well as by invoking data stored in the memory.
The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the technical solution of the present invention, and it should be understood by those skilled in the art that the technical solution can be modified and replaced by a plurality of simple modifications and replacements without departing from the spirit and principle of the present invention, and the modifications and replacements also fall into the protection scope covered by the claims.

Claims (10)

1. A diffusion adaptive network learning method is characterized in that: comprises a random gradient descent process and a time-averaged variance reduction random gradient descent process, wherein each node of the distributed network runs P in the random gradient descent processkThe sub-least mean square strategy and collects this PkInput data received in a secondary run; using previously collected data during the time-averaged decreasing variance stochastic gradient descent for a length of PkAveraging the random gradients in the time window to obtain an estimated value of an average gradient, and updating a weight equation for reducing the variance by using the estimated value in the next m-time iterative computations; at the very beginning of PkAt each moment, executing a random gradient descent process, and when the iteration number is more than PkAnd then, calculating the average gradient under the window function, and further realizing the reduction of the variance of the random gradient, thereby accelerating the convergence speed of the self-adaptive algorithm of the whole diffusion network.
2. The diffusion adaptive network learning method of claim 1, wherein:
executing a random gradient descent strategy through a distributed network, and obtaining an estimation result w of a diffusion strategy by a node k at the moment ik,iAnd collecting input signal stream data x at time ik,iRepeating this strategy until i is greater than the length P of the window functionkAnd then stop.
3. The diffusion adaptive network learning method of claim 2, wherein:
the network of the random gradient descent strategy has a global cost function of the form
Figure FDA0002849218730000011
Wherein
Figure FDA0002849218730000012
N represents the total number of nodes in the network, and the symbol E (-) represents the data xk,iThe distribution of (a) is desired.
4. The diffusion adaptive network learning method of claim 2, wherein:
the first and second convergence step sizes of the node k of the random gradient descent strategy satisfy
Figure FDA0002849218730000013
Wherein, deltakRepresenting a cost function JkThe gradient vector of (a) satisfies δkLipschitz continuous conditions.
5. The diffusion adaptive network learning method of claim 2, wherein:
from i > PkInitially, the distributed network implements a reduced variance stochastic gradient descent strategy, first wk,i-1Is assigned to the inner loop variable
Figure FDA0002849218730000014
Namely, it is
Figure FDA0002849218730000015
The average gradient is then estimated using a window function
Figure FDA0002849218730000016
Wherein
Figure FDA0002849218730000017
Representing a cost function JkAt input as signal xk,iWhen, to wk,i-1Of the gradient of (c).
6. The diffusion adaptive network learning method of claim 5, wherein:
the number m of the inner loop and the length P of the window functionkThere is a set relationship:
Figure FDA0002849218730000021
7. the diffusion adaptive network learning method of claim 5, wherein: in the next m times of iterative calculation, the obtained inner loop variable is utilized
Figure FDA0002849218730000022
And average gradient
Figure FDA0002849218730000023
Calculating a reduced variance random gradient
Figure FDA0002849218730000024
Then the node k obtains an estimation result w of the diffusion strategy at the moment ik,i(ii) a After the m internal loop iterations are executed, the internal loop variables are updated again
Figure FDA0002849218730000025
Until the algorithm converges;
in the random gradient of decreasing variance, the first convergence step of node k is satisfied
Figure FDA0002849218730000026
Second order convergence step size satisfy
Figure FDA0002849218730000027
Wherein v iskRepresenting a cost function
Figure FDA0002849218730000028
Is vkStrongly convex.
8. A diffusion adaptive web learning system, comprising:
a random gradient descent execution module for operating each node of the distributed network by PkThe sub-least mean square strategy and collects this PkInput data received in a secondary run;
a time-averaged variance-reducing stochastic gradient descent execution module for executing data with length P collected by the stochastic gradient descent execution modulekAveraging the random gradients in the time window to obtain an estimated value of an average gradient, and updating a weight equation for reducing the variance by using the estimated value in the next m-time iterative computations;
a timing control module for controlling the timing of the first PkAt each moment, controlling to execute a random gradient descent process, and when the iteration number is more than PkAnd controlling to calculate the average gradient under the window function, thereby reducing the variance of the random gradient.
9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: the processor, when executing the computer program, performs the steps of the diffusion adaptive network learning method of any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: the computer program, when being executed by a processor, carries out the steps of the diffusion adaptive network learning method according to any one of claims 1 to 7.
CN202011521741.6A 2020-12-21 2020-12-21 Diffusion self-adaptive network learning method, system, terminal and storage medium Active CN112688809B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011521741.6A CN112688809B (en) 2020-12-21 2020-12-21 Diffusion self-adaptive network learning method, system, terminal and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011521741.6A CN112688809B (en) 2020-12-21 2020-12-21 Diffusion self-adaptive network learning method, system, terminal and storage medium

Publications (2)

Publication Number Publication Date
CN112688809A true CN112688809A (en) 2021-04-20
CN112688809B CN112688809B (en) 2023-10-03

Family

ID=75450048

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011521741.6A Active CN112688809B (en) 2020-12-21 2020-12-21 Diffusion self-adaptive network learning method, system, terminal and storage medium

Country Status (1)

Country Link
CN (1) CN112688809B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190050727A1 (en) * 2018-01-12 2019-02-14 Timothy Anderson Neural network training
CN109492753A (en) * 2018-11-05 2019-03-19 中山大学 A kind of method of the stochastic gradient descent of decentralization
WO2019235551A1 (en) * 2018-06-05 2019-12-12 Okinawa Institute Of Science And Technology School Corporation Total stochastic gradient estimation method, device and computer program
CN110809772A (en) * 2017-10-27 2020-02-18 谷歌有限责任公司 System and method for improving optimization of machine learning models
CN110929878A (en) * 2019-10-30 2020-03-27 同济大学 Distributed random gradient descent method
CN110930016A (en) * 2019-11-19 2020-03-27 三峡大学 Cascade reservoir random optimization scheduling method based on deep Q learning

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110809772A (en) * 2017-10-27 2020-02-18 谷歌有限责任公司 System and method for improving optimization of machine learning models
US20190050727A1 (en) * 2018-01-12 2019-02-14 Timothy Anderson Neural network training
WO2019235551A1 (en) * 2018-06-05 2019-12-12 Okinawa Institute Of Science And Technology School Corporation Total stochastic gradient estimation method, device and computer program
CN109492753A (en) * 2018-11-05 2019-03-19 中山大学 A kind of method of the stochastic gradient descent of decentralization
CN110929878A (en) * 2019-10-30 2020-03-27 同济大学 Distributed random gradient descent method
CN110930016A (en) * 2019-11-19 2020-03-27 三峡大学 Cascade reservoir random optimization scheduling method based on deep Q learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
刘张虎;程春玲;: "面向大规模数据主题建模的方差减小的随机变分推理算法", 计算机应用, no. 06 *
陈振宏;兰艳艳;郭嘉丰;程学旗;: "基于差异合并的分布式随机梯度下降算法", 计算机学报, no. 10 *

Also Published As

Publication number Publication date
CN112688809B (en) 2023-10-03

Similar Documents

Publication Publication Date Title
Wang et al. Stability of recurrent neural networks with time-varying delay via flexible terminal method
Xie et al. Event‐triggered consensus control for second‐order multi‐agent systems
CN105677489B (en) The dynamic of batch gap size sets system and method under discrete stream process model
CN110456681B (en) Output feedback controller of neutral stable saturation system based on event trigger
CN114851198B (en) Consistent tracking fixed time stable control method for multiple single-link mechanical arms
CN106502100B (en) The single controller for time delay design method of the distribution of multiple mobile robot
CN109032630B (en) Method for updating global parameters in parameter server
CN110703667A (en) Design method of network control system controller with time delay and data packet loss
CN113359463A (en) Cyclic switching scheme for leadership following consistency problem of multi-agent system
Ma et al. Finite‐time average consensus based approach for distributed convex optimization
Wang et al. Fixed‐time event‐triggered consensus tracking control for uncertain nonlinear multiagent systems with dead‐zone constraint
CN112269318B (en) Finite time remote safety state estimation method for time delay uncertain system
CN112688809A (en) Diffusion adaptive network learning method, system, terminal and storage medium
Ramadevi et al. Chaotic Sandpiper Optimization Based Virtual Machine Scheduling for Cyber-Physical Systems.
Tiberi et al. A simple self-triggered sampler for nonlinear systems
CN113515066B (en) Nonlinear multi-intelligent system dynamic event trigger control method
CN115268275A (en) Multi-agent system consistency tracking method and system based on state observer
Zhang et al. Adaptive event‐based finite‐time consensus of nonholonomic multi‐agent systems subject to unknown disturbances
Zhou et al. Piecewise adaptive sliding mode control for aeroengine networked control systems with resource constraints
CN113459083A (en) Self-adaptive fixed time control method and system for mechanical arm under event trigger
Takao et al. Approximate Fault-Tolerant Data Stream Aggregation for Edge Computing
Zeng et al. Distributed moving horizon estimation subject to communication delays and losses
Hou et al. Distributed consensus of multi-agent via aperiodic intermittent sampled-data control
Tang et al. Finite‐Time Simultaneous Stabilization for Stochastic Port‐Controlled Hamiltonian Systems over Delayed and Fading Channels
Zhuang et al. Event-triggered output feedback control for a class of discrete-time nonlinear systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant