Convergence analysis of generalized back-propagation algorithm with modified gradient function

S. C. Ng; S. H. Leung; Andrew Luk; Yunfeng Wu

doi:10.1109/ijcnn.2006.247381

Convergence analysis of generalized back-propagation algorithm with modified gradient function

S. C. Ng
, S. H. Leung
, Andrew Luk
, Yunfeng Wu

Hong Kong Metropolitan University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Citations (Scopus)

Abstract

In this paper, we further investigate the convergence properties of the generalized back-propagation algorithm using magnified gradient function (MGFPROP). The idea of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. The algorithm can effectively speed up the convergence rate and reduce the chance of being trapped in premature saturation. From the convergence analysis, it is shown that MGFPROP retains the gradient descent property, gives faster convergence and has better global searching capability than that of the back-propagation algorithm.

Original language	English
Title of host publication	International Joint Conference on Neural Networks 2006, IJCNN '06
Pages	3672-3678
Number of pages	7
DOIs	https://doi.org/10.1109/ijcnn.2006.247381
Publication status	Published - 2006
Event	International Joint Conference on Neural Networks 2006, IJCNN '06 - Vancouver, BC, Canada Duration: 16 Jul 2006 → 21 Jul 2006

Publication series

Name	IEEE International Conference on Neural Networks - Conference Proceedings
ISSN (Print)	1098-7576

Conference

Conference	International Joint Conference on Neural Networks 2006, IJCNN '06
Country/Territory	Canada
City	Vancouver, BC
Period	16/07/06 → 21/07/06

Access to Document

10.1109/ijcnn.2006.247381

Cite this

@inproceedings{fc7e243ccf7f4c40b2ea43bd8ec1d683,

title = "Convergence analysis of generalized back-propagation algorithm with modified gradient function",

abstract = "In this paper, we further investigate the convergence properties of the generalized back-propagation algorithm using magnified gradient function (MGFPROP). The idea of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. The algorithm can effectively speed up the convergence rate and reduce the chance of being trapped in premature saturation. From the convergence analysis, it is shown that MGFPROP retains the gradient descent property, gives faster convergence and has better global searching capability than that of the back-propagation algorithm.",

author = "Ng, \{S. C.\} and Leung, \{S. H.\} and Andrew Luk and Yunfeng Wu",

year = "2006",

doi = "10.1109/ijcnn.2006.247381",

language = "English",

isbn = "0780394909",

series = "IEEE International Conference on Neural Networks - Conference Proceedings",

pages = "3672--3678",

booktitle = "International Joint Conference on Neural Networks 2006, IJCNN '06",

note = "International Joint Conference on Neural Networks 2006, IJCNN '06 ; Conference date: 16-07-2006 Through 21-07-2006",

}

Ng, SC, Leung, SH, Luk, A & Wu, Y 2006, Convergence analysis of generalized back-propagation algorithm with modified gradient function. in International Joint Conference on Neural Networks 2006, IJCNN '06., 1716603, IEEE International Conference on Neural Networks - Conference Proceedings, pp. 3672-3678, International Joint Conference on Neural Networks 2006, IJCNN '06, Vancouver, BC, Canada, 16/07/06. https://doi.org/10.1109/ijcnn.2006.247381

Convergence analysis of generalized back-propagation algorithm with modified gradient function. / Ng, S. C.; Leung, S. H.; Luk, Andrew et al.
International Joint Conference on Neural Networks 2006, IJCNN '06. 2006. p. 3672-3678 1716603 (IEEE International Conference on Neural Networks - Conference Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Convergence analysis of generalized back-propagation algorithm with modified gradient function

AU - Ng, S. C.

AU - Leung, S. H.

AU - Luk, Andrew

AU - Wu, Yunfeng

PY - 2006

Y1 - 2006

N2 - In this paper, we further investigate the convergence properties of the generalized back-propagation algorithm using magnified gradient function (MGFPROP). The idea of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. The algorithm can effectively speed up the convergence rate and reduce the chance of being trapped in premature saturation. From the convergence analysis, it is shown that MGFPROP retains the gradient descent property, gives faster convergence and has better global searching capability than that of the back-propagation algorithm.

AB - In this paper, we further investigate the convergence properties of the generalized back-propagation algorithm using magnified gradient function (MGFPROP). The idea of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. The algorithm can effectively speed up the convergence rate and reduce the chance of being trapped in premature saturation. From the convergence analysis, it is shown that MGFPROP retains the gradient descent property, gives faster convergence and has better global searching capability than that of the back-propagation algorithm.

UR - https://www.scopus.com/pages/publications/40649083079

U2 - 10.1109/ijcnn.2006.247381

DO - 10.1109/ijcnn.2006.247381

M3 - Conference contribution

AN - SCOPUS:40649083079

SN - 0780394909

SN - 9780780394902

T3 - IEEE International Conference on Neural Networks - Conference Proceedings

SP - 3672

EP - 3678

BT - International Joint Conference on Neural Networks 2006, IJCNN '06

T2 - International Joint Conference on Neural Networks 2006, IJCNN '06

Y2 - 16 July 2006 through 21 July 2006

ER -

Convergence analysis of generalized back-propagation algorithm with modified gradient function

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this