Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

Chi Chung Cheung; Sin Chun Ng; Andrew Kwok Fai Lui

doi:10.1007/978-3-642-31346-2_65

Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

Chi Chung Cheung, Sin Chun Ng, Andrew Kwok Fai Lui

Hong Kong Metropolitan University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Backpropagation (BP) learning algorithm is the most widely supervised learning technique which is extensively applied in the training of multi-layer feed-forward neural networks. Many modifications of BP have been proposed to speed up the learning of the original BP. However, they all have different drawbacks and they cannot perform very well in all kinds of applications. This paper proposes a new algorithm, which provides a systematic approach to make use of the characteristics of different fast learning algorithms so that the learning process can converge to the global minimum. During the training, different fast learning algorithms will be used in different phases to improve the global convergence capability. Our performance investigation shows that the proposed algorithm always converges in different benchmarking problems (applications) whereas other popular fast learning algorithms sometimes give very poor global convergence capabilities.

Original language	English
Title of host publication	Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings
Pages	580-589
Number of pages	10
Edition	PART 1
DOIs	https://doi.org/10.1007/978-3-642-31346-2_65
Publication status	Published - 2012
Event	9th International Symposium on Neural Networks, ISNN 2012 - Shenyang, China Duration: 11 Jul 2012 → 14 Jul 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 1
Volume	7367 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Symposium on Neural Networks, ISNN 2012
Country/Territory	China
City	Shenyang
Period	11/07/12 → 14/07/12

Keywords

backpropagation
local minimum
multi-phase learning algorithms

Access to Document

10.1007/978-3-642-31346-2_65

Cite this

Cheung, C. C., Ng, S. C., & Lui, A. K. F. (2012). Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks. In Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings (PART 1 ed., pp. 580-589). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7367 LNCS, No. PART 1). https://doi.org/10.1007/978-3-642-31346-2_65

Cheung, Chi Chung ; Ng, Sin Chun ; Lui, Andrew Kwok Fai. / Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks. Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings. PART 1. ed. 2012. pp. 580-589 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1).

@inproceedings{c8b7aa1c77744f52bd736ee4449bcece,

title = "Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks",

abstract = "Backpropagation (BP) learning algorithm is the most widely supervised learning technique which is extensively applied in the training of multi-layer feed-forward neural networks. Many modifications of BP have been proposed to speed up the learning of the original BP. However, they all have different drawbacks and they cannot perform very well in all kinds of applications. This paper proposes a new algorithm, which provides a systematic approach to make use of the characteristics of different fast learning algorithms so that the learning process can converge to the global minimum. During the training, different fast learning algorithms will be used in different phases to improve the global convergence capability. Our performance investigation shows that the proposed algorithm always converges in different benchmarking problems (applications) whereas other popular fast learning algorithms sometimes give very poor global convergence capabilities.",

keywords = "backpropagation, local minimum, multi-phase learning algorithms",

author = "Cheung, {Chi Chung} and Ng, {Sin Chun} and Lui, {Andrew Kwok Fai}",

year = "2012",

doi = "10.1007/978-3-642-31346-2_65",

language = "English",

isbn = "9783642313455",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 1",

pages = "580--589",

booktitle = "Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings",

edition = "PART 1",

note = "9th International Symposium on Neural Networks, ISNN 2012 ; Conference date: 11-07-2012 Through 14-07-2012",

}

Cheung, CC, Ng, SC & Lui, AKF 2012, Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks. in Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings. PART 1 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 1, vol. 7367 LNCS, pp. 580-589, 9th International Symposium on Neural Networks, ISNN 2012, Shenyang, China, 11/07/12. https://doi.org/10.1007/978-3-642-31346-2_65

Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks. / Cheung, Chi Chung; Ng, Sin Chun; Lui, Andrew Kwok Fai.
Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings. PART 1. ed. 2012. p. 580-589 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7367 LNCS, No. PART 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

AU - Cheung, Chi Chung

AU - Ng, Sin Chun

AU - Lui, Andrew Kwok Fai

PY - 2012

Y1 - 2012

N2 - Backpropagation (BP) learning algorithm is the most widely supervised learning technique which is extensively applied in the training of multi-layer feed-forward neural networks. Many modifications of BP have been proposed to speed up the learning of the original BP. However, they all have different drawbacks and they cannot perform very well in all kinds of applications. This paper proposes a new algorithm, which provides a systematic approach to make use of the characteristics of different fast learning algorithms so that the learning process can converge to the global minimum. During the training, different fast learning algorithms will be used in different phases to improve the global convergence capability. Our performance investigation shows that the proposed algorithm always converges in different benchmarking problems (applications) whereas other popular fast learning algorithms sometimes give very poor global convergence capabilities.

AB - Backpropagation (BP) learning algorithm is the most widely supervised learning technique which is extensively applied in the training of multi-layer feed-forward neural networks. Many modifications of BP have been proposed to speed up the learning of the original BP. However, they all have different drawbacks and they cannot perform very well in all kinds of applications. This paper proposes a new algorithm, which provides a systematic approach to make use of the characteristics of different fast learning algorithms so that the learning process can converge to the global minimum. During the training, different fast learning algorithms will be used in different phases to improve the global convergence capability. Our performance investigation shows that the proposed algorithm always converges in different benchmarking problems (applications) whereas other popular fast learning algorithms sometimes give very poor global convergence capabilities.

KW - backpropagation

KW - local minimum

KW - multi-phase learning algorithms

UR - http://www.scopus.com/inward/record.url?scp=84865146290&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-31346-2_65

DO - 10.1007/978-3-642-31346-2_65

M3 - Conference contribution

AN - SCOPUS:84865146290

SN - 9783642313455

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 580

EP - 589

BT - Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings

T2 - 9th International Symposium on Neural Networks, ISNN 2012

Y2 - 11 July 2012 through 14 July 2012

ER -

Cheung CC, Ng SC, Lui AKF. Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks. In Advances in Neural Networks, ISNN 2012 - 9th International Symposium on Neural Networks, Proceedings. PART 1 ed. 2012. p. 580-589. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 1). doi: 10.1007/978-3-642-31346-2_65

Multi-phase fast learning algorithms for solving the local minimum problem in feed-forward neural networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this