On modelling using radial basis function networks with structure determined by support vector regression

by Choy, Kin-yee

of thesis entitled

On Modelling Using Radial Basis Function Networks with Structure Determined by Support Vector Regression submitted by

ChoyKinYee

for the Degree of

Master of Philosophy

at

The University of Hong Kong

III

March 2004

Many practical systems are complex and nonlinear, and they cannot be treated satisfactorily using linear systems theory. Neural networks have the ability to approximate any nonlinear function with arbitrary accuracy, and are often used to model complex nonlinear systems. The Radial Basis Function network is well-known for its ability to interpolate in high-dimensional input space, but its performance depends on the choice of the number and the centres of the Radial Basis Function. To avoid the "curse of dimensionality" problem, cluster-partitioned input space is used and the centres are chosen in such a way that the Radial Basis Function will have effect mainly in certain regions of the input space. The major problem is how to select the suitable set

of centres such that the network is both relatively simple and achieves good generalization.

A recent technique to identify the centres is the Support Vector Regression algorithm. For a given error bound ? the centres are selected as the Support Vectors obtained from a constrained optimization. This class of networks is referred to as the Support Vector Radial Basis Function Networks (SVRBFNs) in this study. With sparse structure determined objectively by the Support Vector Regression algorithm, the SVRBFN should be a parsimonious model that can approximate the data with arbitrary accuracy.

The performance and the application procedure of the SVRBFN are illustrated by the modelling of the river discharges of Fuji River, the third steepest river in Japan. Since there are outliers in the modelling errors arising from the data collection process, intervention analysis is utilized to remove the outliers. The improved SVRBFN can then be employed to examine the dynamic effect of rainfall on river discharges, and to forecast river discharges for given rainfalls.