Tools

"... We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/ sampling importance resampling algorithm (HySIR). In terms of computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequentia ..."

We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/ sampling importance resampling algorithm (HySIR). In terms of computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimization strategy that allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear, and nongaussian signal processing. We show how the new algorithm outperforms extended Kalman filter training on several problems. In particular, we address the problem of pricing option contracts, traded in financial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices.

"... In this paper, we show that a hierarchical Bayesian modelling approach to sequential learning leads to many interesting attributes such as regularisation and automatic relevance determination. We identify three inference levels within this hierarchy, namely model selection, parameter estimation and ..."

In this paper, we show that a hierarchical Bayesian modelling approach to sequential learning leads to many interesting attributes such as regularisation and automatic relevance determination. We identify three inference levels within this hierarchy, namely model selection, parameter estimation and noise estimation. In environments where data arrives sequentially, techniques such as cross-validation to achieve regularisation or model selection are not possible. The Bayesian approach, with extended Kalman filtering at the parameter estimation level, allows for regularisation within a minimum variance framework. A multi-layer perceptron is used to generate the extended Kalman filter nonlinear measurements mapping. We describe several algorithms at the noise estimation level, which allow us to implement adaptive regularisation and automatic relevance determination of model inputs and basis functions. An important contribution of this paper is to show the theoretical links between adaptive...

"... We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of both computational time and accuracy, the hybrid SIR is a clear improvement over conventional seque ..."

We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of both computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimisation strategy, which allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear and non-Gaussian signal processing. We show how the new algorithm outperforms extended Kalman filter training on several problems. In particular, we address the problem of pricing option contracts, traded in financial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices.

"... In this paper, we derive an algorithm to train support vector machines sequentially. The algorithm makes use of the Kalman filter and is optimal in a minimum variance framework. It extends the support vector machine paradigm to applications involving real-time and non-stationary signal processing. I ..."

In this paper, we derive an algorithm to train support vector machines sequentially. The algorithm makes use of the Kalman filter and is optimal in a minimum variance framework. It extends the support vector machine paradigm to applications involving real-time and non-stationary signal processing. It also provides a computationally efficient alternative to the problem of quadratic optimisation.

"... The importance of volatility modeling is evidenced by the voluminous literature on temporal dependencies in financial market assets. A substantial body of this literature relies on explorations of daily and lower frequencies using parametric ARCH or stochastic volatility models. In this research we ..."

The importance of volatility modeling is evidenced by the voluminous literature on temporal dependencies in financial market assets. A substantial body of this literature relies on explorations of daily and lower frequencies using parametric ARCH or stochastic volatility models. In this research we compare the model performance of alternate neural network models against that of the (G)ARCH framework as applied to hourly volatility behavior of currency futures options. We report that the results obtained from solving the Bayesian regularization radial basis function neural network are considerably more accurate than those produced by alternate neural network topologies and the (G)ARCH model formulation.

"... We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network's parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by d ..."

We consider sequential or online learning in dynamic neural regression models. By using a state space representation for the neural network&apos;s parameter evolution in time we obtain approximations to the unknown posterior by either deriving posterior modes via the Fisher scoring algorithm or by deriving approximate posterior means with the importance sampling method. Furthermore, we replace the commonly used Gaussian noise assumption in the neural regression model by a more flexible noise model based on the Student t-density. Since the t-density can be interpreted as being an infinite mixture of Gaussians, hyperparameters such as the degrees of freedom of the t-density can be learned from the data based on an online EM-type algorithm. We show experimentally that our novel methods outperform state-of-the art neural network online learning algorithms like the extended Kalman filter method for both, situations with standard Gaussian noise terms and situations with measurement outliers. 1 I...

"... Over the recent past, stylized facts have not yielded a synthesis regarding the predictability of returns for alternative investment assets such as hedge funds. Recent studies on alternative asset return predictability have added to the ambiguity. These studies suggest that classification prediction ..."

Over the recent past, stylized facts have not yielded a synthesis regarding the predictability of returns for alternative investment assets such as hedge funds. Recent studies on alternative asset return predictability have added to the ambiguity. These studies suggest that classification prediction methods may dominate more traditional return-level prediction methodology. This paper examines the predictive accuracy of three alternate radial basis function neural networks when applied to the returns of thirteen Credit Swiss First Boston/Tremont (CSFB) hedge fund indices. We provide evidence that the Kajiji-4 RBF neural network dominates within the RBF topology in the prediction of hedge fund returns by both level and classification. The results also show that the Kajiji-4 method is capable of near perfect directional prediction.

"... Abstract — This paper develops an unscented grid-based filter and a smoother for accurate nonlinear modeling and analysis of time series. The filter uses unscented deterministic sampling during both the time and measurement updating phases, to approximate directly the distributions of the latent sta ..."

Abstract — This paper develops an unscented grid-based filter and a smoother for accurate nonlinear modeling and analysis of time series. The filter uses unscented deterministic sampling during both the time and measurement updating phases, to approximate directly the distributions of the latent state variable. A complementary grid smoother is also made to enable computing of the likelihood. This helps us to formulate an expectation maximisation algorithm for maximum likelihood estimation of the state noise and the observation noise. Empirical investigations show that the proposed unscented grid filter/smoother compares favourably to other similar filters on nonlinear estimation tasks. I.

"... Volatility modeling is the lifeline of the derivative- and asset-pricing evaluation process. As such, it is understandable that a voluminous literature has evolved to discuss the temporal dependencies in financial market volatility. Much of this literature has been directed at daily and lower freque ..."

Volatility modeling is the lifeline of the derivative- and asset-pricing evaluation process. As such, it is understandable that a voluminous literature has evolved to discuss the temporal dependencies in financial market volatility. Much of this literature has been directed at daily and lower frequencies using ARCH and stochastic volatility type models. With access to high frequency and ultra high-frequency databases, more recent research has been able to explain about fifty percent of the interdaily forecasts of latent volatility. Relying upon hourly intervals, the GARCH(1,1) results presented here are consistent with prior studies. However, this paper adds to the tools available for conducting volatility exploration by introducing an adaptive radial basis function neural network that significantly lowers overall prediction error while maintaining a high explanatory ratio. The newly formulated RBF implements a closed-form regularization parameter with Bayesian prior information. It is an algorithmic extension that will permit more accurate and insightful analyses to be performed on high frequency financial time series. Over the past decade, research efforts increased significantly in the area of modeling volatility behavior in capital market high frequency data. Obtaining accurate

"... My companion in love and life on this long and weirdly wonderful, arduous journey. For not undertaking this journey we would not have met. For not meeting, this journey surely would have failed. Dankie vir alles! iv Acknowledgments I would like to thank the following people who all contributed in so ..."

My companion in love and life on this long and weirdly wonderful, arduous journey. For not undertaking this journey we would not have met. For not meeting, this journey surely would have failed. Dankie vir alles! iv Acknowledgments I would like to thank the following people who all contributed in some form to the success of this endevour: My professors and colleagues at OGI for their intellectual support, and for providing a congenial and stimulating academic environment. In particular, I want to thank my advisor, Eric Wan, for his time, patience, insight and knowledge imparted to me over the last five years. Special thanks also go to Todd Leen for sharing his mathematical expertise and wisdom, as well as emphasizing the importance of rigor in research. Finally, I want to thank the remaining members of my thesis committee, Misha Pavel and Dieter Fox, for their input and suggestions along the way, as well as for taking the time to review this work. My family for their love and support. Chris, Henriette, Mary, Christelle, Theron, Rykie &amp; Gary: Thank you for always believing in me. All my wonderful friends (of whom there are simply too many to mention by name 1): Thanks for your friendship and support. It kept me sane.