2013/08/20

Introduction

In this post, we review the sequential importance sampling-resampling for state space models. These algorithms are also known as particle filters. We give a derivation of these filters and their application to the general state space models.

2013/07/22

Introduction

In this post, we review the online maximum-likelihood parameter estimation for GARCH model which is a dynamic variance model. GARCH can be seen as a toy volatility model and used as a textbook example for financial time series modelling.

2013/06/22

Introduction.

In this post, we derive the nonnegative matrix factorization (NMF). We derive the multiplicative updates from a gradient descent point of view by using the treatment of Lee and Seung, Algorithms for Nonnegative Matrix Factorization. The code for this blogpost can be accessed from here.

2013/06/13

Introduction.

In this post, we give the definitions of sample space, probability measure, random variable. We give these definitions on a very simple example of the space of two coin tosses. Note that definitions in this note are for finite probability spaces and the example simplifies everything significantly. This note is mostly based on the Shreve's Stochastic Calculus for Finance, vol. I, Chapter 2 and vol. II, Chapter 2.

2013/05/23

17/01/2017 update: While searching for something else, I came across with my old blogpost on stochastic gradient descent (SGD) dated back to 23/05/2013. I found it a bit low-level and little informative (this, in fact, is true for most posts from that year). Despite there have been many great posts published on SGD since then, I still wanted to update the version in this blog. So I decided to rewrite it from scratch.

2013/05/20

Introduction.

In this post, we show the relationship between Gaussian observation model, Least-squares and pseudoinverse. We start with a Gaussian observation model and then move to the least-squares estimation. Then we show that the solution of the least-squares corresponds to the pseudoinverse operation.

2013/05/03

Introduction

These notes are mostly based on the book Stochastic Calculus for Finance vol. II, Chapter 4. I give a few propositions and focus on exercises of Shreve by make use of the Ito-Doeblin formula. The use of Ito-Doeblin formula is almost purely practical to solve continuous-time stochastic models. My treatment is slightly different from the Shreve since I emphasize on the differential forms of the formulas.