wikidot css and coding solutions since 2008

On this page we are trying to get Firefox to print the entire page and not truncate it

Created by Carl Schmertmann on 8 Sep 2010

This is an attempt to write a generic notation for Bayesian model fitting with least-squares fitting and priors, as in Girosi and King.

In this special framework:

the likelihood L(Parameters | Data) is normal

the posterior likelihood Post(Parameters | Data) is normal

there is a closed-form analytical solution for the posterior mode

the posterior mode and mean are identical

PROBLEM

We have a vector of data $y \in \Re^K$, and a vector of parameters $\theta \in \Re^N$, where N > K in the case of a forecast. We want to find $\theta$ that fits y, and satisfies a set of P priors.

Fit

$\theta$ fits the data y when $\Sigma^{-1/2}(G\theta - y)$ is "small".1G is KxN, so there are K fitting objectives.

As an example, $\theta$ might be a complete set of N (true) past+future fertility rates, y might be the set of K (observed) past rates, and $G\,\theta$ might the subset of $\theta$ that refers to the past.

where $C_1 \equiv \sum \tau_p\,A_p^\prime\,D_p^\prime \,W_p\, D_p\,A_p$ and $c_2 \equiv \sum \tau_p\,A_p^\prime\,D_p^\prime \,W_p\, D_p\,b_p$ effectively summarize what we need to use from all the priors.

This allows easily sampling from the log posterior — for example, to calculate confidence intervals for one or more components of $\theta$.

Footnotes

1. (Josh asks: do you really mean 1/sqrt … seems backwards to me)
Carl answers: Yes, I think the power should be negative: a smaller std deviation for one of the fitting rules should lead to a bigger (not smaller) penalty for missing that target. This is the matrix equiv of $\frac{(x-\mu)}{\sigma}$