Recursions for the Moments of Some Continuous Distributions

What I'll be discussing here are some useful recursion formulae for computing the moments of a number of continuous distributions that are widely used in econometrics. The coverage won't be exhaustive, by any means. I provide some motivation for looking at formulae such as these in the previous post, so I won't repeat it here.

When we deal with the Normal distribution, below, we'll make explicit use of Stein's Lemma. Several of the other results are derived (behind the scenes) by using a very similar approach. So, let's begin by stating this Lemma.Stein's Lemma (Stein, 1973):

"If X ~ N[θ , σ2], and if g(.) is a differentiable function such that E|g'(X)| is finite, then

E[g(X)(X - θ)] = σ2 E[g'(X)]."

It's worth noting that although this lemma relates to a single Normal random variable, in the bivariate Normal case the lemma generalizes to:

"If Xand Y follow a bivariate Normal distribution, and if g(.) is a differentiable function such that E|g'(Y)| is finite, then

Cov.[g(Y ), X] = Cov.(X , Y) E[g'(Y)]."

In this latter form, the lemma is useful in asset pricing models.There are extensions of Stein's Lemma to a broader class univariate and multivariate distributions. For example, see Alghalith (undated), and Landsman et al. (2013), and the references in those papers. Generally, if a distribution belongs to an exponential family, then recursions for its moments can be obtained quite easily.

Now, let's get down to business............

Recall that the rth. "raw moment" (or "moment about zero") for the random variable X, with distribution function F,is defined as μr' = E[X r] = ∫ xrdF(x) , for r = 1, 2, .....; and the "central (centered) moments" of X are defined as μr = E[(X - μ1' )r] , for r = 1, 2, 3, .......
Also, we can express one of these sets of moments in terms of the other set by using the two relationships:μr = Σ{[r! / (i! (r - i)!)] μi' (- μ1' ) r-i } , (1)where the range of summation is from i = 0 to i = r ; andμr' = Σ{[r! / (i! (r - i)!)] μi(μ1' ) r-i } , (2)where, again, the range of summation is from i = 0 to i = r.Normal distribution

We'll use the following standard result for the Chi-square distribution with "v" degrees of freedom:"For any real-valued function, h, E[h(χ2v)] = v E[h(χ2v+2) / χ2v+2], provided that the expectations exist."

(This result can be generalized to the case of a non-central Chi-square distribution. See Appendix B of Judge and Bock (1978).

Let h(x) = xk, for some integer, k. Then, applying the above result repeatedly, we get:

I'll bet that you can see right away what the expressions are for μ4', μ5', etc.!In terms of a genuine recurrence relationship, we see from the above that we can write:

μr' = μr-1' (v + 2(r -1)) ; r = 1, 2, 3, ........

Although we can then use equation (1) to obtain the central moments of X, there's also a separate recursion formula for these moments in the case of the Chi-square distribution This is discussed in the section on the Gamma distribution, below.

Student-t distribution

Suppose that X follows a Student-t distribution, with v degrees of freedom. Then the moments of X can be summarized as follows:μr' = 0 ; if r is odd, and 0 < r < vμr' = v(r/2) Π[(2i - 1) / (v - 2i)] ; if r is even, and 0 < r < v (3)where the product is for i = 1 to i = (r / 2).The inequality, v > r, ensures that the moment "exists" - that is, it is finite. If v = 1, then X follows a Cauchy distribution, and none of its moments "exist".

This recurrence relationship avoids us having to deal with the gamma functions in (4), let alone having to perform any integration, or deal with this distribution's messy characteristic function. (As with the Student-t distribution, the moment generating function isn't defined here, because of the above conditions on the existence of the moments.)
The distributions that we've considered so far are ones that you use every day in your econometrics work. Now let's consider a couple more distributions that arise a bit less frequently.

Gamma distribution

One situation where the Gamma distribution comes up in econometrics is in the context of "count data". This might seem a bit odd, because count data are non-negative integers, and we're talking about continuous random variables here.

However, the Gamma distribution comes into play when we generalize the Poisson distribution to a particular form of the Negative Binomial distribution. Both of these distributions were discussed in my previous, related, post. Looking back at that post, you'll be able to see that the variance of a Poisson random variable equals its mean. We say that the distribution is "equi-dispersed". This is very restrictive, and isn't realistic with a lot of count data in practice. On the other hand, the variance of the Negative Binomial distribution exceeds its mean. We say that this distribution is "over-dispersed", and in practice this is often more reasonable.

To construct the Negative Binomial distribution in the form that we usually use it in econometrics, we take a Poisson random variable and then add in an unobserved random effect to its (conditional) mean. If this random effect follows a Gamma distribution, we end up with the Negative Binomial distribution for the count data. (See Greene, 2012, pp.806-807 for details.)

With this by way of motivation, consider a random variable, X that follows a Gamma distribution with a shape parameter 'a', and a scale parameter 'b'. (Be careful here - there are two forms of the Gamma distribution. The other one has the shape parameter 'a' and the rate parameter, θ = 1 / b.)

Willink (2003) shows that the central moments (moments about the mean) for X can be obtained from the following recursion -

μr = (r - 1)(bμr-1 + ab2μr-2) ; r = 2, 3, ..........

There are two special cases of the Gamma distribution that we might note. First, if a = (v / 2), and b = 2, then X follows a Chi-square distribution with v degrees of freedom. So, the central moments of the Chi-square distribution follow the recursion relationship:

μr = 2(r - 1)(μr-1 + vμr-2) ;r = 2, 3, ..........

Second, if a = b = 1, then X follows an Exponential distribution, and its central moments satisfy:

μr = (r - 1)(μr-1 + μr-2) ; r = 2, 3, .........

In addition, Withers (1992) shows that for the Exponential distribution we have the following simpler recursion:

Remember, the whole point abut these recursion formulae is they help us to rapidly compute all of the moments of a distribution, up to some pre-specified maximum order, using one general formula.

There is an R script file on the code page for this blog that illustrates this point, first for X ~ N[θ, σ2]; and second for X ~ F[v1,v2]. In the first case the first ten raw moments for X when θ = 1 and σ2 = 4 are: 1, 5, 13, 73, 281, 1741, 8485, 57233, 328753, 2389141, ....