Spurious #5: Variance of Autocorrelated Processes

What is the standard deviation (variance) of an autocorrelated series? Sounds like an easy question, but it isn’t. This issue turns out to affect the spurious regression problem, so I’m posting up a short note on the problem. These issues are well-known in econometrics, where they have led to “heteroskedastic-autocorrelation consistent” estimators. There’s an interesting discussion in Percival [1993], who’s involved in climate, who says:

the effect of not knowning the process mean can be rather large: whereas s is an unbiased estimator of à?à’ when µ is known, the commonly used s can severely underestimate à?à’ when µ is unknown.

Percival gives an example where small-sample methods are biased by a factor of 100!

The caption to Figure 2 of Percival [1993] reports that the variance of an autoregressive process with à?à=0.997 is 166.9. This can be derived as follows.

Percival illustrates the very serious finite sample problems of variance estimation in autoregressive processes when you don’t know the true mean as follows in connection with his Figure 2 (s being the estimate of the standard deviation and à?à’ being the true standard deviation of the process (notation slightly different for typology reasons):

For sample size N = 10, we have E{sà’¹”¬➲/à?à’^2} = 0.01; i.e., the sample variance is biased by a factor of 100. The two thin vertical lines enclose one such sample of 10 points, for which s^2= 0.7 (well below à?à’^2 = 166.9). If we sweep across the time series plotted above and compute s^2 for all possible 991 subseries of size 10, we find that s^2 varies from 0.1 to 17.1 and has an average value of 1.8 (again well below the true variance). On the other hand, if we compute s^2 based upon the known process mean of zero, we find that s^2 varies from 0.6 to 764.0 and has an average value of 184.1, which is rather close to à?à’^2= 166.9.

Percival reports a short theorem stating that the situation can get arbitrarily bad, as follows:

For every sample size N à⣃¢’¬°à⣠1 and every àÅ½àⳠ> 0, there exists a stationary process such that E{sà’¹”¬➲/à?à’^2} < àÅ½àⳠ.

I’m going to come back to these issues when I discuss why “scaling” autoregressive processes in a short calibration period (as recently argued by Hockey Team members Briffa and Esper) does not avoid the need to examine statistical issues and is thereby not a magic bullet. You can also picture why a certain amount of care needs to be taken if you are standardizing by dividing by calibration-period standard deviation or even calculating correlations (which include standard deviation terms). Simply transposing techniques from independent samples can be a dangerous recipe.

3 Comments

I always become suspicious when complex statistical techniques are needed to extract “facts” from data that on first examination are rather uniformative – almost as if the a theory was proposed and the data then massaged to support the theory.

As for your initial question, I just wonder whether these issues arise because statistics are being extended to problems that can’t be subject to this type of analysis in the first place.