Thursday, December 18, 2008

Benford's Law Catches Madoff (error!)

Benford's law, also called the first-digit law, states that in lists of numbers from many real-life sources of data, the leading digit is distributed in a specific, non-uniform way. According to this law, the first digit is 1 almost one third of the time, and larger digits occur as the leading digit with lower and lower frequency, to the point where 9 as a first digit occurs less than one time in twenty. Hal Varian noted this could be used to detect accounting fraud way back in 1972. Here are the proportions of the leading digit according to Benford's law compared to Madoff's numbers (sample size, 171):

Earlier I reported there were errors in digits 2 and 3. Someone noted Paul Kedrosky did a similar analysis and got different results, so I redid the analysis, and got different results! My bad. I do note that the number of 1's is now outside of 2 stds. Out of 173 data points (12 years plus 5 months minus two zeros), I have 70 observations with leading 1's. But, if the mean return is really 0.96 with minimal vol, one could say this naturally violates Benford's law.

9 comments:

The SEC could automate the law, but eventually the cheaters would catch on and fake the frequency of their bogus digits. Benford's law can be extended to pairs of digits and higher orders to give even more tests. I once proposed such an approach for faked scientific cheating and I got resistance even from scientists.

The problem with selling this approach is that it seems too esoteric. The SEC consists mostly of lawyers who a notoriously non-quantitative.

Naively, you have a bunch of draws, so the odds aren't terrible than one will be an outlier. The draws are related, so once you have an outlier, the odds of another outlier are higher than in an independent draws.