you probably just need to understand what you are doing, every statistical test is different and you need a good understanding of the underlying assumptions, judging by your open ended questions you might be better off reading some statistics or probability books
–
pyCthonSep 28 '12 at 18:32

I honestly think (and i know that obviously many people like this question) that this question is out of scope (refering to the faq-section "questions i should not ask"). In my time as mathematician i have come across so many statistical tests (and i am not even a statistician) which are always at least somehow tied to their application. I mean there are literally thousands of them and imho it would be useless effort to pot them together in an encyclopedia. Why? If you know some statsitics you can develop a test yourself or look into the specifics if needed. A cookbook does more harm than good
–
vanguard2kOct 2 '12 at 6:28

6 Answers
6

I believe this is very difficult to do because of the different nature of statistical tests. Some of them are used to test the assumption of normality, some of them allow you to compare the volatility of different samples, some of them allow you to determine the suitability of a specific model.

Essentially you will find the basic ones on any good statistical book with a hypothesis testing chapter. For the more advanced one, you will need an introduction to the topic first which require ... a book by itself.

@DangerMouse "Subset Selection in Regression" hm... as far as a major online distributor is concerned the book is not the cheapest of its kind. Do you happen to have a book review for this one available?
–
vanguard2kOct 2 '12 at 8:58

@vanguard2k Not sure I understand your question. The ones on the .com site of a major online distributor are pretty accurate. It is a fairly expensive book, but at least that stops everyone reading it. I came accross the book via a personal recommendation.
–
DangerMouseOct 3 '12 at 14:34

@DangerMouse I dont doubt that its a great book I just thought it would be great to read a book review before ordering it. (And I almost surely will) Just to see if it is too theoretical or too "shallow" material.
–
vanguard2kOct 3 '12 at 15:17

It's very mathematical - you need good level of linear algebra (regression, QR decomposition etc). If you feel comfortable with Hamilton or ts analysis books then you should be okay. He talks about the maths as what to do in solving the problems and doesn't explain the maths in terms of teaching linear algebra. You may find some online excepts if you google carefully.
–
DangerMouseOct 4 '12 at 9:16

While the first quarter of the book is spent establishing the distinction between chart eye-ballers and objective verifiable TA, the latter portion of the book is an absolute goldmine as it starts off with the basics of statistical inference and then covers the monte-carlo permutation method and bootstrap tests extensively. In the end, numerous technical indicators were tested on the S&P and none of them exhibited statistically significant predictive power, although the author did mention that combinations of such indicators could be lucrative.

If you run enough tests, you are guaranteed to find something that "works" in a backtest. The problem is.....Does it work in real time trading? If not, then you were "fooled by the backtest". If it does work, the next question is.....For how long? Markets evolve 100% of the time. So, when the thing that "works" eventually dies out, was it really there or were you just lucky?

The bottom line is.....Is it reasonable to assume that what you're looking for is predictable? If not, then why waste your time. If so, then what advantage do you have over others in exploiting that predictability?

I'm also looking for something like this, a book with setups and examples of statistical tests specifically for trading strategies. I picked up "Probability and Statistics for Finance" (Frank Fabozzi Series). I'm still reading it but I have not yet encountered an example of an application to an actual trading strategy/backtest. However, it's relevance and examples in general finance does help give it a context for the statistics/probability concepts described.