Help With Evaulating Intra-Day Trading Systems

So after I check the curve fit parameters (from testing on 3 year data) on the out of sample (e.g. 7 years) data, and the results are bad, does the mean the curve-fit parameters from 3 year data was over fitted? Just trying to get the terminology correct.

The drawdown is defined as the difference between your current account value and the account's all time high. Often is this expressed as a percentage. So if you have a drawdown of 20% it means that your current account value is 20% lower than the highest value it ever reached.

The drawdown is defined as the difference between your current account value and the account's all time high. Often is this expressed as a percentage. So if you have a drawdown of 20% it means that your current account value is 20% lower than the highest value it ever reached.

Trying to read between the lines of your post above, I suspect that Ralph Vince's excellent book The Mathematics of Money Management: Risk Analysis Techniques for Traders will perhaps be particularly helpful to you (and he posts here sometimes).

More...

I try to read most threads here and on other venues, I'm just often several days behind. If I don't respond, oftentimes it's because I jsut don't want to get drawn into an argument, it's exhausting to me and the markets are exhausting enough to an old mutt like me. But this is a very interesting, very thought provoking thread.

For a long time, I sought this as being the spectrum of parameters, with respect to a criteria (be it Sharpe ratio, drawdown, whatever) as being ideally, a jagged, uniform distribution, and worst-case, a very peaked, or multi, sharp-peaked distribution shape across parameter values.

I'm reminded in this type of thinking however, i.e. robustness as an indication of the reliability of future profitability of the Second Arc Sine Law (2AL for short), which pertains to random walks. (And in terms of trading, I prefer to think of price as a sort of perverted random walk, one that is constantly trying to fake me out, with all kinds of overlays of trend, cycle, seasonality, surprises mixed in, to further try to fake me out. I think it;s a good starting point in thinking of price).

The 2AL tells us that for the equity curve of a random walk (and a trading program's equity curve is simply prices, transformed by the trading rules), wheres we would expect the extreme of high and low in the equity curve to appear towards the center, this is the least likely place for these extremes to occur. In fact, the most likely points for the high or low of an equity curve comprised of random-walk data is towards the endpoints.

So it is very likely, in looking at the equity curve of a random walk, for the low pint to occur at the far left point in time and high point to occur at the far right endpoint ("Yes, this is a very profitable system") or vice versa ("This system is so bad, I wonder if I could flip it upside down?").

But 2AL is a natural law occurring on random data. It's very insidious, and I think, based on personal experience, visits us routinely.

Over the years, I finally came to the conclusion that the only type of trading approach I could count on that would work in the future (and "the future" is something which, however you define it, you should multiply that time by, say, 5, if you want to be a realist) was something that I could see, intuitively, had to work. It wouldn't require any backtesting (though I would backtest it anyhow. On one occasion, with my primary system, Dow Jones Indexes wanted to license it to use on their various indexes. It was a interesting conversation where I explained to them that, despite being a programmer, I had never backtested it but had traded it for over a decade to that time. Of course, they put together the crew to backtest it, as they were entirely thorough.)

Take, for example, a simple trend-following system of a moving average crossover (yes, there are better trend following mechanisms, but just using this to illustrate the point). Now you know, intuitively, it is going to make money, it;s just a matter of time. And yes, you could optimize it all up for the optimal MA length over a given time window with respect to a given criterion, etc., and all the various rubrics that go with all that. Nevertheless, you know a moving average on price will make money. It's simply a matter of time and nerve.

And that's exactly it - "Time and nerve," that distinguishes these types of reliable approaches. There is no "luck," or "if the markets behave as they have," or other dependencies. I know of several such approaches - they are out there, where you can be certain they will work if you apply the necessary "time & nerve."

And it is exhausting.

The problem with "time" is that, the size of the expected drawdown approaches 100% as the length of time an approach is traded gets ever-greater. This is as much a natural fact as the 2AL. (A recent health issue drove me to conjure the expression, "If you live long enough, you get to experience everything. Twice." And given that, I thank God I wont live long enough to experience certain things!).

Drawdown is something I prefer to truncate at a certain percentage with certainty by using a hedge. The construction of proper hedges is a very rich topic, often overlooked, but as expressed in this thread, is the difference between life and death in trading. I like to truncate things worst-case.

But a hedge allows you to do a lot more. Not only does it allow you to proscribe your risk, but it's possible to bookend performance such that if things go against you far enough, your drawdown is less, and, in some cases, if you get very good at your hedge construction (and maintenance) becomes profitable with a market move that otherwise would have been terminal.

Here is an example of the risk profile of one of the hedge funds I manage created by hedge:

Looking at SPY as a proxy for large-cap stocks here,which comprise the portfolio. The red line shows the drawdown with respect to moves downward with this SPY as proxy. The hedge, however, allows me to prosribe risk on the portfolio to about 13% around SPY 177 (about 33% drawdown otherwise). The extreme curvature appears to the far right, however unrealistic it may be by some people's standards), but the point is, at SPY 176 and lower, the drawdown gets ever less than 13%.

All hedges cost money, detracting from performance. But as is visible on the enclosed graph, it helps on the downside, and diminishes return to a degree on the upside. It is a price well-worth it however, as the end result is not only survival, but a diminishment in variance - a broad topic altogether, and a criteria to be pursued by all serious traders.

I try to read most threads here and on other venues, I'm just often several days behind. If I don't respond, oftentimes it's because I jsut don't want to get drawn into an argument, it's exhausting to me and the markets are exhausting enough to an old mutt like me. But this is a very interesting, very thought provoking thread.

For a long time, I sought this as being the spectrum of parameters, with respect to a criteria (be it Sharpe ratio, drawdown, whatever) as being ideally, a jagged, uniform distribution, and worst-case, a very peaked, or multi, sharp-peaked distribution shape across parameter values.

I'm reminded in this type of thinking however, i.e. robustness as an indication of the reliability of future profitability of the Second Arc Sine Law (2AL for short), which pertains to random walks. (And in terms of trading, I prefer to think of price as a sort of perverted random walk, one that is constantly trying to fake me out, with all kinds of overlays of trend, cycle, seasonality, surprises mixed in, to further try to fake me out. I think it;s a good starting point in thinking of price).

The 2AL tells us that for the equity curve of a random walk (and a trading program's equity curve is simply prices, transformed by the trading rules), wheres we would expect the extreme of high and low in the equity curve to appear towards the center, this is the least likely place for these extremes to occur. In fact, the most likely points for the high or low of an equity curve comprised of random-walk data is towards the endpoints.

So it is very likely, in looking at the equity curve of a random walk, for the low pint to occur at the far left point in time and high point to occur at the far right endpoint ("Yes, this is a very profitable system") or vice versa ("This system is so bad, I wonder if I could flip it upside down?").

But 2AL is a natural law occurring on random data. It's very insidious, and I think, based on personal experience, visits us routinely.

Over the years, I finally came to the conclusion that the only type of trading approach I could count on that would work in the future (and "the future" is something which, however you define it, you should multiply that time by, say, 5, if you want to be a realist) was something that I could see, intuitively, had to work. It wouldn't require any backtesting (though I would backtest it anyhow. On one occasion, with my primary system, Dow Jones Indexes wanted to license it to use on their various indexes. It was a interesting conversation where I explained to them that, despite being a programmer, I had never backtested it but had traded it for over a decade to that time. Of course, they put together the crew to backtest it, as they were entirely thorough.)

Take, for example, a simple trend-following system of a moving average crossover (yes, there are better trend following mechanisms, but just using this to illustrate the point). Now you know, intuitively, it is going to make money, it;s just a matter of time. And yes, you could optimize it all up for the optimal MA length over a given time window with respect to a given criterion, etc., and all the various rubrics that go with all that. Nevertheless, you know a moving average on price will make money. It's simply a matter of time and nerve.

And that's exactly it - "Time and nerve," that distinguishes these types of reliable approaches. There is no "luck," or "if the markets behave as they have," or other dependencies. I know of several such approaches - they are out there, where you can be certain they will work if you apply the necessary "time & nerve."

And it is exhausting.

The problem with "time" is that, the size of the expected drawdown approaches 100% as the length of time an approach is traded gets ever-greater. This is as much a natural fact as the 2AL. (A recent health issue drove me to conjure the expression, "If you live long enough, you get to experience everything. Twice." And given that, I thank God I wont live long enough to experience certain things!).

Drawdown is something I prefer to truncate at a certain percentage with certainty by using a hedge. The construction of proper hedges is a very rich topic, often overlooked, but as expressed in this thread, is the difference between life and death in trading. I like to truncate things worst-case.

But a hedge allows you to do a lot more. Not only does it allow you to proscribe your risk, but it's possible to bookend performance such that if things go against you far enough, your drawdown is less, and, in some cases, if you get very good at your hedge construction (and maintenance) becomes profitable with a market move that otherwise would have been terminal.

Here is an example of the risk profile of one of the hedge funds I manage created by hedge:

Looking at SPY as a proxy for large-cap stocks here,which comprise the portfolio. The red line shows the drawdown with respect to moves downward with this SPY as proxy. The hedge, however, allows me to prosribe risk on the portfolio to about 13% around SPY 177 (about 33% drawdown otherwise). The extreme curvature appears to the far right, however unrealistic it may be by some people's standards), but the point is, at SPY 176 and lower, the drawdown gets ever less than 13%.

All hedges cost money, detracting from performance. But as is visible on the enclosed graph, it helps on the downside, and diminishes return to a degree on the upside. It is a price well-worth it however, as the end result is not only survival, but a diminishment in variance - a broad topic altogether, and a criteria to be pursued by all serious traders.

More...

Hello rvince99,

Thank you very much for contributing to thread. I read your comments two times and appreciate your well written story and advice.