Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.

Dean Hoffman wrote:The one stat that I would be vitally interested in would be the average daily change percentage. Larger daily volatilities (percentage changes) foretell of bigger potential risks in my opinion.

I think this metric is one of the most important and simultaneously underused.

I couldn't agree more with this insight Dean!!!! The way I like to think about it is "net profit per night per contract". When we convert any and all trading systems to this metric they all become comparable. An LTTF system that goes long Jan 1 and exits Dec 31 a year later can be compared to a short term swing system on this basis, if you convert the LTTF to daily fluctuations. All other "metrics" (PF, % profit, etc.) can then be recomputed on this basis. This all gets to a very important point that Bob Buran came up with in the early 1990s called "margin efficiency" so it is not my idea - but I think it is a highly overlooked idea. "Margin Efficiency" is the idea that: who wants to tie up their equity (through margin requirments) on a dull market. I for one don't want to be sitting in a trade for months on end when it is going sideways. The way to measure a system that has you doing this is through margin efficiency which you can derive once you convert a system to daily fluctuations - as Dean alludes to above.

I also believe that understanding and measuring daily fluctuation is important. Last year I began to closely track daily returns and the standard deviation of these returns â€“ even though my overall program (4 systems) does not include any â€œday-tradingâ€

In an earlier post, Dean Hoffman wrote:I may be a bit sensitive, but Iâ€™m fairly certain there was a bit of passive aggressive criticism lodged that would not have occurred had I presented all these results in a Trading Blox format (or on a Mechanica forum). Anyway, I wish great trading success to you all!

I was admittedly fairly strong in my wording above. Before I posted I asked myself if I would have written the same things had Dean said that he thought Trading Blox was the better product. I think I would have. I don't consider Mechanica to be a competitor as I think anyone who can afford it generally would buy Trading Blox anyway just to evaluate it.

I believe that the advice Dean gave was dangerous and I strongly disagreed with it, and since Dean is someone who many might listen to, I felt like I had to respond in a manner that would cause the casual reader to realize that I considered this no light matter. This might cause the reader to make his own evaluation of the merits of the argument. I might not have been so harsh had this been written by someone without a reputation or public voice however.

In all this, I am reminded of the debates between the economists Thomas Malthus and David Ricardo who often disagreed sharply in public , and indeed did not hesitate to do so, but who became good friends because they each respected the others outspokenness.

I hope we can feel free to openly voice our opinions here in this forum and that if anyone disagrees with me strongly he does not hesitate to express that disagreement.

Smart people sometimes have erroneous opinions. Some very smart people have been ruined in this industry because they did not manage risks.

In the example given in the presentation, there was a 9% drawdown with an estimated maximum drawdown of 15% leaving an estimated worst-case loss remaining of 6%. Then math was applied to determine the account size where 6% was $20,000. This results in a notional account size of $333,333. Now consider what would happen if the estimated worst-case drawdown were to actually end up being 20% instead of 15% which in my experience is not at all unlikely (a 95% confidence Monte Carlo simulation notwithstanding). In this event, you would end up losing an additional 5% of the new larger notional account size or $16,666.

I am sure that anyone who doesn't want to lose more than $20,000, even if they think of themselves as aggressive, will be very upset to lose $36,666. They might be so discouraged they quit trading. This is especially true if they told their spouse that they would only lose $20,000 in the worst case scenario.

Anyone who traded on the wrong side of a big price shock will not be so inclined to think they know what their worst-case drowdown will be. They might find that the 15% maximum drawdown jumps overnight to 30% or 50% or more. Now that $20,000 becomes $100,000 or $150,000.

I don't think our predictive tools are sufficiently accurate to enable one make anything other than a rough estimate of drawdown sizes. There are far too many variables involved and the game changes constantly as the players implement new strategies based on previous price history.

Forum Mgmnt wrote:Anyone who traded on the wrong side of a big price shock will not be so inclined to think they know what their worst-case drowdown will be. They might find that the 15% maximum drawdown jumps overnight to 30% or 50% or more. Now that $20,000 becomes $100,000 or $150,000.

- Forum Mgmnt

I am sure you talk from some certain experience you've had, could you share more?

There was a posting on Elite Trader recently when someone asked about the worst single day of trading. I did a bit of research to determine the exact prices for my worst day.

The worst day I had trading was a huge price shock I experienced just after the crash of 1987. Eurodollars closed on black Monday, October 19th, 1987 at 90.64 close to their contract lows of 90.15 which had been set two days prior and had been tested earlier that morning with a low at 90.18. I was short 1,200 contracts of December Eurodollars and another 600 T Bills. I also had significant long positions in Gold, Silver, and large positions in a few currencies.

The next morning they opened up at 92.85 which was more than 2 full points higher and about $5,500 per contract without any opportunity to exit. This was a price we hadn't seen in 8 months. Additionally, Gold opened down $25 and Silver opened down over a $1.

All totalled I was down about $11 million on the $20 million account I traded for Richard Dennis overnight. This was essentially my entire years profits which vanished overnight. Had I been trading a larger portfolio or a more aggressive percentage and had this been a normal account this price shock could have wiped the account out overnight.

We are not in disagreement. I did not say, "This is the way to determine account size"." What I said was (paraphrasing) "If you're aggressive and you're looking for a strategy to maximize returns by using a tool to attempt to measure a maximum drawdown then here is an idea"."

I think you're giving your readers far too little credit. Any one of them can look at a spreadsheet and compute what the losses beyond the 95% confidence level would be.

You have to pick a stop in a system or a trade SOMEWHERE. If you base your trading leverage on the worst that can ever happen then the only logical risk level is zero. Even your old mentor William Eckhardt said that you can not base your leverage on the worst possible scenario.

The ENTIRE point of my presentation was how future risk is usually larger than past risk (please read the last sentence again as you seem to have missed it). Your comments seem silly to me only because you're just adding a one paragraph exclamation point to something I spent 30 minutes explaining. (Remember the Fed Ex commercial about the guy who repeats the other guyâ€™s idea and the first guy says "I just said that!")

The difference is that I then provided tools to try to measure and use that information as aggressively OR conservatively as you wish. I thought that was far more interesting than some worn out phrase about how "you never know big risk can be, I remember 20 years ago when....". I'm sure at this point that is an old well-understood point. Readers now need specific tools that can help them when taking action, as opposed to pointing out the incredibly obvious. Maybe I am underestimating the intelligence of those who read your forum but I really don't think so.

For the record, right now on a 125k nominal personal account I have portfolio heat of about $11,693 (what I would lose if every trade got stopped out tomorrow) and, thatâ€™s high for me. So my portfolio heat is about 9% (and again, that is atypically high for me, I am usually closer to 5%). I think many here would agree that puts me firmly in the conservative camp!

I guess I am a little upset not because of your perception of risk (I agree with it and made a 30 minute presentation about it). What is upsetting is that you're trying to paint me as having taken a position that is completely opposite of what my message was! I think I will reproduce the video with the last 30 seconds of the 30 minutes taken off so that this silly mischaracterization can end. It will then be clearer than ever that you're only making the same point I am. The difference is that I am trying to provide specific tools to deal with it as opposed to just pointing out it's existence. If readers would rather have people just simply reporting on potential risk as opposed to offering methods to deal with it I'll change my contributions. (and I can write them in 1/100th the time.)

For the record, I first heard the method from a very successful fund-of-fund manager in Santa Barbara who had about 600 million under management at the time. He told me over dinner one night that he occasionally determines allocation amounts based on similar formulas (I knew I was in for an interesting night when he arrived looking like a Hells Angel on a fully chromed out Harley!) It turned out being one of the most cerebral evenings Iâ€™ve ever had. He eventually asked me to quit what I was doing and work for him internally with initial promises of up to 50 million in equity to manage. However, I would have had to sign away all rights to any past, present or future research. I had to decline.

In summary, there is no way to know risk for sure. Therefore, only risk money that you can afford to lose. Furthermore, use statistics and probabilities to mold a risk to reward matrix that satisfies you. Obviously this matrix will look entirely different depending on many variables such as your net worth, age, risk tolerance, and goals etc.

Forum Mgmnt, it seems to me that maybe you have shifted more into the Nassim Taleb "fooled by randomness" camp that says long only options are the only way to manage risk. (And he may be right.) My question to you is this, if risk is theoretically endless, and Monte Carlo simulations or any other strategies for maximum risk prediction can be vastly wrong, then how do you determine your leverage? What sort of statistics, formulas, and theories do you use to determine what level you should leverage yourself at? Would they really be that much different than looking at properly done worst case Monte Carlo simulations and then adding in a margin of cushion to that?

Regards,
Dean Hoffman

PS I like your story about â€œThomas Malthus and David Ricardo who often disagreed sharply in public , and indeed did not hesitate to do so, but who became good friends because they each respected the others outspokenness.â€

Wow, having 50% of the account wiped out in one day thats most destabilizing thing, espacially when you know you've been working hard for it. I can share my worst trade so far. It was in 2004, I was a student back then and had 1000 us$ monthly paycheck from my job, the account was small: 5000 us $. At that time I lost a bit of it so it became 4000 $US. It was August 2004, and I was very unexperienced, total newbie, I put on a Short position in September Cocoa 204 contract. The next day some news came out that aborigens/bushmen or whoever live there fired the Cocoa fields in Ivory Cost. the result was the market rallied right at the open ( of course going through my Stop loss) and filling me right at the open. I'll never forget that day. I lost 900 us$ in one day. Comparing it to what I was earning at my job was depressing.

Dean wrote

I thought that was far more interesting than some worn out phrase about how "you never know big risk can be, I remember 20 years ago when....".

Its not some worn out phrase, first when someone been trading for 20 years, it tells you that this guy survived and he's a veteran trader ( not many lasts in this business). Another thing, is that all those models you been talking about go to the trash can, lets see now why from the markerts:

in this example the one would lose 5900us$ per contract in Beans.

Why go to 70's some 20 years ago, lets go to our time Dean!

I'm sure many guys were wiped out trading Natural Gas futures that day and had to transfer additional funds to satisfy the margin requirements...

No question scenarios like those happen all the time. I have probably run tens of thousands of hours of analysis over that very data and I would assume any serious traders here have done the same thing.

In my case what I need are specific tools. At this point in my trading having the obvious pointed out does not seem to help my bottom line much. There is no question that 15 years ago when I was looking at charts like that I realized the need to test my ideas.

One of the nice things about properly done Monte Carlo simulations is that it takes those sorts of worse case scenarios you picked and makes them MUCH worse (you can read about it in the last 15 or so messages). Without this you would be dramatically under representing the potential risks in those charts. You need a way to see what happens if those bad runs were to be 5 times longer and compounded with 5 other positions going against you just as bad or worse etc.

However, your chart observation with the limit moves did remind me of a tool I suggest using in testing:

If the open, high, low and close are the same price then assume that a limit day has occurred. Then make sure your back test would not have let you in or out on that day (this is how I have tested for about 10 years now). This will not only properly account for the actual stuck position in back testing that trade, but it will also give a truly nasty data point for the Monte Carlo simulation to compound even worse.

As far as your Natural Gas observation another tool I would suggest you use is

Volatility based position sizing.

The dollar volatility the day before that nasty move was about $10,000. In my case I like to keep my positions no larger than about 2% of dollar volatility. This would mean that I would not have been in that trade unless I had a $500,000 (or so account). Therefore the approx $15,000 move against me would have been about 3% of my account lost that day. Painful yes, but possibly forgotten by other winners that day, and at worse nothing more than a minor setback.

I donâ€™t really have any chart pictures to post for you, but hopefully some of the above suggestions give you some tools to deal with the problems you observed in the chart pictures you shared with us.

I canâ€™t help but being left with a lingering feeling about this whole discussion. Trading futures is not a place to try and figure out how to have certainty. It is where RISK TAKERS exchange UNCERTAINTY and potential losses for the chance of a profit.

If youâ€™re not walking into this knowing full well that this is nothing more than taking chances (gambling) then you are in the wrong place.

If you donâ€™t have any tools that you believe give you any edge in predicting profits or risk then why in the world would you ever participate!? Iâ€™m reminded of the phrase â€œIf you canâ€™t figure out the sucker at a poker table ITS YOU!

I trade with the full acceptance that my analysis may be wrong and I may lose all my money. I readily accept that. Once that has been established I can then comfortably move on to using tools that I think improve (not guarantee) my odds.

Iâ€™m sensing far too much dependence on certainty here. It reminds me of another old saying â€œif you have to ask, you canâ€™t afford itâ€

Novie OJ this morning - what a gap up and then some. I was fortunate to be out of my shorts 2 days ago when I didn't like the dull action (inefficient use of my margin money). But oh boy! To be caught in that updraught would be quite sobering - enough to make one want to stop trading.

I wonder where today's move would sit in a monte carlo backtest analysis in terms of percentages?

More likely traders would be long the Corn and Oats, but the OJ has been trending down, that's why the huge reversal must have caught quite a few - I could easily have been short the OJ coming into today.

Don't want to get off topic, but this has to be an example of when modeling riks just might not capture an event like today in OJ...

Quote:
Oct. 12 (Bloomberg) -- Orange-juice futures soared to a 16- year high after the U.S. government said Florida's orange crop will be the smallest in 17 years as cold temperatures and lingering hurricane damage hampered fruit growth.

Florida, the world's second-biggest orange grower behind Brazil, will produce 135 million boxes in the 2006-2007 season, down from 147.9 million, the final revised estimate for last season's crop, the U.S. Department of Agriculture said today in a report. The forecast was the first for a harvest season that began this month and ends in June.

``The trees are still trying to recover from back-to-back hurricanes,'' said Andy Taylor, vice president of finance at Arcadia, Florida-based Peace River Citrus Products Inc. ``Some trees are very sparsely fruited. That's the hurricane hangover from the last two years.''

Orange juice for November delivery soared 27.5 cents, or 17 percent, to $1.923 a pound on the New York Board of Trade, the highest closing price since July 1990 and the biggest percentage gain since August 1999. Prices reached $1.93 in intraday trading.

These are precisely the kinds of scenarios that MC simulations do a great job with. Assume that on average your daily percentage fluctuations are plus or minus .75%. Lets then assume you were in OJ the wrong way today and this caused you to have a -3% day. What the Monte Carlo will do is randomly pick the percentage change days in your data INCLUDING your bad -3% (or whatever) days like today and then resample them how ever many thousands or millions of times you want.

What this means is that unlike your current experience of only seeing the result of this on a one day basis the Monte Carlo simulation might put 5 of these bad days together back-to-back. This is also the reason you don't want every data point from your sample included in your MC output, otherwise it precludes having the same bad day chosen multiple times. For example, if you have a 3000 data point sample then chose to have the new 3000 data points created from a random sample of those old data points as opposed to just reshuffling all the points. Then repeat this 10,000 to 10,000,000 (however many times you think) in your MC simulation. Again, this will allow the possibility of the same bad days being chosen multiple times in a row etc.

Another idea I like is to sort my data in Excel and find the single worst data point and then fabricate and add a data point much worse. In other words, letâ€™s assume your worst historical data point was a -7% day. Artificially add in an unknown September 11 type scenario and a fabricated -15% (or whatever) day, etc. This new fabricated data point will then be factored into the thousands of new sequence outcomes.

Again, the outlier days like we had in OJ today are PRECISELY why you want to use MC sims.

Hope that gives you some ideas anyway.

PS On a side note, if you look at OJ on a weekly chart its been in a raging bull market for over 2 years now. I guess its all relative depending on your timeframe!

I've been reading this thread with interest. First I was also a bit surprised that the the original question only spawned discussions on MC metrics (centered on drawdowns), risk etc. But then, the discussions at this forum tend to have a bias towards strategies that are applicable for long term traders, or at least most discussions are set in that context.

I believe that part of the edge in a long term system, or composition of long term systems, is psychological and lies within the trader him/herself. This would include the ability to endure large drawdowns and the ability to wait several months if not years for new equity highs. Nothing new to participants in this forum, I'm sure. But because of this I usually think of a long term system as characterized by a rather blunt edge. c.f. may well be right in that such a system will never cease to work (the edge being partly psychological, my view). And in this context, of course MC sims makes sense. It gives you an idea on what you might be in for.

Intraday or interday (a few days) systems that exploit market tendencies (or inefficiencies, as some like to call anything that is not random) mostly trade at a higher frequency when compared to longer term systems. And the edge may actually deteriorate and become useless. A difference when compared to long term systems is that you can see this, because of the shorter timeframe.

From a short term systems point of view, stop trading it "when it stops working". When has it stopped working? When the results are no different than random. And that can be monitored in a suitable rolling timeframe for each system individually.

Griffy wrote:is it typcial for a LTTF trader to employ multiple systems?

The reasons to employ multiple systems are discussed in several threads at this forum. For one thing, multiple (uncorrelated) systems can help you make your equity curve smoother (and perhaps steeper). I believe that is true for LTTF systems as well as for short term systems. But then, it's not easy to find one system to earn your trust. Finding additional - uncorrelated - systems isn't exactly easier (my experience at least).