Charles D. Robusti was the patriarch of a family of mechanical system traders. (He was also a little crazy, and remarried every 10 years, having a new baby with each new wife). Charles was very opinionated, and strongly believed that (1) Mechanical Trading Systems Must Be ROBUST; (2) Once you develop a ROBUST mechanical trading system, and subject it to all of the challenging backtests, and forward tests, and out of sample tests, and stress tests, and perturbation tests, and Whamma Lamma Ding Dong tests that the Robusti family is justifiably famous for, then you should trade that system forever, unmodified, never making a single change. After all, it's ROBUST.

Charles' first son, Richard Donchian Robusti, became a system trader in 1963. Richard Donchian Robusti carefully developed a mechanical trading system, tested it by hand (no access to computers), paper traded it, and ran it through the battery of Robusti family tests. When he became convinced that his system was ROBUST, Richard Donchian Robusti began trading it. He swore an oath and solemnly promised that he would trade this sytem forever, never making a single change.

Charles' second son, Welles Wilder Robusti, became a system trader in 1973. Welles developed his own mechanical trading system, tested it using pencil and paper and adding machines (there were no electronic calculators or personal computers in 1973), and ran it through the battery of Robusti family tests. Carefully note: when Welles Wilder Robusti was developing and testing his system, in 1973, he had ten more years' worth of market history data than his brother did, in 1963. So it isn't surprising that Welles Wilder Robusti's mechanical system (developed 1973) was different than Richard Donchian Robusti's mechanical system (developed 1963). Welles also swore the oath and solemnly promised that he would trade his system forever, never making a single change.

Charles' third son, William Eckhardt Robusti, became a system trader in 1983. Like his two brothers before him, Bill developed his very own mechanical trading system. But he used these newfangled things called Minicomputers to develop his system. Bill bought two "MicroVAX" computers from Digital Equipment Corporation of Hudson, Massachusetts, carefully entered price history data, and developed mechanical trading systems On Computer (!). When he had a pretty good looking trading system, Bill tested it thoroughly, including the famous Robusti family suite of challenging and extensive tests. Carefully note: when William Eckhardt Robusti was developing and testing his system, in 1983, he had ten more years' worth of market history data than his brother Welles had, in 1973, and Bill had twenty more years' worth of market history data than his brother Richard had, in 1963. Unsurprisingly, William Eckhardt Robusti's ROBUST system was different than Welles's ROBUST system, and was different than Richard's ROBUST system. Bill also swore the oath and solemnly promised that he would trade his system forever, never making a single change.

Charles' fourth son, John Henry Robusti, became a system trader in 1993. Following the family tradition, John developed his own mechanical trading system. By this time, personal computers were ubiquitous and fast (300 MHz Pentium CPUs!), and John Henry used them, along with state of the art software, to develop ROBUST trading systems. When he had a pretty good looking system, John tested it thoroughly, including running it through the entire battery of Robusti family tests. Carefully note: John Henry Robusti had 10 more years' worth of price history data than William Eckhardt Robusti, 20 more years' worth of price history data than Welles Wilder Robusti, and 30 more years' worth of price history data than Richard Donchian Robusti. Nobody was surprised to see that John's system was different than the other three. Like his three brothers before him, John swore the oath and solemnly promised that he would trade his system forever, never making a single change.

Charles's fifth son, David Harding Robusti, bacame a system trader in 2003. He developed a trading system using a server farm of 300 computers, each machine with 8 CPU "cores" running their own copy of Veritrader / Blox. Carefully note, DH Robusti had 10 more years of data than JH Robusti, 20 more years than WE Robusti, 30 more years than WW Robusti, and 40 more years of data than RD Robusti. After his system passed all the tests, it turned out to be different than all the predecessor systems. David swore the oath and solemnly promised that he would trade his system forever, never making a single change.

Now here we are in 2009. The Robusti family story may suggest some interesting philosophical questions. It may (or may not!) stimulate thinking about fundamental assumptions and dogmatic beliefs. Questions such as,

Is it "fair" for each successive Robusti son to develop his system using more years of historical data, merely because the data is available? Is it "proper"? It it "correct"? Is it "valid"?

The availability of more data encompassing more market phases and cycles and psychologies, is one reason why the developed systems are all different from each other. But is this "fair" and "proper" and "correct" and "valid"? Is Welles' system (developed 1973) any more robust or valid than Richard's system (developed 1963)? Perhaps you may feel the reverse is true: the 1973 system, having been fitted to more data, is less robust or valid than the 1963 system?

If you were the first son, Richard D. Robusti, would you be tempted to discard your ROBUST system (developed in 1963) and instead use your younger brother David H. Robusti's ROBUST system (developed in 2003)? To do so would violate your sworn oath and invalidate your sacred family motto. On the other hand it might make you a ton more money.

Cousins and nieces and nephews have money to invest. Would you suggest they invest it all with Richard and his 1963-vintage system? With David and his 2003 system? Equally among all five brothers? How do you divide up the pie?

Whatever allocation you chose, would you suggest to the Robusti brothers themselves, that they should quit trading their individual systems, and instead, each of them should trade a Suite of systems according to your allocation percentages in question 4 above? If a certain allocation mix is best for the investors, isn't it also best for the traders too?

The Robustis keep having kids who keep developing new and different ROBUST systems. Do you add each of them to your allocation, your "Suite", as their systems become available? Why or why not?

What happens when one of the brothers retires or dies? Should the family hire somebody to keep trading that ROBUST system? After all, if it's ROBUST it should be profitable forever. Or do you think it's okay to eventually stop trading a system, for some reason or another?

Change the story so the Robusti's have a new baby who becomes a system trader, every 2 years instead of every 10 years. Does this change any of your conclusions? How about if there's a new trader with a new ROBUST system, every 6 months?

I'll probably think of better ways to express these questions, or think of brand new questions, later. So I may edit this post a number of times. +SLUGGO+

Are the "different" systems valid? I suppose that depends on whether the systems are really "different." There are many "different" trend following systems. However, as observed by more than one trend follower they are all essentially profiting from the same market moves. If the systems are indeed "different," the next question is what are the differences. If by different you mean philosophically different in their approach to the market, then perhaps the systems are not valid. If a different system does not ride winners, cut losses, or manage risk, I would question its validity regardless of what the robustness tests show. A system which tests out as robust (regardless of the amount of data used) can nevertheless fail if the philosophy upon which it is based is faulty or impermanent.

As for whether I would switch to a sibling's system, that would depend on my emotional comfort level with a particular system in light of its return characteristics. Ditto for whether I would recommend a system to relatives. A system may be incredibly robust, yet be impossible to trade due to its drawdown characteristics. I would not trade or recommend that system, regardless of its robustness.

In deciding how to allocate across systems, I would make that determination based on how philosophically different the systems are. In other words, what is the risk premia a system is attempting to capture and what is the theory behind the existence and persistence of that premia. I would apply this reasoning to all issues involving adding systems, maintaining older systems, etc.

Finally, I would urge my relatives to continue with whatever faiths, prayers, talismans, lucky charms, or superstitious habits they have been following, as even the most robust systems can be upended by some really bad luck.

You pose an interesting theoretical problem on robustness and data sufficiency, likely to be answered more by personal preference and gut feel than any proven scientific method.

Before providing my answers to the above questions, I would first want to clarify the definition of robust. Dictionary definitions state that robust means sturdy, resilient, strong and healthy. My own personal definition, in a trading sense, would be along the lines of â€œlikely to weather all stormsâ€

Thanks sluggo for bringing forth with greater clarity an issue I was not able to do so in another thread.

I sense there is a kind of taboo in this regard as we're dealing with some of the many facets of fear and uncertainty:

1) first and foremost we want to believe we've done the proper tests and that our system is "robust". Admitting that the system deserves improvement can be hard for we have to admit we were wrong or that we failed;

2) secondly we can never be exactly sure whether markets really "changed" or instead if there is something wrong about our system;

3) third, we can never be certain if we're letting go of discipline when it's required, or instead, we have legitimate reasons to "tinker" with the system.

Perhaps we can come up with some objective rules here such as constantly evaluate our system's real time performance in light of its historical performance so as to find clues about whether it's indeed functioning as it should or if there were false premises in its design.

However, deciding when to abandon the system can be very hard and tricky. I remember that back in 2004 I designed a system with some poor optimization methods and criteria. I expected a maximum drawdown around 30%. And I thought it was robust. I committed a small amount of capital to it as a "test". In real time trading the drawdown approached 50% when I decided to abandon the system. However it happened to be exactly the worst drawdown. Subsequently it would manage to recover the lost capital. This is just an example of what can happen.

Although this might be a bit off-topic, but I feel it's also rather connected to the main issue we're discussing about, is the issue of creativity and imagination in trading.

I feel that trading somehow ends up killing our creativity and imagination because of the random component of markets. It's as if we're always trying to understand and comprehend something that in its essence is unfathomable and at the same time never changing, much like the human nature.

So, what can the Robusti new generations come up with, if markets in their essence are always the same, the only difference being that they have more "history"?

P.S.: I am so convinced about this that I am working on a long term system so that I will dedicate myself to other interests instead such as family, quality of life, spirituality and other areas of study.

This is my first post on this forum and my first week as a new owner of Blox.

Having spent quite some time, however, following the various ideas on this excellent forum, I know that I would be reluctant to jump in and post under certain other topics, presenting new ideas which I could in no way support with quantitative and objective research.

But Sluggo's post was so thought provoking, and since the forum subcategory is Psychology I decided to reply to the questions 1-8 in the form of another question.

The question is: What's to be done if we did in fact have the data?

I think it's an important question, maybe THE MOST important question. And since Sluggo sort of stuck his neck out and raised the subject, and given the astounding depth of expertise to be found on this forum, how about taking the Robusti family example to its logical conclusion?

I mean, let's say we had 1000 years of data. This means 240,000 daily bars. This would be roughly the same as 13.9 years of data made up of 5-minute bars. This is assuming an average of 6 trading hours per day per market. And then let's assume by all means that this data existed for 50 markets and that no markets had been added or dropped during the period.

Where am I going with this? Well, if you can just stick with me for a second, let's make a gigantic leap and say that the data ended up being a whole lot like the 5-minute bars. That is to say that there was a lot of noise in it - a large random component. This being the case, you still find yourself able to create a profitable system (or comination of several systems) by testing and optimizing across the entire 240,000 bars.

The glitch is that (maybe like with the 13.9 years of 5-min bar data) these systems have MAR ratios of 0.3 to 0.5 and such like. They make money, astonomically so over time with the power of compounding. But nothing remotely close to what any of us aspire to achieve over the next five years.

You also notice in the 1000 years of data that certain strategies run very hot at times, for several centuries in some cases and especially for periods as short as a few decades. In the later case, some of these systems show MAR's as high as 1.0 or more over said periods.

So would the right thing to do be to run adaptive systems, constantly looking in the rearview mirror over the recent past and attempting to deploy an array of parameters that we feel is likely hit the mark in the near future, or is it better to accept a one-size-fits all approach and run a system that performs best over the entire 240,000-bar series of data?

Can we use the 5-minute data we have on hand to help answer this question?

Or do we need to aswer this question at all as it seems that the clear answer over the past few decades is hands down in favor of the adaptive approach. As Keynes once famously said, in the long run we are all dead anyway.

J Randall wrote:The glitch is that (maybe like with the 13.9 years of 5-min bar data) these systems have MAR ratios of 0.3 to 0.5 and such like. They make money, astonomically so over time with the power of compounding. But nothing remotely close to what any of us aspire to achieve over the next five years.

You also notice in the 1000 years of data that certain strategies run very hot at times, for several centuries in some cases and especially for periods as short as a few decades. In the later case, some of these systems show MAR's as high as 1.0 or more over said periods.

So would the right thing to do be to run adaptive systems, constantly looking in the rearview mirror over the recent past and attempting to deploy an array of parameters that we feel is likely hit the mark in the near future, or is it better to accept a one-size-fits all approach and run a system that performs best over the entire 240,000-bar series of data?

Can we use the 5-minute data we have on hand to help answer this question?

Or do we need to aswer this question at all as it seems that the clear answer over the past few decades is hands down in favor of the adaptive approach.

Perhaps you could use the 13 years of 5-minute data to test it, reoptimizing the system at fixed time intervals and then check the overall final performance compared to the initial untweaked system.

Some traders argue, for instance, that markets exihibit a cyclical nature. If so there is the risk that by reoptimizing the system to recent price history we end up rendering it unfit to the "next" cycle. Just a side thought though, which could be tested anyway.

Attached is the performance of "LACM Trend Following Diversified Share Class", by Stig Ostgaard, a former turtle himself. His performance seems to be better than his "Robusti" predecessors. Yet, if he's being honest about this philosophy, it still looks like a strategy of the Robusti family with some kind of improvement:

Trading signals are calculated as stops â€“ either above some pre-determined level for a buy, or below for a sell. Stop levels are of three independent types: initiation, liquidation, or re-initiation (i.e., resumption of a trade in the direction of the last liquidated trade). Each trading system is a volatility-based system: i.e., trading levels are determined based on an optimal movement away from a reference price. This reference price may be of a structural nature (based on price patterns) or of a statistical or mathematical nature. The margin to equity ratio for the program is expected to range between fifteen and forty percent. The portfolio is approximately evenly weighted between financial and commodity markets.

I hope C. Faith would shed some light on this in his new book "Inside the Mind of the Turtles".

Let's look at it from a different angle. How does your human
brain work? While I can't remember every lesson I've learned
about riding a bike, I do use my entire history of riding a
bike to make my decisions. Some memories stick with you...
like almost getting hit by a car. I have memories of riding in
different terrain, and of the different bikes I've ridden...all
the way up to my recent experience with a new beach cruiser.

So why shouldn't we use all the experience (data) that is available.
If I could live forever, I would try to remember all the experiences
I've had. Why shouldn't a trading system?

One way to interpret the Robusti story is to imagine that all of the various Robusti family members are, in fact, just one person: You. Charles D. Robusti, the boss of the trading family, is YOU. You are the one that lays down the rules, the system testing procedures. You are also the developer of the first trading system. Some time later (T=10 years? T=2 years? T=6 months?), you develop the second trading system. After another T units of time, you develop the third system. T units of time after that, the fourth system. Etc.

The same psychological questions still apply. When the family (i.e. You) develops a new system, what do you do with it? What do you do with its predecessors? Do you trade them all? Only trade the newest? Only trade the 4 newest? Only trade the oldest*? What is a value of "T" (time between introduction of new systems) you can live with? And so on.

*If you only trade the oldest then you only need to develop one system. After that you can spend your research time pursuing other activities, such as marketing or hot air ballooning or buying baseball teams.

sluggo wrote:You are the one that lays down the rules, the system testing procedures. You are also the developer of the first trading system. Some time later (T=10 years? T=2 years? T=6 months?), you develop the second trading system. After another T units of time, you develop the third system. T units of time after that, the fourth system. Etc.

The same psychological questions still apply.

Indeed. As a matter of fact, if reoptimizing the system is "allowable" then why not doing it on a weekly or daily basis? The same psychological questions still apply.

I would just add that Pardo points to a sober way of dealing with this issue: "If market conditions remain the same, a new reoptimization of the trading system will most likely arrive at the same parameter values that were identified by the previous optimization. Conversely, if market conditions change, especially dramatically, reoptimization will most likely identify new parameter values." After all, there might have been errors in the design and optimization process of the first system which, as a matter of fact, is not quite robust; or else, market history available at the inception was not representative enough of general and recurring market conditions.

sluggo wrote:One way to interpret the Robusti story is to imagine that all of the various Robusti family members are, in fact, just one person: You. Charles D. Robusti, the boss of the trading family, is YOU. You are the one that lays down the rules, the system testing procedures. You are also the developer of the first trading system. Some time later (T=10 years? T=2 years? T=6 months?), you develop the second trading system. After another T units of time, you develop the third system. T units of time after that, the fourth system. Etc.

The same psychological questions still apply. When the family (i.e. You) develops a new system, what do you do with it? What do you do with its predecessors? Do you trade them all? Only trade the newest? Only trade the 4 newest? Only trade the oldest? What is a value of "T" (time between introduction of new systems) you can live with? And so on.
.

Perhaps, the Robusti family has a strict research implementation policy as a part of the system rules of each generation. In other words, they may or may not have a set of specific rules to how, if and when to SYSTEMATICALLY implement findings which may be consider integral part of the commitment to stick to their systematic approach.

True but don't let's forget that Mother's Nature is a very severe parent and that 99% of all species ever evolved are now extinct. Which is why some say it's best to emulate the long surviving cockroach in trading system design! Simple, robust, durable?