Discussions about the testing and simulation of mechanical trading systems using historical data and other methods. Trading Blox Customers should post Trading Blox specific questions in the Customer Support forum.

Great thread! My $0.02 worth: I have not (yet?) found much benefit from either testing entries in isolation or MAE/MFE beyond general interest. I believe that any entry must have a corresponding exit and that therefore the two actions or entities are inextricably linked. With that in mind and also remembering that we are evaluating trades for an individual MarketSystem, I still use Ralph Vince's Weighted Geometric Mean (ie. his GeoMean calc that includes f ) as my primary metric for measuring edge in that context.

That said I do occasionally look at the average percentage of the MFE retained as a measure of the efficiency (or otherwise) of my exits.

Roscoe wrote:I believe that any entry must have a corresponding exit and that therefore the two actions or entities are inextricably linked.

Perhaps true. For a moment let's suppose so. Then if you have six generic entries (e.g. (1)MA cross, (2)Channel Breakout, (3)RSI extreme, (4)Bollinger Band Breakout, (5)Opening Range Breakout, (6)ADX pop) and six generic exits, you need to test 6 x 6 = 36 different combinations of an entry and a corresponding exit.

On the other hand, perhaps the other viewpoint is correct. Perhaps entries and exits are not inextricably linked. Perhaps their goodness can be measured in isolation, and perhaps in practice they operate relatively independently. Now under this second scenario you only need to test 6 entries in isolation, plus 6 exits in isolation, = 12 total tests. A lot less work! The theory profs would call the former "Order N squared" and the latter "Order N".

This little arithmetical exercise doesn't prove one viewpoint or the other. But it might justify wishing for one of the possibilities versus another.

My view is that there may be value to testing each entry method with a fixed/random exit and each exit method with a fixed/random entry. When trying to see how useful particular entries and exits are, in isolation, testing the 6 entries plus 6 exits may be useful.

However, if we narrow down a bunch of entry methods and exit methods, testing them in the way that sluggo suggests, it is still important to test the surviving entry and exit methods in combination, in the way that roscoe suggests. After all, when you trade you have to have your entries and exits paired. So, even if the 6+6 method has value, eventually we still need to test the 6*6 case.

In the end, both methods have merit. Use the 6+6 method to find entries and exits that seem to have merit. Then use the 6*6 method to find the most useful pairings of entry and exit.

Side comment: Actually, 6*6 is an underestimate as you may want to pair more than one entry method and more than one exit method in the same system. Pairing more than one entry and exit becomes a combinatorial problem and the number of tests can get out of hand easily.

sluggo wrote:Then if you have six generic entries (e.g. (1)MA cross, (2)Channel Breakout, (3)RSI extreme, (4)Bollinger Band Breakout, (5)Opening Range Breakout, (6)ADX pop) and six generic exits, you need to test 6 x 6 = 36 different combinations of an entry and a corresponding exit.

That is my thinking anyway and a few years back I did exactly that test (which BTW was a bit bigger than 6x6 and included some parameter ranges for entries and exits as well) the results of which were what I had in mind as I was writing my reply. BTW it was easier for me to test 'everything against everything' in the one automated test sequence than do it all in stages so no problem with the workload, but thanks for your concern.

Previously I was not able to convince myself that I was actually testing an entry in isolation, and I tried the usual things: exit after N bars where N = from 5 to 50 in steps, etc. Random entries as a means to test exits probably has some validity but my conclusions from the above testing series basically confirmed the linkage as well as confirming the oft-quoted "KISS principle" as it relates to entries and exits.

Oh, my tagline is my family motto, which translates as "No Sacred Cow Left Unturned". Perhaps my Latin grammar is rusty or just plain bad?

I think HZ touched on an important point- i.e. consider the path, not just the final outcome.

personally i do not distinguish between the importance of entry and exits in providing edge- they are both just signal filters that should be indicating the expectation of market drift has changed from whatever bias you were holding up until that point(long/flat/short).

The edge of a given signal filter is like a letter in the alphabet, it is how you combine the various letters that ultimately gives you the identity of a "word". saying that a "word" contains the letters d,o,r,w doesn't give the full identity/information of "word".

c.f.! Thank you very much for making available your ER-Blocks. They have animated me to add some more statistic numbers and some more modifications. I do not want to keep the result from you.

The following is different between the original c.f. ER-Block and my modified ER/Block:

-If you want to test your Entry Block with different holding duration, you must use the stepping function of TBB with c.f. original Block. With modified Block you can enter your holding duration before the test starts. Then you get a line-diagramm with the number of holding days at the x-Axis and the ER at the y-Axes. Only one test is necessary.

-The results of the modified c.f. Block do not depend on the Exit Block. (Except the Exit Block depends on the Exit Block.)

-c.f. original Block does not take account of the bar with the Enty for calculating the MFE and MAE. But the modified Block does.

-The holding duration includes the Entry bar and the Exit bar. Example: If you set the holding duration at 3 bars, then the holding duration consits of the Entry bar, the middle bar and the Exit bar.

-The c.f. original Block only shows the total value of the ER. My Block does calculate in addition the long value and the short value separately.

1) Computing the MFE and MAE for each trade.
2) Dividing each by the ATR at entry to adjust for volatility and normalize across markets.
3) Summing these separately and take the difference between the volatility-Adjusted MFE and the volatility-Adjusted MAE.
4) Devide the result by the number of trades.

Prifit in ATR per day: (Should be better than 0)

Is the same like the "Profit in ATR" but does in addition take into consideration the time:

HZ wrote:I was surprised, that the Entry Rules for Short trades are so bad.

That is one surprise. Another surprise is that the green curves keep rising and rising. The longer you hold, the better; which is not the expected result. For a countertrend style of entry, we expect that short holding times (2-6 days) will be optimum: get in quickly and get out quickly.

These surprises make me worry that perhaps there might be a programming glitch in your code. c.f. presented an excellent method for testing Edge Measurement code: apply it to totally random entries.

Forum Mgmnt wrote:So with version 2.1.11, if one sets the slippage to zero one will indeed get a 1.0 Excursion Ratio for random entries at any length of measurement.

- Forum Mgmnt

That would be a quick way to find out whether the two surprises are artifacts of a code mistake, or whether they are real. For truly random entries, long trades have the same "edge" as short trades so the "edge measurements" should be equal. Similarly, for truly random entries, the "edge" is independent of holding time, so the plots of "edge" versus holding time should be flat, neither sloping up nor down.

The test-results with random Entrys were neutral all in all.
The fact, that the difference between the Long- and Short-line is the bigger the longer the holding duration is, should not be a problem in my opinion, as long as the total black line has a neutral value.

I assume, if the market increases (which is case for stocks), then Long-trades must be better than short trades. The longer the holding duration is, the more significant is this behavior.
Assumed, I only take three Long-trades and three Short-trades with a holding duration of 5 years in a increasing market: It seems to be clear, that the LONG-trades must be better.

sluggo wrote:This little arithmetical exercise doesn't prove one viewpoint or the other. But it might justify wishing for one of the possibilities versus another.

I am coming to this thread late but will add anyway.

I always struggled with the usefulness of isolating an edge ratio from the time I first read about it in cf's book. I encountered this confusion again recently when discussing a system idea with someone who insists any system must be tested for an entry edge first.

My understanding is that the edge concept come from gambling - card games etc. Here, the system always has an exit but exit options are very limited (fold or loose/win). In this situation looking at entry edges make sense.

In trading, the game ends when the player decides. Therefore to test an entry for an edge separate from the exit you will ultimately use does not make sense.