Comparing Daily Stock Market Returns to a Coin Flip

So I applied my skills to this problem and came up with a bit of code which attempts to use some feature engineering to predict a coin flip then use the same approach on daily S&P 500 (where 1 is a up day and 0 is a down day). The next day's outcome is the classification label for the current day.

The features designed for this experiment consisted of:
1. Previous, i.e. was yesterday up or down.
2. A count of Heads while in a Heads streak.
3. A count of Tails while in a Tails streak.
4. A count of Heads and Tails in a set of lookback periods. That is in the last 5, 10,20,30, etc days how many heads and how many tails were there. This to capture any observable trends. (not necessarily valid for coin flips, but believed to be a valuable tool in stock trading. Here is a bit of code where I identify the feature labels:

LOOKBACKS = [5,10,20,30,40,50,100]HEADER_LINE = ['label','previous','heads_streak','tails_streak']for i in LOOKBACKS: HEADER_LINE.append('heads_'+str(i))for i in LOOKBACKS: HEADER_LINE.append('tails_'+str(i))

1. As expected, the 100,000 coin flips run shows exactly what you would expect. With an Average AUC of 0.4981, this is almost the definition of the ROC curve. https://en.wikipedia.org/wiki/Receiver_operating_characteristic . For 100k flips and all the generated features, the 5 ROC curves basically follow the 45 degree line. So, there was no benefit found over random guessing what the next flip will be.

2. For 4600 days of S&P 500 'flips', there appears to be a very slight edge from the model, with an average AUC of 0.5419609577. Not enough to risk actual money.

3. Now, the 4600 coin flips output raises some interesting questions. The average AUC was 0.5112 and the five ROC curves somewhat hug the 45 degree line. This raises the question of the validity of the 0.54 found in run #2 (4600 days SP500) ? I wonder what would happen if we had 100,000 days S&P 500 data. That is about 380 years worth. Maybe some motivated enough, could dig up 100,000 hourly readings try this experiment again. I would be curious to see the results.