Sick Alligator

The new Alligator system had one its worst months ever in June. Like anyone, I began to question the system and try to understand why it did so poorly. One of the things that can happen in designing a system is to simply arrive at too good a fit to the historical record. The easiest way to do this is to just keep adding different indicators like moving averages and stochastics and optimizing them until you have a really good looking backtest. With enough degrees of freedom it's possible to build a system from nothing. I don't think that's what we've done, but some things can creep in, even with the best of intentions. To evaluate whether this was the case, I re-optimized most of the optimizable parameters for the past 12 months instead of using the past 12 years as I would normally. I was able to make the system produce a higher net profit, and I could reduce the drawdown over the past two months, but I could not make it go away. And, when I was done, I had a drawdown that was almost as large, but in an earlier period. So, it's not merely a over-fit problem.

This system was built based on studying market characteristics over a long period of time and then trying to build a system that models that behavior. Of course, the past is not always prologue, but I believe markets such as stock index futures are going to exhibit some of the same behaviors in the future as they have in the past.

So what is the behavior we're talking about. Two of the things Alligator is based on the tendency of gaps to fill, and the tendency of markets to have choppy days where the price moves up and down, but does not really go anywhere. Much of this is related to a statistical phenomenon called "regression to the mean", which just means that measured variables tend to go back toward the middle of their typical range when they are near the outside of their range.

June was an unusual month in two important respects; the number of days when the market moved out side of the opening bar range and never came back was unusually high, and the number of days in which gaps were filled were relatively low. You can see the data in the following images (click to view).

In June 2011, there were 14 uni-directional days. This means the market moved away from the opening bar range 14 times and never came back. Uni-directional days are bad for this system because it results in a lack of "chop" which we are depending on for part of our returns. There are only 20 trading days in a month, so 14 is a high percentage, and more importantly, you can see from the histogram that 14 is in the right hand tail of the distribution curve. It's not as if it's never happened before, but it's not normal.

Gaps fills were below normal which you can see in the other chart. Historically, gaps fill about 14 times per month, and in June, 2011 we only had 11. These are not 100% gap fills, but only that which is required to reach the exit price for a trade entered at the open. This system always fades the gap, if it trades them at all. This month was not a complete anomaly, but it was definitely a negative for the system.

Any system is going to have bad spells which are going to result from the kinds of events mentioned here. It cannot be avoided. I'll be watching carefully to see what happens next.