Monday, August 26, 2013

Review of the ES1 system

The poor performance of this system the past two years is the primary reason for the general overall systems performance. ES1 performance chart. This trading concept dates from the beginning of 2006. It has done well in every year since except for 2012 and 2013. The concept is very simple- fading breakouts. The chart image below illustrates how this works (click to enlarge):

The poor performance could be a result of some fundamental change in the market which cannot be accounted for by looking at market history or it could be due to hindsight bias in the system as it exists or both. To address these questions, I decided to go back to square one, so to speak, and rebuild the strategy from scratch and optimize incrementally and only report out of sample results. This is known as walk forward optimization.

My optimization process is a little different than some in that it has two distinct parts. First is the part of the strategy that determines when and how trades will occur within a given day. In the case of ES1 this is mostly determining where to set the buy and sell points. For this study I used a version of this strategy that has only one optimizable parameter (the width of the channel). The more variables you have to optimize, the easier it is to producing a winning strategy, but you can overfit the system to the data which we avoid by using just one parameter and by using only out of sample performance for hypothetical results.

The second part of the optimization involves deciding whether or not to take any trades at all on a given day. This is done with a regression filter which relies on a lot of previous market data  to predict whether a given day is likely to be profitable or not. The regression filter is an array of technical indicators such as moving averages, stochastics, calendar effects, and so forth. Without the filter, this system is not and would never be a profitable system. 

The method I've been using for the regression filter has always used nearly all available data in the calculation. For ES systems I've standardized on starting in the year 2000. I would then use all data from year 2000 up to the end of some past year, typically 2010 or 2011. This base period was used to "train" the filter. Any data following the training period was used as evidence of good performance on the part of the filter. If the out of sample filter performance was comparable to in sample performance, then the method was consider valid. Of course, there is always more out of sample data to come, and this is always the best test. Lately, it has been failing this test.

For this study, I decided to use only the period from 2009 forward. The first reason is that 2008 was a very unusual year and I was concerned that it might be having an undue influence on the filter. The second reasons is that I thought some of the very old market data might no longer be relevant. 

So starting in 2009, the single parameter was optimized for the first three months using only those three months. When the optimization was done the resulting trade sequence was used to design the initial filter. Then the initial optimizations were applied to the next three months which were not in the initial sample (2nd quarter of 2009). When this was done, another 3 months was added to the design period and the results were then applied to the next quarter. This was done for the entire year of 2009. This gave me three quarters of out of sample results. From this point, the in sample period was increased to 12 months going forward 12 months at a time until sampling was done through the end of 2012. This left all 2013 as out of sample. From this work an equity curve was pieced together which used only out of sample results. The chart below is the result of this process:

What the chart shows is the same deterioration in performance since the beginning of 2009. Below is the same chart, but using the actual strategy which is being reported on the website and has been for quite some time.

The two charts do not match up exactly in time or in magnitude, but it makes the point that while this type of trading worked in the past, it is not working now and is not suitable for trading. It may at some point start acting like it has in the past, but only time will tell. I have tried other means to see if could revive this trading method and while I can get the past couple of years to stop losing, I haven't been able to turn them upward. And even if I could, it would require the use of up to date market data, which still leaves one waiting to see if the redesign will hold up. 

That's enough on the trading method for the moment. My next post will be about the other way to trade the ES, and that is to try to capture some portion of an intaday trend.

No comments:

Post a Comment