Wednesday, December 7th, 2011
Interview: Victor Niederhoffer protege Henry Carstens on trading systems testing
This interview first appeared in Your Trading Edge magazine
At some stage in a trader’s career, they’ll be hit by the uncomfortable questions: am I winging it? Do I really have an edge?
The answer, of course, is to begin testing your system or method. But for a discretionary trader, many of whom use charts and intuition, learning how to test existing and fresh ideas can be bamboozling.
One person who is helping demystify the complex world of trading system testing and development is Henry Carstens.
Carstens, who is based in Portland,Oregon, develops and trades systems through his company, Vertical Solutions, and has recently begun trading other people’s money.
A protégé of trading legend Victor Niederhoffer, Carstens is developing a growing reputation as an insightful teacher of systems development and testing.
His ‘Introduction to Testing Trading Ideas’, which is available at http://www.verticalsolutions.com/, is an excellent outline and recommended by trading coach Dr Brett Steenbarger.
Henry spoke with YTE’s Ben Power about how traders can boost their trading through systems testing and development. He has helpfully provided a number of links to papers available on his site that readers can follow up.
How did you get into trading?
I bought my first stock, Anglo Lautro, when I was 12. I it bought at 1 1/8, the high was nine and it was eventually nationalized by the Chilean government for no gain.
Later, I noticed that on some days the market really seemed to take off and run in one direction and I figured out how to predict when that would happen. I’d watch the market in the morning before going to work, place my trades and head off to work.
How did you get into trading system development?
Later I found a book called Trading Systems Toolkit by Joe Krutsinger. I got a copy of TradeStation (trading software) and coded up the idea outlined above – an idea I use to this day. The core of that idea is in the Trend Day or Range Day forecasts on my website.
After that I worked a bit for Dr. Victor Niederhoffer and have been on my own ever since. It was for Dr. Niederhoffer that I built a huge array of systems based on machine learning – all of which ended up being spectacularly curve fit. It was a great, if painful, lesson.
What was it like working for Niederhoffer?
Absolutely fascinating! His ability to generate simple, insightful ideas and test them is unparalleled. Every body should read his book Education of a Speculator.
I assume that developing a trading system begins with an idea or a hypothesis about a potential edge. How do you generate ideas?
Three ways: I watch the market, I play with the data, and I try to cross-pollinate ideas from outside trading, into trading – for example, chess has the idea of critical points, a place where the game turns; a similar idea in trading might be support and resistance or it might be the line where the day changes from being a range day into a trend day.
As an example, here are ten ideas based on an article about a basketball player. Still, I am nothing compared to Dr Niederhoffer when it comes to idea generation. His book, Education of a Speculator is unbelievable.
Once you’ve got an edge, I assume you test it? How do you got about that? What numbers would confirm a solid edge?
I use two numbers: ‘t score’ (or t-Test) and ‘optimal f’. The ‘t score’ tells me how likely the results I’m seeing are statistically significant (or how likely the test results are to have occurred by chance alone); and ‘optimal f’ tells me how much the edge can be leveraged without the risk of ruin.
So having found an edge, you verify that it is ‘real’ using ‘t score’, and then ‘optimal f’ tells you how tradable it is. Generally, the more leverage that can be applied to the system the better the system is (regardless of whether that leverage is ever applied).
Here is a pretty good example of developing an edge, and a tool for evaluating it as a potential trading system.
What kind of investment would it require for a discretionary trader to learn software like TradeStation?
I don’t know. It’s such a continuous process. Maybe not so much to use it as a very simple tool. A bit more to use it to create new tools.
One of the most important things TradeStation does for me is manage historical data. That saves huge amounts of time. The second thing they do is provide a way to run tests against that historical data to determine if an edge/idea is real and tradable. For the discretionary trader, knowing an edge is real is almost an edge on top of an edge.
Personally, I have a mental block when it comes to statistics and hard maths. Do you need to be good at maths for systems development?
Not just for trading system development, but to test discretionary ideas and set-ups too! You need some math because that’s the language you’re going to use to understand and contextualize results and the better you understand the results the more successful you’ll be. It’s like the eskimos having 42 words for snow – math helps you understand the results of strategy testing.
An example: the t score is a very simple calculation (square root of number of observations/standard deviation x average trade), but it tells you something that is of fundamental importance: are the results I’m seeing real, or did they occur by chance alone? A little bit of math providing a lot of insight.
Simple works best in trading systems; hard math isn’t necessary and insight, creativity and hard work are all more important than complexity. Readers can find an overview of the math in Introduction to Testing Trading Systems here.
What are some of the major pitfalls of systems testing? Is there a danger, for example, that you twist the numbers to suit your hypothesis?
Yes, curve-fitting is the great danger and the odds of curve-fitting tend to increase at least linearly with each variable added to a model. This is because, when the relationships in the market change, the odds that the change will affect your system are higher because you have more variables to be affected.
Another pitfall is the ability to anticipate future changes in the market from historical data. Calculating and building tolerances for systems can help with this.
Perhaps the biggest pitfall is thinking that any single trading system is going to perform flawlessly. All trading systems ebb and flow as the market changes and cycles, so having multiple systems to counteract the ebb and flow is very important.
Here are two general rules governing trading system automation.
What are the benefits of automated trading systems versus discretionary trading? Can a trader use both?
Automated trading systems help organize multiple edges. An automated system never forgets and is never afraid. Still they are rough tools, a generalized system never gets the best entry or the best exit so there is always room for improvement. A nice combination of automated and discretionary trading might be to let an automated system fill the holes in a repertoire; i.e., the discretionary trader handles range days and the automated systems handle trend days.
Say you have found an edge and verified it through testing. How would you then trade the system? Would that be automated too?
Yes, although it’s not necessary to automate it, you could have it just generate signals which you followed based on current market conditions. The advantage for the discretionary trader is there is an edge and then they can choose when to exploit it.
I have an idea that discretion may be the most useful in shorter time frames (minutes and hours, but not seconds) because as the profits per trade shrink the importance of maximizing entries and exits increases.
I assume one challenge to systems trading is changes in the market. How quickly does it take to lose an edge in the market? Does it mean you need to constantly be building new systems?
If you build an incredibly curve-fit system you can lose your edge before you ever get started. The changes in the market are, to my eye, more of an ebb and flow. Diversification of systems working within a market helps offset the ebb and flow.
I end up building new systems more because of new ideas than because of market changes…hmmm…but maybe those new ideas are being generated by market changes…It’s all incredibly circular.
In general, I think our edge, as system developers or discretionary traders, is much smaller than we tend to think.
Finally, for the discretionary trader who wants to begin exploring automated trading, how do you recommend they go about it? Are there any courses or internet resources that might help them along?
I wrote a paper called “Introduction to Testing Trading Ideas” that will help.
This entry was posted on Wednesday, December 7th, 2011 at 12:46 am and is filed under Trader Interviews. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
December 14th, 2011 at 8:59 am
I just wanted to say that I really enjoyed your site and this post. You make some very informative points. Keep up the great work!