“It should produce 150 pips per week. With the best indicators that you know. How much does it cost? Please also send live histories of your top systems.”
Although we often get such requests, we still don’t know the best indicators and can’t send live histories. We do not invent algo trading systems, but program them from clients’ specifications. And we do not trade them, except for testing. But after almost 1000 systems, we can see a pattern emerging. Which algo trading strategies do usually work? Which will fall apart already in the backtest? Here’s a ranking of all systems we did so far, with a surprising winner.
One should think that most clients come up with very similar trading systems, so we could meanwhile just click them together from ready code. But it is not so. There’s apparently no limit of trading ideas. Almost any other system uses some new trading method, unusual data source, or exotic indicator. Still, the systems can be classified in several simple categories. Any of them has its specific success and failure rate.
Trading systems categorized
We classify the systems by their market and by their trading rules, both specified by the client. The 4 main markets are Forex/ CFDs, cryptocurrencies, stocks/ETFs, and options. And the 4 main algorithmic trading methods are risk premia, market models, data mining, and indicator soups. To recap:
Risk premium systems gain higher profits by accepting higher risks. In that category fall many stock portfolio rotation and options trading systems.
Market model systems exploit a particular market inefficiency by detecting anomalies in price curves. Trend following, mean reversion, market cycles, statistical arbitrage, or HFT arbitrage are typical model based trade methods.
Data mining systems predict a price trend by evaluating signals with a machine learning algorithm. Those signals are usually derived from the order book or the price curve, but sometimes also from fundamental data or exotic data sources.
Indicator soups do not target a particular market inefficiency. They generate trade signals from a complex combination of traditional or newly invented and fashionable indicators, with no recognizable concept or idea behind.
Some algo trading systems can fall in more than one category. For instance, a grid trader can be considered a risk premium system (high probability of small wins against low probability of high losses), but also a model based system (exploitation of volatility anomalies). If a system cannot be clearly assigned, it is split among several categories in the table below. We can see that clients favor some of the 16 possible combinations, while others are rare:
| Risk premium |
Market model |
Data mining |
Indicator soup |
Sum | |
| Forex/CFDs | 21 | 74 | 121 | 235 | 451 |
| Crypto | 0 | 4 | 55 | 40 | 99 |
| Stocks/ETFs | 68 | 95 | 34 | 14 | 211 |
| Options | 48 | 159 | 16 | 12 | 235 |
| Sum | 137 | 332 | 226 | 301 | 996 |
For determining the success or failure rate, we used 8-years backtests for unoptimized systems, and a walk forward analysis for optimized systems. A successful system had to return at least 12% CAGR for stocks or options, or 30% annual profit for Forex, CFDs, or cryptocurrencies. The R2 parameter had to be above 0.7. If clients ordered a reality check, the system had to pass it at 95% confidence. If one of those conditions was not fulfilled, the system was classified as failure.
The percentages of successful systems:
| Risk premium |
Market model |
Data mining |
Indicator soup |
Success rate |
|
| Forex/CFDs | 88 % | 81 % | 69 % | 31 % | 52 % |
| Crypto | 0 % | 75 % | 62 % | 25 % | 49 % |
| Stocks/ETFs | 92 % | 85 % | 61 % | 35 % | 80 % |
| Options | 96 % | 91 % | 75 % | 58 % | 89 % |
| Success rate | 93 % | 87 % | 67 % | 32 % | 66 % |
The average success rates at the end of the columns and rows are weighted by the number of systems. We can see that the overall success rate was only 66%. In 34% of cases we had to break the bad news to the client that it’s not advised to trade this system live. It produced no, or too little profit in the tests. Sometimes we could see what the problem was, and suggest ways to improve the system. But even total failures were no wasted money. When you know that your favorite manually traded system won’t work in the long run, you’ll save a lot more money than spent for programming and testing.
And the winner is…
The statistics are spoiled by the forex and crypto systems, half of which were losers. This is at least better than most such systems from trading books or trader forums, of which 90% fail either already in a proper backtest, or at least in a reality check. We got a surprising result in the ‘Indicator soup’ systems. You would normally expect that they all fail big time, since they are not based on a market model. But in fact almost every third indicator hodgepodge was successful, even in a reality check. Maybe the clients knew more than we did.
It is also a bit surprising that the most complex systems of all, the data mining systems that usually employ deep learning algorithms, did not fare much better. They have an acceptable success rate, but are easily surpassed by a certain sort of much simpler systems.
Of all systems we tested so far, the big winners were the long-term trading systems for stocks, ETFs, or options. Of the option traders, the simpler systems had often better performance. It’s relatively hard to specify a losing option system, but some still managed it by using short expiration dates, complex entries, fancy rollovers, or intraday buying and selling. One of the very simple, but successful option traders was included in the Zorro scripts.
We found another trend that is not visible in the table: With a few exceptions like HFT or arbitrage, there was an almost linear correlation between time frames and performances. Systems on one, five, or ten minute bars were rarely profitable. The good systems traded mostly on 1-hour, 4-hour, or 24-hour time frames. Faster is not always better.
This does not mean that we all should now abandon forex and cryptos and trade only long-term options or ETF portfolios. Diversification is a key to success. All markets still have long periods of ineffectivity and plenty opportunities of trading profits. Maybe the statistics above help to look for them.
Were the strategy tested on the latest 8-year period, if so then stocks enjoyed a bull market thus the results favors stocks.
Mostly, except for fall 2018 and spring 2020. But the same can be said for almost any test period.
You’re right that the long term bullish market favors most stock portfolio systems. If stocks were direction neutral, many of those systems would not work.
Thank you, JCL, for the post. Would be great if you have more time to write more often 🙂
Just a question: these are the results with trading costs. correct?
Yes, all results are with spread, commission, swaps, and simulated slippage.
JCL, fantastic write up. Do you have a sense of which category / instruments performed well in the last 12-36 months?
Your blog has really interesting and original content not seen very often, I really appreciate it. You should use more social media channels to let us know about this great blog, such as twitter account to be updated. Keep it up!
Fascinating article! May I ask where I can find the options strategy you are referring to, would love to give it a closer look.
Which category / instruments performed well in the last 12-36 months: We have not particularly tested the last months, but I think it’s the same ETFs and options as in the period before.
Where I can find the options strategy: It’s the Z13 system in the Zorro installation.
Thanks for the post jcl. You mention the Z13.exe file in the Zorro installation but there is no such a file. The last file in .exe is Z12. Would you have this file somewhere? Where can I find the options strategy for it?
I’m not jcl, but can tell that the Z strategies that come with Zorro, including Z13, are all in the “Strategy” folder.
Can anyone help? I need the code from zorro for Market Sentiment Index by Larry Williams. Can anyone please send me the script for this indicator. Send to ashleyhedges1028@gmail.com
Where can I find more examples of risk premium and market models strategies? Thanks.
The code of Larry Williams Sentiment is AFAIK in the Zorro indicators.c library. – In the black book, the systems in chapter 3, 4, 5 are model based, chapter 6 is data mining, and chapters 7 and 8 are risk premia.
Can you elaborate which black book you are referring in the above comments ?
The book that you see when you look at the upper right corner of this page.
Hi jcl and the Zorro team, excellent article again, very appreciated.
Just curious, why is the threshold of a successful system different for Stocks/ETFs and for CfD/FX/Crypto (12%vs.30%) ? Isnt that the reason the other group “failed” ?
Also, I am surprised, that Cryptos failed so miserably. Some respectable blogs promote that the „unmature“ markets provide more inefficiencies than „mature“ markets. Probably, there is no backtest evidence for this Crypto promotion ?
The reason of the different threshold is the higher leverage of currency systems, and accordingly higher volatilty and drawdowns. 30% CAGR for a stock portfolio are pretty good, but I would not trade a forex system with less than 30% annual return.
Immature markets are usually more chaotic and less predictable, especially with cryptos. That’s why it’s harder to build a long-term profitable crypto system.
Nice name: indicator soup.
Thanks for reporting! 🙂
Would be interesting to know more about a successful system.
* What is the worst trade in terms of %?
* What is the position sizing strategy – Kelly, fixed fraction, fixed €?
* Max num losses in a row?
Ps: mostly interested in the evaluation of Forex systems
These details vary greatly among systems, but we cannot publish them because they are client owned.
Johann, a friend of mine seems to think that a martingale system is classified as a “Risk Premium Strategy” because it makes big risks for a small reward. I would call it “indicator soup”. Can you break this tie?
For the record, this was my response to him: In a martingale, if you lose, you need a bigger win to compensate. And if you lose again, you need a bigger win. And if you lose again, you need yet another bigger win! A martingale does not accept loss. Instead, it “revenge trades”. Thus your description of a martingale as a risk premium system is wrong.
“System” normally refers to the entry and exit logic. In that sense Martingale is no system at all, but a way of money management.
If the system trades in a way that wins and losses have a strong negative correlation, then Martingale can indeed have a risk premium effect. I know no such system, but it’s theoretically possible. Normally wins and losses have slightly positive correlation, meaning when you lose, you’re more likely to also lose next time. On such a system, Martingale is just bad money management. You lose more money faster.
Hello jcl and Andrew, I am the friend, and yeah there is a slightly negative correlation between the wins and between the losses (didn’t measure it) but what I am sure about is that once a loss occurs there is 53% chance of having a win, that’s why I believe a martingale with a max allowed consecutive losses is useful in this scenario, right? But the question is : How can I stress test this idea please guys, I have historical data for stocks/forex/crypto but I want to test this mainly on forex (because there is no trend bias)
Hello,
What makes you think option based strategies have higher success?
Is it due to A/ an intrinsic quality of options, or B/ to the fact that options are more complex instruments, used by more financially educated people who have (independently from the instrument) a higher ability to formulate strategies?
Neither. It’s just the empirical experience of their higher success that lets me think so.
Thank you, just THank you / you guys for this experience and overall evaluation of all this Systems.
As an intermediat privat algo trader, i can say:For me that is very help full to see. Such inner insights are very helpful.
Thank you Jcl and also thx to Petra