“It should produce 150 pips per week. With the best indicators that you know. How much does it cost? Please also send live histories of your top systems.”
Although we often get such requests, we still don’t know the best indicators and can’t send live histories. We do not invent systems, but program them from clients’ specifications. And we do not trade them, except for testing. But after almost 1000 systems, we can see a pattern emerging. Which strategies do usually work? Which will fall apart already in the backtest? Here’s a ranking of all systems we did so far, with a surprising winner.
One should think that most clients come up with very similar trading systems, so we could meanwhile just click them together from ready code. But it is not so. There’s apparently no limit of trading ideas. Almost any other system uses some new trading method, unusual data source, or exotic indicator. Still, the systems can be classified in several simple categories. Any of them has its specific success and failure rate.
Trading systems categorized
We classify the systems by their market and by their trading rules, both specified by the client. The 4 main markets are Forex/ CFDs, cryptocurrencies, stocks/ETFs, and options. And the 4 main trading methods are risk premia, market models, data mining, and indicator soups.
Risk premium systems gain higher profits by accepting higher risks. In that category fall many stock portfolio rotation and options trading systems.
Market model systems exploit a particular market inefficiency by detecting anomalies in price curves. Trend following, mean reversion, market cycles, statistical arbitrage, or HFT arbitrage are typical model based trade methods.
Data mining systems predict a price trend by evaluating signals with a machine learning algorithm. Those signals are usually derived from the order book or the price curve, but sometimes also from fundamental data or exotic data sources.
Indicator soups do not target a particular market inefficiency. They generate trade signals from a complex combination of traditional or newly invented and fashionable indicators, with no recognizable concept or idea behind.
Some systems can fall in more than one category. For instance, a grid trader can be considered a risk premium system (high probability of small wins against low probability of high losses), but also a model based system (exploitation of volatility anomalies). If a system cannot be clearly assigned, it is split among several categories in the table below. We can see that clients favor some of the 16 possible combinations, while others are rare:
For determining the success or failure rate, we used 8-years backtests for unoptimized systems, and a walk forward analysis for optimized systems. A successful system had to return at least 12% CAGR for stocks or options, or 30% annual profit for Forex, CFDs, or cryptocurrencies. The R2 parameter had to be above 0.7. If clients ordered a reality check, the system had to pass it at 95% confidence. If one of those conditions was not fulfilled, the system was classified as failure.
The percentages of successful systems:
|Forex/CFDs||88 %||81 %||69 %||31 %||52 %|
|Crypto||0 %||75 %||62 %||25 %||49 %|
|Stocks/ETFs||92 %||85 %||61 %||35 %||80 %|
|Options||96 %||91 %||75 %||58 %||89 %|
|Success rate||93 %||87 %||67 %||32 %||66 %|
The average success rates at the end of the columns and rows are weighted by the number of systems. We can see that the overall success rate was only 66%. In 34% of cases we had to break the bad news to the client that it’s not advised to trade this system live. It produced no, or too little profit in the tests. Sometimes we could see what the problem was, and suggest ways to improve the system. But even total failures were no wasted money. When you know that your favorite manually traded system won’t work in the long run, you’ll save a lot more money than spent for programming and testing.
And the winner is…
The statistics are spoiled by the forex and crypto systems, half of which were losers. This is at least better than most such systems from trading books or trader forums, of which 90% already fail in a proper backtest. We got a surprising result in the ‘Indicator soup’ systems. You would normally expect that they all fail big time, since they are not based on a rational market model or trading method. But in fact almost every third indicator hodgepodge was successful, even in a reality check. Maybe the clients knew more than we did.
It is also a bit surprising that the most complex systems of all, the data mining systems that usually employ deep learning algorithms, did not fare much better. They have an acceptable success rate, but are easily surpassed by a certain sort of much simpler systems.
Of all systems we tested so far, the big winners were the long-term trading systems for stocks, ETFs, or options. Of the option traders, the simpler systems had often better performance. It’s relatively hard to specify a losing option system, but some still managed it by using short expiration dates, complex entries, fancy rollovers, or intraday buying and selling. One of the very simple, but successful option traders was included in the Zorro scripts.
We found another trend that is not visible in the table: With a few exceptions like HFT or arbitrage, there was an almost linear correlation between time frames and performances. Systems on one, five, or ten minute bars were rarely profitable. The good systems traded mostly on 1-hour, 4-hour, or 24-hour time frames. Faster is not always better.
This does not mean that we all should now abandon forex and cryptos and trade only long-term options or ETF portfolios. Diversification is a key to success. All markets still have long periods of ineffectivity and plenty opportunities of trading profits. Maybe the statistics above help to look for them.