One should think that most clients come up with very similar trading systems, so we could meanwhile just click them together from ready code. But it is not so. There’s apparently no limit of trading ideas. Almost any other system uses some new trading method, unusual data source, or exotic indicator. Still, the systems can be classified in several simple categories. Any of them has its specific success and failure rate.

Trading systems categorized

We classify the systems by their market and by their trading rules, both specified by the client. The 4 main markets are Forex/ CFDs, cryptocurrencies, stocks/ETFs/futures, and options. And the 4 main algorithmic trading methods are risk premia, market models, data mining, and indicator soups.  To recap:

Risk premium systems gain higher profits by accepting higher risks. In that category fall many stock portfolio rotation and options trading systems.

Market model systems exploit a particular market inefficiency by detecting anomalies in price curves. Mean reversion, market cycles, market events, or statistical arbitrage are typical model based trade methods. 

Data mining systems predict a price trend by evaluating signals with a machine learning algorithm (aka ‘artificial intelligence’). Those signals are usually derived from the order book or the price curve, but sometimes also from fundamental data or exotic data sources.

Indicator soups are the most often demanded systems. They do not target a particular market inefficiency, except maybe for going with the trend. They generate trade signals from a combination of traditional or new invented ‘technical indicators’. 

Some algo trading systems can fall in more than one category. For instance, a grid trader can be considered a risk premium system (high probability of small wins against low probability of high losses), but also a model based system (exploitation of volatility anomalies). If a system cannot be clearly assigned, it is split among several categories in the table below. We can see that clients favor some of the 16 possible combinations, while others are rare:

For determining the success or failure rate, we used 8-years backtests for unoptimized systems, and a walk forward analysis for optimized systems. A successful system had to return at least 12% CAGR for stocks, futures, or options, or 30% annual profit for Forex, CFDs, or cryptocurrencies. The R2 parameter had to be above 0.7. If clients ordered a Montecarlo analysis, the system had to pass it at 95% confidence. If one of those conditions was not fulfilled, the system was classified as failure. 

The percentages of successful systems:

 The average success rates at the end of the columns and rows are weighted by the number of systems. We can see that the overall success rate was only 66%. In 34% of cases we had to break the bad news to the client that it’s not advised to trade this system live. It produced no, or too little profit in the tests. Sometimes we could see what the problem was, and suggest ways to improve the system. But even total failures were no wasted money. When you know that your favorite manually traded system won’t work in the long run, you’ll save a lot more money than spent for programming and testing. 

And the winner is…

The statistics are spoiled by the forex and crypto systems, half of which were losers. This is at least better than most such systems from trading books or trader forums, of which 90% fail already in a simple out-of-sample test, or at least in a cluster or Montecarlo analysis. We got a surprising result in the ‘Indicator soup’ systems. You would normally expect that they all fail big time, since they are not based on a market model. But in fact almost every third indicator hodgepodge was successful, even in live trading on a test server. Maybe the clients knew more than we did. 

It is also a bit surprising that the most complex systems of all, the data mining systems that usually employ deep learning algorithms, did not fare much better. They have an acceptable success rate, but are easily surpassed by a certain sort of much simpler systems. 

Of all systems we tested so far, the big winners were the long-term trading systems for stocks, ETFs, or options. Of the option traders, the simpler systems had often better performance. It’s relatively hard to specify a losing option system, but some still managed it by using short expiration dates, complex entries, fancy rollovers, or intraday buying and selling. One of the very simple, but successful option traders was included in the Zorro scripts.

We found another trend that is not visible in the table: With a few exceptions like HFT or arbitrage, there was an almost linear correlation between time frames and performances. Systems on one, five, or ten minute bars were rarely profitable. The good systems traded mostly on 1-hour, 4-hour, or 24-hour time frames. Faster is not always better.

This does not mean that we all should now abandon forex and cryptos and trade only long-term options or ETF portfolios. Diversification is a key to success. All markets still have long periods of ineffectivity and plenty opportunities of trading profits. Maybe the statistics above help to look for them. 

Update (2025): This article was posted 5 years ago, and we have meanwhile more than 2000 systems programmed for clients. Due to technical progress especially with machine learning systems the overall success rate is now slightly better at 71%. But the success relations between the various trading methods are still the same.

Related articles

⇒ I Hired a Contract Coder

⇒ Build Better Strategies!

Our example will be a research project – a machine learning experiment for answering two questions. Does a more complex algorithm – such as, more neurons and deeper learning – produce a better prediction? And are short-term price moves predictable by short-term price history? The last question came up due to my scepticism about price action trading in the previous part of this series. I got several emails asking about the “trading system generators” or similar price action tools that are praised on some websites. There is no hard evidence that such tools ever produced any profit (except for their vendors) – but does this mean that they all are garbage? We’ll see.

Our experiment is simple: We collect information from the last candles of a price curve, feed it in a deep learning neural net, and use it to predict the next candles. My hypothesis is that a few candles don’t contain any useful predictive information. Of course, a nonpredictive outcome of the experiment won’t mean that I’m right, since I could have used wrong parameters or prepared the data badly. But a predictive outcome would be a hint that I’m wrong and price action trading can indeed be profitable.

Machine learning strategy development
Step 1: The target variable

To recap the previous part: a supervised learning algorithm is trained with a set of features in order to predict a target variable. So the first thing to determine is what this target variable shall be. A popular target, used in most papers, is the sign of the price return at the next bar. Better suited for prediction, since less susceptible to randomness, is the price difference to a more distant prediction horizon, like 3 bars from now, or same day next week. Like almost anything in trading systems, the prediction horizon is a compromise between the effects of randomness (less bars are worse) and predictability (less bars are better).

Sometimes you’re not interested in directly predicting price, but in predicting some other parameter – such as the current leg of a Zigzag indicator – that could otherwise only be determined in hindsight. Or you want to know if a certain market inefficiency will be present in the next time, especially when you’re using machine learning not directly for trading, but for filtering trades in a model-based system. Or you want to predict something entirely different, for instance the probability of a market crash tomorrow. All this is often easier to predict than the popular tomorrow’s return.

In our price action experiment we’ll use the return of a short-term price action trade as target variable. Once the target is determined, next step is selecting the features.

Step 2: The features

A price curve is the worst case for any machine learning algorithm. Not only does it carry little signal and mostly noise, it is also nonstationary and the signal/noise ratio changes all the time. The exact ratio of signal and noise depends on what is meant with “signal”, but it is normally too low for any known machine learning algorithm to produce anything useful. So we must derive features from the price curve that contain more signal and less noise. Signal, in that context, is any information that can be used to predict the target, whatever it is. All the rest is noise.

Thus, selecting the features is critical for success – even more critical than deciding which machine learning algorithm you’re going to use. There are two approaches for selecting features. The first and most common is extracting as much information from the price curve as possible. Since you do not know where the information is hidden, you just generate a wild collection of indicators with a wide range of parameters, and hope that at least a few of them will contain the information that the algorithm needs. This is the approach that you normally find in the literature. The problem of this method: Any machine learning algorithm is easily confused by nonpredictive predictors. So it won’t do to just throw 150 indicators at it. You need some preselection algorithm that determines which of them carry useful information and which can be omitted. Without reducing the features this way to maybe eight or ten, even the deepest learning algorithm won’t produce anything useful.

The other approach, normally for experiments and research, is using only limited information from the price curve. This is the case here: Since we want to examine price action trading, we only use the last few prices as inputs, and must discard all the rest of the curve. This has the advantage that we don’t need any preselection algorithm since the number of features is limited anyway. Here are the two simple predictor functions that we use in our experiment (in C):

var change(int n)
{
	return scale((priceClose(0) - priceClose(n))/priceClose(0),100)/100;
}

var range(int n)
{
	return scale((HH(n) - LL(n))/priceClose(0),100)/100;
}

The two functions are supposed to carry the necessary information for price action: per-bar movement and volatility. The change function is the difference of the current price to the price of n bars before, divided by the current price. The range function is the total high-low distance of the last n candles, also in divided by the current price. And the scale function centers and compresses the values to the +/-100 range, so we divide them by 100 for getting them normalized to +/-1. We remember that normalizing is needed for machine learning algorithms.

Step 3: Preselecting predictors

When you have selected a large number of indicators or other signals as features for your algorithm, you must determine which of them is useful and which not. There are many methods for reducing the number of features, for instance:

• Determine the correlations between the signals. Remove those with a strong correlation to other signals, since they do not contribute to the information.
• Compare the information content of signals directly, with algorithms like information entropy or decision trees.
• Determine the information content indirectly by comparing the signals with randomized signals; there are some software libraries for this, such as the R Boruta package.
• Use an algorithm like Principal Components Analysis (PCA) for generating a new signal set with reduced dimensionality.
• Use genetic optimization for determining the most important signals just by the most profitable results from the prediction process. Great for curve fitting if you want to publish impressive results in a research paper.

Reducing the number of features is important for most machine learning algorithms, including shallow neural nets. For deep learning it’s less important, since deep nets  with many neurons are normally able to process huge feature sets and discard redundant features. For our experiment we do not preselect or preprocess the features, but you can find useful information about this in articles (1), (2), and (3) listed at the end of the page.

Step 4: Select the machine learning algorithm

R offers many different ML packages, and any of them offers many different algorithms with many different parameters. Even if you already decided about the method – here, deep learning – you have still the choice among different approaches and different R packages. Most are quite new, and you can find not many empirical information that helps your decision. You have to try them all and gain experience with different methods. For our experiment we’ve choosen the Deepnet package, which is probably the simplest and easiest to use deep learning library. This keeps our code short. We’re using its Stacked Autoencoder (SAE) algorithm for pre-training the network. Deepnet also offers a Restricted Boltzmann Machine (RBM) for pre-training, but I could not get good results from it. There are other and more complex deep learning packages for R, so you can spend a lot of time checking out all of them.

How pre-training works is easily explained, but why it works is a different matter. As to my knowledge, no one has yet come up with a solid mathematical proof that it works at all. Anyway, imagine a large neural net with many hidden layers:

Training the net means setting up the connection weights between the neurons. The usual method is error backpropagation. But it turns out that the more hidden layers you have, the worse it works. The backpropagated error terms get smaller and smaller from layer to layer, causing the first layers of the net to learn almost nothing. Which means that the predicted result becomes more and more dependent of the random initial state of the weights. This severely limited the complexity of layer-based neural nets and therefore the tasks that they can solve. At least until 10 years ago.

In 2006 scientists in Toronto first published the idea to pre-train the weights with an unsupervised learning algorithm, a restricted Boltzmann machine. This turned out a revolutionary concept. It boosted the development of artificial intelligence and allowed all sorts of new applications from Go-playing machines to self-driving cars. Meanwhile, several new improvements and algorithms for deep learning have been found. A stacked autoencoder works this way:

1. Select the hidden layer to train; begin with the first hidden layer. Connect its outputs to a temporary output layer that has the same structure as the network’s input layer.
2. Feed the network with the training samples, but without the targets. Train it so that the first hidden layer reproduces the input signal – the features – at its outputs as exactly as possible. The rest of the network is ignored. During training, apply a ‘weight penalty term’ so that as few connection weights as possible are used for reproducing the signal.
3. Now feed the outputs of the trained hidden layer to the inputs of the next untrained hidden layer, and repeat the training process so that the input signal is now reproduced at the outputs of the next layer.
4. Repeat this process until all hidden layers are trained. We have now a ‘sparse network’ with very few layer connections that can reproduce the input signals.
5. Now train the network with backpropagation for learning the target variable, using the pre-trained weights of the hidden layers as a starting point.

The hope is that the unsupervised pre-training process produces an internal noise-reduced abstraction of the input signals that can then be used for easier learning the target. And this indeed appears to work. No one really knows why, but several theories – see paper (4) below – try to explain that phenomenon.

Step 5: Generate a test data set

We first need to produce a data set with features and targets so that we can test our prediction process and try out parameters. The features must be based on the same price data as in live trading, and for the target we must simulate a short-term trade. So it makes sense to generate the data not with R, but with our trading platform, which is anyway a lot faster. Here’s a small Zorro script for this, DeepSignals.c:

function run()
{
	StartDate = 20140601; // start two years ago
	BarPeriod = 60; // use 1-hour bars
	LookBack = 100; // needed for scale()

	set(RULES);   // generate signals
	LifeTime = 3; // prediction horizon
	Spread = RollLong = RollShort = Commission = Slippage = 0;
	
	adviseLong(SIGNALS+BALANCED,0,
		change(1),change(2),change(3),change(4),
		range(1),range(2),range(3),range(4));
	enterLong(); 
}

We’re generating 2 years of data with features calculated by our above defined change and range functions. Our target is the result of a trade with 3 bars life time. Trading costs are set to zero, so in this case the result is equivalent to the sign of the price difference at 3 bars in the future. The adviseLong function is described in the Zorro manual; it is a mighty function that automatically handles training and predicting and allows to use any R-based machine learning algorithm just as if it were a simple indicator.

In our code, the function uses the next trade return as target, and the price changes and ranges of the last 4 bars as features. The SIGNALS flag tells it not to train the data, but to export it to a .csv file. The BALANCED flag makes sure that we get as many positive as negative returns; this is important for most machine learning algorithms. Run the script in [Train] mode with our usual test asset EUR/USD selected. It generates a spreadsheet file named DeepSignalsEURUSD_L.csv that contains the features in the first 8 columns, and the trade return in the last column.

Step 6: Calibrate the algorithm

Complex machine learning algorithms have many parameters to adjust. Some of them offer great opportunities to curve-fit the algorithm for publications. Still, we must calibrate parameters since the algorithm rarely works well with its default settings. For this, here’s an R script that reads the previously created data set and processes it with the deep learning algorithm (DeepSignal.r): 

library('deepnet', quietly = T) 
library('caret', quietly = T)

neural.train = function(model,XY) 
{
  XY <- as.matrix(XY)
  X <- XY[,-ncol(XY)]
  Y <- XY[,ncol(XY)]
  Y <- ifelse(Y > 0,1,0)
  Models[[model]] <<- sae.dnn.train(X,Y,
      hidden = c(50,100,50), 
      activationfun = "tanh", 
      learningrate = 0.5, 
      momentum = 0.5, 
      learningrate_scale = 1.0, 
      output = "sigm", 
      sae_output = "linear", 
      numepochs = 100, 
      batchsize = 100,
      hidden_dropout = 0, 
      visible_dropout = 0)
}

neural.predict = function(model,X) 
{
  if(is.vector(X)) X <- t(X)
  return(nn.predict(Models[[model]],X))
}

neural.init = function()
{
  set.seed(365)
  Models <<- vector("list")
}

TestOOS = function() 
{
  neural.init()
  XY <<- read.csv('C:/Zorro/Data/DeepSignalsEURUSD_L.csv',header = F)
  splits <- nrow(XY)*0.8
  XY.tr <<- head(XY,splits);
  XY.ts <<- tail(XY,-splits)
  neural.train(1,XY.tr)
  X <<- XY.ts[,-ncol(XY.ts)]
  Y <<- XY.ts[,ncol(XY.ts)]
  Y.ob <<- ifelse(Y > 0,1,0)
  Y <<- neural.predict(1,X)
  Y.pr <<- ifelse(Y > 0.5,1,0)
  confusionMatrix(Y.pr,Y.ob)
}

We’ve defined three functions neural.train, neural.predict, and neural.init for training, predicting, and initializing the neural net. The function names are not arbitrary, but follow the convention used by Zorro’s advise(NEURAL,..) function. It doesn’t matter now, but will matter later when we use the same R script for training and trading the deep learning strategy. A fourth function, TestOOS, is used for out-of-sample testing our setup.

The function neural.init seeds the R random generator with a fixed value (365 is my personal lucky number). Otherwise we would get a slightly different result any time, since the neural net is initialized with random weights. It also creates a global R list named “Models”. Most R variable types don’t need to be created beforehand, some do (don’t ask me why). The ‘<<-‘ operator is for accessing a global variable from within a function.

The function neural.train takes as input a model number and the data set to be trained. The model number identifies the trained model in the “Models” list. A list is not really needed for this test, but we’ll need it for more complex strategies that train more than one model. The matrix containing the features and target is passed to the function as second parameter. If the XY data is not a proper matrix, which frequently happens in R depending on how you generated it, it is converted to one. Then it is split into the features (X) and the target (Y), and finally the target is converted to 1 for a positive trade outcome and 0 for a negative outcome. 

The network parameters are then set up. Some are obvious, others are free to play around with:

• The network structure is given by the hidden vector:  c(50,100,50) defines 3 hidden layers, the first with 50, second with 100, and third with 50 neurons. That’s the parameter that we’ll later modify for determining whether deeper is better.
• The activation function converts the sum of neuron input values to the neuron output; most often used are sigmoid that saturates to 0 or 1, or tanh that saturates to -1 or +1.

We use tanh here since our signals are also in the +/-1 range. The output of the network is a sigmoid function since we want a prediction in the 0..1 range. But the SAE output must be “linear” so that the Stacked Autoencoder can reproduce the analog input signals on the outputs. Recently in fashion came RLUs, Rectified Linear Units, as activation functions for internal layers. RLUs are faster and partially overcome the above mentioned backpropagation problem, but are not supported by deepnet.

• The learning rate controls the step size for the gradient descent in training; a lower rate means finer steps and possibly more precise prediction, but longer training time.
• Momentum adds a fraction of the previous step to the current one. It prevents the gradient descent from getting stuck at a tiny local minimum or saddle point.
• The learning rate scale is a multiplication factor for changing the learning rate after each iteration (I am not sure for what this is good, but there may be tasks where a lower learning rate on higher epochs improves the training).
• An epoch is a training iteration over the entire data set. Training will stop once the number of epochs is reached. More epochs mean better prediction, but longer training.
• The batch size is a number of random samples – a mini batch – taken out of the data set for a single training run. Splitting the data into mini batches speeds up training since the weight gradient is then calculated from fewer samples. The higher the batch size, the better is the training, but the more time it will take.
• The dropout is a number of randomly selected neurons that are disabled during a mini batch. This way the net learns only with a part of its neurons. This seems a strange idea, but can effectively reduce overfitting.

All these parameters are common for neural networks. Play around with them and check their effect on the result and the training time. Properly calibrating a neural net is not trivial and might be the topic of another article. The parameters are stored in the model together with the matrix of trained connection weights. So they need not to be given again in the prediction function, neural.predict. It takes the model and a vector X of features, runs it through the layers, and returns the network output, the predicted target Y. Compared with training, prediction is pretty fast since it only needs a couple thousand multiplications. If X was a row vector, it is transposed and this way converted to a column vector, otherwise the nn.predict function won’t accept it.

Use RStudio or some similar environment for conveniently working with R. Edit the path to the .csv data in the file above, source it, install the required R packages (deepnet, e1071, and caret), then call the TestOOS function from the command line. If everything works, it should print something like that:

> TestOOS()
begin to train sae ......
training layer 1 autoencoder ...
####loss on step 10000 is : 0.000079
training layer 2 autoencoder ...
####loss on step 10000 is : 0.000085
training layer 3 autoencoder ...
####loss on step 10000 is : 0.000113
sae has been trained.
begin to train deep nn ......
####loss on step 10000 is : 0.123806
deep nn has been trained.
Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 1231  808
         1  512  934
                                          
               Accuracy : 0.6212          
                 95% CI : (0.6049, 0.6374)
    No Information Rate : 0.5001          
    P-Value [Acc > NIR] : < 2.2e-16       
                                          
                  Kappa : 0.2424          
 Mcnemar's Test P-Value : 4.677e-16       
                                          
            Sensitivity : 0.7063          
            Specificity : 0.5362          
         Pos Pred Value : 0.6037          
         Neg Pred Value : 0.6459          
             Prevalence : 0.5001          
         Detection Rate : 0.3532          
   Detection Prevalence : 0.5851          
      Balanced Accuracy : 0.6212          
                                          
       'Positive' Class : 0               
                                          
> 

TestOOS reads first our data set from Zorro’s Data folder. It splits the data in 80% for training (XY.tr) and 20% for out-of-sample testing (XY.ts). The training set is trained and the result stored in the Models list at index 1. The test set is further split in features (X) and targets (Y). Y is converted to binary 0 or 1 and stored in Y.ob, our vector of observed targets. We then predict the targets from the test set, convert them again to binary 0 or 1 and store them in Y.pr. For comparing the observation with the prediction, we use the confusionMatrix function from the caret package.

A confusion matrix of a binary classifier is simply a 2×2 matrix that tells how many 0’s and how many 1’s had been predicted wrongly and correctly. A lot of metrics are derived from the matrix and printed in the lines above. The most important at the moment is the 62% prediction accuracy. This may hint that I bashed price action trading a little prematurely. But of course the 62% might have been just luck. We’ll see that later when we run a WFO test.

A final advice: R packages are occasionally updated, with the possible consequence that previous R code suddenly might work differently, or not at all. This really happens, so test carefully after any update.

Step 7: The strategy

Now that we’ve tested our algorithm and got some prediction accuracy above 50% with a test data set, we can finally code our machine learning strategy. In fact we’ve already coded most of it, we just must add a few lines to the above Zorro script that exported the data set. This is the final script for training, testing, and (theoretically) trading the system (DeepLearn.c):

#include <r.h>

function run()
{
	StartDate = 20140601;
	BarPeriod = 60;	// 1 hour
	LookBack = 100;

	WFOPeriod = 252*24; // 1 year
	DataSplit = 90;
	NumCores = -1;  // use all CPU cores but one

	set(RULES);
	Spread = RollLong = RollShort = Commission = Slippage = 0;
	LifeTime = 3;
	if(Train) Hedge = 2;
	
	if(adviseLong(NEURAL+BALANCED,0,
		change(1),change(2),change(3),change(4),
		range(1),range(2),range(3),range(4)) > 0.5) 
		enterLong();
	if(adviseShort() > 0.5) 
		enterShort();
}

We’re using a WFO cycle of one year, split in a 90% training and a 10% out-of-sample test period. You might ask why I have earlier used two year’s data and a different split, 80/20, for calibrating the network in step 5. This is for using differently composed data for calibrating and for walk forward testing. If we used exactly the same data, the calibration might overfit it and compromise the test. 

The selected WFO parameters mean that the system is trained with about 225 days data, followed by a 25 days test or trade period. Thus, in live trading the system would retrain every 25 days, using the prices from the previous 225 days. In the literature you’ll sometimes find the recommendation to retrain a machine learning system after any trade, or at least any day. But this does not make much sense to me. When you used almost 1 year’s data for training a system, it can obviously not deteriorate after a single day. Or if it did, and only produced positive test results with daily retraining, I would strongly suspect that the results are artifacts by some coding mistake.

Training a deep network takes really a long time, in our case about 10 minutes for a network with 3 hidden layers and 200 neurons. In live trading this would be done by a second Zorro process that is automatically started by the trading Zorro. In the backtest, the system trains at any WFO cycle. Therefore using multiple cores is recommended for training many cycles in parallel. The NumCores variable at -1 activates all CPU cores but one. Multiple cores are only available in Zorro S, so a complete walk forward test with all WFO cycles can take several hours with the free version.

In the script we now train both long and short trades. For this we have to allow hedging in Training mode, since long and short positions are open at the same time. Entering a position is now dependent on the return value from the advise function, which in turn calls either the neural.train or the neural.predict function from the R script. So we’re here entering positions when the neural net predicts a result above 0.5. 

The R script is now controlled by the Zorro script (for this it must have the same name, DeepLearn.r, only with different extension). It is identical to our R script above since we’re using the same network parameters. Only one additional function is needed for supporting a WFO test:

neural.save = function(name)
{
  save(Models,file=name)  
}

The neural.save function stores the Models list – it now contains 2 models for long and for short trades – after every training run in Zorro’s Data folder. Since the models are stored for later use, we do not need to train them again for repeated test runs.

This is the WFO equity curve generated with the script above (EUR/USD, without trading costs):

Although not all WFO cycles get a positive result, it seems that there is some predictive effect. The curve is equivalent to an annual return of 89%, achieved with a 50-100-50 hidden layer structure. We’ll check in the next step how different network structures affect the result.

Since the neural.init, neural.train, neural.predict, and neural.save functions are automatically called by Zorro’s adviseLong/adviseShort functions, there are no R functions directly called in the Zorro script. Thus the script can remain unchanged when using a different machine learning method. Only the DeepLearn.r script must be modified and the neural net, for instance, replaced by a support vector machine. For trading such a machine learning system live on a VPS, make sure that R is also installed on the VPS, the needed R packages are installed, and the path to the R terminal set up in Zorro’s ini file. Otherwise you’ll get an error message when starting the strategy.

Step 8: The experiment

If our goal had been developing a strategy, the next steps would be the reality check, risk and money management, and preparing for live trading just as described under model-based strategy development. But for our experiment we’ll now run a series of tests, with the number of neurons per layer increased from 10 to 100 in 3 steps, and 1, 2, or 3 hidden layers (deepnet does not support more than 3). So we’re looking into the following 9 network structures: c(10), c(10,10), c(10,10,10), c(30), c(30,30), c(30,30,30), c(100), c(100,100), c(100,100,100). For this experiment you need an afternoon even with a fast PC and in multiple core mode. Here are the results (SR = Sharpe ratio, R2 = slope linearity): 

	* 10 neurons	* 30 neurons	* 100 neurons
1
2
3

We see that a simple net with only 10 neurons in a single hidden layer won’t work well for short-term prediction. Network complexity clearly improves the performance, however only up to a certain point. A good result for our system is already achieved with 3 layers x 30 neurons. Even more neurons won’t help much and sometimes even produce a worse result. This is no real surprise, since for processing only 8 inputs, 300 neurons can likely not do a better job than 100.  

Conclusion

Our goal was determining if a few candles can have predictive power and how the results are affected by the complexity of the algorithm. The results seem to suggest that short-term price movements can indeed be predicted sometimes by analyzing the changes and ranges of the last 4 candles. The prediction is not very accurate – it’s in the 58%..60% range, and most systems of the test series become unprofitable when trading costs are included. Still, I have to reconsider my opinion about price action trading. The fact that the prediction improves with network complexity is an especially convincing argument for short-term price predictability.

It would be interesting to look into the long-term stability of predictive price patterns. For this we had to run another series of experiments and modify the training period (WFOPeriod in the script above) and the 90% IS/OOS split. This takes longer time since we must use more historical data. I have done a few tests and found so far that a year seems to be indeed a good training period. The system deteriorates with periods longer than a few years. Predictive price patterns, at least of EUR/USD, have a limited lifetime.

Where can we go from here? There’s a plethora of possibilities, for instance:

• Use inputs from more candles and process them with far bigger networks with thousands of neurons.
• Use oversampling for expanding the training data. Prediction always improves with more training samples.
• Compress time series f.i. with spectal analysis and analyze not the candles, but their frequency representation with machine learning methods.
• Use inputs from many candles – such as, 100 – and pre-process adjacent candles with one-dimensional convolutional network layers.
• Use recurrent networks. Especially LSTM could be very interesting for analyzing time series – and as to my knowledge, they have been rarely used for financial prediction so far.
• Use an ensemble of neural networks for prediction, such as Aronson’s “oracles” and “comitees”.

Papers / Articles

(1) A.S.Sisodiya, Reducing Dimensionality of Data 
(2) K.Longmore, Machine Learning for Financial Prediction 
(3) V.Perervenko, Selection of Variables for Machine Learning
(4) D.Erhan et al, Why Does Pre-training Help Deep Learning?

I’ve added the C and R scripts to the 2016 script repository. You need both in Zorro’s Strategy folder. Zorro version 1.474, and R version 3.2.5 (64 bit) was used for the experiment, but it should also work with other versions. 

It is certainly tempting to earn profits within minutes. Additionally, short time frames produce more bars and trades – a great advantage for strategy development. The quality of test and training depends on the amount of data, and timely price data is always in short supply. Still, scalping – opening and closing trades in minutes or seconds – is largely considered nonsense and irrational by algo traders. Four main reasons are given:

1. Short time frames cause high trading costs – slippage, spread, commission – in relation to the expected profit.
2. Short time frames expose more ‘noise’, ‘randomness’ and ‘artifacts’ in the price curve, which reduces profit and increases risk.
3. Any algorithms had to be individually adapted to the broker or price data provider due to price feed dependency in short time frames.
4. Algorithmic strategies usually cease working below a certain time frame.

Higher costs, less profit, more risk, feed dependency, no working strategies – seemingly good arguments against scalping (HFT is a very different matter). But never trust common wisdom, especially not in trading. That’s why I had not yet added scalping to my list of irrational trade methods. I can confirm reasons number 3 and 4 from my own experiences: Below bar periods of about 10 minutes, backtests with price histories from different brokers began to produce noticeably different results. And I never managed to develop a strategy with a significantly positive walk-forward test on bar periods less than 30 minutes. But this does not mean that such a strategy does not exist. Maybe short time frames just need special trade methods?

So I’ve programmed an experiment for finding out once and for all if scalping is really as bad as it’s rumored to be. Then I can at least give some reasoned advice to the next client who desires a tick-triggered short-term trading strategy.

Trading costs examined

The first part of the experiment is easily done: a statistic of the impact of trading costs. Higher costs obviously require more profits for compensation. How many trades must you win for overcoming the trading costs at different time frames? Here’s a short script (in C, for Zorro) for answering this question:

function run()
{
  BarPeriod = 1;
  LookBack = 1440;
  Commission = 0.60;
  Spread = 0.5*PIP;

  int duration = 1, i = 0;
  if(!is(LOOKBACK))
    while(duration <= 1440)
  { 
    var Return = abs(priceClose(0)-priceClose(duration))*PIPCost/PIP;
    var Cost = Commission*LotAmount/10000. + Spread*PIPCost/PIP;
    var Rate = ifelse(Return > Cost, Cost/(2*Return) + 0.5, 1.);

    plotBar("Min Rate",i++,duration,100*Rate,AVG+BARS,RED); 
 
    if(duration < 10) duration += 1;
    else if(duration < 60) duration += 5;
    else if(duration < 180) duration += 30;
    else duration += 60;
  }
  Bar += 100; // hack!
}

This script calculates the minimum win rate to compensate the trade costs for different trade durations. We assumed here a spread of 0.5 pips and a round turn commission of 60 cents per 10,000 contracts – that’s average costs of a Forex trade. PIPCost/PIP in the above script is the conversion factor from a price difference to a win or loss on the account. We’re also assuming no win/loss bias: Trades shall win or lose on average the same amount. This allows us to split the Return of any trade in a win and a loss, determined by WinRate. The win is WinRate * Return and the loss is (1-WinRate) * Return. For breaking even, the win minus the loss must cover the cost. The required win rate for this is

WinRate = Cost/(2*Return) + 0.5

The win rate is averaged over all bars and plotted in a histogram of trade durations from 1 minute up to 1 day. The duration is varied in steps of 1, 5, 30, and 60 minutes. We’re entering a trade for any duration every 101 minutes (Bar += 100 in the script is a hack for running the simulation in steps of 101 minutes, while still maintaining the 1-minute bar period).

The script needs a few seconds to run, then produces this histogram (for EUR/USD and 2015):

You need about 53% win rate for covering the costs of 1-day trades (rightmost bar), but 90% win rate for 1-minute trades! Or alternatively, a 9:1 reward to risk ratio at 50% win rate. This exceeds the best performances of real trading systems by a large amount, and seems to confirm convincingly the first reason why you better take tales by scalping heroes on trader forums with a grain of salt.

But what about reason number two – that short time frames are plagued with ‘noise’ and ‘randomness’? Or is it maybe the other way around and some effect makes short time frames even more predictable? That’s a little harder to test.

Measuring randomness

‘Noise’ is often identified with the high-frequency components of a signal. Naturally, short time frames produce more high-frequency components than long time frames. They could be detected with a highpass filter, or eliminated with a lowpass filter. Only problem: Price curve noise is not always related to high frequencies. Noise is just the part of the curve that does not carry information about the trading signal. For cycle trading, high frequencies are the signal and low-frequency trend is the noise. So the jaggies and ripples of a short time frame price curve might be just the very inefficiencies that you want to exploit. It depends on the strategy what noise is; there is no ‘general price noise’.

Thus we need a better criteria for determining the tradeability of a price curve. That criteria is randomness. You can not trade a random market, but you can potentially trade anything that deviates from randomness. Randomness can be measured through the information content of the price curve. A good measure of information content is the Shannon Entropy. It is defined this way:

This formula basically measures disorder. A very ordered, predictable signal has low entropy. A random, unpredictable signal has high entropy. In the formula, P(s_i) is the relative frequency of a certain pattern s_i in the signal S. The entropy is at maximum when all patterns are evenly distributed and all P(s_i) have about the same value. If some patterns appear more frequently than other patterns, the entropy goes down. The signal is then less random and more predictable. The Shannon Entropy is measured in bit.

The problem: Zorro has tons of indicators, even the Shannon Gain, but not the Shannon Entropy! So I have no choice but to write a new indicator, which fortunately is my job anyway. This is the source code of the Shannon Entropy of a char string:

var ShannonEntropy(char *S,int Length)
{
  static var Hist[256];
  memset(Hist,0,256*sizeof(var));
  var Step = 1./Length;
  int i;
  for(i=0; i<Length; i++) 
    Hist[S[i]] += Step;
  var H = 0;
  for(i=0;i<256;i++) {
    if(Hist[i] > 0.)
      H -= Hist[i]*log2(Hist[i]);
  }
  return H;
}

A char has 8 bit, so 2⁸ = 256 different chars can appear in a string. The frequency of each char is counted and stored in the Hist array. So this array contains the P(s_i) of the above entropy formula. They are multiplied with their binary logarithm and summed up; the result is H(S), the Shannon Entropy.

In the above code, a char is a pattern of the signal. So we need to convert our price curve into char patterns. This is done by a second ShannonEntropy function that calls the first one:

var ShannonEntropy(var *Data,int Length,int PatternSize)
{
  static char S[1024]; // hack!
  int i,j;
  int Size = min(Length-PatternSize-1,1024);
  for(i=0; i<Size; i++) {
    int C = 0;
    for(j=0; j<PatternSize; j++) {
    if(Data[i+j] > Data[i+j+1])
      C += 1<<j;
    }
    S[i] = C;
  }
  return ShannonEntropy(S,Size);
}

PatternSize determines the partitioning of the price curve. A pattern is defined by a number of price changes. Each price is either higher than the previous price, or it is not; this is a binary information and constitutes one bit of the pattern. A pattern can consist of up to 8 bits, equivalent to 256 combinations of price changes. The patterns are stored in a char string. Their entropy is then determined by calling the first ShannonEntropy function with that string (both functions have the same name, but the compiler can distinguish them from their different parameters). Patterns are generated from any price and the subsequent PatternSize prices; then the procedure is repeated with the next price. So the patterns overlap.

An unexpected result

Now we only need to produce a histogram of the Shannon Entropy, similar to the win rate in our first script:

function run()
{
  BarPeriod = 1;
  LookBack = 1440*300;
  StartWeek = 10000;
 
  int Duration = 1, i = 0;
  while(Duration <= 1440)
  { 
    TimeFrame = frameSync(Duration);
    var *Prices = series(price(),300);

    if(!is(LOOKBACK) && 0 == (Bar%101)) {
      var H = ShannonEntropy(Prices,300,3);
      plotBar("Randomness",i++,Duration,H,AVG+BARS,BLUE);	
    }
    if(Duration < 10) Duration += 1;
    else if(Duration < 60) Duration += 5;
    else if(Duration < 240) Duration += 30;
    else if(Duration < 720) Duration += 120;
    else Duration += 720;
  }
}

The entropy is calculated for all time frames at every 101th bar, determined with the modulo function. (Why 101? In such cases I’m using odd numbers for preventing synchronization effects). I cannot use here the hack with skipping the next 100 bars as in the previous script, as skipping bars would prevent proper shifting of the price series. That’s why this script must really grind through any minute of 3 years, and needs several minutes to complete.

Two code lines should be explained because they are critical for measuring the entropy of daily candles using less-than-a-day bar periods:

StartWeek = 10000;

This starts the week at Monday midnight (1 = Monday, 00 00 = midnight) instead of Sunday 11 pm. This line was missing at first and I wondered why the entropy of daily candles was higher than I expected. Reason:  The single Sunday hour at 11 pm counted as a full day and noticeably increased the randomness of daily candles.

TimeFrame = frameSync(Duration);

This synchronizes the time frame to full hours respectively days. If this is missing, the Shannon Entropy of daily candles gets again a too high value since the candles are not in sync with a day anymore. A day has often less than 1440 one-minute bars due to weekends and irregularities in the historical data.

The Shannon Entropy is calculated with a pattern size of 3 price changes, resulting in 8 different patterns. 3 bit is the maximum entropy for 8 patterns. As price changes are not completely random, I expected an entropy value slightly smaller than 3, steadily increasing when time frames are decreasing. However I got this interesting histogram (EUR/USD, 2013-2015, FXCM price data):

The entropy is almost, but not quite 3 bit. This confirms that price patterns are not absolutely random. We can see that the 1440 minutes time frame has the lowest Shannon Entropy at about 2.9 bit. This was expected, as the daily cycle has a strong effect on the price curve, and daily candles are thus more regular than candles of other time frames. For this reason price action or price pattern algorithms often use daily candles. The entropy increases with decreasing time frames, but only down to time frames  of about ten minutes. Even lower time frames are actually less random!

This is an unexpected result. The lower the time frame, the less price quotes does it contain, so the impact of chance should be in fact higher. But the opposite is the case. I could produce similar results with other patterns of 4 and 5 bit, and also with other assets. For making sure I continued the experiment with a different, tick-based price history and even shorter time frames of 2, 5, 10, 15, 30, 45, and 60 seconds (Zorro’s “useless” micro time frames now came in handy, after all):

The x axis is now in second units instead of minutes. We see that price randomness continues to drop with the time frame.

There are several possible explanations. Price granularity is higher at low time frames due to the smaller number of ticks. High-volume trades are often split into many small parts (‘iceberg trades‘) and may cause a sequence of similar price quotes in short intervals. All this reduces the price entropy of short time frames. But it does not necessarily increase trade opportunities:  A series of identical quotes has zero entropy and is 100% predictable, but can not be traded. Of course, iceberg trades are still an interesting inefficiency that could theoretically be exploited – if it weren’t for the high trading costs. So that’s something to look further into only when you have direct market access and no broker fees.

I have again uploaded the scripts to the 2015 scripts collection. You’ll need Zorro 1.36 or above for reproducing the results. Zorro S and tick based data are needed for the second time frames.

Conclusions

• Scalping is not completely nuts. Very low time frames expose some regularity.
• Whatever the reason, this regularity can not be exploited by retail traders due to the high costs of short term trades.
• On time frames above 60 minutes prices become less random and more regular. This recommends long time frames for algo trading.
• The most regular price patterns appear with 1-day bars. They also cause the least trading costs.

Papers

Shannon Entropy: Lecture

Suppose you want to trade by moon phases. But you’re not sure if you shall buy at full moon and sell at new moon, or the other way around. So you do a series of moon phase backtests and find out that the best system, which opens positions at any first quarter moon, achieves 30% annual return. Is this finally the proof that astrology works?

A trade system based on a nonexisting effect has normally a profit expectancy of zero (minus trade costs). But you won’t get zero when you backtest variants of such a system. Due to statistical fluctuations, some of them will produce a positive and some a negative return. When you now pick the best performer, such as the first quarter moon trading system, you might get a high return and an impressive equity curve in the backtest. Sadly, its test result is not necessarily caused by clever trading. It might be just by clever selecting the random best performer from a pool of useless systems.

For finding out if the 30% return by quarter moon trading are for real or just the fool’s gold of Data Mining Bias, Halbert White (1959-2012) invented a test method in 2000.  White’s Reality Check (aka Bootstrap Reality Check) is explained in detail in the book ‘Evidence-Based Technical Analysis’ by David Aronson. It works this way:

1. Develop a strategy. During the development process, keep a record of all strategy variants that were tested and discarded because of their test results, including all abandoned algorithms, ideas, methods, and parameters. It does not matter if they were discarded by human decision or by a computer search or optimizing process.
2. Produce balance curves of all strategy variants, using detrended trade results and no trade costs. Note down the profit P of the best strategy.
3. Detrend all balance curves by subtracting the mean return per bar (not to be confused with detrending the trade results!). This way you get a series of curves with the same characteristics of the tested systems, but zero profit.
4. Randomize all curves by bootstrap with replacement. This produces new curves from the random returns of the old curves. Because the same bars can be selected multiple times, most new curves now produce losses or profits different from zero.
5. Select the best performer from the randomized curves, and note down its profit.
6.  Repeat steps 4 and 5 a couple 1000 times.
7. You now have a list of several 1000 best profits. The median M of that list is the Data Mining Bias by your strategy development process.
8. Check where the original best profit P appears in the list. The percentage of best bootstrap profits greater than P is the so-called p-Value of the best strategy. You want the p-Value to be as low as possible. If P is better than 95% of the best bootstrap profits, the best strategy has a real edge with 95% probability.
9. P minus M minus trade costs is the result to be expected in real trading the best strategy.

The method is not really intuitive, but mathematically sound. However, it suffers from a couple problems that makes WRC difficult to use  in real strategy development:

• You can see the worst problem already in step 1. During strategy development you’re permanently testing ideas, adding or removing parameters, or checking out different assets and time frames. Putting aside all discarded variants and producing balance curves of all combinations of them is a cumbersome process. It gets even more diffcult with machine learning algorithms that optimize weight factors and usually do not produce discarded variants. However, the good news are that you can easily apply the WRC when your strategy variants are produced by a transparent mechanical process with no human decisions involved. That’s fortunately the case for our trend experiment.
• WRC tends to type II errors. That means it can reject strategies although they have an edge. When more irrelevant variants – systems with random trading and zero profit expectancy – are added to the pool,  more positive results can be produced in steps 4 and 5, which reduces the probability that your selected strategy survives the test. WRC can determine rather good that a system is profitable, but can not as well determine that it is worthless.
• It gets worse when variants have a negative expectancy.  WRC can then over-estimate Data Mining Bias (see paper 2 at the end of the article). This could theoretically also happen with our trend systems, as some variants may suffer from a phase reversal due to the delay by the smoothing indicators, and thus in fact trade against the trend instead of with it.

The Experiment

First you need to collect daily return curves from all tested strategies. This requires adding a few lines to the Trend.c script from the previous article:

// some global variables
int Period;
var Daily[3000];
...
// in the run function, set all trading costs to zero
 Spread = Commission = RollLong = RollShort = Slippage = 0;
...
// store daily results in an equity curve
  Daily[Day] = Equity;
}
...
// in the objective function, save the curves in a file for later evaluation
string FileName = "Log\\TrendDaily.bin";
string Name = strf("%s_%s_%s_%i",Script,Asset,Algo,Period);
int Size = Day*sizeof(var); 
file_append(FileName,Name,strlen(Name)+1);
file_append(FileName,&Size,sizeof(int));
file_append(FileName,Daily,Size);

The second part of the above code stores the equity at the end of any day in the Daily array. The third part stores a string with the name of the strategy, the length of the curve,  and the equity values itself in a file named TrendDaily.bin in the Log folder. After running the 10 trend scripts, all 900 resulting curves are collected together in the file.

The next part of our experiment is the Bootstrap.c script that applies White’s Reality Check. I’ll write it in two parts. The first part just reads the 900 curves from the TrendDaily.bin file, stores them for later evaluation, finds the best one and displays a histogram of the profit factors. Once we got that, we did already 80% of the work for the Reality Check. This is the code:

void _plotHistogram(string Name,var Value,var Step,int Color)
{
  var Bucket = floor(Value/Step);
  plotBar(Name,Bucket,Step*Bucket,1,SUM+BARS+LBL2,Color);
}

typedef struct curve
{
  string Name;
  int Length;
  var *Values;
} curve;

curve Curve[900];
var Daily[3000];

void main()
{
  byte *Content = file_content("Log\\TrendDaily.bin");
  int i,j,N = 0;
  int MaxN = 0;
  var MaxPerf = 0.0;
	
  while(N<900 && *Content)
  {
// extract the next curve from the file
    string Name = Content;
    Content += strlen(Name)+1;
    int Size = *((int*)Content);
    int Length = Size/sizeof(var); // number of values
    Content += 4;
    var *Values = Content;
    Content += Size;

// store and plot the curve		
    Curve[N].Name = Name;
    Curve[N].Length = Length;
    Curve[N].Values = Values;
    var Performance = 1.0/ProfitFactor(Values,Length);
    printf("\n%s: %.2f",Name,Performance);
    _plotHistogram("Profit",Performance,0.005,RED);

// find the best curve		
    if(MaxPerf < Performance) {
      MaxN = N; MaxPerf = Performance;
    }
    N++;
  }
  printf("\n\nBenchmark: %s, %.2f",Curve[MaxN].Name,MaxPerf); 
}

Most of the code is just for reading and storing all the equity curves. The indicator ProfitFactor calculates the profit factor of the curve, the sum of all daily wins divided by the sum of all daily losses. However, here we need to consider the array order. Like many platforms, Zorro stores time series in reverse chronological order,  with the most recent data at the begin. However we stored the daily equity curve in straight chronological order. So the losses are actually wins and the wins actually losses, which is why we need to inverse the profit factor. The curve with the best profit factor will be our benchmark for the test.

This is the resulting histogram, the profit factors of all 900 (or rather, 705 due to the trade number minimum) equity curves:

Note that the profit factors are slightly different to the parameter charts of the previous article because they were now calculated from daily returns, not from trade results. We removed trading costs, so the histogram is centered at a profit factor 1.0, aka zero profit. Only a few systems achieved a profit factor in the 1.2 range, the two best made about 1.3. Now we’ll see what White has to say to that. This is the rest of the main function in Bootstrap.c that finally applies his Reality Check:

plotBar("Benchmark",MaxPerf/0.005,MaxPerf,50,BARS+LBL2,BLUE);	
printf("\nBootstrap - please wait");
int Worse = 0, Better = 0;
for(i=0; i<1000; i++) {
  var MaxBootstrapPerf = 0;
  for(j=0; j<N; j++) {
    randomize(BOOTSTRAP|DETREND,Daily,Curve[j].Values,Curve[j].Length);
    var Performance = 1.0/ProfitFactor(Daily,Curve[j].Length);
    MaxBootstrapPerf = max(MaxBootstrapPerf,Performance);
  }
  if(MaxPerf > MaxBootstrapPerf)
    Better++;
  else
    Worse++;
  _plotHistogram("Profit",MaxBootstrapPerf,0.005,RED);
  progress(100*i/SAMPLES,0);
}
printf("\nBenchmark beats %.0f%% of samples!",
  (var)Better*100./(Better+Worse));

This code needs about 3 minutes to run; we’re sampling the 705 curves 1000 times. The randomize function will shuffle the daily returns by bootstrap with replacement; the DETREND flag tells it to subtract the mean return from all returns before. The number of random curves that are better and worse than the benchmark is stored, for printing the percentage at the end. The progress function moves the progress bar while Zorro grinds through the 705,000 curves. And this is the result:

Hmm. We can see that the best system – the black bar – is at the right side of the histogram, indicating that it might be significant. But only with about 80% probability (the script gives a slightly different result every time due to the randomizing). 20% of the random curves achieve better profit factors than the best system from the experiment. The median of the randomized samples is about 1.26. Only the two best systems from the original distribution (first image) have profit factors above 1.26 – all the rest is at or below the bootstrap median.

So we have to conclude that this simple way of trend trading does not really work. Interestingly, one of those 900 tested systems is a system that I use for myself since 2012, although with additional filters and conditions. This system has produced good live returns and a positive result by the end of every year so far. And there’s still the fact that EUR/USD and silver in all variants produced better statistics than S&P500. This hints that some trend effect exists in their price curves, but the profit factors by the simple algorithms are not high enough to pass White’s Reality Check. We need a better approach for trend exploitation. For instance, a filter that detects if trend is there or not. This will be the topic of the next article of this series. We will see that a filter can have a surprising effect on reality checks. Since we now have the Bootstrap script for applying White’s Reality Check, we can quickly do further experiments.

The Bootstrap.c script has been added to the 2015 script collection downloadable on the sidebar.

Conclusion

• None of the 10 tested low-lag indicators, and none of the 3 tested markets shows significant positive expectancy with trend trading.
• There is evidence of a trend effect in currencies and commodities, but it is too weak or too infrequent for being effectively exploited with simple trade signals by a filtered price curve.
• We have now a useful code framework for comparing indicators and assets, and for further experiments with trade strategies.

Papers

1. Original paper by Dr. H. White:  White2000
2. WRC modification by P. Hansen: Hansen2005
3. Stepwise WRC Testing by J. Romano, M. Wolf: RomanoWolf2005
4. Technical Analysis examined with WRC, by P. Hsu, C. Kuan: HsuKuan2006
5. WRC and its Extensions by V. Corradi, N. Swanson: CorradiSwanson2011

• Trend exists, but is difficult to detect and to exploit. The experiment will find only a few profitable and a lot more unprofitable systems (with realistic trade costs).
• More sophisticated indicators, such as the Decycler or the Lowpass filter, will produce better results than the old SMA, EMA, or HMA. Modern indicators have less lag and thus react faster on the begin and end of a trend.
• White’s Reality Check will reveal a data mining bias rather close to the returns of the best systems from the experiment. If algorithmic trend trading were easy, anyone would be doing it.
• EUR/USD will return better results because according to trader’s wisdom, currencies are “more trending”.

However the results were quite a surprise.

The Trend Test System

This is the script Trend.c used for the trend experiment:

var objective() 
{ 
  return WinTotal/max(1,LossTotal); // Profit factor
}

var filter(var* Data,int Period);

void run()
{
  set(PARAMETERS|LOGFILE);
  PlotHeight1 = 200;
  PlotScale = 8;
  ColorBars[1] = BLACK; // winning trades
  ColorBars[2] = GREY; // losing trades

  StartDate = 2010;
  BarPeriod = 15;
  LookBack = 80*4*24; // 80 trading days ~ 4 months
	
  while(asset(loop("EUR/USD","SPX500","XAG/USD")))
  while(algo(loop("M15","H1","H4")))
  {
    if(Algo == "M15")
      TimeFrame = 1; // 15 minutes
    else if(Algo == "H1")
      TimeFrame = 4; // 1 hour
    else if(Algo == "H4")
      TimeFrame = 16;	// 4 hours

    int Periods[10] = { 10,20,50,100,200,500,1000,2000,5000,10000 };
    int Period = Periods[optimize(0,0,9,1)];
		
    vars Prices = series(price());
    vars Smoothed = series(filter(Prices,Period));

    if(valley(Smoothed))
      enterLong();
    else if(peak(Smoothed))
      enterShort();
  } 
}

This script is supposed to run in Zorro’s “Train” mode. We’re not really training the systems, but Train mode produces parameter histograms, and that’s what we want. The script is basically the same as described in the previous article, but two asset and algo loops are now used for cycling through three different assets (EUR/USD, S&P 500, and Silver) and three time frames (15 minutes, 1 hour, 4 hours). The optimize function selects the time period of the smoothing indicator, in 10 steps from 10 to 10,000 time frames. The meaning of the Zorro-specific functions like asset, algo, loop, optimize can be found in the Zorro manual.

6 years of price history from 2010 to 2015 are used for the simulation. The default trade costs – commission, bid/ask spread, and rollover – are taken from the average 2015 parameters of a FXCM microlot account. FXCM is not the cheapest broker, so the expectation value of a random-trading simulation is not zero, but a loss, dependent on the number of trades. For comparing results we’re using the profit factor, calculated in the objective function. The script calls a filter function that is defined as a prototype, and contained in a main script that includes Trend.c. So any of the indicators is represented by a very short main script like this (TrendLowpass.c):

#include "Strategy\Trend.c"

var filter(var* Data,int Period)
{
  return LowPass(Data,Period);
}

All the scripts can be downloaded from the scripts2015 archive at the sidebar. There’s also a batch file included that starts them all and lets them run in parallel.

The Results

When you select one of the scripts and click Zorro’s  [Train] button, it will grind its wheels about 10 minutes and then spit out a sheet of histograms like this (for the ZMA):

We got 9 histograms from the combinations of EUR/USD, S&P 500, and Silver (horizontally) with bar sizes of 15 minutes, 1 hour, and 4 hours. The red bars are the objective return – in this case, the profit factor,  total win divided by total loss. If they are above 1.0 (left scale), the system was profitable. The black and grey bars are the number of winning and losing trades (right scale). The steps on the X axis are the indicator time period; they normally don’t go all the way to 10 because systems are discarded when the time period exceeded 4 months or the system entered less than 30 trades. That’s why the experiment didn’t produce 900 systems, but only 705.

We can see that the profit generally increases, and the number of trades decreases with the time frame. As typical for trend trading systems we got far more losing trades (grey) than winning trades (black). 13 of the 82 systems in the histograms above have been profitable, 69 were losers. Most of the profitable systems traded EUR/USD. Only one S&P 500 system, and only three silver trading systems produced any profit. Here’s a statistic of all winners and losers from the 705 tested systems:

Since the results are for 2010-2015, you might get different figures when testing different time periods. The Rate column shows the percentage of successful systems. We got a mix of expected and unexpected results. Almost 40% of the EUR/USD systems are winners,  which seems to confirm that currencies tend to trend. The S&P 500 index however is not trending at all –  and this despite the fact that it shows the most significant trend in the long run, as visible in the title image. But here we’re looking for shorter term trends that can be exploited by day trading. Silver, with 20% winners, has at least some trend. Longer time frames produce dramatically better results, mostly due to less trade costs, but also partly because trend seems to prefer long cycles and low frequencies. (You can see that the difference comes not from trade costs alone when you test the systems with spread, commission, slippage, and rollover set to zero). 1-day bar periods would be even more profitable. But I haven’t included them in the test because they produce not enough trades for statistical significance.

The indicators show a surprising behavior. The old traditional indicators put up a good fight. Ehler’s smoothing, decycling, and zero-lag indicators, produce less profitable systems than their classic counterparts. However we’ll see later that the modern indicators are in fact more profitable, although within a smaller parameter range. But they all occupy the top of the profit list. The best systems, with modern indicators and long time frames, have profit factors up to 2.

Does this mean that we can put them together in a compound system and will get rich? I don’t know, because I haven’t determined yet the Data Mining Bias produced by this experiment. For this we need to store and randomize the balance curves, and apply White’s Reality Check. This will be the topic of the next article of the Trend Experiment series.

Conclusion

• Different markets show very different behavior. EUR/USD and silver are significantly more profitable with short-term trend following than S&P500.
• Low-lag smoothing indicators have a smaller profitable parameter range than traditional indicators, but produce higher profits.