Let’s first code the bands. They are simply a regression line moved up or down so that it crosses through the highest and lowest price peaks and valleys. Here’s the piece of code in C:

var Slope = LinearRegSlope(seriesC(),N);
var Intercept = LinearRegIntercept(seriesC(),N);
var LinVal, HighDev = 0, LowDev = 0;
for(i=N; i>0; i--) {
  LinVal = Intercept + Slope*(N-i);
  HighDev = max(HighDev,priceC(i)-LinVal);
  LowDev = min(LowDev,priceC(i)-LinVal);
}

N is the number of bars for which the regression line is calculated. The line has the formula y = b + m*x, where b is the intercept and m the slope. The code generates both for the previous N bars, then calculates in the loop the maximum and minimum deviations (HighDev, LowDev). The regression value (LinVal) is calculated from the intercept and slope with the above formula. Since the bar offset i runs backwards from the current bar, the bar number that’s multiplied with the slope runs from N down to 0. Here’s the code applied to a SPY chart from 2025:

The candles can exceed the upper and lower bands because only the close price is used for the bands. It would probably improve the system, at least in theory, when we used the high and low prices instead.

Kaufman suggests several methods of trading with these bands; we’re here using the ‘Inside Channel’ method since it is, according to Kaufman, the most profitable. We open a long position when the price comes within a zone around the lower band, and close the position (or open a short position) when the price comes within a zone around the upper band. Here’s the complete trading system in C for Zorro, using the above code to calculate the bands.

void run()
{
  BarPeriod = 1440;
  StartDate = 20100101;
  LookBack = 150;
  assetList("AssetsIB");
  asset("SPY");
  if(is(LOOKBACK)) return;

  int i, N = 40;
  var Factor = 0.2;
  var Slope = LinearRegSlope(seriesC(),N);
  var Intercept = LinearRegIntercept(seriesC(),N);
  var LinVal, HighDev = 0, LowDev = 0;
  for(i=N; i>0; i--) {
    LinVal = Intercept + Slope*(N-i);
    HighDev = max(HighDev,priceC(i)-LinVal);
    LowDev = min(LowDev,priceC(i)-LinVal);
  }
  var Zone = Factor*(HighDev+LowDev);
  if(!NumOpenLong && priceC(0) < LinVal+LowDev+Zone)
    enterLong();
  if(!NumOpenShort && priceC(0) > LinVal+HighDev-Zone)
    exitLong();
}

We’ve selected the AssetsIB asset list, which contains the margins, commissions and other parameters from an US broker (IBKR). So the backtest simulates trading with IBKR. The resulting equity curve with the default parameters, N = 40 and Zone Factor = 20%, already shows promise with a 2.8 profit factor:

However, Kaufman mentioned that he tested N values from 20 to 150, and zone factors from 5% to 50%. We’ll do the same by optimizing these parameters. Of course a backtest with the best optimization result would be meaningless due to bias (see https://zorro-project.com/backtest.php). Therefore we’re using walk forward optimization for an out-of-sample backtest. Since anything with Zorro is done in code, we’ll insert 4 lines of C code for the optimization:

set(PARAMETERS); // optimize parameters
NumWFOCycles = 5;
N = optimize(40,20,150,10);
Factor = optimize(0.2,0.05,0.5,0.05);

We also changed the trading from only long positions to long and short by replacing exitLong with enterShort By default, entering a short position automatically exits the long one, and vice versa. So the system is always in the market with 1 share, either long or short. The walk forward optimization takes about 3 seconds. This is the resulting equity curve:

The profit factor rose to 7, with a 76% win rate – not bad for such a simple system. The code can be downloaded from the 2025 script repository on https://financial-hacker.com. 

The indicator is a straighforward conversion of Kaufman’s EasyLanguage code to C:

var GAPM(int Period, int SignalPeriod)
{
    var UpGaps = 0, DnGaps = 0;
    int ix;
    for(ix = 0; ix < Period; ix++) {
        var Gap = priceO(ix) - priceC(ix+1);
        if(Gap > 0) UpGaps += Gap;
        else if(Gap < 0) DnGaps -= Gap;
    }
    var GapRatio = ifelse(DnGaps == 0,1,100*UpGaps/DnGaps);
    return SMA(series(GapRatio),SignalPeriod);
}

The system trades QQQ, a NASDAQ ETF. The historical data for the backtest is loaded from Yahoo. It enters a long position when upward gaps are on the rise, and closes the position otherwise. The code in C for the Zorro platform:

function run()
{
    StartDate = 2011;
    EndDate = 2023;
    BarPeriod = 1440;
    Fill = 3;    // trade on next open
    var Investment = 100000;
    assetAdd("QQQ","YAHOO:*");
    asset("QQQ");
    vars Signals = series(GAPM(40,20));
    if(!NumOpenLong && rising(Signals))
        enterLong(Investment/priceC());
    if(NumOpenLong && falling(Signals))
        exitLong();
}

The resulting equity curve:

The result is not sensational, but the system achieved a solid 12% return on investment, well above buy-and-hold, with 56% winning trades. The GAPM indicator and the trading system can be downloaded from the 2023 script repository.

Developing a model-based strategy begins with the market inefficiency that you want to exploit. The inefficiency produces a price anomaly or price pattern that you can describe with a qualitative or quantitative model. Such a model predicts the current price y_t from the previous price y_t-1 plus some function f of a limited number of previous prices plus some noise term ε:

[pmath size=16]y_t ~=~ y_{t-1} + f(y_{t-1},~…, ~y_{t-n}) + epsilon[/pmath]

The time distance between the prices y_t is the time frame of the model; the number n of prices used in the function f is the lookback period of the model. The higher the predictive f term in relation to the nonpredictive ε term, the better is the strategy. Some traders claim that their favorite method does not predict, but ‘reacts on the market’ or achieves a positive return by some other means. On a certain trader forum you can even encounter a math professor who re-invented the grid trading system, and praised it as non-predictive and even able to trade a random walk curve. But systems that do not predict y_t in some way must rely on luck; they only can redistribute risk, for instance exchange a high risk of a small loss for a low risk of a high loss. The profit expectancy stays negative. As far as I know, the professor is still trying to sell his grid trader, still advertising it as non-predictive, and still regularly blowing his demo account with it. 

Trading by throwing a coin loses the transaction costs. But trading by applying the wrong model – for instance, trend following to a mean reverting price series – can cause much higher losses. The average trader indeed loses more than by random trading (about 13 pips per trade according to FXCM statistics). So it’s not sufficient to have a model; you must also prove that it is valid for the market you trade, at the time you trade, and with the used time frame and lookback period.

Not all price anomalies can be exploited. Limiting stock prices to 1/16 fractions of a dollar is clearly an inefficiency, but it’s probably difficult to use it for prediction or make money from it. The working model-based strategies that I know, either from theory or because we’ve been contracted to code some of them, can be classified in several categories. The most frequent are:

1. Trend 

Momentum in the price curve is probably the most significant and most exploited anomaly. No need to elaborate here, as trend following was the topic of a whole article series on this blog. There are many methods of trend following, the classic being a moving average crossover. This ‘hello world’ of strategies (here the scripts in R and in C) routinely fails, as it does not distinguish between real momentum and random peaks or valleys in the price curve.

The problem: momentum does not exist in all markets all the time. Any asset can have long non-trending periods. And contrary to popular belief this is not necessarily a ‘sidewards market’. A random walk curve can go up and down and still has zero momentum. Therefore, some good filter that detects the real market regime is essential for trend following systems. Here’s a minimal Zorro strategy that uses a lowpass filter for detecting trend reversal, and the MMI indicator for determining when we’re entering trend regime:

function run()
{
  vars Price = series(price());
  vars Trend = series(LowPass(Price,500));
	
  vars MMI_Raw = series(MMI(Price,300));
  vars MMI_Smooth = series(LowPass(MMI_Raw,500));
	
  if(falling(MMI_Smooth)) {
    if(valley(Trend))
      reverseLong(1);
    else if(peak(Trend))
      reverseShort(1);
  }
}

The profit curve of this strategy:

(For the sake of simplicity all strategy snippets on this page are barebone systems with no exit mechanism other than reversal, and no stops, trailing, parameter training, money management, or other gimmicks. Of course the backtests mean in no way that those are profitable systems. The P&L curves are all from EUR/USD, an asset good for demonstrations since it seems to contain a little bit of every possible inefficiency). 

2. Mean reversion

A mean reverting market believes in a ‘real value’ or ‘fair price’ of an asset. Traders buy when the actual price is cheaper than it ought to be in their opinion, and sell when it is more expensive. This causes the price curve to revert back to the mean more often than in a random walk. Random data are mean reverting 75% of the time (proof here), so anything above 75% is caused by a market inefficiency. A model:

[pmath size=16]y_t ~=~ y_{t-1} ~-~ 1/{1+lambda}(y_{t-1}- hat{y}) ~+~ epsilon[/pmath]

[pmath size=16]y_t[/pmath] = price at bar t
[pmath size=16]hat{y}[/pmath] = fair price
[pmath size=16]lambda[/pmath] = half-life factor
[pmath size=16]epsilon[/pmath] = some random noise term

The higher the half-life factor, the weaker is the mean reversion. The half-life of mean reversion in price series ist normally in the range of  50-200 bars. You can calculate λ by linear regression between y_t-1 and (y_t-1-y_t). The price series need not be stationary for experiencing mean reversion, since the fair price is allowed to drift. It just must drift less as in a random walk. Mean reversion is usually exploited by removing the trend from the price curve and normalizing the result. This produces an oscillating signal that can trigger trades when it approaches a top or bottom. Here’s the script of a simple mean reversion system:

function run()
{
  vars Price = series(price());
  vars Filtered = series(HighPass(Price,30));
  vars Signal = series(FisherN(Filtered,500));
  var Threshold = 1.0;

  if(Hurst(Price,500) < 0.5) { // do we have mean reversion?
    if(crossUnder(Signal,-Threshold))
      reverseLong(1); 
    else if(crossOver(Signal,Threshold))
      reverseShort(1);
  }
} 

The highpass filter dampens all cycles above 30 bars and thus removes the trend from the price curve. The result is normalized by the Fisher transformation which produces a Gaussian distribution of the data. This allows us to determine fixed thresholds at 1 and -1 for separating the tails from the resulting bell curve. If the price enters a tail in any direction, a trade is triggered in anticipation that it will soon return into the bell’s belly. For detecting mean reverting regime, the script uses the Hurst Exponent. The exponent is 0.5 for a random walk. Above 0.5 begins momentum regime and below 0.5 mean reversion regime.

3. Statistical Arbitrage

Strategies can exploit the similarity between two or more assets.  This allows to hedge the first asset by a reverse position in the second asset, and this way derive profit from mean reversion of their price difference:

[pmath size=16]y ~=~ h_1 y_1 – h_2 y_2[/pmath]

where y₁ and y₂ are the prices of the two assets and the multiplication factors h₁ and h₂ their hedge ratios. The hedge ratios are calculated in a way that the mean of the difference y is zero or a constant value. The simplest method for calculating the hedge ratios is linear regression between y₁ and y₂. A mean reversion strategy as above can then be applied to y.  

The assets need not be of the same type; a typical arbitrage system can be based on the price difference between an index ETF and its major stock. When y is not stationary – meaning that its mean tends to wander off slowly – the hedge ratios must be adapted in real time for compensating. Here is a proposal using a Kalman Filter by a fellow blogger.

The simple arbitrage system from the R tutorial:

require(quantmod)

symbols <- c("AAPL", "QQQ")
getSymbols(symbols)

#define training set
startT  <- "2007-01-01"
endT    <- "2009-01-01"
rangeT  <- paste(startT,"::",endT,sep ="")
tAAPL   <- AAPL[,6][rangeT]
tQQQ   <- QQQ[,6][rangeT]
 
#compute price differences on in-sample data
pdtAAPL <- diff(tAAPL)[-1]
pdtQQQ <- diff(tQQQ)[-1]
 
#build the model
model  <- lm(pdtAAPL ~ pdtQQQ - 1)
 
#extract the hedge ratio (h1 is assumed 1)
h2 <- as.numeric(model$coefficients[1])

#spread price (in-sample)
spreadT <- tAAPL - h2 * tQQQ
 
#compute statistics of the spread
meanT    <- as.numeric(mean(spreadT,na.rm=TRUE))
sdT      <- as.numeric(sd(spreadT,na.rm=TRUE))
upperThr <- meanT + 1 * sdT
lowerThr <- meanT - 1 * sdT
 
#run in-sample test
spreadL  <- length(spreadT)
pricesB  <- c(rep(NA,spreadL))
pricesS  <- c(rep(NA,spreadL))
sp       <- as.numeric(spreadT)
tradeQty <- 100
totalP   <- 0

for(i in 1:spreadL) {
     spTemp <- sp[i]
     if(spTemp < lowerThr) {
        if(totalP <= 0){
           totalP     <- totalP + tradeQty
           pricesB[i] <- spTemp
        }
     } else if(spTemp > upperThr) {
       if(totalP >= 0){
          totalP <- totalP - tradeQty
          pricesS[i] <- spTemp
       }
    }
}

4. Price constraints

A price constraint is an artificial force that causes a constant price drift or establishes a price range, floor, or ceiling. The most famous example was the EUR/CHF price cap mentioned in the first part of this series. But even after removal of the cap, the EUR/CHF price has still a constraint, this time not enforced by the national bank, but by the current strong asymmetry in EUR and CHF buying power. An extreme example of a ranging price is the EUR/DKK pair (see below). All such constraints can be used in strategies to the trader’s advantage. 

5. Cycles

Non-seasonal cycles are caused by feedback from the price curve. When traders believe in a ‘fair price’ of an asset, they often sell or buy a position when the price reaches a certain distance from that value, in hope of a reversal. Or they close winning positions when the favorite price movement begins to decelerate. Such effects can synchronize entries and exits among a large number of traders, and cause the price curve to oscillate with a period that is stable over several cycles. Often many such cycles are superposed on the curve, like this:

[pmath size=16]y_t ~=~ hat{y}  ~+~ sum{i}{}{a_i sin(2 pi t/C_i+D_i)} ~+~ epsilon[/pmath]

When you know the period C_i and the phase D_i of the dominant cycle, you can calculate the optimal trade entry and exit points as long as the cycle persists. Cycles in the price curve can be detected with spectral analysis functions – for instance, fast Fourier transformation (FFT) or simply a bank of narrow bandpass filters. Here is the frequency spectrum of the EUR/USD in October 2015:

Exploiting cycles is a little more tricky than trend following or mean reversion. You need not only the cycle length of the dominant cycle of the spectrum, but also its phase (for triggering trades at the right moment) and its amplitude (for determining if there is a cycle worth trading at all). This is a barebone script:

function run()
{
  vars Price = series(price());
  var Phase = DominantPhase(Price,10);
  vars Signal = series(sin(Phase+PI/4));
  vars Dominant = series(BandPass(Price,rDominantPeriod,1));
  ExitTime = 10*rDominantPeriod;
  var Threshold = 1*PIP;
	
  if(Amplitude(Dominant,100) > Threshold) {
    if(valley(Signal))
      reverseLong(1); 
    else if(peak(Signal))
      reverseShort(1);
  }
}

The DominantPhase function determines both the phase and the cycle length of the dominant peak in the spectrum; the latter is stored in the rDominantPeriod variable. The phase is converted to a sine curve that is shifted ahead by π/4. With this trick we’ll get a sine curve that runs ahead of the price curve. Thus we do real price prediction here, only question is if the price will follow our prediction. This is determined by applying a bandpass filter centered at the dominant cycle to the price curve, and measuring its amplitude (the a_i in the formula). If the amplitude is above a threshold,  we conclude that we have a strong enough cycle. The script then enters long on a valley of the run-ahead sine curve, and short on a peak. Since cycles are shortlived, the duration of a trade is limited by ExitTime to a maximum of 10 cycles. 

We can see from the P&L curve that there were long periods in 2012 and 2014 with no strong cycles in the EUR/USD price curve. 

6. Clusters

The same effect that causes prices to oscillate can also let them cluster at certain levels. Extreme clustering can even produce “supply” and “demand” lines (also known as “support and resistance“), the favorite subjects in trading seminars. Expert seminar lecturers can draw support and resistance lines on any chart, no matter if it’s pork belly prices or last year’s baseball scores. However the mere existence of those lines remains debatable: There are few strategies that really identify and exploit them, and even less that really produce profits. Still, clusters in price curves are real and can be easily identified in a histogram similar to the cycles spectogram.

7. Curve patterns

They arise from repetitive behavior of traders. Traders not only produce, but also believe in many curve patterns; most – such as the famous ‘head and shoulders’ pattern that is said to predict trend reversal – are myths (at least I have not found any statistical evidence of it, and heard of no other any research that ever confirmed the existence of predictive heads and shoulders in price curves). But some patterns, for instance “cups” or “half-cups”, really exist and can indeed precede an upwards or downwards movement. Curve patterns – not to be confused with candle patterns – can be exploited by pattern-detecting methods such as the Fréchet algorithm. 

A special variant of a curve pattern is the Breakout – a sudden momentum after a long sidewards movement. Is can be caused, for instance, by trader’s tendency to place their stop losses as a short distance below or above the current plateau. Triggering the first stops then accelerates the price movement until more and more stops are triggered. Such an effect can be exploited by a system that detects a sidewards period and then lies in wait for the first move in any direction.

8. Seasonality

“Season” does not necessarily mean a season of a year. Supply and demand can also follow monthly, weekly, or daily patterns that can be detected and exploited by strategies. For instance, the S&P500 index is said to often move upwards in the first days of a month, or to show an upwards trend in the early morning hours before the main trading session of the day. Since seasonal effects are easy to exploit, they are often shortlived, weak, and therefore hard to detect by just eyeballing price curves. But they can be found by plotting a day, week, or month profile of average price curve differences.

9. Gaps

When market participants contemplate whether to enter or close a position, they seem to come to rather similar conclusions when they have time to think it over at night or during the weekend. This can cause the price to start at a different level when the market opens again. Overnight or weekend price gaps are often more predictable than price changes during trading hours. And of course they can be exploited in a strategy. On the Zorro forum was recently a discussion about the “One Night Stand System“, a simple currency weekend-gap trader with mysterious profits.

10. Autoregression and heteroskedasticity

The latter is a fancy word for: “Prices jitter a lot and the jittering varies over time”. The ARIMA and GARCH models are the first models that you encounter in financial math. They assume that future returns or future volatility can be determined with a linear combination of past returns or past volatility. Those models are often considered purely theoretical and of no practical use. Not true: You can use them for predicting tomorrow’s price just as any other model. You can examine a correlogram – a statistics of the correlation of the current return with the returns of the previous bars – for finding out if an ARIMA model fits to a certain price series. Here are two excellent articles by fellow bloggers for using those models in trading strategies:  ARIMA+GARCH Trading Strategy on the S&P500 and Are ARIMA/GARCH Predictions Profitable?

11. Price shocks

Price shocks often happen on Monday or Friday morning when companies or organizations publish good or bad news that affect the market. Even without knowing the news, a strategy can detect the first price reactions and quickly jump onto the bandwagon. This is especially easy when a large shock is shaking the markets. Here’s a simple Forex portfolio strategy that evaluates the relative strengths of currencies for detecting price shocks:

function run()
{
  BarPeriod = 60;
  ccyReset();
  string Name;
  while(Name = (loop(Assets)))
  {
    if(assetType(Name) != FOREX) 
      continue; // Currency pairs only
    asset(Name);
    vars Prices = series(priceClose());
    ccySet(ROC(Prices,1)); // store price change as strength
  }
  
// get currency pairs with highest and lowest strength difference
  string Best = ccyMax(), Worst = ccyMin();
  var Threshold = 1.0; // The shock level

  static char OldBest[8], OldWorst[8];	// static for keeping contents between runs
  if(*OldBest && !strstr(Best,OldBest)) { // new strongest asset?
    asset(OldBest);
    exitLong();
    if(ccyStrength(Best) > Threshold) {
      asset(Best);
      enterLong();
    }
  } 
  if(*OldWorst && !strstr(Worst,OldWorst)) { // new weakest asset?
    asset(OldWorst);
    exitShort();
    if(ccyStrength(Worst) < -Threshold) {
      asset(Worst);
      enterShort();
    }
  }

// store previous strongest and weakest asset names  
  strcpy(OldBest,Best);
  strcpy(OldWorst,Worst);
}

The equity curve of the currency strength system (you’ll need Zorro 1.48 or above):

The blue equity curve above reflects profits from small and large jumps of currency prices. You can clearly identify the Brexit and the CHF price cap shock. Of course such strategies would work even better if the news could be early detected and interpreted in some way. Some data services provide news events with a binary valuation, like “good” or “bad”. Especially of interest are earnings reports, as provided by data services such as Zacks or Xignite. Depending on which surprises the earnings report contains, stock prices and implied volatilities can rise or drop sharply at the report day, and generate quick profits.

For learning what can happen when news are used in more creative ways, I recommend the excellent Fear Index by Robert Harris – a mandatory book in any financial hacker’s library.

This was the second part of the Build Better Strategies series. The third part will deal with the process to develop a model-based strategy, from inital research up to building the user interface. In case someone wants to experiment with the code snippets posted here, I’ve added them to the 2015 scripts repository. They are no real strategies, though. The missing elements – parameter optimization, exit algorithms, money management etc. – will be the topic of the next part of the series.

⇒ Build Better Strategies – Part 3

There are already several methods for differentiating trending and nontrending market regimes. Some of them are rumored to really work, at least occasionally. John Ehlers proposed the Hilbert Transform or a Cycle / Trend decomposition, Benoit Mandelbrot the Hurst Exponent. In comparison, the source code of the Market Meanness Index is relatively simple:

// Market Meanness Index
double MMI(double *Data,int Length)
{
  double m = Median(Data,Length);
  int i, nh=0, nl=0;
  for(i=1; i<Length; i++) {
    if(Data[i] > m && Data[i] > Data[i-1]) // mind Data order: Data[0] is newest!
      nl++;
    else if(Data[i] < m && Data[i] < Data[i-1])
      nh++;
  }
  return 100.*(nl+nh)/(Length-1);
}

This code is in C for Zorro, but meanwhile also versions for MT4, Amibroker, Ninja Trader, and other platforms have been programmed by users. As the name suggests, the indicator measures the meanness of the market – its tendency to revert to the mean after pretending to start a trend. If that happens too often, all trend following systems will bite the dust.

The Three-Quarter Rule

Any series of independent random numbers reverts to the mean – or more precisely, to their median – with a probability of 75%. Assume you have a sequence of random, uncorrelated daily data – for instance, the daily rates of change of a random walk price curve. If Monday’s data value was above the median, then in 75% of all cases Tuesday’s data will be lower than Monday’s. And if Monday was below the median, 75% chance is that Tuesday will be higher. The proof of the 75% rule is relatively simple and won’t require integral calculus. Consider a data series with median M. By definition, half the values are less than M and half are greater (for simplicity’s sake we’re ignoring the case when a value is exactly M). Now combine the values to pairs each consisting of a value Py and the following value Pt. Thus each pair represents a change from Py to Pt. We now got a lot of changes that we divide into four sets:

1. (Pt < M, Py < M)
2. (Pt < M, Py > M)
3. (Pt > M, Py < M)
4. (Pt > M, Py > M)

These four sets have obviously the same number of elements – that is, 1/4 of all Py->Pt changes – when Pt and Py are uncorrelated, i.e. completely independent of one another. The value of M and the kind of data in the series won’t matter for this. Now how many data pairs revert to the median? All pairs that fulfill this condition: (Py < M and Pt > Py) or (Py > M and Pt < Py) The condition in the first bracket is fulfilled for half the data in set 1 (in the other half is Pt less than Py) and in the whole set 3 (because Pt is always higher than Py in set 3). So the first bracket is true for 1/2 * 1/4 + 1/4 = 3/8 of all data changes. Likewise,  the second bracket is true in half the set 4 and in the whole set 2, thus also for 3/8 of all data changes. 3/8 + 3/8 yields 6/8, i.e. 75%. This is the three-quarter rule for random numbers.

The MMI function just counts the number of data pairs for which the conditition is met, and returns their percentage. The Data series may contain prices or price changes. Prices have always some serial correlation: If EUR / USD today is at 1.20, it will also be tomorrow around 1.20. That it will end up tomorrow at 70 cents or 2 dollars per EUR is rather unlikely. This serial correlation is also true for a price series calculated from random numbers, as not the prices themselves are random, but their changes. Thus, the MMI function should return a smaller percentage, such as 55%, when fed with prices.

 Unlike prices, price changes have not necessarily serial correlation. A one hundred percent efficient market has no correlation between the price change from yesterday to today and the price change from today to tomorrow. If the MMI function is fed with perfectly random price changes from a perfectly efficient market, it will return a value of about 75%. The less efficient and the more trending the market becomes, the more the MMI decreases. Thus a falling MMI is a indicator of an upcoming trend. A rising MMI hints that the market will get nastier, at least for trend trading systems.

Using the MMI in a trend strategy

One could assume that MMI predicts the price direction. A high MMI value indicates a high chance of mean reversion, so when prices were moving up in the last time and MMI is high, can we expect a soon price drop? Unfortunately it doesn’t work this way. The probability of mean reversion is not evenly distributed over the Length of the Data interval. For the early prices it is high (since the median is computed from future prices), but for the late prices, at the very time when MMI is calculated, it is down to just 50%. Predicting the next price with the MMI would work as well as flipping a coin.

Another mistake would be using the MMI for detecting a cyclic or mean-reverting market regime. True, the MMI will rise in such a situation, but it will also rise when the market becomes more random and more effective. A rising MMI alone is no promise of profit by cycle trading systems.

So the MMI won’t tell us the next price, and it won’t tell us if the market is mean reverting or just plain mean, but it can reveal information about the success chance of trend following. For this we’re making an assumption: Trend itself is trending. The market does not jump in and out of trend mode suddenly, but with some inertia. Thus, when we know that MMI is rising, we assume that the market is becoming more efficient, more random, more cyclic, more reversing or whatever, but in any case bad for trend trading. However when MMI is falling, chances are good that the next beginning trend will last longer than normal.

This way the MMI can be an excellent trend filter – in theory. But we all know that there’s often a large gap between theory and practice, especially in algorithmic trading. So I’m now going to test what the Market Meanness Index does to the collection of the 900 trend following systems that I’ve accumulated. For a first quick test, this was the equity curve of one of the systems, TrendEMA, without MMI (44% average annual return):

This is the same system with MMI (55% average annual return):

We can see that the profit has doubled, from $250 to $500. The profit factor climbed from 1.2 to 1.8, and the number of trades (green and red lines) is noticeable reduced. On the other hand, the equity curve started with a drawdown that wasn’t there with the original system. So MMI obviously does not improve all trades. And this was just a randomly selected system. If our assumption about trend trendiness is true, the indicator should have a significant effect also on the other 899 systems.

This experiment will be the topic of the next article, in about a week. As usually I’ll include all the source code for anyone to reproduce it. Will the MMI miserably fail? Or improve only a few systems, but worsen others? Or will it light up the way to the Holy Grail of trend strategies? Let the market be the judge.

Next: Testing the Market Meanness Index

Addendum (2022). I thought my above proof of the 3/4 rule was trivial and no math required, but judging from some comments, it is not so. The rule appears a bit counter-intuitive at first glance. Some comments also confuse a random walk (for example, prices) with a random sequence (for example, price differences). Maybe I have just bad explained it – in that case, read up the proof anywhere else. I found the two most elaborate proofs of the 3/4 rule, geometrical and analytical, in this book:

(1)  Andrew Pole, Statistical Arbitrage, Wiley 2007

And with no reading and no math: Take a dice, roll it 100 times, write down each result, get the median of all, and then count how often a pair reverted to the median.

Below you can see the MMI applied to a synthetic price curve, first sidewards (black) and then with added trend (blue). For using the MMI with real prices or price differences, read the next article.

The code:

function run()
{
	BarPeriod = 60;
	MaxBars = 1000;
	LookBack = 500;
	asset(""); // dummy asset
	ColorUp = ColorDn = 0; // don't plot a price curve
	set(PLOTNOW);
	
	if(Init) seed(1000);
	int TrendStart = LookBack + 0.4*(MaxBars-LookBack);
	vars Signals = series(1 + 0.5*genNoise());
	if(Bar > TrendStart) 
		Signals[0] += 0.003*(Bar-TrendStart);
	int Color = RED;
	int TimePeriod = 0.5*LookBack;
	if(Bar < TrendStart)
		plot("Sidewards",Signals,MAIN,BLACK);
	else
		plot("Trending",Signals,MAIN,BLUE);
	plot("MMI",MMI(Signals,TimePeriod),NEW|LINE,Color);
}

Suppose you want to trade by moon phases. But you’re not sure if you shall buy at full moon and sell at new moon, or the other way around. So you do a series of moon phase backtests and find out that the best system, which opens positions at any first quarter moon, achieves 30% annual return. Is this finally the proof that astrology works?

A trade system based on a nonexisting effect has normally a profit expectancy of zero (minus trade costs). But you won’t get zero when you backtest variants of such a system. Due to statistical fluctuations, some of them will produce a positive and some a negative return. When you now pick the best performer, such as the first quarter moon trading system, you might get a high return and an impressive equity curve in the backtest. Sadly, its test result is not necessarily caused by clever trading. It might be just by clever selecting the random best performer from a pool of useless systems.

For finding out if the 30% return by quarter moon trading are for real or just the fool’s gold of Data Mining Bias, Halbert White (1959-2012) invented a test method in 2000.  White’s Reality Check (aka Bootstrap Reality Check) is explained in detail in the book ‘Evidence-Based Technical Analysis’ by David Aronson. It works this way:

1. Develop a strategy. During the development process, keep a record of all strategy variants that were tested and discarded because of their test results, including all abandoned algorithms, ideas, methods, and parameters. It does not matter if they were discarded by human decision or by a computer search or optimizing process.
2. Produce balance curves of all strategy variants, using detrended trade results and no trade costs. Note down the profit P of the best strategy.
3. Detrend all balance curves by subtracting the mean return per bar (not to be confused with detrending the trade results!). This way you get a series of curves with the same characteristics of the tested systems, but zero profit.
4. Randomize all curves by bootstrap with replacement. This produces new curves from the random returns of the old curves. Because the same bars can be selected multiple times, most new curves now produce losses or profits different from zero.
5. Select the best performer from the randomized curves, and note down its profit.
6.  Repeat steps 4 and 5 a couple 1000 times.
7. You now have a list of several 1000 best profits. The median M of that list is the Data Mining Bias by your strategy development process.
8. Check where the original best profit P appears in the list. The percentage of best bootstrap profits greater than P is the so-called p-Value of the best strategy. You want the p-Value to be as low as possible. If P is better than 95% of the best bootstrap profits, the best strategy has a real edge with 95% probability.
9. P minus M minus trade costs is the result to be expected in real trading the best strategy.

The method is not really intuitive, but mathematically sound. However, it suffers from a couple problems that makes WRC difficult to use  in real strategy development:

• You can see the worst problem already in step 1. During strategy development you’re permanently testing ideas, adding or removing parameters, or checking out different assets and time frames. Putting aside all discarded variants and producing balance curves of all combinations of them is a cumbersome process. It gets even more diffcult with machine learning algorithms that optimize weight factors and usually do not produce discarded variants. However, the good news are that you can easily apply the WRC when your strategy variants are produced by a transparent mechanical process with no human decisions involved. That’s fortunately the case for our trend experiment.
• WRC tends to type II errors. That means it can reject strategies although they have an edge. When more irrelevant variants – systems with random trading and zero profit expectancy – are added to the pool,  more positive results can be produced in steps 4 and 5, which reduces the probability that your selected strategy survives the test. WRC can determine rather good that a system is profitable, but can not as well determine that it is worthless.
• It gets worse when variants have a negative expectancy.  WRC can then over-estimate Data Mining Bias (see paper 2 at the end of the article). This could theoretically also happen with our trend systems, as some variants may suffer from a phase reversal due to the delay by the smoothing indicators, and thus in fact trade against the trend instead of with it.

The Experiment

First you need to collect daily return curves from all tested strategies. This requires adding a few lines to the Trend.c script from the previous article:

// some global variables
int Period;
var Daily[3000];
...
// in the run function, set all trading costs to zero
 Spread = Commission = RollLong = RollShort = Slippage = 0;
...
// store daily results in an equity curve
  Daily[Day] = Equity;
}
...
// in the objective function, save the curves in a file for later evaluation
string FileName = "Log\\TrendDaily.bin";
string Name = strf("%s_%s_%s_%i",Script,Asset,Algo,Period);
int Size = Day*sizeof(var); 
file_append(FileName,Name,strlen(Name)+1);
file_append(FileName,&Size,sizeof(int));
file_append(FileName,Daily,Size);

The second part of the above code stores the equity at the end of any day in the Daily array. The third part stores a string with the name of the strategy, the length of the curve,  and the equity values itself in a file named TrendDaily.bin in the Log folder. After running the 10 trend scripts, all 900 resulting curves are collected together in the file.

The next part of our experiment is the Bootstrap.c script that applies White’s Reality Check. I’ll write it in two parts. The first part just reads the 900 curves from the TrendDaily.bin file, stores them for later evaluation, finds the best one and displays a histogram of the profit factors. Once we got that, we did already 80% of the work for the Reality Check. This is the code:

void _plotHistogram(string Name,var Value,var Step,int Color)
{
  var Bucket = floor(Value/Step);
  plotBar(Name,Bucket,Step*Bucket,1,SUM+BARS+LBL2,Color);
}

typedef struct curve
{
  string Name;
  int Length;
  var *Values;
} curve;

curve Curve[900];
var Daily[3000];

void main()
{
  byte *Content = file_content("Log\\TrendDaily.bin");
  int i,j,N = 0;
  int MaxN = 0;
  var MaxPerf = 0.0;
	
  while(N<900 && *Content)
  {
// extract the next curve from the file
    string Name = Content;
    Content += strlen(Name)+1;
    int Size = *((int*)Content);
    int Length = Size/sizeof(var); // number of values
    Content += 4;
    var *Values = Content;
    Content += Size;

// store and plot the curve		
    Curve[N].Name = Name;
    Curve[N].Length = Length;
    Curve[N].Values = Values;
    var Performance = 1.0/ProfitFactor(Values,Length);
    printf("\n%s: %.2f",Name,Performance);
    _plotHistogram("Profit",Performance,0.005,RED);

// find the best curve		
    if(MaxPerf < Performance) {
      MaxN = N; MaxPerf = Performance;
    }
    N++;
  }
  printf("\n\nBenchmark: %s, %.2f",Curve[MaxN].Name,MaxPerf); 
}

Most of the code is just for reading and storing all the equity curves. The indicator ProfitFactor calculates the profit factor of the curve, the sum of all daily wins divided by the sum of all daily losses. However, here we need to consider the array order. Like many platforms, Zorro stores time series in reverse chronological order,  with the most recent data at the begin. However we stored the daily equity curve in straight chronological order. So the losses are actually wins and the wins actually losses, which is why we need to inverse the profit factor. The curve with the best profit factor will be our benchmark for the test.

This is the resulting histogram, the profit factors of all 900 (or rather, 705 due to the trade number minimum) equity curves:

Note that the profit factors are slightly different to the parameter charts of the previous article because they were now calculated from daily returns, not from trade results. We removed trading costs, so the histogram is centered at a profit factor 1.0, aka zero profit. Only a few systems achieved a profit factor in the 1.2 range, the two best made about 1.3. Now we’ll see what White has to say to that. This is the rest of the main function in Bootstrap.c that finally applies his Reality Check:

plotBar("Benchmark",MaxPerf/0.005,MaxPerf,50,BARS+LBL2,BLUE);	
printf("\nBootstrap - please wait");
int Worse = 0, Better = 0;
for(i=0; i<1000; i++) {
  var MaxBootstrapPerf = 0;
  for(j=0; j<N; j++) {
    randomize(BOOTSTRAP|DETREND,Daily,Curve[j].Values,Curve[j].Length);
    var Performance = 1.0/ProfitFactor(Daily,Curve[j].Length);
    MaxBootstrapPerf = max(MaxBootstrapPerf,Performance);
  }
  if(MaxPerf > MaxBootstrapPerf)
    Better++;
  else
    Worse++;
  _plotHistogram("Profit",MaxBootstrapPerf,0.005,RED);
  progress(100*i/SAMPLES,0);
}
printf("\nBenchmark beats %.0f%% of samples!",
  (var)Better*100./(Better+Worse));

This code needs about 3 minutes to run; we’re sampling the 705 curves 1000 times. The randomize function will shuffle the daily returns by bootstrap with replacement; the DETREND flag tells it to subtract the mean return from all returns before. The number of random curves that are better and worse than the benchmark is stored, for printing the percentage at the end. The progress function moves the progress bar while Zorro grinds through the 705,000 curves. And this is the result:

Hmm. We can see that the best system – the black bar – is at the right side of the histogram, indicating that it might be significant. But only with about 80% probability (the script gives a slightly different result every time due to the randomizing). 20% of the random curves achieve better profit factors than the best system from the experiment. The median of the randomized samples is about 1.26. Only the two best systems from the original distribution (first image) have profit factors above 1.26 – all the rest is at or below the bootstrap median.

So we have to conclude that this simple way of trend trading does not really work. Interestingly, one of those 900 tested systems is a system that I use for myself since 2012, although with additional filters and conditions. This system has produced good live returns and a positive result by the end of every year so far. And there’s still the fact that EUR/USD and silver in all variants produced better statistics than S&P500. This hints that some trend effect exists in their price curves, but the profit factors by the simple algorithms are not high enough to pass White’s Reality Check. We need a better approach for trend exploitation. For instance, a filter that detects if trend is there or not. This will be the topic of the next article of this series. We will see that a filter can have a surprising effect on reality checks. Since we now have the Bootstrap script for applying White’s Reality Check, we can quickly do further experiments.

The Bootstrap.c script has been added to the 2015 script collection downloadable on the sidebar.

Conclusion

• None of the 10 tested low-lag indicators, and none of the 3 tested markets shows significant positive expectancy with trend trading.
• There is evidence of a trend effect in currencies and commodities, but it is too weak or too infrequent for being effectively exploited with simple trade signals by a filtered price curve.
• We have now a useful code framework for comparing indicators and assets, and for further experiments with trade strategies.

Papers

1. Original paper by Dr. H. White:  White2000
2. WRC modification by P. Hansen: Hansen2005
3. Stepwise WRC Testing by J. Romano, M. Wolf: RomanoWolf2005
4. Technical Analysis examined with WRC, by P. Hsu, C. Kuan: HsuKuan2006
5. WRC and its Extensions by V. Corradi, N. Swanson: CorradiSwanson2011

• Trend exists, but is difficult to detect and to exploit. The experiment will find only a few profitable and a lot more unprofitable systems (with realistic trade costs).
• More sophisticated indicators, such as the Decycler or the Lowpass filter, will produce better results than the old SMA, EMA, or HMA. Modern indicators have less lag and thus react faster on the begin and end of a trend.
• White’s Reality Check will reveal a data mining bias rather close to the returns of the best systems from the experiment. If algorithmic trend trading were easy, anyone would be doing it.
• EUR/USD will return better results because according to trader’s wisdom, currencies are “more trending”.

However the results were quite a surprise.

The Trend Test System

This is the script Trend.c used for the trend experiment:

var objective() 
{ 
  return WinTotal/max(1,LossTotal); // Profit factor
}

var filter(var* Data,int Period);

void run()
{
  set(PARAMETERS|LOGFILE);
  PlotHeight1 = 200;
  PlotScale = 8;
  ColorBars[1] = BLACK; // winning trades
  ColorBars[2] = GREY; // losing trades

  StartDate = 2010;
  BarPeriod = 15;
  LookBack = 80*4*24; // 80 trading days ~ 4 months
	
  while(asset(loop("EUR/USD","SPX500","XAG/USD")))
  while(algo(loop("M15","H1","H4")))
  {
    if(Algo == "M15")
      TimeFrame = 1; // 15 minutes
    else if(Algo == "H1")
      TimeFrame = 4; // 1 hour
    else if(Algo == "H4")
      TimeFrame = 16;	// 4 hours

    int Periods[10] = { 10,20,50,100,200,500,1000,2000,5000,10000 };
    int Period = Periods[optimize(0,0,9,1)];
		
    vars Prices = series(price());
    vars Smoothed = series(filter(Prices,Period));

    if(valley(Smoothed))
      enterLong();
    else if(peak(Smoothed))
      enterShort();
  } 
}

This script is supposed to run in Zorro’s “Train” mode. We’re not really training the systems, but Train mode produces parameter histograms, and that’s what we want. The script is basically the same as described in the previous article, but two asset and algo loops are now used for cycling through three different assets (EUR/USD, S&P 500, and Silver) and three time frames (15 minutes, 1 hour, 4 hours). The optimize function selects the time period of the smoothing indicator, in 10 steps from 10 to 10,000 time frames. The meaning of the Zorro-specific functions like asset, algo, loop, optimize can be found in the Zorro manual.

6 years of price history from 2010 to 2015 are used for the simulation. The default trade costs – commission, bid/ask spread, and rollover – are taken from the average 2015 parameters of a FXCM microlot account. FXCM is not the cheapest broker, so the expectation value of a random-trading simulation is not zero, but a loss, dependent on the number of trades. For comparing results we’re using the profit factor, calculated in the objective function. The script calls a filter function that is defined as a prototype, and contained in a main script that includes Trend.c. So any of the indicators is represented by a very short main script like this (TrendLowpass.c):

#include "Strategy\Trend.c"

var filter(var* Data,int Period)
{
  return LowPass(Data,Period);
}

All the scripts can be downloaded from the scripts2015 archive at the sidebar. There’s also a batch file included that starts them all and lets them run in parallel.

The Results

When you select one of the scripts and click Zorro’s  [Train] button, it will grind its wheels about 10 minutes and then spit out a sheet of histograms like this (for the ZMA):

We got 9 histograms from the combinations of EUR/USD, S&P 500, and Silver (horizontally) with bar sizes of 15 minutes, 1 hour, and 4 hours. The red bars are the objective return – in this case, the profit factor,  total win divided by total loss. If they are above 1.0 (left scale), the system was profitable. The black and grey bars are the number of winning and losing trades (right scale). The steps on the X axis are the indicator time period; they normally don’t go all the way to 10 because systems are discarded when the time period exceeded 4 months or the system entered less than 30 trades. That’s why the experiment didn’t produce 900 systems, but only 705.

We can see that the profit generally increases, and the number of trades decreases with the time frame. As typical for trend trading systems we got far more losing trades (grey) than winning trades (black). 13 of the 82 systems in the histograms above have been profitable, 69 were losers. Most of the profitable systems traded EUR/USD. Only one S&P 500 system, and only three silver trading systems produced any profit. Here’s a statistic of all winners and losers from the 705 tested systems:

Since the results are for 2010-2015, you might get different figures when testing different time periods. The Rate column shows the percentage of successful systems. We got a mix of expected and unexpected results. Almost 40% of the EUR/USD systems are winners,  which seems to confirm that currencies tend to trend. The S&P 500 index however is not trending at all –  and this despite the fact that it shows the most significant trend in the long run, as visible in the title image. But here we’re looking for shorter term trends that can be exploited by day trading. Silver, with 20% winners, has at least some trend. Longer time frames produce dramatically better results, mostly due to less trade costs, but also partly because trend seems to prefer long cycles and low frequencies. (You can see that the difference comes not from trade costs alone when you test the systems with spread, commission, slippage, and rollover set to zero). 1-day bar periods would be even more profitable. But I haven’t included them in the test because they produce not enough trades for statistical significance.

The indicators show a surprising behavior. The old traditional indicators put up a good fight. Ehler’s smoothing, decycling, and zero-lag indicators, produce less profitable systems than their classic counterparts. However we’ll see later that the modern indicators are in fact more profitable, although within a smaller parameter range. But they all occupy the top of the profit list. The best systems, with modern indicators and long time frames, have profit factors up to 2.

Does this mean that we can put them together in a compound system and will get rich? I don’t know, because I haven’t determined yet the Data Mining Bias produced by this experiment. For this we need to store and randomize the balance curves, and apply White’s Reality Check. This will be the topic of the next article of the Trend Experiment series.

Conclusion

• Different markets show very different behavior. EUR/USD and silver are significantly more profitable with short-term trend following than S&P500.
• Low-lag smoothing indicators have a smaller profitable parameter range than traditional indicators, but produce higher profits.

• Information spreads slowly, thus producing a delayed, prolonged reaction and price movement. (There are examples where it took the majority of traders more than a year to become aware of essential, fundamental information about a certain market!)
• Events trigger other events with a similar effect on the price, causing a sort of avalanche effect.
• Traders see the price move, shout “Ah! Trend!” and jump onto the bandwagon, thus creating a self fulfilling prophecy.
• Trade algorithms detect a trend, attempt to exploit it and amplify it this way.
• All of this, but maybe on different time scales.

Whatever the cause, hackers prefer experiment over speculation, so let’s test once and for all if trend in price curves really exists and can be exploited in an algorithmic strategy. Exploiting trend obviously means buying at the begin of a trend, and selling at the end. For this, we simply assume that trend begins at the bottom of a price curve valley and ends at a peak, or vice versa. Obviously this assumption makes no difference between ‘real’ trends and random price fluctuations. Such a difference could anyway only be determined in hindsight. However, trade profits and losses by random fluctuations should cancel out each other in the long run, at least when trade costs are disregarded. When trends exist in price curves, this method should produce an overall profit from the the remaining trades that are triggered by real trends and last longer than in random walk curves. And hopefully this profit will exceed the costs of the random trades. At least – that’s the theory. So we’re just looking for peaks and valleys of the price curve. The script of the trend following experiment would look like this (all scripts here are in C for the Zorro platform; var is typedef’d to double):

void run()
{
  var *Price = series(price());
  var *Smoothed = series(Filter(Price,Period));
  
  if(valley(Smoothed))
    enterLong();
  else if(peak(Smoothed))
    enterShort();
}

This strategy is simple enough. It is basically the first strategy from the Zorro tutorial. The valley function returns true when the last price was below the current price and also below the last-but-one price; the peak function returns true when the last price was above the current and the last-but-one. Trades are thus entered at all peaks and valleys and closed by reversal only. For not getting too many random trades and too high trade costs, we’re removing the high frequencies from the price curve with some smoothing indicator, named Filter in the code above. Trend, after all, is supposed to be longer-lasting and should manifest itself in the lower frequencies. This is an example trade produced by this system: The black jagged line in the chart above is the raw price curve, the red line is the filtered curve. You can see that it lags behind the black curve a little, which is typical of smoothing indicators. It has a peak at the end of September and thus entered a short trade (tiny green dot). The red line continues going down all the way until November 23, when a valley was reached. A long trade (not shown in this chart) was then entered and the short trade was closed by reversal (other tiny green dot). The green straight line connects the entry and exit points of the trade. It was open almost 2 months and made in that time a profit of ~ 13 cents per unit, or 1300 pips.

The only remaining question is – which indicator shall we use for filtering out the high frequencies, the ripples and jaggies, from the price curve? We’re spoilt for choice.

The Smoothing Candidates

Smoothing a curve in some way is a common task of all trend strategies. Consequently, there’s a multitude of smoothing, averaging, low-lag and spectral filter indicators at our disposal. We have to test them. The following candidates were selected for the experiment, traditional indicators as well as fancier algorithms:

• SMA, the simple moving average, the sum of the last n prices divided by n.
• EMA, exponential moving average, the current price multiplied with a small factor plus the last EMA multiplied with a large factor. The sum of both factors must be 1. This is a first order lowpass filter.
• LowPass, a second order lowpass filter with the following source code:

  var LowPass(var *Data,int Period)
  {
    var* LP = series(Data[0]);
    var a = 2.0/(1+Period);
    return LP[0] = (a-0.25*a*a)*Data[0]
      + 0.5*a*a*Data[1]
      - (a-0.75*a*a)*Data[2]
      + 2*(1.-a)*LP[1]
      - (1.-a)*(1.-a)*LP[2];
  }

• HMA, the Hull Moving Average, with the following source code:

  var HMA(var *Data,int Period)
  {
    return WMA(series(2*WMA(Data,Period/2)-WMA(Data,Period),sqrt(Period))
  }

• ZMA, Ehler’s Zero-Lag Moving Average, an EMA with a correction term for removing lag. This is the source code:

  var ZMA(var *Data,int Period)
  {
    var *ZMA = series(Data[0]);
    var a = 2.0/(1+Period);
    var Ema = EMA(Data,Period);
    var Error = 1000000;
    var Gain, GainLimit=5, BestGain=0;
    for(Gain = -GainLimit; Gain < GainLimit; Gain += 0.1) {
       ZMA[0] = a*(Ema + Gain*(Data[0]-ZMA[1])) + (1-a)*ZMA[1];
       var NewError = Data[0] - ZMA[0];
       if(abs(Error) < newError) {
         Error = abs(newError);
         BestGain = Gain;
       }
    }
    return ZMA[0] = a*(Ema + BestGain*(Data[0]-ZMA[1])) + (1-a)*ZMA[1];
  }

• ALMA, Arnaud Legoux Moving Average, based on a shifted Gaussian distribution (described in this paper):

  var ALMA(var *Data, int Period)
  {
    var m = floor(0.85*(Period-1));
    var s = Period/6.0;
    var alma = 0., wSum = 0.;
    int i;
    for (i = 0; i < Period; i++) {
      var w = exp(-(i-m)*(i-m)/(2*s*s));
      alma += Data[Period-1-i] * w;
      wSum += w;
    }
    return alma / wSum;
  }

• Laguerre, a 4-element Laguerre filter:

  var Laguerre(var *Data, var alpha)
  {
    var *L = series(Data[0]);
    L[0] = alpha*Data[0] + (1-alpha)*L[1];
    L[2] = -(1-alpha)*L[0] + L[1] + (1-alpha)*L[2+1];
    L[4] = -(1-alpha)*L[2] + L[2+1] + (1-alpha)*L[4+1];
    L[6] = -(1-alpha)*L[4] + L[4+1] + (1-alpha)*L[6+1];
    return (L[0]+2*L[2]+2*L[4]+L[6])/6;
  }

• Linear regression, fits a straight line between the data points in a way that the distance between each data point and the line is minimized by the least-squares rule.
• Smooth, John Ehlers’ “Super Smoother”, a 2-pole Butterworth filter combined with a 2-bar SMA that suppresses the Nyquist frequency:

  var Smooth(var *Data,int Period)
  {
    var f = (1.414*PI) / Period;
    var a = exp(-f);
    var c2 = 2*a*cos(f);
    var c3 = -a*a;
    var c1 = 1 - c2 - c3;
    var *S = series(Data[0]);
    return S[0] = c1*(Data[0]+Data[1])*0.5 + c2*S[1] + c3*S[2];
  }

• Decycle, another low-lag indicator by John Ehlers. His decycler is simply the difference between the price and its fluctuation, retrieved with a highpass filter:

  var Decycle(var* Data,int Period)
  {
    return Data[0]-HighPass2(Data,Period);
  }

The above source codes have been taken from the indicators.c file of the Zorro platform, which contains source codes of all supplied indicators.

Comparing Step Responses

What is the best of all those indicators? For a first impression here’s a chart showing them reacting on a simulated sudden price step. You can see that some react slow, some very slow, some fast, and some overshoot: You can generate such a step response diagram with this Zorro script:

// compare the step responses of some low-lag MAs
function run()
{
  set(PLOTNOW);
  BarPeriod = 60;
  MaxBars = 1000;
  LookBack = 150;
  asset("");   // don't load an asset
  ColorUp = ColorDn = 0;  // don't plot a price curve
  PlotWidth = 800;
  PlotHeight1 = 400;

  var *Impulse = series(ifelse(Bar>200 && Bar<400,1,0)); // 0->1 impulse
  int Period = 50;
  plot("Impulse",Impulse[0],0,GREY);
  plot("SMA",SMA(Impulse,Period),0,BLACK);
  plot("EMA",EMA(Impulse,Period),0,0x808000);
  plot("ALMA",ALMA(Impulse,Period),0,0x008000);
  plot("Laguerre",Laguerre(Impulse,4.0/Period),0,0x800000);
  plot("Hull MA",HMA(Impulse,Period),0,0x00FF00);
  plot("Zero-Lag MA",ZMA(Impulse,Period),0,0x00FFFF); 
  plot("Decycle",Decycle(Impulse,Period),0,0xFF00FF);
  plot("LowPass",LowPass(Impulse,Period),0,0xFF0000);
  plot("Smooth",Smooth(Impulse,Period),0,0x0000FF);
}

We can see that from all the indicators, the ALMA seems to be the best representation of the step, while the ZMA produces the fastest response with no overshoot, the Decycler the fastest with some overshoot, and Laguerre lags a lot behind anything else. Obviously, all this has not much meaning because the time period parameters of the indicators are not really comparable. So the step response per se does not reveal if the indicator is well suited for trend detection. We have no choice: We must use them all.

The Trend Experiment

For the experiment, all smoothing indicators above will be applied to a currency, a stock index, and a commodity, and will compete in exploiting trend. Because different time frames can represent different trader groups and thus different markets, the indicators will be applied to price curves made of 15-minutes, 1-hour, and 4-hours bars. And we’ll also try different indicator time periods in 10 steps from 10 to 10000 bars.

So we’ll have to program 10 indicators * 3 assets * 3 bar sizes * 10 time periods = 900 systems. Some of the 900 results will be positive, some negative, and some zero. If there are no significant positive results, we’ll have to conclude that trend in price curves either does not exist, or can at least not be exploited with this all-too-obvious method. However we’ll likely get some winning systems, if only for statistical reasons.

The next step will be checking if their profits are for real or just caused by a statistical effect dubbed Data Mining Bias. There is a method to measure Data Mining Bias from the resulting equity curves: the notorious White’s Reality Check. We’ll apply that check to the results of our experiment. If some systems survive White’s Reality Check, we’ll have the once-and-for-all proof that markets are ineffective, trends really exist, and algorithmic trend trading works.

Next Step…