The Market Meanness Index

This indicator can improve – sometimes even double – the profit expectancy of trend following systems. The Market Meanness Index tells whether the market is currently moving in or out of a “trending” regime. It can this way prevent losses by false signals of trend indicators. It is a purely statistical algorithm and not based on volatility, trends, or cycles of the price curve.

There are already several methods for differentiating trending and nontrending market regimes. Some of them are rumored to really work, at least occasionally. John Ehlers proposed the Hilbert Transform or a Cycle / Trend decomposition, Benoit Mandelbrot the Hurst Exponent. In comparison, the source code of the Market Meanness Index is relatively simple:

// Market Meanness Index
double MMI(double *Data,int Length)
{
  double m = Median(Data,Length);
  int i, nh=0, nl=0;
  for(i=1; i<Length; i++) {
    if(Data[i] > m && Data[i] > Data[i-1]) // mind Data order: Data[0] is newest!
      nl++;
    else if(Data[i] < m && Data[i] < Data[i-1])
      nh++;
  }
  return 100.*(nl+nh)/(Length-1);
}

This code is in C for Zorro, but meanwhile also versions for MT4, Amibroker, Ninja Trader, and other platforms have been programmed by users. As the name suggests, the indicator measures the meanness of the market – its tendency to revert to the mean after pretending to start a trend. If that happens too often, all trend following systems will bite the dust.

The Three-Quarter Rule

Any series of independent random numbers reverts to the mean – or more precisely, to their median – with a probability of 75%. When you have a sequence of random, uncorrelated daily data, if Monday’s data value was above the median, then in 75% of all cases Tuesday’s data will be lower than Monday’s. And if Monday was below the median, 75% chance is that Tuesday will be higher. The proof of the 75% rule is relatively simple and won’t require integral calculus. Consider a data series with median M. By definition, half the values are less than M and half are greater (for simplicity’s sake we’re ignoring the case when a value is exactly M). Now combine the values to pairs each consisting of a value Py and the following value Pt. Thus each pair represents a change from Py to Pt. We now got a lot of changes that we divide into four sets:

  1. (Pt < M, Py < M)
  2. (Pt < M, Py > M)
  3. (Pt > M, Py < M)
  4. (Pt > M, Py > M)

These four sets have obviously the same number of elements – that is, 1/4 of all Py->Pt changes – when Pt and Py are uncorrelated, i.e. completely independent of one another. The value of M and the kind of data in the series won’t matter for this. Now how many data pairs revert to the median? All pairs that fulfill this condition: (Py < M and Pt > Py) or (Py > M and Pt < Py) The condition in the first bracket is fulfilled for half the data in set 1 (in the other half is Pt less than Py) and in the whole set 3 (because Pt is always higher than Py in set 3). So the first bracket is true for 1/2 * 1/4 + 1/4 = 3/8 of all data changes. Likewise,  the second bracket is true in half the set 4 and in the whole set 2, thus also for 3/8 of all data changes. 3/8 + 3/8 yields 6/8, i.e. 75%. This is the three-quarter rule for the differences of random numbers.

The MMI function just counts the number of data differences for which the conditition is met, and returns their percentage. The Data series may contain prices or price changes. Prices have always some serial correlation: If EUR / USD today is at 1.20, it will also be tomorrow around 1.20. That it will end up tomorrow at 70 cents or 2 dollars per EUR is rather unlikely. This serial correlation is also true for a price series calculated from random numbers, as not the prices themselves are random, but their changes. Thus, the MMI function should return a smaller percentage, such as 55%, when fed with prices.

 Unlike prices, price changes have not necessarily serial correlation. A one hundred percent efficient market has no correlation between the price change from yesterday to today and the price change from today to tomorrow. If the MMI function is fed with perfectly random price changes from a perfectly efficient market, it will return a value of about 75%. The less efficient and the more trending the market becomes, the more the MMI decreases. Thus a falling MMI is a indicator of an upcoming trend. A rising MMI hints that the market will get nastier, at least for trend trading systems.

Using the MMI in a trend strategy

One could assume that MMI predicts the price direction. A high MMI value indicates a high chance of mean reversion, so when prices were moving up in the last time and MMI is high, can we expect a soon price drop? Unfortunately it doesn’t work this way. The probability of mean reversion is not evenly distributed over the Length of the Data interval. For the early prices it is high (since the median is computed from future prices), but for the late prices, at the very time when MMI is calculated, it is down to just 50%. Predicting the next price with the MMI would work as well as flipping a coin.

Another mistake would be using the MMI for detecting a cyclic or mean-reverting market regime. True, the MMI will rise in such a situation, but it will also rise when the market becomes more random and more effective. A rising MMI alone is no promise of profit by cycle trading systems.

So the MMI won’t tell us the next price, and it won’t tell us if the market is mean reverting or just plain mean, but it can reveal information about the success chance of trend following. For this we’re making an assumption: Trend itself is trending. The market does not jump in and out of trend mode suddenly, but with some inertia. Thus, when we know that MMI is rising, we assume that the market is becoming more efficient, more random, more cyclic, more reversing or whatever, but in any case bad for trend trading. However when MMI is falling, chances are good that the next beginning trend will last longer than normal.

This way the MMI can be an excellent trend filter – in theory. But we all know that there’s often a large gap between theory and practice, especially in algorithmic trading. So I’m now going to test what the Market Meanness Index does to the collection of the 900 trend following systems that I’ve accumulated. For a first quick test, this was the equity curve of one of the systems, TrendEMA, without MMI (44% average annual return):

EMA strategy without MMI

This is the same system with MMI (55% average annual return):

EMA strategy with MMI

We can see that the profit has doubled, from $250 to $500. The profit factor climbed from 1.2 to 1.8, and the number of trades (green and red lines) is noticeable reduced. On the other hand, the equity curve started with a drawdown that wasn’t there with the original system. So MMI obviously does not improve all trades. And this was just a randomly selected system. If our assumption about trend trendiness is true, the indicator should have a significant effect also on the other 899 systems.

This experiment will be the topic of the next article, in about a week. As usually I’ll include all the source code for anyone to reproduce it. Will the MMI miserably fail? Or improve only a few systems, but worsen others? Or will it light up the way to the Holy Grail of trend strategies? Let the market be the judge.

Next: Testing the Market Meanness Index


Addendum (2022). I thought my above proof of the 3/4 rule was trivial and no math required, but judging from many ongoing comments, it is not so. Maybe the rule appears a bit counter-intuitive to the layman. Or I have just bad explained it. In that case, read up the proof anywhere else. I found the two most elaborate proofs of the 3/4 rule, geometrical and analytical, in this book:

(1)  Andrew Pole, Statistical Arbitrage, Wiley 2007

And if you don’t want to read: Take a dice, roll it a 100 times, get the resulting  median, and simply count how often a pair reverted to the median.

Here’s the MMI applied to a random noise generator, sidewards (black) and with added trend (blue):

MMI

The code:

function run()
{
	BarPeriod = 60;
	MaxBars = 1000;
	LookBack = 500;
	asset(""); // dummy asset
	ColorUp = ColorDn = 0; // don't plot a price curve
	set(PLOTNOW);
	
	if(Init) seed(1000);
	int TrendStart = LookBack + 0.4*(MaxBars-LookBack);
	vars Signals = series(1 + 0.5*genNoise());
	if(Bar > TrendStart) 
		Signals[0] += 0.003*(Bar-TrendStart);
	int Color = RED;
	int TimePeriod = 0.5*LookBack;
	if(Bar < TrendStart)
		plot("Sidewards",Signals,MAIN,BLACK);
	else
		plot("Trending",Signals,MAIN,BLUE);
	plot("MMI",MMI(Signals,TimePeriod),NEW|LINE,Color);
}

123 thoughts on “The Market Meanness Index”

  1. Thanks. Your code looks good to me except for the med[bar]. If I understand AFL right, that’s a past array value, but the median must not change inside the loop. So use med[BarCount], not med[bar].

  2. Just wanted to say that I absolutely love it. MMI is the backbone of my algo.

  3. I think your proof is wrong.
    I have some ideas of the mistakes in the proof, but it’s kind of obvious that the conclusion is wrong even if you don’t know why. Obviously you don’t have more then 50% in predicting tomorrow in random.
    Now to the proof, I think there are atleast two mistakes, one is that the median is not known till the pt day, but more importantly, if pt is the following day to py than the groups are not of same size. The group with one point above m and one below m will be smaller than the other two groups.

  4. Thank you for your comment. You can read up the proof in many statistics books – I did not invent it – but your “median” concept looks wrong. A median of a data sample is simply the 50% divide. It has nothing to do with “past or present days”, and is always known since you know all data values of the sample. If something else is unclear, just ask.

    If you want to learn more about the 75% rule and its use in Finance, read “Statistical Arbitrage” by Andrew Pole. He explains the rule on many pages with several proofs.

  5. this is just the classic academic stereotype that love intellectual masturbation rather than application or result.

  6. ^ my comment referring back to the discussion back in 2020 with the “masters degree” guy. MMI is just fantastic. thanks for sharing

  7. jcb I read through your conversation with “lurker” and tried their code and it looks right to me. Why would we use MMI when we can just use serial correlation and why would we use if random walks just give the same MMI as trend?

  8. I can at least answer the second part of your question: A purely random walk has serial correlation, and a trending time series has even more serial correlation due to the trend. The purpose of the MMI is differentiating between the two. As to the first part, since the MMI is a measure of serial correlation, you’re in fact using it when using the MMI -if that was the question.

  9. Ok thank you but I just try calculate the MMI of a random walk with trend and random walk without trend and the MMI the same for both. Why are they not different?

  10. Thank you very much jcb and please excuse if I’m not clear but I think there’s two problems.

    Main problem is you are using median for entire price series to calculate MMI and so it look like it go down a little. But in reality we don’t know median of whole price series. On Feb 12 when trend start, as traders we don’t know trend is about to start. At that time we only know median of series up to Feb 12, not median of series including Feb 13, Feb 14 etc. I think this what Sharon mean when she try to talk about lookahead bias. So we have to use rolling median over some span of time from Feb 12 back to some day like Feb 1 or something. If we do that the MMI quickly show no signal for random walk without trend v random walk with trend. I have code below which show.

    Second problem even in your example we see MMI barely change. It is at 52 when you start the Trend part on Feb 12 and drop to 50 on Feb 19 then drop to 48 on Feb 22 then back up to 50 on Feb 23 then down to 46 on Feb 26 then at 49 on Mar 2. So I dont see how we can use this to tell when random stock price movements about to turn into a trend.

    Code:

    import matplotlib.pyplot as plt
    import numpy as np
    import pandas as pd

    def MMI(series):
    nh = 0
    nl = 0

    for i in range(1, len(series)):
    m = np.median(series.iloc[:i])
    if series.iloc[i-1] > m and series.iloc[i] series.iloc[i-1]:
    nh += 1

    return 100 * (nh + nl) / len(series)

    def MMI_ma(series, timesteps):
    series = series.dropna()
    list_of_lists = [series.iloc[x[0]:x[1]] for x in zip(range(len(series) – timesteps + 1), range(timesteps, len(series) + 1))]
    idx = series.index[timesteps-1:]
    ma = pd.Series([MMI(x) for x in list_of_lists], index = idx)
    return ma

    np.random.seed(16)

    diffs_no_trend = np.random.normal(size = 1000)
    diffs_with_trend = np.random.normal(size = 1000) + np.ones(shape = 1000) / 4
    rwalk_no_trend = np.cumsum(diffs_no_trend)
    rwalk_with_trend = np.cumsum(diffs_with_trend)
    complete_series = pd.Series(np.concatenate([rwalk_no_trend, rwalk_with_trend]))

    plt.figure(figsize=(10,2))
    plt.plot(complete_series)
    plt.show()

    running_MMI = MMI_ma(complete_series, 100)

    plt.figure(figsize=(10,2))
    plt.plot(running_MMI)
    plt.show()

  11. Indeed. Since you now found out that we don’t have the median of tomorrow’s prices, you know the sad truth of finance: the future is unknown. All indicators, including the MMI, use past prices. If I knew the median of tomorrow’s prices, I’d be billionaire and would need no MMI anymore.

    The MMI is quite sensitive, as you see from its reaction on regime change in the charts. This is not ‘barely’, but dramatic, at least for a statistical indicator. For its practical use, check out the examples in the subsequent articles. The MMI is meanwhile long established and is one of the most frequent indicators that we use for market change detection. If you haven’t noticed: This article was from 2015.

  12. How can it be useful if its showing 50 when there’s a random walk and then shows 50 again later during when the trend is supposed to be happening?

  13. Price changes are no random walk, they are random noise. And you’re anyway not checking some value like 50, but the MMI direction. Normally after lowpass filtering to remove random fluctuations. Look in the code examples. 20% improvement is, at least in finance, normally considered useful. Does this answer your question?

  14. We discovered this site at our fund a few days back, and a number of us have been obsessively reading through the comments with bemused fascination (and some amusement). I wanted to throw in my two cents to try to clear up some of the confusion that seems to be prevalent.

    1. The author seems not to understand that there is a difference between random noise and random walks. This is evident by the graphs they keep displaying.

    2. The author therefore thinks that the (correct) proof about distribution around the median in random noise can be extrapolated to random walks. It can’t. After all… what is the “median” of a random walk?

    3. Several commenters have pointed out this flaw to the author and tried to explain that, since no median exists for the price series as a whole, a median from some arbitrary finite time window in the past of the price series needs to be used, at which point the MMI becomes completely useless, as has been repeatedly demonstrated.

    4. I am not aware of any StatArb professionals who use the MMI in their calculations.

    5. The book by Andrew Pole that the author seems to revere, “Statistical Arbitrage: Algorithmic Trading Insights and Techniques” is regarded as something of a laughing stock among StatArb guys, and you can even see why in the caustic reviews of the book on Amazon: https://www.amazon.co.uk/Statistical-Arbitrage-Algorithmic-Insights-Techniques/dp/0470138440

  15. For some reason, this article, even after almost a decade, seems to still evoke heavy emotions in some readers. I take this as a compliment. And share amusement about certain comments. But without now again explaining a random walk and how its median is calculated – which you can easily read up in Wikipedia – just out of curiosity: who is “we”?

  16. As someone already mentioned with an example, a perfect trend gives a MMI value of 50%.

    In the book (3rd edition, p. 78) you write: “At 100%, any price movement would be compensated immediately by a counter-movement, and at 0% a trend would continue indefinitely”.

    One example where I could obtain a value close to 0% was a series consisting of [100, 100,…., 100, 102, 102…, 102] which “feels” less trendy than y=x with 50%.

    A series with a value of 100% would be alternating 100, 102, 100, 102… which makes perfect sense.

    How would you interpret an MMI of 60% vs one of 45%? Does a lower value mean a longer-term trend is likely to start? Or checking the rate of change in decreasing MMI values would be more appropriate?

  17. “””
    How would you interpret an MMI of 60% vs one of 45%? Does a lower value mean a longer-term trend is likely to start? Or checking the rate of change in decreasing MMI values would be more appropriate?
    “””

    Just to clarify: assumption is we have a peak/valley forming in the low pass filter series as well.

  18. We found that evaluating the MMI slope works normally better for trend regime detection than its absolute value.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.