The C code of the used Hann filter, straight from Ehler’s article:

var Hann(vars Data,int Length) { var Filt = 0, Coeff = 0; int i; for(i=1; i<=Length; i++) { Filt += (1-cos(2*PI*i/(Length+1)))*Data[i-1]; Coeff += 1-cos(2*PI*i/(Length+1)); } return Filt/fix0(Coeff); }

The **fix0** function in the denominator is a convenience function for fixing division by zero issues.

We will now apply this Hann filter to the undersampled SPY price curve. Undersampling means: Take only every n-th price, and throw the rest away. We use the modulo operator to detect every 5th bar:

void run() { StartDate = 20220101; EndDate = 20221231; BarPeriod = 1440; set(PLOTNOW); asset("SPY"); vars Samples = series(); if(Init || Bar%5 == 0) // sample only every 5th bar Samples[0] = priceC(0); else Samples[0] = Samples[1]; plot("Hann6",Hann(Samples,6),LINE,MAGENTA); plot("Hann12",Hann(Samples,12),LINE,BLUE); plot("Hann",Hann(seriesC(),12),0,DARKGREEN); }

The resulting chart:

The blue line is the Hann filter of the undersampled curve with period 12, the magenta line with period 6. For comparison I’ve added the Hann filter output from the original curve (thin green line). We can see that the green line has less lag, but is also less smooth. Will using undersampled data improve a trading system? Well… I guess it depends on the system.

The Hann indicator and undersampling test script can be downloaded from the 2022 script repository.

]]>The script below, in C for the Zorro platform, compares the standard RSI with the open-close average RSI on the S&P 500 index with 15-minute bars:

void run() { BarPeriod = 15; StartDate = 20220629; EndDate = 20220712; asset("SPX500"); vars OC = series((priceO()+priceC())/2); plot("RSI(Close)",RSI(seriesC(),14),NEW,RED); plot("RSI(OC)",RSI(OC,14),0,BLUE); }

We can indeed see some noise reduction in the resulting chart:

The obvious question: Will the smoother curve compensate for the additional lag in a trading system? For testing this, I added the following 5 lines to the script:

vars RSIs = series(RSI(OC,14)); if(crossUnder(RSIs,70)) enterShort(); if(crossOver(RSIs,30)) enterLong();

That’s a simple RSI trading system: Enter a short position when the RSI crosses below 70, and enter a long position when it crosses above 30. Any position is closed when an opposite position is opened. I found indeed that that using the open-close average produced a better result with some instruments and time periods, but a worse result with others. There was no clear tendency. However, it’s certainly worth a try when you’re anyway developing an indicator based trading system.

The script can be downloaded from the 2023 script repository.

]]>For connecting the Zorro platform to a particular broker API, a DLL must be dropped in its Plugin folder. The DLL is then automatically recognized and appears in Zorro’s Broker selection box. For trading with the broker, the DLL exports functions for getting prices and sending orders. These functions are standardized and described in the Zorro manual. They are the same for all brokers. The following 4 functions are the most important:

**BrokerOpen** – identify the DLL.**BrokerLogin** – initialize and store login credentials.**BrokerAsset** – retrieve price and other parameters of an asset.**BrokerBuy2** – send a buy or sell order.

These four are sufficient for running most trading strategies. The DLL can export additional functions that are useful, but not mandatory, since Zorro works around them when they are not implemented:

**BrokerAccount** -get the account state.**BrokerTrade** – get the trade or order state.**BrokerHistory2** -get price history.**BrokerCommand** – set special modes, get order book, etc.

Zorro users have written DLLs for several broker APIs, but the code is often redundant and invents the wheel many times again. I’ll give here an example of a relatively lean broker DLL that can be used as template for individual REST API implementations. Bittrex is a US based digital currency exchange that supports several hundred crypto currencies and provides free API access. We’re first going to implement the 4 mandatory functions, following the API description on the Bittrex website:

https://bittrex.github.io/api/v3

We will use the free Visual C++ Community Edition from Microsoft. Aside from the exported functions, we need some internal functions to print messages, send HTTP requests, parse a JSON response, and generate a digital signature. For all this there’s ready code available on the Internet, but there are also ready functions in Zorro’s library. To make life easier, we simply import the Zorro function library to the DLL. The VC++ setup for generating such a DLL is described here:

https://zorro-project.com/manual/en/dlls.htm

The setup is for a C++ trading strategy, but works as well for a broker API DLL. First we’re going to implement the adminstrative stuff, the **BrokerOpen** and **BrokerLogin** functions:

#define PLUGIN_TYPE 2

#define PLUGIN_NAME "Bittrex V3"

#define DLLFUNC extern "C" __declspec(dllexport)

int (__cdecl *BrokerMessage)(const char *Text);

int (__cdecl *BrokerProgress)(const int Progress);

...

DLLFUNC int BrokerOpen(char* Name,FARPROC fpMessage,FARPROC fpProgress)

{

strcpy_s(Name,32,PLUGIN_NAME);

(FARPROC&)BrokerMessage = fpMessage;

(FARPROC&)BrokerProgress = fpProgress;

return PLUGIN_TYPE;

}

The **BrokerOpen** function is Zorro specific. It sets up two function pointers for printing message and sending heartbeats to Zorro. It’s needed for recognizing the DLL in the **Plugin** folder and letting the plugin name, “Bittrex V3”, appear in the broker scrollbox.

Next we need the login function that’s called at the begin and the end of any algo trading session

struct GLOBAL {

int PriceType,VolType,OrderType;

double Unit; // smallest trade unit

char Path[256]; // send path buffer

char Key[256],Secret[256]; // credentials

char Symbol[64],Uuid[256]; // last trade symbol and UUID

char AccountId[16]; // account currency

} G; // parameter singleton

...

DLLFUNC int BrokerLogin(char* User,char* Pwd,char* Type,char* Accounts)

{

if(User) { // login

memset(&G,0,sizeof(G));

strcpy_s(G.Key,User);

strcpy_s(G.Secret,Pwd);

return 1;

} else // logout

return 0;

}

This function only sets up the **G** struct, a singleton that holds all global variables of the DLL. The **User** and **Pwd** arguments contain the Bittrex key and secret. They are taken from the Zorro login fields or from the account list. **Type** is the account type, ignored here because Bittrex has no demo accounts. **Account** is the account identifier and also ignored since we’re not using sub-accounts.

Next comes the function to retrieve prices and other parameters of a particular cryptocurrency. It’s a bit more complex and requires a HTTP request to the API. Some requests require authentication, some not. For all this we first implement a **send()** function in the DLL that uses Zorro’s functions for sending http requests and generating a HMAC signature. The authentication method by the Bittrex REST API V3 is a bit complex:

*To properly sign an authenticated request for the Bittrex v3 API, the following headers must be included: Api-Key, Api-Timestamp, Api-Content-Hash, Api-Signature. Api-Timestamp is the current time as a UNIX timestamp in epoch-millisecond format. Api-Content-Hash is a SHA512 hash of the request body, Hex-encoded (with no request body, a SHA512 hash of an empty string). For creating the Api-Signature, first generate a string by concatenating the following items: Contents of the Api-Timestamp header; the full URI used to make the request, including query string; the HTTP method of the request, in all caps (GET, POST, DELETE, etc.); Contents of the Api-Content-Hash header. Sign this string via HmacSHA512, using the API secret as the signing secret. Hex-encode the result and populate the Api-Signature header with it.*

This is the implementation:

#define RHEADER "https://api.bittrex.com/v3/"

...

int sleep(int ms)

{

Sleep(ms);

return BrokerProgress(0);

}

char* send(const char* Url,

int Mode = 0,

const char* Method = NULL,

const char* Body = NULL) { static char Url[1024], Header[2048], Signature[1024], Buffer1[1024*1024], Buffer2[2048]; *Signature = *Header = 0; sprintf_s(Url,"%s%s",RHEADER,Url); if (Mode & 1) { // Authentication required strcpy_s(Header, "Content-Type:application/json"); strcat_s(Header, "\nAccept:application/json"); strcat_s(Header, "\nApi-Key: "); strcat_s(Header,G.Key); strcat_s(Header, "\nApi-Timestamp: "); __time64_t Time; _time64(&Time); static __time64_t Offset = 0; char* TimeStamp = i64toa(Time*1000 + Offset++); strcat_s(Header, TimeStamp); strcat_s(Header, "\nApi-Content-Hash: "); char* Hash = hmac(Body, 0, 0, 512); strcat_s(Header,Hash); strcpy_s(Signature, TimeStamp); strcat_s(Signature, Url); if(Method && *Method) strcat_s(Signature, Method); else if(!Body) strcat_s(Signature,"GET"); else strcat_s(Signature,"POST"); strcat_s(Signature, Hash); strcat_s(Header, "\nApi-Signature: "); strcat_s(Header, hmac(Signature, 0, G.Secret, 512)); } char* Response = Mode & 2? Buffer2 : Buffer1;

int MaxSize = Mode & 2? sizeof(Buffer2) : sizeof(Buffer1);

int Id = http_request(Url,Body,Header,Method); if(!Id) goto send_error; // wait 30 seconds for the server to reply int Size = 0, Wait = 3000; while (!(Size = http_status(Id)) && --Wait > 0) if(!sleep(10)) goto send_error; if (!Size) goto send_error; if(!http_result(Id,Response,MaxSize)) goto send_error; Response[MaxSize-1] = 0; // prevent buffer overrun http_free(Id); return Response;

// transfer unsuccessful? send_error: if(Id) http_free(Id); return NULL; }

The above code adds an authentication header with signature when **Mode** == 1. **Mode == 2** selects a second, smaller response buffer and thus allows to evaluate two API responses at the same time. The **sleep** function generates a delay with a heartbeat to keep the Zorro window responsive during long requests. If it returns **0**, someone has hit Zorro’s [Stop] key and all operations should be aborted. The Windows **_time64** function has only 1-second resolution, so we add an incremented **Offset** for generating unique timestamps.

We’re now prepared to implement the **BrokerAsset** function:

double fixAmount(double Value)

{

int Exp = log10(Value);

if (Exp >= 0) Exp++;

return pow(10,Exp);

}

DLLFUNC int BrokerAsset(char* Symbol,double* pPrice,double* pSpread,

double *pVolume, double *pPip, double *pPipCost, double *pMinAmount,

double *pMargin, double *pRollLong, double *pRollShort, double *pCommission)

{

sprintf_s(G.Url,"markets/%s/ticker",fixSymbol(Symbol));

char* Response = send(G.Url,2);

if(!Response) return 0;

double Bid = strvar(Response, "bidRate", 0.),

Ask = strvar(Response,"askRate",0.);

if (Ask > 0. Bid > 0. && pSpread)

*pSpread = Ask - Bid;

double Last = strvar(Response, "lastTradeRate", 0.);

if(Ask == 0. || G.PriceType == 2)

Ask = Last;

if (Ask == 0.) return 0; // symbol does not exist

if(pPrice) *pPrice = Ask;

if(pVolume) {

sprintf_s(G.Url,"markets/%s/summary",fixSymbol(Symbol));

Response = send(G.Url,2);

if (Response) {

if (G.VolType == 4)

*pVolume = strvar(Response, "volume", 0);

else

*pVolume = strvar(Response, "quoteVolume", 0);

}

}

if (pMinAmount) { // get lot amount

sprintf_s(G.Url,"markets/%s",fixSymbol(Symbol));

Response = send(G.Url,2);

if (Response) {

*pMinAmount = fixAmount(strvar(Response,"minTradeSize",0.000001));

if (pPip) {

int Exp = strvar(Response,"precision",8);

*pPip = pow(10,-Exp);

while (*pPip * *pMinAmount < 0.000000001)

*pPip *= 10; // avoid too small pip cost

if (pPipCost)

*pPipCost = *pPip * *pMinAmount;

}

}

}

if (pMargin)

*pMargin = -100; // no leverage

return 1;

}

This function is supposed to return current price, current ask-bid spread, and current volume. It can also optionally request other asset specific parameters when available from the API. Otherwise Zorro will replace them with parameter values from the asset list. Broker APIs rarely provide all requested parameters. Make sure to only calculate and fill a parameter when its pointer passed to the function is nonzero. Most of the pointers are NULL most of the time.

The **send** function uses the second buffer because we’ll need price requests internally for calculating the account state. For getting lot amount, pip size, and pip cost, two more API requests are needed. The values from the response string are parsed with Zorro’s **strvar** function. The **fixAmount** function converts a value to its next-higher power of 10 – for instance, 7 is converted to 10 and 0.07 to 0.1. We’re doing this because we don’t want a strange number for the lot amount. The pip size is calculated from the precision, but we prevent it from going too small. Bittrex has no leverage, so the margin cost is always 100%.

The next and final function sends an order to the API:

DLLFUNC int BrokerBuy2(char* Symbol,int Volume,double StopDist,double Limit,double *pPrice,int *pFill)

{

if(!isConnected() || !Volume) return 0;

// compose the body

char Body[256] = "{\n";

strcat_s(Body,"\"marketsymbol\": \"");

strcat_s(Body,fixSymbol(Symbol));

strcat_s(Body,"\",\n\"direction\": \"");

strcat_s(Body,Volume > 0? "BUY" : "SELL");

strcat_s(Body,"\",\n\"type\": \"");

strcat_s(Body,Limit > 0. ? "LIMIT" : "MARKET");

strcat_s(Body,"\",\n\"quantity\": \"");

double Size = labs(Volume);

if(G.Amount < 1.) Size *= G.Amount;

strcat_s(Body,ftoa(Size));

if (Limit > 0.) {

strcat_s(Body,"\",\n\"limit\": \"");

strcat_s(Body,ftoa(Limit));

}

strcat_s(Body,"\",\n\"timeInForce\": \"");

if ((G.OrderType&2) && Limit > 0.)

strcat_s(Body,"GOOD_TIL_CANCELLED"); // limit orders only

else if (G.OrderType&1)

strcat_s(Body,"FILL_OR_KILL"); // fill all or nothing

else

strcat_s(Body,"IMMEDIATE_OR_CANCEL");

strcat_s(Body,"\"\n}");

char* Response = send("orders",1,0,Body);

if(!Response) return 0;

char* Uuid = strtext(Response,"id","");

if(!*Uuid) return 0; // failed

strcpy(G.Uuid,Uuid);

double Filled = strvar(Response,"fillQuantity",0);

if(Filled == 0. && !(G.OrderType&2))

return 0; // unfilled FOK/IOC order

double Price = strvar(Response,"proceeds",0);

if (Filled > 0. && Price > 0. && pPrice)

*pPrice = Price/Filled;

if (G.Unit < 1.) Filled *= G.Unit;

if (*pFill) *pFill = Filled;

return -1;

}

The function first composes a message body in this JSON format (example):

`{`

"marketSymbol": "ETH/BTC",

"direction": "BUY",

"type": "LIMIT",

"quantity": "0.1",

"limit": "0.008",

"timeInForce": "IMMEDIATE_OR_CANCEL"

}

If the order is accepted, the response has this JSON format:

{

"id": "12345-6789-007-4711",

"marketSymbol": "ETH/BTC",

"direction": "BUY",

"type": "LIMIT",

"quantity": "0.1",

"limit": "0.008",

"timeInForce": "IMMEDIATE_OR_CANCEL",

"fillQuantity": "0.05",

"commission": "0.000002",

"proceeds": "0.00001",

"status": "CLOSED"

}

The relevant fields, especially the id, the fill amount, and the fill price, are parsed from the response and returned. The return value **-1** indicates to Zorro that the order generated no identifer number, but an UUID. It was stored and can be retrieved with a subsequent command. **G.Unit **is the lot amount that was generated with the previous BrokerAsset call.

In part 2 of this series we’ll implement the remaining functions **BrokerTrade**, **BrokerAccount**, **BrokerHistory2**, and **BrokerCommand**. We’ll then have everything together for a relatively complex and rich broker API implementation. The full source code of the Bittrex DLL is available in the **Source** folder of the latest Zorro beta version.

The LR_EMA indicator can be directly converted from Apirine’s MetaStock to C:

```
var LREMA(int Periods,int Pds,var Mltp)
{
var Mltp1 = 2./(Periods+1);
var LR = LinearReg(seriesC(),Pds);
vars Dists = series(abs(LR-priceC()));
var ST = (Dists[0]-MinVal(Dists,Pds))/
max(MaxVal(Dists,Pds)-MinVal(Dists,Pds),0.01);
var Rate = min(1.,Mltp1*(1+ST*Mltp));
return EMA(priceC(),Rate);
}
```

The max and min functions prevent divisions by zero and EMA alpha parameters (**Rate**) above 1. This can otherwise happen at the begin of the backtest when price series are still flat.

The LREMA(20,20,5) and EMA(20) applied on SPY replicates the chart in Apirine’s article:

We can see that the LREMA (blue line) follows the price more closely than the EMA (red line). Which is not really surprising, because LREMA period is smaller at most places. But how good can the LREMA identify trend changes in a real trading system?

For testing this, we write a simple SPY system that concurrently trades with EMA, and with LREMA. A single position is entered when the faster EMA crosses over a slower EMA, and closed when it crosses back. The LREMA is used likewise. For not comparing apples with oranges, we optimize all time periods and test with walk-forward analysis. The code of this trading system in C:

void run() { BarPeriod = 1440; LookBack = 200; StartDate = 2005; EndDate = 2022; assetList("AssetsIB"); asset("SPY"); // walk-forward optimization setup set(PARAMETERS,TESTNOW,PLOTNOW); NumWFOCycles = 8; // run a loop over the two algos while(algo(loop("LR_EMA","EMA"))) { vars Signals1, Signals2; int Color; // optimize time periods int Period1 = optimize(20,10,50,10); int Period2 = Period1*optimize(2.5,1.5,3,0.5); if(Algo == "LR_EMA") { Signals1 = series(LREMA(Period1,Period1,5)); Signals2 = series(LREMA(Period2,Period2,5)); Color = BLUE; } else if(Algo == "EMA") { Signals1 = series(EMA(seriesC(),Period1)); Signals2 = series(EMA(seriesC(),Period2)); Color = GREY; } // trade on fast/slow crossovers if(crossOver(Signals1,Signals2)) enterLong(); if(crossUnder(Signals1,Signals2)) exitLong(); // plot equity curve plot(Algo,ProfitOpen+ProfitClosed,LINE+AXIS2,Color); } }

This script trades two different algos and compares the resulting equity curves. The fast and slow periods are optimized separately for both algorithms. The slow period is the fast period multiplied with a factor. This guarantees that the slow period is always bigger than the fast period during the optimization. The system must be first trained by clicking the [Train] button on the Zorro panel. After training, it automatically starts a backtest with walk-forward analysis.

Training and analysis take about 2 seconds. The resulting equity curve is plotted in a blue line for the LR_EMA algo, and in a grey line for the standard EMA algo:

We can see that both algorithms yield a positive result, but the standard EMA a bit more so than the LR_EMA. Not only the profit is smaller, the drawdowns are also worse with the new invented indicator – which is probably the reason why the article, like almost any article about a new indicator, did not contain an example system or a backtest. But maybe the walk-forward period optimization was simply more effective on the fixed-period EMA, than on the variable-period LR_EMA. If so, then the LR_EMA would fare better with unoptimized systems.

The LREMA indicator and the EMA/LREMA trading system can be downloaded from the 2022 script repository.

]]>

For his loop chart, Ehlers filtered the low and high frequencies out of the two data series with a roofing filter. Its code in C for Zorro:

var Roofing(var *Data,int HPeriod,int LPeriod) { var f = 1.414*PI/HPeriod; var hpa1 = exp(-f); var hpc2 = 2*hpa1*cos(f/2); var hpc3 = -hpa1*hpa1; var hpc1 = (1 + hpc2 - hpc3) / 4; f = 1.414*PI/LPeriod; var ssa1 = exp(-f); var ssc2 = 2*ssa1*cos(f/2); var ssc3 = -ssa1*ssa1; var ssc1 = 1 - ssc2 - ssc3; vars HP = series(0,3); HP[0] = hpc1*(Data[0] - 2*Data[1] + Data[2]) + hpc2*HP[1] + hpc3*HP[2]; vars SS = series(HP[0],3); SS[0] = ssc1*(HP[0] + HP[1])/2 + ssc2*SS[1] + ssc3*SS[2]; var Scaled = EMA(SS[0]*SS[0],.0242); return SS[0]/sqrt(Scaled); }

For demonstration purposes we apply that filter to the FDX price and volume series. The filtered data points serve as XY coordinates for our curve – the “Ehlers Loop”. The following script reads 3 months stock data from an online source and displays it in a scatter plot where the XY points are connected with splines:

function run() { StartDate = ymd(wdate(NOW)-90); // 90 days before today BarPeriod = 1440; asset("FDX"); var PriceRMS = Roofing(seriesC(),125,20); var VolRMS = Roofing(series(marketVol()),125,20); if(is(LOOKBACK)) return; // don't plot the lookback period plotGraph("Loop",VolRMS,PriceRMS,DOT|GRAPH,BLUE); plotGraph("Loops",VolRMS,PriceRMS,SPLINE|GRAPH,TRANSP|GREY); if(is(EXITRUN)) plotGraph("Last",VolRMS,PriceRMS,SQUARE|GRAPH,RED); }

The plotGraph function is used to display each coordinate with a blue dot. The last day is marked with a red square.

Ehlers intended his loops for discretionary trading, but it can of course also be automated. For instance, the last N coordinates could be used as inputs for Zorro’s neural net, which can then be trained to predict tomorrow’s price. Or even simpler, the slope of a polynomial regression through the last points could trigger a buy order when positive, or a sell order when negative. You can use Zorro’s polyfit function and filter with the regression error. I leave this trading system as an exercise to the reader.

The same method can be used to compare a stock with an index for a pairs rotation strategy. We’re using RTX and SPY:

function run() { BarPeriod = 1440; LookBack = 300; StartDate = 20210801; EndDate = 20211130; assetAdd("RTX","YAHOO:*"); asset("RTX"); var Y = Roofing(seriesC(),125,20); assetAdd("SPY","YAHOO:*"); asset("SPY"); var X = Roofing(seriesC(),125,20); if(is(LOOKBACK)) return; plotGraph("Loop",X,Y,DOT|GRAPH,BLUE); plotGraph("Loops",X,Y,SPLINE|GRAPH,TRANSP|GREY); if(is(EXITRUN)) plotGraph("Last",X,Y,SQUARE|GRAPH,RED); }

The resulting chart for the period from August to November 2021:

I found that Ehlers’ roofing filter is sensitive to the lookback period. It needs some time to ‘swing in’. A too short lookback period, like Zorro’s default 80 bars, produces a visibly different loop. With 300 bars we’re on the safe side.

What can you now do with this loop chart? Ehlers proposed to switch between stock and index depending on the loop rotation and angle quadrant. Our example above, with the SPY index on the Y axis and the stock on the X axis, has a clockwise rotating loop. On clockwise rotations, buy the stock and sell the index from 5 to 10 on the clock face. Sell the stock and buy the index from 11 to 4. Be out of the market between 4 and 5, and have both positions between 10 and 11.

On a counter clockwise rotation, do just the opposite. This was intended for discretionary trading, but can of course be automated with a script. Which I also gracefully leave to the reader.

The Roofing indicator and the Ehlers Loop scripts can be downloaded from the 2022 script repository. The Zorro software can be downloaded from https://zorro-project.com.

]]>

The trading algorithm “Sell in August and buy back in October” could have been realized with the Zorro software in just 5 lines of C:

asset("SPY");

if(month() == 8 && tdm() == 1) // sell 1st trading day of August

exitLong();

else if(month() == 10 && tdm() == 1) // buy back 1st trading day of October

enterLong();

Alas, this seasonality trading system was apparently way too simplistic for Markos Katsanos. His version is a veritable monster with many, many trade entry and exit conditions. The trading script below is a 1:1 translation from his AmiBroker code to C for the Zorro platform. I only moved his **VFI indicator** – a variant of the On Balance Volume, but with a lot more signals – in a separate indicator function because I didn’t like trading logic cluttered with indicator code.

var priceAvg(int Offset) {

return (priceC(Offset)+priceH(Offset)+priceL(Offset))/3;

}

var VFI(var Period,var Coef, var VCoef)

{

vars Inters = series(log(priceAvg(0))-log(priceAvg(1)));

var Vinter = StdDev(Inters,30);

var Cutoff = Coef * Vinter * priceC();

vars Volumes = series(marketVol());

var Vave = SMA(Volumes+1,Period);

var Vmax = Vave * VCoef;

var VC = min(Volumes[0],Vmax);

var MF = priceAvg(0)-priceAvg(1);

vars VCPs = series(ifelse(MF > Cutoff,VC,ifelse(MF < -Cutoff,-VC,0)));

var VFI1 = Sum(VCPs,Period)/Vave;

return EMA(VFI1,3);

}

function run()

{

StartDate = 2006;

EndDate = 2022;

BarPeriod = 1440; // 1 day

LookBack = 150;

assetList("AssetsIB");

MaxLong = 1;

Capital = 100000;

Margin = Equity; // invest all you have

Leverage = 1;

BarZone = EST;

Fill = 3; // enter/exit at next day open

set(PARAMETERS,TESTNOW,PLOTNOW);

asset("VIX");

var VIXdn = (priceC(0)/HH(25,0)-1)*100;

var VIXup = (priceC(0)/LL(25,0)-1)*100;

asset("SPY");

int SellMonth = optimize(8,5,8,1);

var VIXupMax = optimize(60,50,60,10);

var Crit = -optimize(20,15,20,5); //VFI SELL

var K = optimize(1.5,1.3,1.7,.2); // ATR/VIX RATIO

vars ATRs = series(ATR(15));

var ATRDn = (ATRs[0]/MaxVal(ATRs,25)-1)*100;

var ATRUp = (ATRs[0]/MinVal(ATRs,25)-1)*100;

vars VFIs = series(VFI(130,0.2,2.5));

vars SMAVFIs = series(SMA(VFIs,10));

bool VolCondition = (VIXup < VIXupMax || ATRUp < K*VIXupMax ) && VFIs[0] > Crit;

bool Buy = (month() >= 10 || month() < SellMonth) && ref(VolCondition,1) != 0;

bool SellSeasonal = month() == SellMonth ; //SEASONAL

bool SellVolatility = VIXup > 2*VIXupMax ; //VOLATILITY EXIT

bool SellMF = crossUnder(VFIs,Crit) && SMAVFIs[0] < SMAVFIs[1] ;

bool Sell = SellSeasonal || ref(SellVolatility,1) != 0 || ref(SellMF,1) != 0;

if(Sell)

exitLong();

else if(Buy)

enterLong();

}

Phew. And who said that you need at least 30 historical trades per optimized parameter? We’re optimizing the heck out of sell month, threshold, ratio, and other parameters, and won’t care that this strategy trades only once per year. All is in-sample of course, since the low number of trades prevents walk-forward optimization. As can be expected, our reward is a splendid backtest:

We can see that the system managed to avoid the 2008 market drop, since it doubtlessly knew it in advance due to the in-sample optimization. So take the result with a grain – or better, a bag – of salt. For the audacious experimenter, the VFI indicator and the seasonality trading system can be downloaded from the 2022 script repository. You need Zorro S 2.47 or above for supporting volume and for the ref macro. And since the question recently came up: you can find the script repositories under “Links & Download” on the right side of this page.

]]>Suppose you’re developing an algorithmic trading strategy, following all rules of proper system development. But you are not aware that your trading algorithm has no statistical edge. The strategy is worthless, the trading rules equivalent to random trading, the profit expectancy – aside from transaction costs – is zero. The problem: you will rarely get a zero result in a backtest. A random trading strategy will in 50% of cases produce a negative backtest result, in 50% a positive result. But if the result is negative, you’re normally tempted to tweak the code or select assets and time frames until you finally got a profitable backtest. Which will happen relatively soon even when applying random modifications to the system. That’s why there are so many unprofitable strategies around, with nevertheless great backtest performances.

Does this mean that backtests are worthless? Not at all. But it is essential to know whether you can trust the test, or not.

There are several methods for verifying a backtest. None of them is perfect, but all give insights from different viewpoints. We’ll use the Zorro algo trading software, and run our experiments with the following test system that is optimized and backtested with walk-forward analysis:

function run() { set(PARAMETERS,TESTNOW,PLOTNOW,LOGFILE); BarPeriod = 1440; LookBack = 100; StartDate = 2012; NumWFOCycles = 10; assetList("AssetsIB"); asset("SPY"); vars Signals = series(LowPass(seriesC(),optimize(10,2,20,2))); vars MMIFast = series(MMI(seriesC(),optimize(50,40,60,5))); vars MMISlow = series(LowPass(MMIFast,100)); MaxLong = 1; if(falling(MMISlow)) { if(valley(Signals)) enterLong(); else if(peak(Signals)) exitLong(); } }

This is a classic trend following algorithm. It uses a lowpass filter for trading at the peaks and valleys of the smoothed price curve, and a MMI filter (Market Meanness Index) for distinguishing trending from non-trending market periods. It only trades when the market has switched to rend regime, which is essential for profitable trend following systems. It opens only long positions. Lowpass and MMI filter periods are optimized, and the backtest is a walk-forward analysis with 10 cycles.

It is standard for experiments to compare the real stuff with a placebo. For this we’re using a trading system that has obviously no edge, but was tweaked with the evil intention to appear profitable in a walk-forward analysis. This is our placebo system:

void run() { set(PARAMETERS,TESTNOW,PLOTNOW,LOGFILE); BarPeriod = 1440; StartDate = 2012; setf(TrainMode,BRUTE); NumWFOCycles = 9; assetList("AssetsIB"); asset("SPY"); int Pause = optimize(5,1,15,1); LifeTime = optimize(5,1,15,1); // trade after a pause... static int NextEntry; if(Init) NextEntry = 0; if(NextEntry-- <= 0) { NextEntry = LifeTime+Pause; enterLong(); } }

This system opens a position, keeps it a while, then closes it and pauses for a while. The trade and pause durations are walk-forward optimized between 1 day and 3 weeks. LifeTime is a predefined variable that closes the position after the given time. If you don’t believe in lucky trade patterns, you can rightfully assume that this system is equivalent to random trading. Let’s see how it fares in comparison to the trend trading system.

This is the equity curve with the trend trading system from a walk forward analysis from 2012 up to 3/2022:

The plot begins 2015 because the preceding 3 years are used for the training and lookback periods. SPY follows the S&P500 index and rises in the long term, so we could expect anyway some profit with a long-only system. But this system, with profit factor 3 and R2 coefficient 0.65 appears a lot better than random trading. Let’s compare it with the placebo system:

The placebo system produced profit factor 2 and R2 coefficient 0.77. Slightly less than the real system, but in the same performance range. And this result was also from a walk-forward analysis, although with 9 cycles – therefore the later start of the test period. Aside from that, it seems impossible to determine solely from the equity curve and performance data which system is for real, and which is a placebo.

Methods to verify backtest results are named ‘reality check’. They are specific to the asset and algorithm; in a multi-asset, multi-algo portfolio, you need to enable only the component you want to test. Let’s first see how the WFO split affects the backtest. In this way we can find out whether our backtest result was just due to lucky trading in a particular WFO cycle. We’re going to plot a **WFO profile** that displays the effect of the number of walk-forward cycles on the result. For this we outcomment the **NumWFOCycles = …** line in the code, and run it in training mode with the **WFOProfile.c** script:

#define run strategy #include "trend.c" // <= your script #undef run #define CYCLES 20 // max WFO cycles function run() { set(TESTNOW); NumTotalCycles = CYCLES-1; NumWFOCycles = TotalCycle+1; strategy(); } function evaluate() { var Perf = ifelse(LossTotal > 0,WinTotal/LossTotal,10); if(Perf > 1) plotBar("WFO+",NumWFOCycles,NumWFOCycles,Perf,BARS,BLACK); else plotBar("WFO-",NumWFOCycles,NumWFOCycles,Perf,BARS,RED); }

We’re redefining the **run** function to a different name. This allows us to just include the tested script and train it with WFO cycles from 2 up to the number defined by CYCLES. A backtest is executed after training. If an **evaluate** function is present, Zorro runs it automatically after any backtest. It plots a histogram bar of the profit factor (y axis) from each number of WFO cycles. First, the WFO profile of the trend trading system:

We can see that the performance rises with the number of cycles. This is typical for a system that adapts to the market. All results are positive with a profit factor > 1. Our arbitrary choice of 10 cycles produced a less than average result. So we can at least be sure that this backtest result was not caused by a particularly lucky number of WFO cycles.

The WFO profile of the placebo system:

This time the number of WFO cycles had a strong random effect on the performance. And it is now obvious why I used 9 WFO cycles for that system. For the same reason I used brute force optimization, since it increases WFO variance and thus the chance to get lucky WFO cycle numbers. That’s the opposite of what we normally do when developing algorithmic trading strategies.

WFO profiles give insight into WFO cycle dependency, but not into randomness or overfitting by other means. For this, more in-depth tests are required. Zorro supports two methods, the Montecarlo Reality Check (MRC) with randomized price curves, and White’s Reality Check (WRC) with detrended and bootstrapped equity curves of strategy variants. Both methods have their advantages and disadvantages. But since strategy variants from optimizing can only be created without walk-forward analysis, we’re using the MRC here.

First we test both systems with random price curves. Randomizing removes short-term price correlations and market inefficiencies, but keeps the long-term trend. Then we compare our original backtest result with the randomized results. This yields a **p-value**, a metric of the probability that our test result was caused by randomness. The lower the p-Value, the more confidence we can have in the backtest result. In statistics we normally consider a result significant when its p-Value is below 5%.

The basic algorithm of the Montecarlo Reality Check (MRC):

- Train your system and run a backtest. Store the profit factor (or any other performance metric that you want to compare).
- Randomize the price curve by randomly swapping price changes (shuffle without replacement).
- Train your system again with the randomized data and run a backtest. Store the performance metric.
- Repeat steps 2 and 3 1000 times.
- Determine the number N of randomized tests that have a better result than the original test. The p-Value is N/1000.

If our backtest result was affected by an overall upwards trending price curve, which is certainly the case for this SPY system, the randomized tests will be likewise affected. The MRC code:

#define run strategy #include "trend.c" // <= your script #undef run #define CYCLES 1000 function run() { set(PRELOAD,TESTNOW); NumTotalCycles = CYCLES; if(TotalCycle == 1) // first cycle = original seed(12345); // always same random sequence else Detrend = SHUFFLE; strategy(); set(LOGFILE|OFF); // don't export files } function evaluate() { static var OriginalProfit, Probability; var PF = ifelse(LossTotal > 0,WinTotal/LossTotal,10); if(TotalCycle == 1) { OriginalProfit = PF; Probability = 0; } else { if(PF < 2*OriginalProfit) // clip image at double range plotHistogram("Random",PF,OriginalProfit/50,1,RED); if(PF > OriginalProfit) Probability += 100./NumTotalCycles; } if(TotalCycle == NumTotalCycles) { // last cycle plotHistogram("Original", OriginalProfit,OriginalProfit/50,sqrt(NumTotalCycles),BLACK); printf("\n-------------------------------------------"); printf("\nP-Value %.1f%%",Probability); printf("\nResult is "); if(Probability <= 1) printf("highly significant") ; else if(Probability <= 5) printf("significant"); else if(Probability <= 15) printf("maybe significant"); else printf("statistically insignificant"); printf("\n-------------------------------------------"); } }

This code sets up the Zorro platform to train and test the system 1000 times. The **seed** setting ensures that you get the same result on any MRC run. From the second cycle on, the historical data is shuffled without replacement. For calculating the p-value and plotting a histogram of the MRC, we use the **evaluate** function again. It calculates the p-value by counting the backtests resulting in higher profit factors than the original system. Depending on the system, training and testing the strategy a thousand times will take several minutes with Zorro. The resulting MRC histogram of the trend following system:

The height of a red bar represents the number of shuffled backtests that ended at the profit factor shown on the x axis. The black bar on the right (height is irrelevant, only the x axis position matters) is the profit factor with the original price curve. We can see that most shuffled tests came out positive, due to the long-term upwards trend of the SPY price. But our test system came out even more positive. The p-Value is below 1%, meaning a high significance of our backtest. This gives us some confidence that the simple trend follower can achieve a similar result in real trading.

This cannot be said from the MRC histogram of the placebo system:

The backtest profit factors now extend over a wider range, and many were more profitable than the original system. The backtest with the real price curve is indistinguishable from the randomized tests, with a p-value in the 40% area. The original backtest result of the placebo system, even though achieved with walk-forward analysis, is therefore meaningless.

It should be mentioned that the MRC cannot detect all invalid backtests. A system that was explicitly fitted to a particular price curve, for instance by knowing in advance its peaks and valleys, would get a low p-value by the MRC. No reality check could distinguish such a system from a system with a real edge. Therefore, neither MRC nor WRC can give absolute guarantee that a system works when it passes the check. But when it does not pass, you’re advised to better not trade it with real money.

I have uploaded the strategies to the 2022 script repository. The MRC and WFOProfile scripts are included in Zorro version 2.47.4 and above. You will need Zorro S for the brute force optimization of the placebo system.

]]>

The algorithm from Vitali’s article, slightly reformulated for clarity:

**RSEMA[0] = Alpha * Price + (1-Alpha) * RSEMA[1]**

This is a classical EMA, but with a volatility dependent alpha factor:

**Alpha = 2/(TimePeriod+1) * (1 + 10*abs(EMAUp-EMADn)/(EMAUp+EMADn))**

**EMAUp** and **EMADn** are the VIX EMA of days with positive returns and the VIX EMA of days with negative returns. The separation into positive and negative days resembles the RSI. And we have a double EMA in this formula, which will hopefully render the result twice as good. The factor **10** is inserted for getting an even better result.

VIX historical data is not available via API from free online sources as far as I know, but it can be downloaded from most brokers with the Zorro Download script. The resulting RSEMA code in C for Zorro:

var RSEMA(int Periods, int PdsEMA, var Mltp) { asset("VIX"); var VolatC = priceC(); asset(AssetPrev); // re-select asset var VolatUp = series(ifelse(RET(1)>0,VolatC,0)), VolatDn = series(ifelse(RET(1)<0,VolatC,0)); var EMAUp = EMA(VolatUp,PdsEMA), EMADn = EMA(VolatDn,PdsEMA); var RS = abs(EMAUp-EMADn)/(EMAUp+EMADn+0.00001)*Mltp; var Rate = 2./(Periods+1)*(RS+1); return EMA(priceC(),Rate); }

**AssetPrev** is the asset that was selected when the **RSEMA** function is called. Adding **0.00001** to the divisor prevents a crash by division by zero. Note that the often used EMA is available in two versions, to be called with a data series or with a single variable. I’m using the variable version here.

The RSEMA(10,10,10) applied on a SPX500 chart replicates the chart in the TASC article:

We can see that the RSEMA (blue) has noticeably less lag than the EMA (red), which is not really surprising because the higher alpha rate is equivalent to a smaller EMA time period. The author believed that EMA crossovers of this indicator can be used as entry points for long trades. But with the caveat that they *“produce lots of whipsaws […] and should be used in conjunction with price analysis”*.

He’s certainly right. A long-only SPY trading system with RSEMA-EMA crossovers at default parameter values produced a positive end result, as seen in the chart below. But with large drawdowns and, sadly, far less profitable than buy-and-hold. So, be sure to use it only in conjunction with price analysis. Whatever that means. And better not on a real money account.

For your own experiments, the RSEMA indicator and the SPY trading system can be downloaded from the 2022 script repository.

]]>

First, the Fisher Transform and its inverse in code:

var Fisher(vars Data) { var V = clamp(Data[0],-0.999,0.999); return 0.5*log((1.+V)/(1.-V)); } var FisherInv(vars Data) { var V = exp(2.*Data[0]); return (V-1.)/(V+1.); }

The Inverse Fisher Transform (IFT) is used in the Elegant Oscillator to convert a normalized derivative of a price series to the +/-1 range. The normalization is done by dividing by the square root of the sum of squares, a sort of N-dimensional distance. Using the IFT is supposedly a better method than simply clipping the data. The result is smoothed by Ehlers’ ‘SuperSmoother’, a low-lag lowpass filter. The Elegant Oscillator code in C for Zorro:

var EO(vars Data,int Length) { vars Derivs = series(Data[0]-Data[2]); var RMS = sqrt(SumSq(Derivs,Length)/Length); var NDeriv = Derivs[0]/RMS; vars IFishs = series(FisherInv(&NDeriv)); return Smooth(IFishs,20); }

**SumSq()** is a helper function for summing up the squares of a data series, and **Smooth()** is Ehlers’ ‘SuperSmoother’ that’s already in the Zorro library. Since the **FisherInv** function needs only a data series of length 1, I could simply pass a pointer to the **NDeriv** variable instead of a series of it. For comparison, here’s the simple code for hard clipping the data:

var HardClip(vars Data) { var Deriv = Data[0]-Data[2]; vars Clips = series(clamp(Deriv,-1,1)); return FIR6(Clips); }

The clipped data is this time smoothed with a finite response filter (FIR6). Here’s a SPY chart with the Elegant Oscillator (upper red line), the Inverse Fisher Transform smoothed with a finite response filter (lower red line), and the hard clipped variant (blue line).

Of the 3, the super smoothed Elegant Oscillator makes the best impression and appears to generate the best signals. According to Ehlers, its peaks and valleys that exceed a threshold can be used for mean reversion trading. Let’s put that to the test. The code:

void run() { StartDate = 20200301; EndDate = 20210501; BarPeriod = 1440; asset("SPY"); vars Signals = series(EO(seriesC(),50)); var Threshold = 0.5; if(Signals[0] > Threshold && peak(Signals)) enterShort(); else if(Signals[0] < -Threshold && valley(Signals)) enterLong(); }

The resulting chart:

Indeed, 5 of 7 trades in that time period were winning, producing an overall positive result with a profit factor above 6. Of course, 7 trades won’t tell much, so I encorage anyone to experiment with different instruments and different time periods for determining the real value of the EO oscillator for mean reversion trading.

The oscillator and trading strategy can be downloaded from the 2021 script repository.

(1) John Ehlers, Cybernetic Analysis of Stocks and Futures, Wiley 2004

]]>

The RSI is basically the normalized difference of price up/down movements. And its here presented Hann variant filters the price differences with a Hann window that was described in a previous article on this blog. The RSIH code in C for Zorro is not very complex:

var RSIH(vars Data, int Length) { var CU = 0, CD = 0; int i; for(i=1; i<Length; i++) { var D = Data[i-1]-Data[i]; var Hann = 1-cos(2*PI*i/(Length+1)); if(D > 0) CU += Hann*D; else if(D < 0) CD -= Hann*D; } if(CU+CD != 0) return (CU-CD) / (CU+CD); else return 0; }

According to Ehlers, the optimal **Length** parameter is the dominant cycle of the price series. Here’s the RSIH (red) applied to a SPY chart, in comparison with a standard RSI (blue):

Indeed the RSIH curve looks a lot smoother and better to trade than the original RSI curve. But the usual question for any new indicator is “What can I do with it?” and as usual the indicator inventor remains silent about that. I can only say what you cannot do: replacing the RSI in a working strategy with RSIH. Even when adapting the scale and thresholds, in all RSI based strategies that I tested the RSIH produced a worse result. Maybe John Ehlers gives a practical example in a future article.

The RSIH indicator and test script can be downloaded from the 2021 script repository.

]]>