Comments on: Hacking a HFT system

By: William

William — Tue, 23 Feb 2021 00:54:01 +0000

Well, indeed MT4/MT5 (with a little preference for MT4 for the moment). Actually, their prices are all linked. If we base our analysis on a macro view, we will see that the candlesticks are quite the same. When we zoom to the millisecond scale, it is all something else. I have even spotted discrepancies between different servers of a single broker.
Regarding the fact of FPGA based system, it was to avoid the time of transaction of Metatrader. They say MT5 can open more than 100 orders in 1 sec or one order every 9~10ms. It is quite nice but unfortunately, it will be too long for a solid HFT arbitrage strategy most of all if you sum it with latency of back and forth signal. This field of trading requires to save as many ms as possible for which giving this task to the FPGA/SoC would be very competitive. This applies for both latency and triangular arbitrage even if I can barely see a situation in which triangular arbitrage could work considering the high spreads (even 1pip on EURUSD and on USDJPY is quite high if we want to take advantage of currency quotation discrepancies)! So with the fact that we have a substantial latency and the spread, HFT is not the best choice I have at my disposal. For the moment and after not having solved my price acquisition system, I am pursuing another avenue.
By the way, Jcl, please tell me if I am polluting your articles with my comments. So far, I try to stick to the topic.

By: jcl

jcl — Thu, 18 Feb 2021 15:09:02 +0000

Since you mention MT4 and MT5, I suppose you want to arbitrage Forex. MT4/MT5 brokers are usually market makers. They don’t have liquidity providers, but produce the prices on their servers. So their prices are all different. Normally they make sure by artificial spread settings and transaction delays that there is no arbitrage opportunity – but there may be exceptions. If you analyzed their price algorithms and knew their data sources, you can maybe take advantage of them even without an FPGA based system.

By: William

William — Thu, 18 Feb 2021 02:12:58 +0000

Hi thank you for your detailed answer. Well, regarding broker arbitrage policy, I already heard stuff like that but didn’t really give a glance at the rules. I hope they are not dishonest enough to kick you out with neither notice, nor mention in the agreement…
As far as I am concerned, there may be 2 main arbitrage techniques both require a very low latency and a very fast system (that is the where the FPGA could intervene even if there is more to take with a latency than a FPGA over a PC system) :
Latency arbitrage which makes you take opportunities on a slow broker with signals coming from a faster broker. The ideal condition would be that all the brokers’ prices were exactly the same but with a little time discrepancy between them. As you will have seen in my pictures on the link, this is not that simple… Sometimes a broker shows a price that evolves in a clear direction and the other one doesn’t follow and sometimes they have the same behaviour. So it becomes complicated to infer from this an algorithm that would offer you a 99% winrate. In this case, maybe the AI may give a good help.
The other form of arbitrage that could be used on retail trading systems is triangular arbitrage for which for a very short period of time you will have two of three correlated currency pairs (let’s say EURUSD/GBPUSD/EURGBP) that offers an opportunity .
But when trade on PC, with MT4 , you can’t take arbitrage opportunities fast enough, whereas they say MT5 is suited for HFT and arbitrage techniques. The FPGA (SoC) would be a very good assistant but as you said, we would need the API.

Best regards

By: jcl

jcl — Sun, 14 Feb 2021 12:06:28 +0000

Thank you. I think forex brokers prevent broker arbitrage mainly by analyzing suspicious trading behavior, and by their business terms. They might kick you out when they suspect you of broker arbitrage. Usually they notice it after a couple days and the arbitrage opportunity will then disappear.

For connecting a FPGA based system to a MT4 or MT5 server you need to analyze their traffic with the terminal. I don’t know if they have a documented API for connecting to their server, but I doubt it.

By: William

William — Sat, 13 Feb 2021 19:51:35 +0000

Hi jcl,

So bad that I have found your article this late. As long as I was looking for more answers, I googled “FPGA metatrader4”.

Anyways, this article/blog/study is very interesting.

“Broker arbitrage is theoretically possible with Forex, but difficult because Forex brokers are doing all they can for preventing it.”. What do you mean by that? How would they avoid to follow real price action?

So actually, I am interested in this arbitrage thing and think that the only real way to address arbitrage is FPGAs and SoCs and obviously coupled to a very low latency. I have started a little work in a first place to define whether a strategy can be applied and which should be the brokers to choose to apply it. I have published it on this forum (https://www.forexfactory.com/thread/post/13314153) but I am facing the nasty obstacle of the 15ms resolution, I will probably work out later.

JCL, do you have any idea whether an FPGA/SoC can directly connect to a MetaTrader broker server without using MT4 or MT5?

Thank you for work anyway.

By: jcl

jcl — Tue, 17 Nov 2020 14:20:10 +0000

I don’t know if MT4/5 poll data from their servers, but they use polling to pass the data to their EAs or to other platforms.

By: Tom

Tom — Tue, 17 Nov 2020 11:15:23 +0000

You are mentioning the Zorro polling cycle as well as the MT4 polling cycle. Does this mean that the MT4 client actually polls the data from the broker’s server or is it being pushed by the server?

Do you happen to know how the MT5 terminal handles this topic? Is the tickdata being pushed to the MT5 client and subsequently to the Zorro client or is there polling happening as well?

Thanks a lot for the sharing such informative details!

By: jcl

jcl — Mon, 16 Nov 2020 14:40:45 +0000

MT4 ticks have no exchange timestamps. That means the time stamps that you see are the time when Zorro received them. Because the MT4 plugin is not streaming but polled, multiple ticks can come in at the same time. This happens when either Zorro’s polling cycle or the MT4 polling cycle took longer than the time between the ticks. For Zorro you can set up the cycle with the TickTime variable, for MT4 there’s a setting in the EA code. I believe it’s 20 ms by default. It cannot be set much smaller because MT4 will then start choking.

By: Tom

Tom — Sun, 15 Nov 2020 15:47:28 +0000

I was referring to live data not historical data. Zorro prints multiple/duplicate ticks with the same timestamp when using below’s code to receive tickdata from two MT4 terminals (this is basically the code from your Simple Broker Arb example).

function tick()
{
asset(ASSET_A);
var AskA = priceClose(), BidA = priceClose() – Spread;
asset(ASSET_B);
var AskB = priceClose(), BidB = priceClose() – Spread;

printf(“\n[%s.%.0f],%.2f,%.2f,%.2f,%.2f,%.i”,strdate(HMS,0),1000.*modf(second(),0),BidA,AskA,BidB,AskB);
}

I know MT4 is too slow for actual HFT trading, but still it would be good to know that tickdata received from MT4 via Zorro is reliable and that there is not something wrong. Though, most likely I am the one doing something wrong here ;).

I hope this is not leading too off topic. I would very much appreciate hearing back from you!

Thanks
Tom

By: jcl

jcl — Wed, 11 Nov 2020 13:54:14 +0000

MT4 terminals are not suited for HFT arbitrage, but I think you know that. The reason of the multiple ticks in a backtest is likely that these ticks are also repeated in the historical data – you can check the data with the History script. If there are multiple ticks, there’s a dataparse flag that can be used to filter them out.