Illuminating the Dark Forest!

Following our previous post, Coding a DeFi Arbitrage Bot, many readers report an issue preventing the arbitrages executed by our bot from completing successfully. As it turns out, this issue is caused by frontrunners, executing the trades before our trading bot itself, so that the order on the 0x exchange isn’t available anymore by the time our bot is about to finish the arbitrage successfully. What is happening here is better described by what has become the essential textbook on arbitrage bots, Flash Boys 2.0:

Like high-frequency traders on Wall Street, these bots exploit inefficiencies in DEXes, paying high transaction fees and optimizing network latency to frontrun, i.e., anticipate and exploit, ordinary users’ DEX trades.

According to MEV-explore, a website showing data about flashbots, the total MEV that has been extracted on Ethereum in the last year and a half is $750M, and that’s a conservative estimate.

Technical readers may be asking themselves how these bots manage to find profitable trades, the answer is, they don’t; in fact there are generalized frontrunner bots looking for any transaction that they could profitably frontrun by copying it and replacing addresses with their own. I.e. frontrunners can determine whether a transaction is profitable while it is still in the mempool, without having to know what the transaction actually does to the Ethereum state.

There are many articles and videos on frontrunning, but getting frontrun first-hand is a different feeling, so we have written this article with the hope to provide readers with an explanation of the root problem of frontrunning known as MEV, and outlining what mitigation tools are currently out there for illuminating the dark forest and more. We conclude with some thoughts on the evolution of frontrunning in regards to the upcoming Eth 2.0 update and L2 scaling solutions.

MEV

The concept of MEV was first introduced by Phil Daian in “Flash Boys 2.0”, known as miner-extractable value (MEV):

value that is extractable by miners directly from smart contracts as cryptocurrency profits

The idea of MEV was more recently popularized by Dan Robinson and samczsun in “Ethereum is a Dark Forest” and “Escaping the Dark Forest”, in which MEV…