All news
May 13, 2022

The Ethereum Merge

Ethereum’s market cap is currently ~$250B and with ETH being a core component of DeFi, there are strong incentives to complete The Merge bug-free and as smoothly as possible. Since the shadow fork 4 test of The Merge from the 12th of May 2022 went very well, it is now likely The Merge is happening this summer.

This year, Ethereum is expected to complete The Merge, which will:

  • reduce Ethereum’s energy consumption by ~99.95%,
  • cause ETH to become a less inflationary asset, potentially even deflationary, and
  • allow validators to earn a higher yield as they replace miners.

What does not happen as a result of The Merge, but is still expected to happen eventually, is:

  • an increase in scalability, and
  • a reduction in gas fees.

In this article we describe:

  • what The Merge is,
  • how it is tested,
  • the associated risks, and 
  • what comes next.

Although related, there is a large and near-religious Proof-of-Work (PoW) vs Proof-of-Stake (PoS) debate, which is beyond the scope of this article.

What is The Merge?

Currently, there are two separate Ethereum chains — the Mainnet, which is the PoW chain we use today, and the Beacon Chain that has used PoS since its genesis. The Merge is an upgrade in which the PoW Mainnet merges into the PoS system that’s coordinated by the Beacon Chain. The consensus mechanism of the Ethereum Mainnet that currently relies on miners will suddenly fully rely on validators.

Image: Danny Ryan

As The Merge happens, Beacon nodes will watch the current PoW chain and wait for it to hit a predefined terminal total difficulty (TTD). Once a block with total difficulty larger than TTD is produced, it’s considered the final PoW block. Subsequent blocks will be produced and attested to by the validators on the Beacon Chain and The Merge is finished.

The Beacon Chain was launched in December 2020 and its planned purpose was always to be the successor consensus layer chain. It is possible for Ether to be transferred from the Mainnet to the Beacon Chain and staked, although it cannot be unstaked until soon after The Merge is complete. There’s already over 11.5M of Ether staked on the Beacon Chain.

Image: beaconcha.in

Currently, all transactions performed on Ethereum are handled by the PoW execution layer, while the PoS Beacon Chain has been coming to consensus on empty (from end users’ standpoint) blocks. One way to think about the Beacon Chain is that it is currently practicing some of its post-merge tasks, but the real activity continues to happen on the PoW Mainnet.

The staking projections after The Merge, when the validators start collecting a portion of transaction fees, are the following:

Source: Justin Drake

 

Testing The Merge

The Kintsugi merge testnet, launched December 2021, has been a valuable testing ground for The Merge. Through various test suites, multi-client devnets, and application deployments, Ethereum has arrived at a set of stable and robust protocol specifications. There are four major execution layer clients (Geth, Erigon, Nethermind, Besu) and five major consensus layer clients (Prysm, Lighthouse, Teku, Nimbus, Grandine). Since all the execution clients have to talk to all the consensus clients, it is one of the main reasons why The Merge takes so long to happen despite the specifications for it being already set.

After the clients have implemented these latest specifications, a new Merge testnet, Kiln, was launched in March 2022. Kiln is expected to be the last Merge testnet created before existing public testnets are upgraded. The aim of the Kiln merge testnet is to allow the community to practice running their nodes, deploying contracts, and testing infrastructure. However, since it is a fresh testnet with little activity, there is a need for a way to stress test the assumptions around syncing and state growth.

Enter shadow forking

The shadow fork is a new testnet created by forking a live network with a small number of nodes to simulate stress testing in the existing network environment.

The diagram below shows what the network looks like after a shadow fork.

Image: parithos

  • The top row of Goerli blocks shows a node on the canonical chain, which are not aware of the shadow fork.
  • The middle row of Goerli blocks shows a node on the shadow-forked chain, which has a modified configuration telling it to fork once the TTD is hit.
  • The bottom row shows a Beacon Chain which was launched for the purposes of the shadow fork only: it will provide consensus to the chain when TTD is hit.
  • After TTD is hit, the nodes on the canonical chain continue producing blocks normally: nothing has happened “for them”.
  • After TTD is hit, nodes with the modified configuration fork off and run through The Merge. The first post-merge block is produced by the next validator in the Beacon Chain.

Shadow forks allow a more realistic environment to perform tests compared to launching new testnets, because existing testnets already have transactions happening organically on them, and a large state size and block history, which put nodes under more stress compared to new testnets. This therefore facilitates real-world performance metrics on nodes, without potentially affecting the canonical network’s operations. Once the shadow forks go smoothly across all implementations, the devs can confidently run existing testnets through The Merge. And once the testnets are upgraded and stable, the devs can plan for The Merge to happen on the Mainnet.

What does the development process look like?

Everything about the Ethereum development, including The Merge, is very transparent. The Merge meetings are open to all client developers and community members. They are all streamed and recorded. The latest Ethereum core devs meeting took place on the 29th April 2022 in which 43 core Ethereum devs were giving updates on the latest Merge developments.

Spoiler alert: the Kiln Testnet Merge happens at 1:20:20. Image: Youtube (Ethereum Foundation)

Another layer of transparency and openness to the larger public is the Eth R&D Discord with over 9000 members and specific and open channels related to The Merge. Most of the Ethereum research discussions take place at this forum and EIPs are discussed here.

There’s also a public github page called The Merge Mainnet Readiness Checklist where anyone can monitor the progress and see what’s left to be implemented and tested.

How is The Merge date determined?

According to Ethereum coordinator Tim Beiko the outcome of the shadow forks is key for determining when the final merge takes place. The Merge date will only be determined when the client teams are confident that the software implementation has been thoroughly tested and free of bugs.

All client teams are working on a merged implementation and testing of shadow forks. Among them, the shadow fork running against the existing testnet and Ethereum mainnet has revealed implementation issues on the client side, which are currently being fixed by the team, and the shadow fork is periodically rerun to test the fixes.

The shadow fork 2 test of The Merge from the 23rd April was a success joined by all clients and with limited identified issues. The shadow fork 3 test from the 5th of May 2022 and the shadow fork 4 test from the 12th of May 2022 both went very well, this is the main reason to think The Merge is coming this summer.

The path to fully transitioning Ethereum to PoS is clearly in sight. What is needed is now defined as:

  • A few mainnet shadow forks without issues
  • Clients passing the various merge test suites
  • Smooth deployments across existing public testnets

Image source: Tim Beiko

Once these three things happen and they are stable for a few weeks, the Ethereum core devs will be ready for The Merge.

 

What are the risks?

Risk can be viewed as a product of the likelihood of some negative event happening and its impact. Below are some of the known risks.

Consensus bug

The least expected, but the risk having the largest impact would be a bug in the Ethereum consensus layer. Both Polygon and Cosmos had consensus bugs before. A critical Ethereum consensus bug that would lead to an invalid state transition (for example giving yourself ETH out of thin air) that gets finalized, would be catastrophic and would cause many validators to get slashed in order to revert back to the original state. A consensus bug in a client with >2/3 share could cause the Beacon Chain to finalize incorrect information and memorialize it forever in Ethereum’s history. There are only two outcomes of this: one is that the remaining clients accept the bug as expected behavior from then on, or the corrupted validators start a slow and very expensive exit from the chain.

Network attacks

Other possible risks are resource exhaustion attacks (e.g. filling victim’s RAM at a fast rate) on a specific client. A temporary resolution of such an attack would be to simply switch the client. This is yet another reason why the Ethereum community cares deeply about client diversity. Validators on the Beacon Chain can choose from multiple different clients, but the majority (>57%) currently run Prysm. This is fine while Prysm runs faultlessly but in the event of a bug or attack, all the validators on the network would become victims of Prysm’s success. The existence of multiple client implementations is a very strong defense against many attacks, but only if the validators distribute somewhat evenly across them. This means that the best possible course of action to strengthen Ethereum as it moves from PoW to PoS is to encourage the major exchanges and staking pools to migrate some of their validators to minority clients and even out the client diversity.

Ethereum Ice Age

The difficulty bomb was created by Ethereum developers in 2016 with The Merge in mind. This mechanism increases the PoW mining difficulty exponentially over time and eventually leads to what is referred to as the “Ice Age” — that is, the chain becomes so difficult to mine that it grinds to a halt and stops producing blocks (freezes). The purpose of the difficulty bomb is to encourage miners to transition into the much less energy-intensive PoS consensus when ETH mining becomes less profitable as block times increase.

The difficulty bomb is currently expected to start being noticeable around May 2022 and to make blocks much slower around August 2022 (read >20 seconds from the current 12 seconds). Historically, there were already times when the difficulty bomb’s effect started to be noticeable, even more than doubling block time. The difficulty bomb has so far always been diffused by the core dev team by delaying it through a network upgrade.

The difficulty bomb can always be delayed, but at the latest Ethereum core devs meeting on 29th April, the devs have decided not to delay the difficulty bomb at least for now. The thinking is that there’s hope for The Merge to be activated in the summer.

“The concern with taking this see as we go approach is that it can get pretty dicey pretty quickly for devs once the bomb is activated. It would be unfortunate to make a mistake there as far as getting to a situation where you have to force yourself to delay the bomb”, said Thomas Rush on the call.

Even though a delay of the difficulty bomb is as simple as a code change, it would still require at least a few weeks to coordinate among Ethereum execution layer clients due to backwards incompatible nature of the change.

Miner revoke

There is not much the miners can do to stop The Merge, but what can be expected is that someone will fork the PoW Mainnet Ethereum at the last block before The Merge. This would allow the PoW chain to survive and continue producing blocks, effectively creating something like another Ethereum Classic.

One of the other reasons why Ethereum devs decided not to go ahead with the latest difficulty bomb delay was that pushing it too far back could make it easier for a PoW version of Ethereum to live on after The Merge upgrade is activated. The thinking behind this is that if The Merge and the difficulty bomb are activated around the same time, then any fork of Ethereum that keeps the PoW consensus algorithm would have to quickly coordinate to diffuse the bomb in order to keep the chain progressing.

Risks conclusion

Ethereum has multiple clients for both the execution layer and consensus layer. This builds greater resilience into Ethereum, but it also brings more risk and the upgrades take much longer. Each client team must fix its problems, so progress is only as fast as the slowest team. However, the key risk is pushing this upgrade before it is ready. Instead of asking “when” people should be asking “what’s missing”.

What’s after The Merge?

The Merge is done in a very minimalistic and simple way. It basically takes the existing PoW chain, strips out the PoW, and makes it continue as a chain inside the PoS Beacon chain. This is a very ‘no frills’ Merge, with high technical debt where things like the execution data voting mechanism (this is how Beacon chain is currently aware of the PoW Mainnet) still exists, unstaking is still impossible, and there are a lot of format issues.

Post-merge cleanup fork

In order to fix these and other issues, a post-merge cleanup fork will take place soon after The Merge. Since users have been unable to withdraw the ETH staked on the Beacon Chain before The Merge, this post-merge cleanup fork will make it possible to unstake ETH again.

Sharding

The Merge is only one of five tracks in the Ethereum development roadmap. These upgrades are being researched and implemented in parallel with different priorities.

Image: Vitalik

The next most anticipated update is sharding, which will help Ethereum scale without compromising decentralization and is expected to ship in 2023. Originally, sharding was on the roadmap before The Merge but priorities have shifted towards achieving sustainability first. Additionally, PoS allows for much more secure sharding. Shard chains allow PoS Ethereum to create multiple blocks at the same time, increasing transaction throughput, while sharding the network in a PoW system would also lower the power needed to compromise a portion of the network.

Post-sharding future

Other notable future upgrades should be:

  • General improvements (e.g. single slot finality, proposer/builder separation, history expiry)
  • Security improvements (e.g. single secret leader election, data availability sampling)
  • Execution layer improvements (e.g. Verkle trees, address space extension, EVM upgrades)
  • STARK everything (for better scalability, e.g. zk-EVM; and quantum security, e.g. STARKed Merkle tree)

Conclusion

The Ethereum Merge primarily addresses the issue of sustainability and is an important step towards the remaining items in the roadmap. It has been a long time in the making but there are strong reasons to think we are at the final stage of Ethereum’s PoW era.

 

– written by Filip Siroky from the RBF investments team