Ethereum stumbles with settlements as validators go missing

The Ethereum network has experienced two glitches in the past week with settling transactions on its Beacon Chain. The two delays occurred late last week after validators failed to confirm valid blocks, causing some exchange services to pause deposits. Developers are still looking for the cause.

Those working to fix the problem downplayed the seriousness of the issue, noting there were no cases of transactions being lost or remaining unsettled for long. However, they did admit it was something that required fixing and that developers faced technical challenges in ensuring the network didn’t suffer longer-term and more severe settlement delays in the future.

While glitches have occurred on all blockchains of all types, Ethereum is a constantly-shifting ocean of complexity with new rules and definitions. Even some of its closest followers confess they struggle to keep up with the latest terminology. While the technical inner circle might not consider an issue serious, they may be potentially so, leading to problems explaining Ethereum’s security to potential users.

This ever-changing technical environment leads to Ethereum being called “experimental” and even a “hobby platform” that isn’t ready for large-scale applications where billions of dollars in money and asset value could be at risk.

Last week’s issues affected Ethereum’s on-chain “finalization,” an explicit settlement concept and “checkpointing” technique that has existed in Ethereum since its recent switch to a proof-of-stake (PoS) processing algorithm. Designed to prevent blockchain “re-orgs” (re-organizations), it determines that a transaction is considered officially “finalized” if at least 2/3 of all validators vote (or “attest”) for consensus after a defined period has passed. Ethereum has unique names for these processes, and finalization occurs following consensus over two “epochs” (one epoch is 32 12-second “slots,” meaning a transaction should be officially final after about 12.8 minutes).

In other words, to most ordinary users, transactions would have appeared to proceed as normal, but the blockchain itself delayed in considering them permanent and unalterable. This caused some exchange services to pause deposits. While the first delay lasted just a few minutes, the second lasted over an hour, reportedly due to 60% of validator nodes appearing absent.

Commenting on the issue, Ethereum client developer at ConsenSys Ben Edgington said Ethereum was still functionally working, maybe with degraded performance for a short time, but not one that stopped the network itself. The root cause, he said in a video interview, was “still under analysis,” and no developer intervention was required to fix the delay. However, once the cause is determined, they will likely issue patches to prevent future incidents.

He described it as a panic situation only on the technical back-end, as validators worked to rebuild older network “states” to check past attestations while new slots continued to pour in every 12 seconds. Old attestations are discarded so they don’t clog up space, but the challenge was working out which ones were important for finalizations and which weren’t. Different validators have their own ways of performing these processes, so it was more of a crisis for some than others.

In theory, a chain re-org is possible within the two-epoch period or before finalization if enough validators vote/attest with their stakes to do so.

In “probabilistic” blockchain networks such as Bitcoin (and Ethereum when it still ran on proof-of-work), there are recommendations on how many additional transaction blocks must pass (if any) before a previous one is considered “final,” but the protocol itself does not define this.

Ethereum completely changed its protocol rules and economic incentives when it switched to proof-of-stake (PoS) transaction validation late in 2022. Though this nominally more “environmentally friendly” move has generated favorable media attention, it also injected a lot of uncertainty over whether Ethereum could remain “safe” or trustworthy in the longer term.

Application developers, especially enterprise-tier ones, don’t want to administer their own systems under rule uncertainty or ongoing technical experiments; they want stable systems that are proven to work. This is why BSV retains Bitcoin’s original, “set in stone” protocol rules and proof-of-work (PoW) based confirmations and economic incentives. On a global scale, this is by far the more trustworthy option for blockchain.

Watch The Bitcoin MasterClasses: Party-to-party is revolutionary, and that’s because Bitcoin is traceable