Ethereum Faces Temporary Validator Drop Following Prysm Client Bug

The Ethereum (ETH) network recently had a sudden drop in validator activity because of a bug in the Prysm consensus client. This happened soon after the network launched its latest upgrade, Fusaka.

The network has now mostly recovered, but the incident shows why it is important to use different clients and closely watch the system to keep Ethereum stable.

Prysm Bug Temporarily Hits Ethereum Validators

The problem began when Prysm developers announced they found a bug in version 7.0.0 of their client. The bug made the client unnecessarily process old data, which caused some nodes to stop working properly. As a result, fewer nodes were able to vote or stay in sync with the network for a short time. 

Developers suggested a temporary fix by asking users to start the client with a special setting. According to Beaconcha.in data, at epoch 411,448, Ethereum had only 75% of nodes in sync and 74.7% participating in votes. 

Voting dropped by about 25%, getting close to the point where the network could lose the two-thirds majority needed to confirm blocks. This drop could cause problems for how Ethereum reaches an agreement.

Without this large majority, new blocks can still be added, but the blockchain is no longer fully finalized. This could create risks for users and applications that depend on Ethereum running smoothly.

Ethereum Network Recovers After Temporary Drop in Validator Participation

At the time of writing, the network had recovered significantly. At epoch 411,712, voting participation had risen to nearly 99%, and sync participation reached 97 percent. Prior to the incident, Ethereum routinely saw vote participation rates above 99%. 

The temporary decline largely corresponded with the share of validators using the Prysm client, which fell from about 22.71% to 18% following the outage. This indicates that the attestation failures were concentrated among Prysm validators.

Experts say a sustained drop in voting participation below the two-thirds threshold could have serious risks. Layer-two bridges may freeze, and rollup withdrawals could pause. Also, crypto exchanges might increase block confirmation requirements to protect against chain reorganizations. 

Even though the recent problem did not get this serious, it shows how bugs in a single client can still cause big issues.

Ethereum Faces Risks from Client Concentration

Ethereum has faced similar risks before. In May 2023, the mainnet lost finality twice in 24 hours because of bugs in the Prysm and Teku clients when handling old attestations. At that time, Prysm ran on many nodes, showing the danger of relying too much on one client.

Even though client diversity has improved, some risks still exist. Current data from MigaLabs shows Lighthouse runs 52.55% of nodes, while Prysm has 18%. Before the recent incident, Lighthouse had less than 48.5% and Prysm about 22.71%. 

If the bug had hit the client with the most nodes, Ethereum could have lost finality completely.