Solana Releases Mainnet Beta Outage Report

Solana has officially released a report of a mainnet Beta outage issue, pointing to a runtime bug triggered by the durable nonce transactions feature allowed, under a specific set of circumstances, for a failed durable nonce transaction to be processed twice. This led to nondeterminism, when a validator processed the transaction a second time and some nodes rejected the subsequent block, while others accepted it. Critically, more than 33% of validators accepted the block, but that number fell short of the 66% required to reconcile the nondeterminism. The durable nonce transaction feature was disabled in releases v1.9.28/v1.10.23 to prevent the network from halting if the same situation were to arise again. Durable nonce transactions will not process until the mitigation has been applied, and the feature re-activated in a forthcoming release. Previously, the Solana mainnet Beta suspended blocks at 16:30 UTC on June 1st, with an outage of over 4 hours.