This post is made on behalf of The Security Council following the Emergency action initiated on 25 September 2024.
On 25 September 2024 the Security Council initiated an Emergency Action to execute a software upgrade on ArbitrumDAO-governed chains (Arbitrum One and Nova). The software upgrade is a response to vulnerabilities that could have affected the availability of the chains. This document describes what was done and why the actions were justified (as required per Section 3 of the ArbitrumDAO Constitution).
The Arbitrum Foundation notified the Security Council of two vulnerabilities affecting the ArbitrumDAO-governed chains. The vulnerabilities made it possible to halt the ability of normal users to transact on the network and ultimately the execution of DeFi applications. The security council successfully mitigated all concerns before any exploitation occurred. The issues consisted of:
- Node Crash. ArbOS Panic due to specially crafted Stylus Program
- Impact: All nodes will crash if a transaction with this program is included in a block, including the sequencer.
- Mispriced Opcodes. Storage opcode is mispriced in Stylus and it is effectively free to store values.
- Impact: An attacker can craft a transaction that bloats state and effectively perform a denial of service attack on the network.
Both issues posed significant risk to the availability of the chains, and thus the Security Council deemed it necessary to execute an Emergency Action to upgrade the chain software. The source code for the software upgrade is now publicly available (Release Arbitrum Nitro v3.2.0 · OffchainLabs/nitro · GitHub).
Great care has been taken to minimize the impact on current operations on the ecosystem whilst not putting them at risk of the vulnerability being exploited. The vulnerability had to be contained and handled on a need-to-know basis until mitigations were ready to be deployed. To help ensure node operators had actionable time to upgrade, a node software binary (but not its source code) was released with support for the fixes. In case some attacker was able to identify the vulnerability as result of the updated and released node software, the Security Council also was ready with an Emergency Action that could pause all Stylus smart contracts, thus mitigating the vulnerability without impact on any EVM smart contracts. Fortunately there was no need to employ this recourse.
Timeline of events
- September 18, 2024: The Arbitrum Foundation disclosed to the Security Council that an issue had been identified in the ArbitrumDAO-governed chains.
Security Council members were alerted to be on-call to evaluate a proposed software upgrade that was being finalized. - September 21, 2024 (morning UTC): Source code for the proposed software upgrade, as well as an audit report by Trail of Bits, were shared with the Security Council for their evaluation.
The Foundation proposed an action plan on how to effectively mitigate risks during the upgrade and minimize impact. - September 21, 2024 (evening UTC): A transaction was initiated and signatures started being collected by members as they completed their respective due diligence processes.
Fellow Security Council members collaborated to ensure the upgrade was sound and justified. - September 22, 2024 (morning UTC): Member signatures were progressively collected as they verified the proposed upgrade and transactions.
- September 22, 2024 (evening UTC): Initial comms were shared with the public for them to update their node software to support the upcoming upgrade. The source code was not released and it was not mentioned that this was a security patch.
A sufficient quorum of signatures were available to initiate the upgrade, but it was decided to wait and give the community actionable time to upgrade their node software, thus mitigating any potential disruption of service. - September 23, 2024: Public comms were shared making it known that the upcoming upgrade was a security patch, but no details shared as to the nature of the vulnerability.
- September 25, 2024: The Emergency Action was initiated onchain and upgraded the software for ArbitrumDAO-governed chains.
Transactions: