BSC Token Hub Compromise investigation and analysis

Tl;dr: Building a better crypto ecosystem means building a better, more equitable future for us all. That’s why we are investing in the larger community to make sure anyone who wants to participate in the crypto economy can do so in a secure way. In this blog post, we share lessons about the nature of the vulnerability, exploitation methodology, as well as on-chain analysis of attacker behavior during the BSC Token Hub incident. 

While the compromise does not directly affect Coinbase, we strongly believe that attacks on any crypto business are bad for the industry as a whole and hope the information in the blog will help strengthen and inform similar projects about threats and techniques used by malicious actors.

By The Coinbase Threat Intelligence and Special Investigations Teams

Engineering

, October 31, 2022

Global Networks.jpg

On October 6th, 2022, Binance’s BSC Token hub, a bridge between the old Binance Beacon Chain and the Binance Smart Chain (BSC) also known as BNB Chain, was exploited by an attacker resulting in the unauthorized transfer of 2M BNB (~$586M). 

Vulnerability root cause analysis confirmed a flaw in the IAVL verification implementation developed by Cosmos, and incorporated by BSC into their contract’s proof verification process. The same vulnerability affected other projects implementing the flawed IAVL library which resulted in an emergency patch from Cosmos.

The attacker swapped stolen assets using Venus Finance and bridged through Stargate Finance and Anyswap to Ethereum, Avalanche, Polygon, Fantom, and Optimism networks. The attacker was careful to swap any assets that implement blocklisting (e.g. USDC, USDT) before consolidating the majority of them on the Ethereum blockchain. However, a large sum of funds remain on various L2 and L1 chains in three distinct addresses.

Following the discovery of the compromise, BNB Chain halted operations for approximately 8 hours by requesting all validators temporarily suspend BSC in order to investigate the reported activity. Upon identifying the root cause, BNB Chain released an updated version of BSC which hard forked the blockchain, patched the vulnerability, and made node upgrade a requirement. As a result of the BNB Chain halt and assistance from asset issuers, a portion of the funds were frozen and transfers were prevented, limiting attacker funds taken off of BSC to an estimated $110M. 

The complexity of the vulnerability, exploitation, and cross-chain bridging/swapping indicates the threat actor is likely well versed in various DeFi protocols as well as blockchain exploitation. The responsible actor(s) are still unknown as of this report's initial publication.

If any dapps or service providers think they’ve been impacted by a frontend hijack like this, please reach out to us at security@coinbase.com

Special thanks to Peter Kacherginsky, Zuhdi Abdelkarim, and Heidi Wilder for their research on this incident. 

Analysis

BSC Contracts

Binance Smart Chain (BSC) implements a number of dedicated built-in system contracts to assist with cross-chain bridging. Unlike smart contracts deployed by blockchain users, built-in system contracts were deployed at genesis time. For example, BSC implements a built-in CrossChain Contract at 0x0000…2000 to handle cross-chain token transfers. BSC also implements a collection of built-in precompiled contracts to facilitate utility functions such as hash calculations, merkle tree proof validation, and others used by cross-chain system contracts.

Precompiled Contracts

Precompiled contracts are embedded within blockchain node software. Unlike user and system contracts, precompiled contracts run native code and can be implemented in any language such as Go. For example, the aforementioned CrossChain Contract triggers precompiled contract 0x65 (101) to validate a Merkle proof message submitted by blockchain users:

Screenshot 2022-10-31 at 9.53.16 AM

Figure 1 - Sample precompiled contract call (Source: 0x0000…2000)

The Ethereum Virtual Machine (EVM) embedded in BSC interprets the above call by forwarding it to the appropriate precompiled contract. In this specific case, the CrossChain system contract calls to the 0x65 (101) address to trigger the iavlMerkleProofValidate precompiled contract as highlighted in the code snippet below:

Screenshot 2022-10-31 at 9.54.04 AM

Figure 2 - Precompiled contract table (Source: bnb-chain/bsc/core/vm/contracts.go)

CrossChain System Contract

CrossChain system contract0x0000…2000 is designed to process cross-chain transfer, staking, governance, and other user transactions. Relayers can submit transactions to the CrossChain contract using the handlePackage method: 

Screenshot 2022-10-31 at 9.54.47 AM

Figure 2 - Precompiled contract table (Source: 0x0000…2000)

The handlePackage method can only be invoked by accounts registered as relayers. A registry of relayers is kept by RelayerHub 0x0000…1006 system contract. Anyone can become a relayer by calling the registermethod and paying a 100 BNB deposit:

Screenshot 2022-10-31 at 9.55.36 AM

Figure 3 - Minimum deposit to become a relayer (Source: Coinbase Threat Intelligence)

All transactions submitted to CrossChain’s handlePackage method consist of several parameters: 

Screenshot 2022-10-31 at 9.56.31 AM

Table 1 - handlePackage parameters (Source: Coinbase Threat Intelligence)

Payloads include variable size msgBytes fields which are interpreted based on the packageType and channelId. For example, 0x7957…f433 transaction submits the following payload to channelId 0x02 (TRANSFER_IN_CHANNELID):

Screenshot 2022-10-31 at 9.57.24 AM

Figure 4 - Sample handlePackage payload  (Source: 0x7957…f433)

CrossChain System contract first looks up an appropriate channel handler contract using an internal map:

Screenshot 2022-10-31 at 9.57.58 AM

Figure 5 - TRANSFER_IN_CHANNELID handler contract  (Source: Coinbase Threat Intelligence)

TokenHub (0x0000…1004) in turn processes msgBytes as an RLP encoded payload:

Screenshot 2022-10-31 at 9.58.31 AM

Figure 6 - RLP encoded payload msgBytes  (Source: Coinbase Threat Intelligence)

The above can be interpreted using the following structure:

Screenshot 2022-10-31 at 9.59.12 AM

Figure 7 - Payload msgBytes structure  (Source: Coinbase Threat Intelligence)

Putting everything together, the sample transaction 0x7957…f433 payload instructs the TokenHub contract to transfer 0.05 BNB to0x4e65…645d.

iavlMerkleProofValidate Precompiled Contract

The iavlMerkleProofValidate reference is a Go library embedded in all instances of BSC node software critical to ensuring that submitted merkle proofs are valid. The library has a number of dependencies including Merkle Tree implementation from Tendermint and IAVL+ Tree implementation from Cosmos. IAVL Tree stands for Immutable Adelson-Velsky and Landis (AVL) data structure which implements a self-balancing Binary Search Tree (BST). IAVL trees allow for efficient lookups to quickly prove existence or non-existence of values stored in leaf nodes. 

In the case of the CrossChain contract, IAVL lookups are used to prove or disprove the presence of a cross-chain transaction and a respective payload hash. CrossChain contract triggers the iavlMerkleProofValidate precompiled library by making staticcall as illustrated in Image 1 in the BSC System and Precompiled Contracts section above.

The iavlMerkleProofValidate precompiled contract decodes the provided proof buffer as a Merkle Tree proof data structure and forwards it to the IAVL library for verification:

Screenshot 2022-10-31 at 10.00.01 AM

Figure 8 - IAVL value proof code snippet  (Source: github.com/cosmos/iavl/proof_iavl_value.go)

The verification process relies on two key checks. First, we ensure that the provided proof was not modified by making sure that the provided tree root hash value is equal to the one we computed independently. This is done by calling op.Proof.Verify(root):

Screenshot 2022-10-31 at 10.01.15 AM

Figure 9 - IAVL root hash verification  (Source: github.com/cosmos/iavl/proof_range.go)

Root verification involves computing the root hash from scratch using computeRootHash and comparing it against the root hash specified in the proof.

Screenshot 2022-10-31 at 10.01.59 AM

Figure 10 - IAVL root hash computation  (Source: github.com/cosmos/iavl/proof_path.go)

Once the root hash in the proof is verified, the IAVL library can safely proceed to the second step and search for the presence of the payload hash in the proof tree using the op.Proof.VerifyItem method:

Screenshot 2022-10-31 at 10.02.35 AM

Figure 11 - IAVL item verification  (Source: github.com/cosmos/iavl/proof_range.go)

IAVL library computes a SHA256 hash of the supplied value and tries to locate it in tree leaves.

To best illustrate the above flow consider the previously mentioned transaction 0x7957…f433 with the following parameters:

Screenshot 2022-10-31 at 10.04.17 AM

Table 2 - Decoded 0x7957…f433 transaction parameters (Source: Coinbase Threat Intelligence)

The proof parameter can be interpreted as follows:

Screenshot 2022-10-31 at 10.05.05 AM

Figure 12 - IAVL Proof Tree (Source: Coinbase Threat Intelligence)

The IAVL library will compute the rootHash from the above tree structure to come up with the same value specified in the proof:

E09159530585455058CF1785F411EA44230F39334E6E0F6A3C54DBF069DF2B62

After determining that the proof tree is trustworthy, the IAVL library will compute the hash of the sample payload for the transaction (listed in Figure 6) and search for it in the proof tree:

Screenshot 2022-10-31 at 10.06.04 AM

Figure 13 - Sample transaction payload hash (Source: Coinbase Threat Intelligence)

Vulnerability

The vulnerability in the IAVL processing of proofs arises in the way the root hash of the proof is calculated and verified. The computeRootHash relies on the Hash method to process proof nodes to compute the hash:

Screenshot 2022-10-31 at 10.06.51 AM

Figure 14 - IAVL tree hash calculation  (Source: github.com/cosmos/iavl/proof.go)

The Hash function logic correctly computes the root hash value when the left leaf is null. However, the hash produced when the left leaf is already defined ignores whether or not the right leaf is also defined. Thus, the root hash value for proofs with the left leaf defined will be the same whether or not the right leaf is also defined. This creates an opportunity for attackers to inject an arbitrary payload in the right leaf, since it will not affect the final hash calculation. If the injected right leaf contains a correct SHA256 value of an arbitrary payload, then the IAVL verification logic may incorrectly approve arbitrary transactions.

Attacker Sequence of Events

The following section describes all attacker steps from the initial wallet and exploitation through cross-chain bridging and swapping to avoid blocklisting. At the time of the writing, the attacker has not attempted laundering their funds through Tornado Cash or similar anonymization platforms.

Initial Funding 

On October 5, 2022, prior to the compromise, the attacker's address 0x489a87 on BSC, received two transactions from ChangeNow (tx 1, tx 2) for 101.5 BNB in total. These assets were used for the next step of their attack flow in registering as a BSC relayer and issuing test transactions.

Screenshot 2022-10-31 at 10.08.01 AM

Figure 15 - Initial funding of the attacker’s wallet  (Source: BSCScan 0xa84f…0d1d)

Test Transactions

On October 6, 2022 the attacker was trialing various protocols and testing possibly for liquidity and traceability of funds following the compromise:

  • They deposited 0.1 BNB to Venus and received vBNB in return borrowed BUSD (tx)

  • They also deposited 0.1 BNB to Alpaca Finance to receive 0.1 ibBNB (tx)

  • Deposited 0.4 BNB to the WBNB contract (tx)

  • Borrowed 116 BUSD from Alpaca (tx)

  • Swapped BNB for 15 BUSD on Stargate (tx

  • Transfer to ChangeNow (tx)

Screenshot 2022-10-31 at 10.09.39 AM

Figure 16 - Test transactions  (Source: BSCScan)

Registering as a Relayer

After the trials, the attacker registered a relayer for 100 BNB (tx). In order to successfully submit transactions to the hub, the attacker needed to satisfy the onlyRelayer modifier required by the handlePackage method in 0x0000…2000. Anyone can register with BSC as a relayer provided they deposit the required amount.

Screenshot 2022-10-31 at 10.10.17 AM

Figure 17 - Attacker registering as a relayer  (Source: BSCScan 0xe1fe…96d6)

Exploitation

On October 6, 2022 at 06:26:46 PM +UTC the attacker submitted the first exploit transaction 0xebf8…3b8b which takes advantage of the vulnerability described above. The transaction is based on an earlier transaction 0x7957…f433 with a modified legitimate proof:

Screenshot 2022-10-31 at 10.14.37 AM

TX Hash [Original, Modified]

Screenshot 2022-10-31 at 10.16.15 AM

Table 3 - Original and modified cross-chain transactions (Source: Coinbase Threat Intelligence)

The attacker modified the original payload to transfer significantly more assets. It can be decoded using the same method used in Figure 6 above:

Screenshot 2022-10-31 at 10.17.06 AM

Figure 18 - Decoded attacker payload  (Source: Coinbase Threat Intelligence)

The payload instructs the CrossChain contract to issue 1,000,000 BNB tokens to the attacker address0x489a…9bec.

To ensure that the above payload is accepted as valid, the attacker also modified the proof to add an additional leaf with a hash value corresponding to the new transfer request payload and an empty inner node to compensate for an increased length:

Screenshot 2022-10-31 at 10.17.46 AM

Table 4 - Original and modified cross-chain transaction proofs (Source: Coinbase Threat Intelligence)

As described in the Vulnerability section above, the additional leaf is not used in the calculation of the root hash, so the proof appears unaltered and successfully passes verification. The additional leaf includes a SHA256 hash of the payload which will be used to pass a second check that the payload is recorded in the proof.

Screenshot 2022-10-31 at 10.18.24 AM

Figure 19 - Original and modified proof trees  (Source: Coinbase Threat Intelligence)

After successfully exploiting the vulnerability in IAVL verification logic, the CrossChain contract instructed the TokenHub contract to transfer 1,000,000 BNB tokens:

Screenshot 2022-10-31 at 10.20.24 AM

Figure 20 - First exploit transaction  (Source: BscScan)

On October 6, 2022 at 08:43:18 PM +UTC the attacker sent a second successful exploit transaction 0xebf8…3b8b to steal another 1,000,000 BNB tokens four hours after the initial exploit. Interestingly, the attacker experienced a number of failed exploitation attempts prior to 0xebf8…3b8b due to the transactions falling behind the current packageSequence number:

Screenshot 2022-10-31 at 10.20.57 AM

Figure 21 - Failed exploitation attempts  (Source: BscScan)

Exfiltration

Following the exploit, the attacker immediately started moving assets off BSC, often correctly assessing that the chain may freeze and claw back assets following the discovery.

The attacker used Venus Finance to mint vBNB and then used it as collateral to borrow BP USD (Binance’s USDT proxy) and BUSD (Binance’s stablecoin). Once they had BP USD/ BUSD, they used Stargate and Anyswap to bridge over funds to various chains, including Ethereum, Avalanche, Fantom, Polygon, Arbitrum, and Optimism. 

Note, BP USD does not have a freeze function associated with it.

bnbanalysis

Figure 22 - Bridging stolen assets  (Source: Coinbase Special Investigations)

After moving assets to various L1/L2s chains, the attacker performed further swaps to avoid assets which could freeze or blocklist addresses (e.g. USDC, USDT). In particular, it seems the attacker selected chains and protocols with high TVL, but also those that included USDT and USDC contracts that do no include blocklist functionality, as they were set up by various separate  projects as mere proxies of USDT and USDC.

Below is a per chain description of further swaps.

Ethereum

The attacker’s address had already received several million USDC and USDT via Stargate as described above. Additionally, the attacker performed a number cross-chain transfers and swaps to avoid blocklisting:

  • Swapped 3.99m USDC for DAI using Curve (to prevent blocklisting) (tx)

  • Swapped DAI for ETH on Uniswap (tx)

  • Received 8.99m USDC from Fantom Bridge (tx)

    • Swapped these using Curve for DAI (tx)

  • Swapped 12.99m USDC for DAI using Uniswap (tx)

  • Used Curve & Synthetix to swap ~9m USDT for sETH (tx, tx2)

  • Received various funds from Avalanche Bridge

    • 174k USDT (tx)

    • 299 WETH (tx)

    • ~2k WETH (tx)

    • 681k USDT (tx)

    • ~3m USDC (tx)

    • 9.8m USDC (tx)

    • Swapped 12.9m USDC using Uniswap for ETH (tx)

  • Received additional funds from Fantom:

    • Received 3 txs worth ~2.5k WETH each (tx1, tx2, tx3)

About 33.8k ETH were later transferred to the fresh address 0xfa0a32…13e9 where they still sit. Some 1.6k WETH was also bridged from Fantom to 0x4ad64…29cb.

Below is a simplified graphic displaying the flow of funds on Ethereum:

ethereuman

Figure 23 - Swapping and consolidating stolen assets on Ethereum (Source: Coinbase Special Investigations)

Avalanche

After receiving USDT and USDC via Stargate, the attacker swapped and borrowed USDT and USDT for bridged forms of these tokens (USDCe and USDTe) as well as eWETH.

The attacker managed to do this using TraderJoe and Platypus. They then bridged the funds over to the Ethereum network via Avalanche Bridge and Stargate.

avalanche

Fantom

After receiving bridged fUSDC and fUSDT assets from Stargate and Anyswap, the attacker swapped these for Geist-based tokens and then borrowed ETH from Geist. Some of this ETH was then bridged over to the Ethereum network via Stargate. Some gUSDC was additionally bridged over to the Ethereum network via Stargate; however, some remain on Fantom’s 0x489a…9bec.

The majority of the leftover ETH was bridged over to Ethereum; however, some were sent to 0x4ad6…29cb, where they still remain.

fantom

Polygon

After receiving bridged USDC and USDT assets from Stargate, the attacker withdrew 3.6m USDT to Ethereum (tx); however, some remain on Polygon’s 0x489a…9bec.

polygon

Optimism

The bridged USDC from BSC continues to remain on Optimism’s 0x489a…9bec.

Arbitrum

After receiving the bridged USDC and USDT from Stargate, the attacker swapped USDT and USDC for ETH via Uniswap (tx1, tx2).

The ETH was then transferred to 0xfa0a32  where it still remains(tx).

arbitrum

Where are the funds now?

The attacker has not begun the laundering process through Tornado Cash or equivalent anonymization protocol. Stolen assets are currently located at the following addresses:

Asset and Blockchain response

Following the compromise, multiple asset issuers and BSC responded by freezing the attacker's assets. Binance halting the BSC network had the most impact preventing the attacker from exfiltrating more than $400M in fraudulently obtained assets. Tether froze the attacker's USDT address at 2022-10-06 21:24:59 UTC which had a moderate impact on the attacker's ability to launder assets on Ethereum blockchain. USDC was the last asset to institute a freeze at 2022-10-07 01:34:34 UTC, at which time the attacker had already swapped most of their USDC assets.

Incident Timeline and Attacker Flow

The following timeline highlights key moments during the incident.

Attacker Preparation

2022-10-05 22:11:04 UTC - Attacker 0x489a…9bec receives about 101.54 BNB from 0x975d…cd94b

2022-10-06 15:19:57 UTC - Attacker 0x489a…9bec tests swapping and bridging assets prior to the exploit.

Exploit - Wave 1

2022-10-06 16:55:06 UTC - Attacker 0x489a…9becregisters as a relay on BNB Bridge.

2022-10-06 18:26:46 UTC - Attacker 0x489a…9becsteals 1,000,000 BNB from the BNB Chain leveraging the vulnerability.

2022-10-06 18:30:13 UTC - Attacker 0x489a…9becfunds Venus with 600,000 BNB.exploit.

2022-10-06 18:31:04 UTC - Attacker borrows 62,500,000 BUSD from Venus.

2022-10-06 18:31:49 UTC - Attacker borrows 50,000,000 USDT from Venus.

2022-10-06 18:32:58 UTC - Attacker starts bridging assets to Ethereum, Polygon, Fantom, Optimism, Arbitrum, Avalanche, and other chains using Stargate Finance.

2022-10-06 18:36:07 UTC - Attacker 0x489a…9becfunds Venus with 300,000 BNB.

2022-10-06 18:37:28 UTC - Attacker borrows 35,000,000 USDC from Venus.

2022-10-06 18:52:59 UTC - Attacker begins bridging assets to Fantom using AnySwap.

Post Exploitation - Wave 1

2022-10-06 18:57:35 UTC - Attacker begins swapping USDC on Ethereum Curve to prevent blocklisting.

2022-10-06 19:18:23 UTC - Attacker begins swapping USDT on Curve and Synthetics to prevent blocklisting.

Exploit - Wave 2

2022-10-06 20:43:18 UTC - Attacker 0x489a…9becsteals another 1,000,000 BNB using the exploit

2022-10-06 20:50:00 UTC - Attacker begins exchanging BNB for BSC-USD on PancakeSwap.

2022-10-06 20:50:41 UTC - Attacker begins bridging BSC-USD to other chains using Stargate Finance.

Post-Exploitation - Wave 2

2022-10-06 21:24:59 UTC - Tether freezes the attacker’s address on Ethereum.

2022-10-06 21:43:59 UTC - Attacker sends the last bridging transaction on BSC.

2022-10-06 22:19:00 UTC - Binance announces halting of BSC network and blocklists 0x489a…9bec from further cross-chain transfers.

2022-10-07 01:34:34 UTC - USDC freezes the attacker’s address on Ethereum.

2022-10-07 05:02:00 UTC - Binance announces BSC validators restarting the network.

2022-10-07 18:09:23 UTC - Attacker0x489a…9bectransfers 33K ETH to 0xfa0a…14e9.

Consolidation

2022-10-07 18:33:47 UTC - Attacker bridges 1.5K WETH to 4ad6…29cb from Fantom to Ethereum.

2022-10-07 19:32:35 UTC - Attacker begins bridging various assets to 0x489a…9bec from Avalanche to Ethereum.

2022-10-07 19:40:30 UTC - Attacker bridges 3.6M USDT to 0x489a…9bec from Polygon to Ethereum.

2022-10-07 19:48:23 UTC - Attacker bridges USDC and USDT to 0x489a…9bec from Arbitrum to Ethereum.

2022-10-07 19:49:53 UTC - Attacker 0x489a…9bec sends 1.457K ETH to 0xfa0a…14e9 on Arbitrum.

---

References

Appendix A: Analyst Notes and Indicators

  • 0x489a8756c18c0b8b24ec2a2b9ff3d4d447f79bec [BSC, Ethereum, Polygon, Optimism, Fantom, Avalanche, Arbitrum]

  • 0xfa0a32e5c33b6123122b6b68099001d9371d14e9 [Ethereum, Arbitrum]

  • 0x4ad64fd7ca6d6150614179b9bce4094bc18f29cb [Ethereum, Fantom]

Coinbase logo
The Coinbase Threat Intelligence and Special Investigations Teams

About The Coinbase Threat Intelligence and Special Investigations Teams

The Coinbase Threat Intelligence and Special Investigations Teams

Take control of your money. Start your portfolio today and get crypto.

Sign up for a Coinbase account today and see what the world of decentralized finance can do for you.