Security PSA: The Blockchain Stack

Disclaimer: Blog posts in this series are not intended to capture the complete security attack surface of Coinbase products.

There are many security considerations and unique potential pitfalls when interacting with blockchains. There are risks associated with the blockchains themselves, as well as risks that occur when building and operating systems that interact with blockchains. The posts in this series will primarily cover the latter, where we focus on the security aspects of services within Coinbase’s control.  

At Coinbase, our top priority is ensuring secure custody of your digital assets. That’s why we have a dedicated security team within Coinbase, Blockchain Security, that specializes in the security of cryptocurrency assets.  The Blockchain Security team utilizes various mechanisms to ensure the security of Crypto services, and one of these mechanisms involves partnering with engineering teams to conduct threat modeling, advising on potential threats, and subsequent necessary mitigations.

We define Crypto services as any service that enables interactions with blockchains. These services help bridge the gap between traditional Web2 services and the Web3 services required to operate on public blockchains. As Coinbase continues to launch new products which interact with blockchains, it is the Blockchain Security team’s responsibility to review these crypto services before they are integrated into new product offerings. 

For every part of the Blockchain stack we spend a lot of time and effort determining which security postures are best for each type of service implementation. In this and subsequent articles we will be covering crypto service best practices at a high-level. The best-practices covered in this post reference security features applicable across all crypto services.  Subsequent blog posts will do a deep dive into best practices that are scoped to each of the major 3 categories of services that make up the blockchain stack. 

Classifying and Defining Crypto Services 

We classify most of our crypto services within one of 3 major categories: signer, wallet, and protocol.

By defining each crypto service into a classification, we can build deeper and more specific guidelines. Our classifications of crypto services across a blockchain technology stack are as follows:

Universal Crypto Service Security Best Practices

Across the Blockchain stack there are various security features that we consider to be universal (i.e. applicable across all crypto services).  We’ve selected 3 notable guidelines, described below.

Resilient Key Storage 🔑

Having access to the private key for any cryptocurrency wallet is what allows the sending of funds.  Thus, any custodian of cryptocurrency needs to ensure proper safeguards are taken against the two main types of private key loss: theft and destruction.

Scenario: Rogue employee at cloud computing provider steals the hard drive from a server that they deduced was a part of Coinbase’s wallets.

Not-so-good: Without resilient key storage, this scenario would pose the risk of both key theft and key destruction.

What does Coinbase do?

Coinbase encrypts all private keys at rest and maintains multiple geographically distributed backups of its private keys.  In the extremely unlikely event that any individual was able to gain physical access to a server that is related to Coinbase, all cryptographic private keys would be fully encrypted, making them completely useless to the attacker.  Additionally, Coinbase maintains backup access to key material independent of any single cloud provider. This does not, however, come at the cost of reduced security of the key material, as the same level of encryption and general security controls apply to the replicas as they do to the actual keys. Coinbase also invests in cutting edge cryptography such as MPC to provide additional security for cryptographic private key material.

Air Gapped Funds 🥶

It’s possible to go even further than taking the utmost care in protecting private keys.  For many use cases, cryptocurrency custodians such as Coinbase only need rapid access to a small percentage of the cryptocurrency in their custody.  Given the criticality of the private keys described above, as well as the irrevocable nature of blockchain transactions, this opens a door for additional risk mitigation: offline storage. 

Scenario: A cryptocurrency custodian stores 100% of funds on a system where the keys are online and available and a vulnerability in a dapp causes all of the funds to be drained. 

Not-so-good:  Even if the system has best-in-class security controls, all the eggs are in one basket.  A successful breach drains 100% of the funds.

What does Coinbase do?

Coinbase ensures that the smallest portion necessary of funds in our custody are readily available via online systems.  The vast majority of funds are stored in a way that requires some offline interaction, typically called “cold storage.” This ensures that our customers have adequate funds quickly available when using our services without exposing any more funds than necessary to various online attack scenarios. Coinbase extends this principle into our new products such as Wallet as a Service (WaaS) that is exclusively built on MPC, which separates keys into multiple distinct locations and provides users more control over the security of their keys.

Consensus Driven Code Reviews 👀

Coinbase protects cryptocurrency private keys with the best in class encryption and operational security, but at the end of the day, software needs to utilize these keys in order to serve our customers. Thus, the security of the code that defines this software is arguably every bit as important as the security of the keys themselves.

Scenario: An employee has their laptop stolen with their yubiKey attached, allowing anyone to have full access to the device.

Not-so-good: Any code that the employee was working on can now be modified by whoever has access to the computer, perhaps resulting in changes to the security of how private keys are handled.

What does Coinbase do?

Any sensitive action within Coinbase requires sufficient consensus from a quorum of multiple individuals.  In the case of making changes to critical code, Coinbase requires formal code review and multiple approvals from engineers familiar with the operation of the system as well as members of other teams that have wider insights into the broader dependencies of the critical systems. In addition, Coinbase has the ability to rapidly restrict the access of any stolen device. All employees undergo mandatory training on how to reach core security teams in case of emergencies.

Providing Best in Class Security

In this blog post, we have provided an overview of how we classify and define crypto services, as well as some of the universal best practices we follow to ensure the security of our customer funds.  We have touched upon the importance of resilient key storage, air-gapped funds, and consensus-driven code reviews in safeguarding the different layers of the blockchain stack. It's crucial to understand that the security landscape is constantly evolving, and so are the threats we face. That's why our team remains vigilant and proactive in identifying and mitigating potential risks.

In the upcoming posts, we will delve deeper into the specific best practices for each of the major categories of services that make up the blockchain stack: signer, wallet, and protocol services. Our goal is to keep you informed about the security measures we implement to maintain a robust and secure environment for your digital assets.

Stay tuned for our forthcoming blog posts that will further explore the security intricacies of the blockchain stack. At Coinbase, we are committed to safeguarding your assets and ensuring a reliable, secure, and seamless experience for all our users.