Introducing Salus: How Coinbase scales security automation

Security at Coinbase is a top priority, and we’re always working to make sure our customers’ information and assets are completely protected at every corner. Many of the software security issues we hear about today come from problems that, in hindsight, could have been easily prevented, like outdated dependencies and obvious anti-patterns. Code-review processes are a standard part of many engineering teams, but engineering teams are always trying to move faster and humans can occasionally miss some issues.

Introducing Salus

At Coinbase, we use a combination of human-driven code reviews and automated scans to ensure all our production deployments are as secure as possible — and when the right tools don’t exist to help us do the work we need to, we build them. One of those tools that we recently built is called Salus, a docker container that decides which security scanners to run, coordinates their configuration, and compiles the output into a single report. And today we’re open sourcing it on our Github for other teams to use too.

All software companies leverage open source software, and common languages and frameworks often have security scanners which can tremendously improve security. For example, for Ruby projects, bundle-audit can alert you of known vulnerabilities in the libraries you are using and for Rails, Brakeman will do AST level analysis to identify possible RCEs, SQL injections and more. Tools like these help us to ship faster, and we are tremendously grateful for these open source efforts. It was in this spirit that Coinbase started its open source fund, a token of gratitude for this type of community-oriented work.

To use these tools, a common initial deployment pattern for a security team is to include the scanner into the repository’s test suite. The continuous integration (CI) pipeline will execute both the project’s tests and the scanner. If there is a security issue, the build will fail and the developer can investigate the build logs to identify what needs to improve in their pull request.

However, this strategy quickly fails at company scale where there can be hundreds of repositories, each with their own security scanning configurations. Updating a scanner, or introducing a new scanner, will then require updating every project and quickly you have an O(n) problem. This is where our new tool Salus — named after the Roman goddess of safety — can make a difference.

Salus coordinates security scanning across all the services we deploy at Coinbase. It helps us enforce security policies for each change made to a codebase and ensures there is a quick feedback loop with the developer about potential vulnerabilities.

Architecture of the Salus container

How it works

During each CI build, the repository source code is volumed into the Salus container and Salus begins executing. An individual scanner might conduct static analysis, dependency checks, anti-pattern (e.g. grep) checks, or anything else that improves security.

To update a scanner or experiment with new scanners, we update the Salus container. Since each CI build pulls and runs the latest container, all builds immediately inherit these changes. If issue are found, the build fails and the scanner’s output is shown to the developer immediately so that they can self service their fix.

###### Salus Scan v1.0.0 for engineering/proxy ######

BundleAudit => passed

PatternSearch => failed

 Forbidden pattern in public/offline.html:204:

   <p>We are currently offline - <br/>

   <a href="https://status.internal.net">status page</a>

 Use `link_to('<link>', target: '_blank')` for rendering links so

 that the appropriate security features can be applied.

overall => failed

Salus also compiles reports about the results of each scanner and which dependencies are being used by a project. At Coinbase, we consume these reports into our logging pipeline to allow us to quickly identify which projects might be using a package that recently had a vulnerability released and from there, we can efficiently move into incident response mode.

Screenshot of Kibana displaying the results of Salus scans.

Salus can be run out of the box with strong default configuration but also support powerful customization to ensure that you can pick which scanners will run, which scanner will fail builds when finding issues, and where to send reports. We use this functionality at Coinbase so that we can enforce a global security policy for all projects, but also apply special configuration at the repository level if a certain project needs it.

For local customization, multiple configuration files can be concatenated. For example, if a project’s dependency is carrying a CVE with no available patch and we have confirmed that the vulnerability is not exploitable, we can use a local configuration file to ignore this concern.

docker run             \

 --rm                 \

 -v $(pwd):/home/repo \

 coinbase/salus --config "https://salus-config.internal.net/salus-global.yaml file://local-salus-config.yaml"

Pointing to remote configuration files also allows a security team to introduce new security policies into an organization and identify where there are gaps without failing builds and surprising developers. A scanner can run in soft mode, Salus will provide data on repos that are not compliant, and then those projects can be patched before enforcing a new, higher global security policy.

Salus currently runs the following checks:

• CVE checks for Ruby gems and Node modules via BundleAudit and NSP respectively.

• Brakeman for Ruby on Rails vulnerabilities.

• Reports which Ruby gems, Node modules, Go packages and Python packages are used by the repo.

• Pattern matching on regular expressions of your choice — for example, this could look for the use of poor cryptographic primitives or potentially dangerous code like React’s dangerouslySetInnerHTML.

Try the tool

If you manage many repositories in your infrastructure, or want a single command to run all relevant scanners on your codebase, you may want to consider running Salus during your test suite. Salus is an important tool to us, and we plan to expand it over time to cover more languages and types of static analysis. If you have feature requests for Salus, or would like to discuss use cases, please see our repository.

Coinbase works hard to provide the most-secure cryptocurrency services in the world, and we’re building and creating highly empowered development environments. If you’re passionate about building security solutions and want to join our team, please take a look at our open careers. We’re hiring for a Application Security Engineers, and many more positions across the board. Get in touch.

— — —

Unless otherwise indicated, all images and diagrams provided herein are by Coinbase.

This website may contain links to third-party websites or other content for information purposes only (“Third-Party Sites”). The Third-Party Sites are not under the control of Coinbase, Inc., and its affiliates (“Coinbase”), and Coinbase is not responsible for the content of any Third-Party Site, including without limitation any link contained in a Third-Party Site, or any changes or updates to a Third-Party Site. Coinbase is not responsible for webcasting or any other form of transmission received from any Third-Party Site. Coinbase is providing these links to you only as a convenience, and the inclusion of any link does not imply endorsement, approval or recommendation by Coinbase of the site or any association with its operators.