Charlton's Blog

Today, I published my first Ruby gem: Sssecrets (RubyGems, GitHub, RubyDoc), a library for generating secrets (like API tokens, etc) in line with best practices. Let me tell you all about it.

Sssecrets stands for Simple Structured Secrets. It’s a reusable implementation of GitHub’s API token format (which is also used by NPM), and it’s designed to make it simple for developers to issue secure secret tokens that are easy to detect when leaked.

You can learn more about GitHub’s design process and the properties of this API token format on the GitHub blog.

Why Structured Secrets?

If you’re a developer and your application issues some kind of access tokens (API keys, PATs, etc), it’s important to format these in a way that both identifies the string as a secret token and provides insight into its permissions. For bonus points, you should also provide example (dummy) tokens and regexes for them in your documentation.

Simple Structured Secrets help solve this problem: They’re a compact format with properties that are optimized for detection with static analysis tools. That makes it possible to automatically detect when secrets are leaked in a codebase using features like GitHub Secret Scanning or GitLab Secret Detection.

Here’s an example. HashiCorp Vault’s API access tokens look like this (ref):

f3b09679-3001-009d-2b80-9c306ab81aa6

You might think that this is pretty is a pretty easy pattern to search for, but here’s the issue: It’s just a UUID string.

While random, strings in this format are used in many places for non-sensitive purposes. Meaning that, given a random UUID formatted string, it’s impossible to know whether it’s a sensitive API credential or a garden-variety identifier for something mundane. In cases like these, secret scanning can’t help much.

What’s in a Structured Secret?

Structured secrets have three parts:

• A prefix (2-10 characters, defined by you)
• 30 characters of randomness
• A 6 character checksum

That’s it!

Here’s the format:

[prefix]_[randomness][checksum]

An example Sssecret, with an org of t and a type of k, looks like this:

tk_GNrRoBa1p9nuwm7XrWkrhYUNQ7edOw4GUp8I

Prefix

Token prefixes are a simple and effective method to make tokens identifiable. Slack, Stripe, GitHub, and others have adopted this approach to great effect.

Sssecrets allows you to provide two abbreviated strings, org and type, which together make up the token prefix. Generally, org would be used to specify an overarching identifier (like your company or app), while type is intended to identify the token type (i.e., OAuth tokens, refresh tokens, etc) in some way. To maintain a compact and consistent format for Sssecret tokens, org and type together should not exceed 10 characters in length.

Entropy

Simple Structured Secret tokens have an entropy of 178:

Math.log(((“a”..“z”).to_a + (“A”..“Z”).to_a + (0..9).to_a).length)/Math.log(2) * 30 = 178

See the GitHub blog.

Checksum

The random component of the token is used to calculate a CRC32 checksum. This checksum is encoded in Base62 and padded with leading zeroes to ensure it’s always 6 characters in length.

The token checksum can be used as a first-pass validity check. Using these checksums, false positives can be more or less eliminated when a codebase is being scanned for secrets, as fake tokens can be ignored without the need to query a backend or database.

Note that this library can only check whether a given token is in the correct form and has a valid checksum. To fully determine whether a given token is active, you’ll still need to implement your own logic for checking the validity of tokens you’ve issued.

Another note: Because Sssecrets uses the same format as GitHub tokens, you can also perform offline validation of GitHub-issued secrets with SimpleStructuredSecrets#validate.

Installation

Add this gem to your application’s Gemfile:

gem 'sssecrets'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install sssecrets

Usage

Sssecrets is designed to be simple and straightforward to use. Here’s an example:

require 'sssecrets'

test = SimpleStructuredSecrets.new("t", "k")
tok = test.generate

puts "#{tok} is valid!" if test.validate(tok)