Detecting Secrets in Your Source Code

Exposed secrets in source code pose a risk to you, your team and your entire organization. But what are secrets exactly? How do they become exposed? And what can you do about them?

Secrets are any kind of data that is sensitive to an organization or person and something that should not be exposed publicly. It can be a password, an access key, an API token, a credit card number, and more. 

Modern applications are built from hundreds or even thousands of parts; databases, cloud infrastructure, independent microservices, and much more. To allow all those parts to interact securely, different keys and authentication methods are used between application components. For example, a microservice may need a database password to query for needed data, or an API key could be required to connect with a 3rd party service. You can imagine the hundreds or thousands of these secrets that might be embedded in a modern, complex business application. However, all these keys, passwords and tokens are secrets for a reason – we want to keep them private to avoid our secure connection and sensitive data from being compromised by a malicious third party. 

The move to microservices and cloud-native development means there are many more secrets and passwords to manage. These secrets should be properly stored, safely transferred to the app, and be accessible to internal teams to develop and debug the software. However, the complexity associated with implementing this can lead to human error. All too often, these secrets are exposed in code and placed in our code repositories, sent over IM apps, or just saved locally on development machines that could be more easily compromised.  

Exposed Secrets - A Vulnerable Attack Surface

Everyone knows the dangers of having your personal credit card information exposed to strangers. Eventually, someone will use it to steal your hard-earned money. Exposed secrets in your source code are similar but pose a broader and more serious risk to you, your team, and your entire organization. With an exposed secret, an attacker could access applications, machines, API's and more. For example, if an SSH key is leaked, an attacker can connect to a server and do pretty much anything; mine for bitcoin, steal data from the server, or encrypt the hard drive and demand ransom. And of course, after achieving access to a single entity or service, a skilled attacker can also attempt to move laterally across the network to obtain control of larger environments or resources. 

Why Secrets End Up in Your Code Repository

Developers already know that pushing secrets into code repositories is a bad practice. But in most cases, secrets eventually make their way into repositories. There are many reasons for this. One reason is that properly transferring secrets to an app in run-time using secret vault software can be complicated and can also complicate the ability to debug the app later. Some development teams choose to release software as fast as possible and leave secret handling to a later downstream task. Whether it is human error, laziness, or the only approach that a developer could find, you can safely assume that some secrets will end up in your repositories.  

Why Secrets in Git Repositories are a big problem

When a secret makes its way to a Git repository, it stays there forever, sitting in one or more of your commits, waiting to be found and used against you. Developers often forget that Git-based repository history is never deleted. Not only do the secrets sit in a Git server, but every clone and fork save this secret on different machines without the clone or fork creator being aware of it. Even on private repositories, one compromised account in our organization can lead to many secrets being leaked over time. 

When using public repositories, a pushed secret can have more immediate and dire ripple effects. Often by the time an exposed secret is found, the secret has already been picked by automated bots and the repository has been archived by multiple archiving tools. Unfortunately, that means you should consider it exposed and compromised from the first minute it was public.

hidden Secrets

While secrets in your main branch might be “easier” to keep track of, the secrets hiding in older commits of your repository are almost impossible to identify. The more developers working on a project means more commits to look in and greater chance of a secret accidentally making its way into your Git history. 

One might argue that secrets entering your Git repository are not a significant threat because all code in your repository goes through a Code Review (CR) process that should pick up on any secret that might enter the repository. While a CR process is very important for several reasons, it is not as effective in preventing secrets for two main reasons: 

1. The code displayed in the CR process only includes the changes between the last commit and the main branch. If some code that included a secret was added, committed, deleted and then committed again, the code reviewer will not see the code in the “diff” he performs the code review on. 
   
   
   
   
   
2. If a code reviewer identifies a secret in the process, the author will probably create another commit that removes the secret from the code, but will not rewrite the history of the previous commit, leaving the secret just one commit behind plain site, where anyone can find it. 

A better way to detect secrets

We've discussed the many ways that hard coded secrets can remain in your code, and that finding all the secrets in a Git repository can be a challenging task. So, what can you do?  

We’ve worked extensively on this problem at Legit Security and feel that removal of hidden secrets should be part of any comprehensive solution to software supply chain security. We’ve created a platform to scan for secrets, among many other capabilities, and can detect hard-coded secrets and PII before they are pushed into the SDLC, as well the ability to scan through Git histories to find secrets that crawled their way into older commits. We can also detect secret-related misconfigurations, like ensuring a Key Vault has an expiration date set. The good news is that these are automated scans that can run continuously in the background.

I found a secret, Now what?

If a secret has been exposed for a reasonable period of time, we must assume that someone has found it and has intentions to use it. Once a secret is exposed, the best thing to do is to revoke it ASAP. Change the password, regenerate the key, contact your bank to issue a new card, and so on to ensure the exposed data is no longer relevant. At the same time, monitor access to the resources that the exposed secret could have given an attacker access to and investigate any suspicious behavior. 

There are other options to remove a secret from your Git history by using tools that alter it but note that this action will change your repository history entirely, which is not trivial and should not be taken without serious consideration. Instead, we recommend doing your best to simply revoke the exposed secret.

Living with your secrets

Secrets are highly valued but they can also be dangerous in the wrong hands. Secrets provide access to valuable services, APIs, machines and more. Due to the nature of modern software development, secrets can very easily make their way into your Git repository without the right security tools in place. Once a secret is in your Git repository it is usually hard to find all instances of it.  

To learn more about other attack patterns documented by popular frameworks check out The 3 Riskiest Software Supply Chain Attack Patterns Across Frameworks.