Prefer video tutorials? Find a video walk through at the bottom of the page
Why set up a pre-commit git hook?
We have covered many articles in this blog that look at why secrets inside source code are dangerous and why automated secrets detection is essential for the security of applications [Read More]. But there are different places you can add secrets detection within the software development lifecycle. It is essential to have secrets detection within your git server or remote git host because this is where the active threats live. But secrets that end up within these git repositories must be considered compromised and revoking them can take time and resources. You now see why it is always best to capture secrets before they enter your repository. This can be achieved with a pre-commit git hook. For this purpose, we will use GitGuardian open-source CLI tool, ggshield.
Before starting, it should be noted that pre-commit hooks should not be taken as a substitute for detection on the git server because local install of git hooks are the developers' individual responsibility, thus hard to enforce as an organization and easy to bypass. But it is an additional layer of defense that can help prevent big issues.
How does ggshield work?
ggshield leverages the GitGuardian detection engine, which uses the GitGuardian API. Detecting secrets in this method has both advantages and disadvantages.
Advantages
- Advanced detection capabilities. Detecting secrets is very difficult and, if not done properly, can create large numbers of false positives. Using an external engine means that we can rely on the most advanced detection and where possible, check with the API providers if the key is in fact valid.
- Up-to-date detection. We are constantly adding new detectors to our engine as well as constantly testing and modifying our algorithms. Using an API means you always have the most up-to-date detection without needing to upgrade systems.
Disadvantages
- Sending data externally. Although this is a secure process, sending source code to an external third party can be a concern.
- Unable to directly edit detection. We cannot directly modify or edit the detection engine for the specific needs of our project or organization.
While it is important to weigh the pros and cons of all our tools, ggshield does provide the best detection capabilities by using this method.
Step 1. Getting and storing the API key
Generating API Key
Because the tool uses an API, we need an API key to establish a secure connection. We can generate it simply and for free from the GitGuardian Dashboard.
You can log in either with GitHub or using your email and password. If you do not want to connect any repositories at the moment, you can skip this step.
Once in the dashboard navigate to the API tab and generate a new API key. There are some different options regarding the lifespan of the key, just remember you will need to update this in the future if you choose a short lifespan key.
Storing API key
Next we need to save our API key as an environment variable. You can do this however you usually store environment variables, but there is a nice feature with ggshield where it will automatically search for a .env file within your project, if it can identify a variable GitGuardian_API_Key this variable it will store it as an environment variable for you, nice huh!
Step 2. Installing ggshield
The next step is to install ggshield. This can be installed either using PIP, the python package manager, or brew if you're on Mac.
To install using PIP simply write the command
pip3 install ggshield
For brew
brew install ggshield
And that's it!
Read more about configuring ggshield in our documentation.
Step 3. Installing the pre-commit hook
Now we need to install the pre-commit hook.
There are two options for installing a pre-commit hook. Locally or Globally. The difference is local will install just on the repository you are navigated with and globally means it will be installed on all repositories.
To do this, we just need to run the command
ggshield install -m local
(replace local for global if desired)
Pre-commit hooks are executable files that are installed in the folder repository/.git/hooks/pre-commit for local and root/.git/hooks/pre-commit if a global pre-commit hook.
Read more about commands in the ggshield documentation.
Testing the pre-commit hook
Now we have installed the pre-commit hook we can test it to see if it works as expected.
Because GitGuardian checks some API keys against the providers to see if they are valid, altering a valid API key may not trigger an alert. I recommend generating a key from https://canarytokens.org. These tokens will trigger as valid but have no risk.
Commit the changes as normal; if working correctly, you should be prompted in your terminal.
🛡️ ⚔️ 🛡️ 2 incidents have been found in file python_server.py
>>> Incident 1(Secrets detection): Slack User Token (Ignore with SHA: ecb4d7fe4d92281b501b14391898eee78e73841c6e58c787fc5d147a7b23a1ee) (1 occurrence)
8 8 | import logging
9 9 |
10 | …ack_key = xoxp-90543978****-************-*************-*******************ae1e754799759
|___________________________________apikey__________________________________|
10 | contributed_slack_key = xoxp-90543978****-************-*************-*************…
11 11 |
Ignoring secrets
If the secret is a false positive or a placeholder you want to ignore, you can ignore secrets quickly and easily.
You can ignore secrets within your code using the tag #ggignore within your code in line with your secret.
But you can also securely save a safe list of secrets that GitGuardian will ignore. When ggshield discovers a secret, you will notice an ignore SHA in the output. This SHA can be used to identify your secret without revealing its identity.
The ignore command in ggshield can help you manage secrets you want to ignore in the future.
By running the command
ggshield ignore --last-found
it will take the last secret discovered and add it to the .gitguardian.yml file which will look like below, notice the SHA value in this document will match the SHA value ggshield provides. This makes the file safe to share with colleagues and can be committed to the git repository.
all-policies: false
api-url: https://api.gitguardian.com
exit-zero: false
matches-ignore:
- match: ecb4d7fe4d92281b501b14391898eee78e73841c6e58c787fc5d147a7b23a1ee
name: Slack User Token - python_server.py
paths-ignore: []
show-secrets: false
verbose: false
Read more about the Ignore function in the ggshield documentation.
Wrap up
That's it. With a few commands, it is easy to set up a pre-commit hook to scan for secrets. Next, we will have a tutorial on how to add GG-shield to your GitHub actions workflow.
