In the Detector of the month series, we feature our favorite projects or services GitGuardian’s secrets detection engine supports. For this first installment in 2022, we’re showcasing the Datadog API and Application keys detector!

What does Datadog do?

Datadog users and customers, you can skip this section and jump to the next one for a step-by-step guide on how to remediate exposed Datadog API and Application keys.

This would’ve been a short answer question 10 years ago but, since its inception, Datadog’s platform has grown very large. It now solves more use cases than we could describe in one paragraph so we’ll keep this short and cover the platform’s core features.

For most software-driven organizations, the last decade has been marked by a once-in-a-generation shift to cloud and microservices-oriented architectures, running on increasingly complex infrastructure (K8s anyone?). This has happened in concert with the adoption of DevOps principles as well as a huge growth in demand for always-on, highly performant services. For all this to run smoothly, developers and DevOps engineers need to be able to see what’s happening inside their stacks and applications all day and all night. In the DevOps world, this is commonly referred to as the observability trifecta – with logs, metrics, and traces.

Datadog’s platform helps developers and cloud operations engineers:

  1. Pull metrics from their infrastructure components (VMs, Docker containers, Kubernetes);
  2. Monitor their applications end-to-end from the frontend up to the databases;
  3. Manage all their logs and traces.

If your users and customers are increasingly relying on you to get things done, it might be a good idea to look into observability and monitoring for your business-critical apps. The Datadog for startups program is a good place to start!

Keep Datadog API keys out of your code with GitGuardian!

Developers hardcode secrets more often than they would like to admit but this doesn’t mean there’s nothing we can do to help. The first step is to alert developers whenever they slip up and leave credentials in Git repositories, GitGuardian’s library of built-in specific and generic detectors is perfect for the job at hand. The second step is to educate developers on the best practices of secrets management and implement them at both the individual and organization levels.

The risks of secrets-in-code can never be overstated. Observability and monitoring tools like Datadog help engineering teams see through all their organization’s infrastructure and the business applications running on them. Attackers with unauthorized access to valuable information (in the form of logs and traces) can possibly map an organization’s entire cloud infrastructure, identify its critical components, and possibly, poke holes in the systems.

Also, keep your logs clean.

Datadog’s log management capabilities can centralize log data from any source in the stack. While rich logs provide engineering teams with relevant details for practical troubleshooting, it is essential they don't fall into the wrong hands. If this were ever to happen, we can imagine a scenario where attackers eavesdropping on the logs will try to look for secrets and credentials for further lateral movement. Datadog’s sensitive data scanner comes as a great add-on here – helping security and compliance teams identify and mask sensitive data or secrets exposed in logs – effectively mitigating the threats associated with the attack vector described here.

How to handle hardcoded Datadog API keys?

Here's what to do when you receive an alert from GitGuardian that mentions compromised Datadog credentials.

  1. Understand the implications of the incident. A bad situation can be made worse if a secret is revoked without understanding how it is currently being used.

  2. Rotate and revoke the secret!

    • If it’s a Datadog API key
      Go to your Datadog dashboard. Under the Organization settings section, you will find API keys in the Access menu:
      a. First, create a new key to replace the compromised one;
      b. Second, select the exposed key and delete it so it can no longer be exploited.
    datadog_api_key_revoke

    To learn more about Datadog's API keys, please visit the developer docs.

    • If it’s a Datadog Application key
      Go to your Datadog dashboard. Under the Organization settings section, you will find API keys in the Access menu:
      a. First, create a new key to replace the compromised one;
      b. Second, select the exposed key and delete it so it can no longer be exploited.
    datadog_app_key_revoke

    To learn more about Datadog Application keys, please visit the developer docs.

  3. Manage your secrets properly. Use a dotenv file or look for alternative Secrets Management solutions to avoid hardcoding credentials in the future.

  4. Optional – Remove all evidence from your Git history. Rewrite the Git history or delete the entire repository. ⚠️ Such operations are not trivial and need to be discussed with your team members.

  5. Review access logs. Check for suspicious activity in the log data of your services impacted by the secret.

What about other secrets?

GitGuardian helps developers keep 350+ types of secrets out of source code. GitGuardian’s automated secrets detection and remediation solution secure every step of the development lifecycle, from code to cloud:

  • On developer workstations with Git hooks (pre-commit and pre-push);
  • On code sharing platforms like GitHub, GitLab, and Bitbucket;
  • In CI environments (Circle CI, Travis CI, Jenkins CI, GitHub Actions, and many more);
  • In Docker images.