On August 21, 2023, security researcher and HackerOne Advisory Board Member Corben Leo announced on social media that he had "hacked a car company" and went on to post a thread explaining how he "gained access to hundreds of their codebases."

first Tweet from Corben's thread

Corben was participating in a bug bounty program run by the car maker. This is a very common practice across industries, as it rewards ethical hackers for finding issues and reporting them in a responsible way. This path has been time-tested and produced some fantastic results for many companies. At the same time, there is some reporting that says that compared to other industries, car manufacturers tend to pay far less for bug bounties. In this case, the car company had the right incentives, and Corben had the right motivation to find and report this potentially crisis-inducing vulnerability.

In the thread, Corbin laid out his attack approach. In short, through trial and error, he found what he believed to be an ingress controller, a load balancer for Kubernetes environments. Through brute force attacks, manipulating the host headers, he discovered a misconfigured Spring Boot Actuator, which uses HTTP endpoints to expose operational information about running applications. The endpoints he discovered were for `/env` and a `/heapdump` endpoints. This is where credentials come into the story.

The fourth tweet in Corbin's thread

The `/env` endpoint had properly redacted passwords, meaning a dead end. However, the ` /heapdump` endpoint contained snapshots of application objects stored in memory, exposing the credentials in plaintext. After just a few minutes of searching for keywords, he found oauth2 credentials. He leveraged those to gain access to a previously discovered `config-server` instance, and again, thanks to another misconfigured Spring Boot Actuator, he was able to find another set of `/env` and a `/heapdump` endpoints.

This time, though, the `/env` endpoint did not redact the credentials. Corbin now had the private key and password for a GitHub repository admin. This admin had access to over 30 GitHub organizations and read/write access to hundreds of repositories.

the ninth tweet in Corbin's thread

If an unethical hacker had discovered this vulnerability first, this car company could have had all its private repositories leaked or ransomed. As we have seen not all companies, including Samsung, Nvidia, Microsoft, and Uber were not so lucky.

Breaking down what went wrong

Multiple issues are at play in this scenario: misconfiguration, exposed plaintext passwords, and scoping.

Misconfigurations are an attacker's best friends

While we can't be certain how this particular car company's DevOps team deployed its infrastructure, there is a good chance that they would use Infrastructure as Code, IaC, such as Terraform, Crossplane, or CloudFormation. There are some serious advantages to IaC, as the config can be versioned and reviewed quickly and makes the entire setup extremely scalable. However, one of the largest downsides is a misconfiguration in your code can mean the same issue is rolled out to every instance throughout your environments. This is likely why the same misconfiguration was present on multiple servers.

Testing for misconfigurations early in the process is the best way to ensure they do not find their way to production. GitGuardian customers have been able to use gghield for IaC scanning for a long while now. Now, we have added this same scanning ability to the GitGuardian platform with our new Infra as Code Security module. Now, customers can see the results of our more than 100 automated scans in their dashboard, as well as coordinate the remediation process.

No matter how you deploy your infrastructure, manually or via IaC, make sure you are adding a review process to cover common scenarios. We also encourage anyone who is just getting started with IaC to review the OWASP Cloud-Native Application Security Top 10 to know what to be on the lookout for when you are testing.

Exposed passwords

By now, we hope everyone knows that storing passwords in plaintext is a bad idea. However, we also know that every year, the number of exposed passwords that make it into public GitHub repos is growing yearly, as reported in the State of Secrets Sprawl. In this story, we did see evidence there was some thought put into secrets security, as the initial `/env` endpoint he encountered did have redacted credentials. However, they overlooked the log files in the `heapdump` endpoint.

It is vital to check for secrets everywhere, not just in the code you write but in any files that code produces, including logs. This is one of the reasons ggshield, the GitGuardian CLI, can scan any arbitrary file or path for secrets. Manual code reviews of logs might have caught that plaintext credentials were present, but teams that consistently use tools to test are more consistently going to find issues early. This would have been a very different social media thread if they had sanitized the logs on that endpoint.

Super admin credentials

The GitHub credentials Corbin discovered didn't just grant access to a single private repo, or even a single private org. He reported gaining access to over 30 organizations and hundreds of repos. There might be a legitimate reason that this level of cross-organizational collaboration could be needed; however, this feels like a case of someone using root user credentials to handle everything. The common consensus in the security world is to always work towards a Zero-Trust architecture and apply the principle of least privilege.

This principle of least privilege is discussed by Mackenzie Jackson in his post, "Best Practices for Managing and Storing Secrets Including API Keys and Other Credentials." all credentials should be tightly scoped for their intended purpose, granting only minimal permissions needed to complete a specific task. Creating a new user who only had access to very specific repos might have been a better path, meaning Crobin would have had some access but not the literal keys to the kingdom.

The importance of bug bounties and security research

In all reality, every application on the web is constantly being security tested. It is just a matter of who is testing it and for what purpose. We are in a constant cat-and-mouse game with attackers, always trying to stay one step ahead of them. Participating in bug bounty programs and hiring pentesters to find and exploit vulnerabilities in your apps and environments are great ways to make sure you find issues before attackers can.

There are a lot of ethical hackers out there who are waiting to work with you to help keep you safe. If you are looking for more information about what bug bounty platforms offer, check out HackerOne, Bugcrowd, or Open Bug Bounty, which are three of the most popular platforms. You can also check out this recent Security Repo Podcast with Jason Haddix, where he talks about his time as a bug bounty hunter as well as being CIO of UbiSoft.

Keeping your apps and, therefore, your customers safe is a journey and one we are all on together. There are a lot of lessons we can take away from this and other examples. In this case, it serves as a reminder to follow the principle of least privilege when scoping our credentials, not to add passwords in plaintext, and test to ensure our Infrastructure as Code is free of misconfigurations. No matter where you are on your path to better security, we encourage you to keep learning and striving to stay safe.