Pulumi VS Terraform: The Definitive Guide to Choosing Your IaC Tool

In the cloud-native era, Infrastructure as Code (IaC; read more about it in this blog here) has become the de-facto standard for managing cloud infrastructure, and more.

While Terraform has been around for almost a decade, and it had been the one-and-only cloud-agnostic option for a couple of years before competitors emerged, now the landscape is a whole lot more diverse: we've got AWS CDK, CDK for Terraform, and there is a relatively new kid on the block: Pulumi.

Having more choices, however, doesn't necessarily mean they'll make our lives easier because the decision-making process can be daunting: we have to spend days, if not weeks, investigating all the options before well-informed decisions. In the fast-paced cloud-native/DevOps era, though, not everyone has the luxury to do just that.

Worry no more: in this blog, we will do a deep dive into Pulumi V.S. Terraform (and slightly touch the mechanism of AWS CDK/CDK for Terraform, for that matter). A comparison chart, a decision tree, some tips, and even an FAQ will be included (TL;DR: jump to the last two sections) to further help you choose the right tool for the job.

Without further adieu, let's get started now.

1. Terraform

There is no getting around Terraform since we are talking about IaC tools because it's the longest-established one. Yes, there are cloud-specific solutions that even have a longer history (e.g., AWS CloudFormation), but no, they are not cloud-agnostic.

So, before touching on Pulumi, let's take a look at Terraform first.

1.1. A Brief History of Terraform

First released by HashiCorp in 2014, Terraform started to gain a lot of momentum in 2016 and 2017, during which period basically all DevOps engineers were either excited trying it out or at least talking about it.

Although the generally available version v1.0 isn't released until much later in 2021, previous versions, especially v0.11 and v0.12, between 2017 and 2019, are already greatly accepted by different companies in various business sectors and widely used in not only development environment, but also production environment.

All this history means we can have some confidence in Terraform: it has been tested in real-world production environments for a long time, and with a proven track record like that, you can't go wrong with Terraform even if you don't have the time or can't be bothered to try out the other alternatives.

1.2. Under the Hood of Terraform

To better understand Terraform (and all IaC tools, for that matter), next, let's have a look at how exactly Terraform works: the core-plugin architecture.

Simply put, the core is a state machine. It manages the infrastructure lifecycle by comparing the current state of your infrastructure with the desired state defined as declarative code, then coming up with a plan to do changes/operations accordingly to bring the infrastructure to the declaratively defined state.

The actual changes/operations, like creating, updating, or deleting some infrastructure, are carried out by the plugins (or providers, which, in this context, is just another name for the same thing). The core communicates with the plugins, telling them what to do in a given state.

To sum up, the core manages the state, and the plugins do the operations: this is basically how all IaC tools work: you've got to manage the state, and you've got to have some stuff doing the heavy lifting that is operating the cloud infrastructure.

Some extra information worth mentioning about Terraform, most plugins are implemented in Golang (although the core-plugin framework by Terraform makes it possible to use plugins written in other programming languages), so the plugin needs cloud Go SDK to interact with the cloud to actually do the CRUD work.

If this detail seems intimidating to you, don't worry: as Terraform users (not developers/contributors), we don't need to think about the plugin implementation; and we don't write Go code, we only write HCL. We define infrastructure in HCL, under the hood, Terraform does some conversion to call those Go plugins which in turn uses cloud Go SDKs.

1.3. Terraform HCL

Defining your infrastructure in Terraform is straightforward: you define your infra with a configuration language that is HCL (HashiCorp Configuration Language).

Let's have a look at an HCL example, which creates an S3 bucket in AWS:

resource "aws_s3_bucket" "example" {
  bucket = "my-tf-test-bucket"

  tags = {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

If you are familiar with any markup language (or markdown, or JSON, or YAML, literally just about anything), the above syntax won't seem too alien, and it's quite clear that HCL is built around two concepts: blocks and attributes. In the example above:

• The whole resource ... {} is a block, which defines a resource, as the first keyword notes.
• aws_s3_bucket is the type of resource. By referring to the AWS provider documentation, we can get a list of all supported AWS resources.
• The example part is the name of the resource.
• Inside the block, we have key-value pairs, which are attributes, or arguments for this resource. Again, we need to refer to the provider's documentation to figure out supported arguments, which are mandatory, etc.

There is a learning curve regarding HCL, but it's not as steep a curve as learning yet another programming language, because HCL isn't really a full-blown programming language; it's only for configuration purposes. This can be a pro or a con depending on your philosophical bend:

• On the one hand, precisely because it's not a full-blown programming language but only a simple configuration language, there comes great benefits: it's basically key-value pairs, which are nice and easy, very human-readable.
• On the other hand, because of its simplicity, you can't do complex operations easily like loops and branches (these can be achieved with special syntax, but not quite as simple as you write for ... or if/else in a real programming language such as Python.)

So, as an IaC tool, Terraform has a somewhat moderate learning curve and limitations (both because of HCL: you must learn it, and you must define your infra with it). Many other IaC tools have emerged to improve this situation. Read on.

2. Pulumi

As mentioned in the previous section, Terraform isn't perfect. Many tools have emerged just to solve these issues, and Pulumi is one of the latest attempts.

What is Pulumi? Simply put, it's an IaC tool, just like Terraform; but unlike Terraform which uses a specific syntax (HCL), Pulumi allows you to define your infrastructure using ANY (well, almost) programming languages.

I know it would be not extremely accurate and a big understatement to say this, but for beginners who want to learn what Pulumi is in just one sentence, simply put: Pulumi is Terraform in Python/Go/Java/Node.js/etc., so to speak.

2.1. A Brief History of Pulumi

Pulumi was first open-sourced in 2018, which seems not so new, but, the current version v3, which has a few breaking changes compared to the previous version, was released in 2021, and after that, people started to notice it a lot more than before (ask Google Trend if you don't believe me.)

Today, compared to Terraform, Pulumi attracts a lot less attention in Google searches. According to Pulumi's blog, it had less than 2000 customers in 2023, which is orders of magnitude less than Terraform (according to multiple data sources from the internet.)

So, it's new and not as widely accepted. Why Pulumi, then? Well, it's because of its strongest feature: multi-language support.

2.2. Pulumi: Multi-Language Support

If you choose Terraform, you have to write HCL. For many people, this can be an overhead.

For example, for backend engineers who mainly write programs in Go and occasionally manage the cloud infrastructure, why learn HCL when you can define your infra in Go?

The same goes for frontend/full-stack engineers who mainly write in JavaScript/TypeScript: learning HCL on top of what they are already using is not only an overhead, but also increases the complexity of the tech stack, and when it comes to tech stack, more often than not, less is more.

With Pulumi, it's a different story: you can define your infrastructure in any of the following languages:

• TypeScript (Node.js)
• Python
• C#, VB, F# (.NET)
• Go
• Java
• YAML

For example, if we want to create exactly the same AWS S3 bucket mentioned in previous sections using Pulumi with Python, we simply write:

import pulumi
import pulumi_aws as aws

bucket = aws.s3.Bucket("bucket",
    acl="private",
    tags={
        "Environment": "Dev",
        "Name": "My bucket",
    })

Or maybe Python isn't your thing and you want to write TypeScript:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("bucket", {
    acl: "private",
    tags: {
        Environment: "Dev",
        Name: "My bucket",
    },
});

Voila!

2.3. Under the Hood of Pulumi

Long stories short, Pulumi works the same way as Terraform: they both have the aforementioned core-plugin architecture.

Like Terraform, Pulumi also uses cloud SDKs and libraries internally. Pulumi supports multiple languages because the plugins themselves are implemented in multiple languages. For example, if we have a look at the AWS provider for Pulumi here, you can see it has multiple implementations in different languages. And that's why when defining your infra in Python, you import pulumi_aws as aws, and with TypeScript, you use a completely different package for Node.js: import * as aws from "@pulumi/aws";.

It's worth mentioning that there are other options that allow you to define your infrastructure code in many languages, like AWS Cloud Development Kit (AWS CDK) and CDK for Terraform (CDKTF), but fundamentally, their multi-language support is implemented in a completely different method: both AWS CDK and CDKTF depends on a library jsii which allows code in any language to naturally interact with JavaScript classes. So, AWS CDK (and CDKTF) uses jsii to translate TypeScript code into different languages to support polyglot (just a hard word for multi-language), but under the hood, Pulumi just has those providers written in different languages.

3. Pulumi V.S. Terraform: Key Similarities and Differences

3.1. Key Similarities of Pulumi and Terraform

The biggest similarity is how they work: the core-plugin architecture.

First of all, it's the core, or to put it in another way, it's actually the state. They both use the core to maintain the state so that it can be computed based on the current infrastructure and what's defined as code to generate a plan of operations to bring the infra to the defined state, and they both can store the state in different places, like locally, in an S3 bucket, or a cloud/SaaS solution for it.

Second, it is the plugins. As aforementioned, they both use a core-plugin architecture to manage the state and carry out changes.

3.2. Key Differences between Pulumi and Terraform

The biggest difference, of course, is the multi-language support.

Terraform uses HCL, which is not a full-blown programming language, but only a configuration language. So, by nature, it can't do as many things as other programming languages; but also as mentioned previously, this could be a good thing for you, because it is easier to read and simple can be better.

Pulumi supports multiple languages and this is by far the single most important difference. If, for any reason, you have to define your infrastructure in Python/Go/Java or other major programming languages, between Terraform and Pulumi, there is no competition.

Another difference worth mentioning is the testability of your infrastructure code because with Pulumi you benefit from the unit tests and functional tests and tools that come with the programming languages. With Terraform, the way you can run your test is limited, mainly to integration tests.

Of course, multiple minor features differ between Terraform and Pulumi (don't be surprised that Pulumi official doc manages to find more than a dozen), but none of them are real deal-breakers. For example, what open-source license they have, or where the default config for the state probably isn't on the top of your list.

4. Misconceptions about Pulumi

If you are reading this blog, chances are, this isn't the very first article you read on Pulumi V.S. Terraform; you probably have done quite some investigation and read tons of pros and cons of each.

However, I find a few most commonly mentioned disadvantages about Pulumi compared to Terraform in many other articles which are in fact misconceptions and are not true anymore. I want to address those so that we can obtain a fair and square view of Pulumi. It's so important that it deserves a separate chapter.

4.1. Misconception 1: Pulumi Lacks Documentation

Not true anymore (maybe so when it just got started).

Pulumi has very detailed, step-by-step documentation on how to install it and how to get started. If you want to go deeper, they've got a great section on core concepts of Pulumi. Plus, Pulumi has detailed docs and examples on multiple cloud providers.

For plugins/providers, if you search for a specific provider, like AWS (or maybe even a less-popular one, like the PagerDuty Pulumi provider), and compare it to Terraform's documentation, you won't reach to the conclusion that "Pulumi lacks documentation" because it's not.

4.2. Misconception 2: Pulumi Has a Small Community

Not true anymore, again.

According to Pulumi's official blog, it has 2000 customers and 150k end users. According to GitHub, the main repo pulumi/pulumi alone got more than 18.8k stars, 1.9k issues, and 184 open pull requests.

No matter which metrics you are looking at, and no matter against what standard you measure the size of the community, Pulumi's community is definitely a big one. For sure, smaller than Terraform, comparatively, but absolutely speaking, by no means is that community small.

4.3. Misconception 3: Pulumi Is Not Universally Applicable

Not true, again.

Many would conclude that since Pulumi is newer, it isn't universally applicable, which isn't the case.

If we are talking about programming languages, Pulumi in fact supports most of the major ones.

If we are talking about plugins and providers, one might think a new tool probably has fewer plugins since it existed for a shorter period, hence fewer plugins, but the reality doesn't agree with this. Pulumi supports all the major public cloud providers; what's more, for managing non-cloud infrastructure, like managing teams and users in certain software, Pulumi covers as wide as Terraform.

For example, in a big company, you may have many DevOps tools to manage, like GitHub, PagerDuty, DataDog, Sentry, etc., and maybe you want to use IaC to manage users/teams/permissions for those tools as well. In this case, you'd be amazed that if you search plugins for these tools, Pulumi has them all, just like Terraform does, even though they are not extremely popular, conventional cloud-related plugins.

That said, I hope we can evaluate Pulumi/Terraform objectively and do not deny Pulumi simply because we read it somewhere that it "lacks documentation" or it "has a small community", or even think "newer equals less universal".

5. Pulumi V.S. Terraform: A Closer Real-World Look

In previous sections, some code snippets are provided to illustrate the syntax and usage of both Terraform and Pulumi, but they are too simple and too "helloworld" to mean anything in real-world, where things tend to grow exponentially, which means it's challenging to keep the code simple and readable, and at the same time, scalable.

So, next, let's have a look at Pulumi V.S. Terraform and do a comparison of how they tackle these real-world challenges.

5.1. Pulumi V.S. Terraform: Code Structure, Readability and Scalability

For Terraform, we can define modules and reuse modules to achieve maximum code reusability. A typical monolithic Terraform repository might look like this:

.
├── dev
│   ├── config.tf
│   ├── main.tf
│   ├── output.tf
│   └── variables.tf
├── modules
│   ├── module_a
│   └── module_b
└── prod
    ├── config.tf
    ├── main.tf
    ├── output.tf
    └── variables.tf

A single repo has its strengths: very human-readable and easy to understand without any further explanation. And because of Terraform's nature, you have two levels of directory structure, maybe three, tops, which all mean it's easy to get an overview at a glance and easy to manage. Even creating another environment is no more complicated than creating yet another folder, for example, named "test" in the same repo.

Based on the clean code base above, when the project grows bigger, there are multiple ways to improve: separate different parts of the infra into different repos; move the modules out, into one or a few other repos; put different env into different repos. All choices are flexible, and they all result in new repos in more or less the same nice and clean, and easy-to-understand directory structure.

With Pulumi, things can be tricker. A quintessential monolithic Pulumi project may look like this:

.
├── Pulumi.dev.yml
├── Pulumi.prod.yml
├── Pulumi.yml
├── api-gateway
│   ├── index.ts
│   └── micro-service-01
│       └── index.ts
├── database
│   └── table-01.ts
├── index.ts
├── package-lock.json
├── package.json
├── sns
│   └── topics.ts
└── queues.ts
├── pkg
│   └──application
│     └── app.go
└── .etc

Like Terraform, we can package some common stuff into a, package, just like any frontend or backend project you've worked on.

When projects grow bigger, we can use the micro-stacks approach, where the project is broken into separately managed smaller projects, and each project could be the same as above.

However, when things grow, the directory structure will become more complicated with a lot more directories and much more levels, which can be confusing. Think Java, or any real-world project you've ever been in, and ask yourself: is it easy to get a quick understanding of the whole thing? No, because there are so many folders and so many levels of directories that you don't even know which is what and where to get started.

Pulumi's strongest selling point is its multi-language support, but with great power, comes great responsibilities: organizing your code in a way that makes it easy to understand and maintain is crucial, and this is true no matter which tool you use.

5.2. Pulumi V.S. Terraform: Integration

More often than not, your IaC doesn't just do the IaC then it's all done. The infrastructure part needs to be integrated with other things like CI/CD pipelines. Luckily, both Terraform and Pulumi only need one command to deploy the changes, which makes them ideal to be integrated. However, there are differences.

In some cases, you want to do something before the IaC pipeline starts.

For example, let's say we want to use IaC to manage users, teams, memberships, and permissions. When adding a new user, I don't want to open my code base, copy-paste, modify, and commit; it's too much of a hassle. Let's say, maybe we have a list of users stored somewhere, and we can retrieve the file and use some templating tools to generate IaC code automatically.

In this case, for Terraform, you might need to rely on extra tools, like some Python script to download the file, parse, template, then commit the generated IaC files, before running the IaC pipeline. For Pulumi, things can be much simpler, because you can do everything in one pass: use the programming language of your choice to download/parse, then use the same language to do a simple for loop to create stuff.

In some other cases, you might want to do something after the IaC finishes. For example, the output of the IaC part contains a load balancer URL that you want to use in your Helm chart. Again, with Terraform, you probably need another step where you run some script, but with Pulumi you can continue writing your code to do just that after the IaC code (some examples here).

As a quick summary, if you are struggling with integrating some scripts with Terraform, it's probably worth it to give Pulumi a try.

5.3. Pulumi V.S. Terraform: Security

Security is always an important topic for code, and infra code is still code.

For code security, the most fundamental principle is probably not to store secrets in clear text in the code. On this front, both Terraform and Pulumi perform well. Terraform can be integrated with different secrets managers, and with Pulumi it's also easy to read from secrets managers using a single line of code. For example, here's a blog for managing secrets in Terraform.

More can be said about code security: With Terraform, you write HCL, and the configuration code gets translated into API calls to create, read, update, and delete resources. Of course, there can be security issues and CVEs with Terraform core and its plugins themselves, but the same can be said about any other IaC options. With Pulumi, things can get a bit more complicated, because your IaC code can be written in many languages and can do a whole lot more, the attack surface is increased. While this seems to be a downside for Pulumi, luckily, there are best practices and tools (like SAST & DAST) for scanning infra code to enhance security.

If you are interested in Terraform and IaC security, here's a blog on IaC Security with Terraform, and here's a blog on some Terraform best practices (don't worry, it's nothing like what you must have read before).

Code security aside, since IaC tools manage infrastructure and critical information is actually stored in the state, the security of the state is crucial, too. Both Terraform and Pulumi can encrypt sensitive information in the state so that they are not printed as clear text, and both of them support different backends to store the state encrypted.

Some recommendations to wrap up this section:

• If you manage any sensitive data with IaC tools (like database passwords, user passwords, or private keys), treat the state itself as sensitive data, this means:
• Storing the state remotely can provide better security, try not to use local disks for the state.
• Use backends that can be configured to encrypt the state data at rest.

6. Summary: Choosing Your IaC Tool

6.1. Comparison Table

To make this easier, let's look at the following table for a quick comparison of the major features of the two:

6.2. Choosing Your IaC Tool

To make this fun, I've created this flow chart to further help you choose the right tool for you:

1. Just get started with IaC? Yes: goto 2. No: goto 7.
2. Do you believe less is more? Yes: goto 11. No: goto 3.
3. Are you a DevOps engineer? Yes: goto 5. No: goto 4.
4. Are you a developer who occasionally manages infrastructure? Yes: goto 5. No: goto 11.
5. Do you already code in JavaScript/Python/Go/Java? Yes: goto 12. No: goto 6.
6. Do you want to learn a new configuration language (much less difficult than learning a new programming language)? Yes: goto 11. No: goto 12.
7. Have you already used Terraform? Yes: goto 8. No: goto 9.
8. Is there a pain point you are struggling with Terraform? Like, you are not happy with HCL, or you want to do more complicated things? Yes: goto 10. No: goto 11.
9. Try Terraform.
10. Try Pulumi.
11. Stay with Terraform.
12. Sorry but neither is suitable.

6.3. Choosing the Right Tool for the Job

Fun aside, here's some tips for choosing the right tool:

• If your main job description is still writing app code, and managing infra code is only your part-time job, maybe Pulumi can be a better choice for you.
• If you have experience with Terraform and it bothers you, go ahead and give Pulumi a try: rest assured, there isn't something critical that Terraform can do but Pulumi can't.
• You DO NOT have to choose one tool to rule them all. Pulumi V.S. Terraform isn't a battle; it's not like you must decide which one is better and which one is wrong. You can actually use both. Monolithic infra repo is hard to manage when a project grows, and when you have a microservice-like infra project, why not make the best out of it by using the right tool for each part of the infra? Certain things can be done easier if the right tool is chosen.
• Get some first-hand experience. Read all the blogs and articles you want, at the end of the day, if you give both of them a quick try, you will know what the heart wants.

FAQ

Q: Is Terraform Old?

Yes and No.

Yes, it's been around for many years; and yes, it has its own limitations.

But no, except for the fact that you have to use HCL, it handles pretty much whatever you throw at it, and it handles them well.

It's worth mentioning that now Terraform has CDKTF which supports defining your infra in other programming languages.

Q: Is Pulumi better than Terraform?

Yes and No.

Yes, because you can actually choose other programming languages to define your infrastructure.

No, because a big project in a full-blown programming language can be less clear and harder to read than some simple configuration language.

Each tool has its strong suits, and it's not possible to say which beats the other.

Q: Can Pulumi do everything that Terraform can?

Yes.

Well, sort of. Although each has its quirks and features, the basic things you are looking for when talking about IaC are all there for both tools. You won't find yourself in an awkward situation if you choose Pulumi then find out it can't do some magic tricks that Terraform performs greatly.

Q: What are the disadvantages of Pulumi?

Nothing, to be honest.

It does what it is supposed to do quite nicely, and you can do it in the programming language of your choice. Plus, the documentation and the community are both nice.

If I have to nitpick on Pulumi, it's a lot trickier to manage a big code base with Python or Go or JavaSciprt compared to a repo of config files that is HCL. That said, it's not that Pulumi brings this challenge; it's the programming part. And, using HCL won't automatically mean your code base becomes easier to read and manage: you can definitely make a mess out of it. In the end, it's the programmers who write clean code and maintain it so.