Infrastructure provisioning practices have always been behind in relation to the maturity and standardization of application development practices. But Pulumi is set to bring infrastructure provisioning to the same level of maturity.

Let me explain

The evolution of infrastructure provisioning has come a long way. It started with physically installing hardware. Then, thanks to virtualization, it became a few clicks in a UI. Then cloud providers started to offer REST APIs for provisioning services. It is now possible to declaratively define the end state of an environment using code. Infrastructure as code became a thing. Some examples include Terraform, Cloudformation, ARM templates, etc.

The thing about infrastructure as code

Infrastructure as code is great for reproducibility. It minimizes human errors and increases immutability. As a bonus, it also serves as documentation. Terraform enables you to provision and make changes to infrastructure incrementally, and stores state of execution after each run. But when used in anger in an enterprise setting, it starts to show its kinks.

For starters, introducing any intelligence in the form of if statements is difficult. Eventually, Terraform’s HCL language got support for if statements and for loops, but they feel more like an afterthought than first-class citizens in HCL. Infrastructure as code frameworks don’t have any native testing capabilities. HCL, Cloudformation, Azure Resource Manager templates have their own learning curves. They also lock you into respective ecosystems.

Pulumi is a happy compromise

On the other end of the spectrum, there are cloud SDKs for many popular programming languages. All major cloud providers offer such SDKs called as CDKs, short for cloud development kit. CDKs are fantastic because they offer full benefits of a programming language; however, CDKs don’t have a concept of state management like Terraform does. State management makes incremental changes to infrastructure manageable and predictable.

This is where Pulumi comes in with these combined benefits. It offers both a fully featured programming language and state management.

For example, vanilla JavaScript code looks like this for multiple if conditions:

switch (ostype){
        case 'linux':
          Userdata = userdata
          break;
          case 'windows':
            Userdata = windowsuserdata
            break;
      }

The same logic expressed in Terraform’s HCL looks like this:

Linux = var.ostype == "linux" ? true : false
Windows = var.ostype == "windows" ? true  : false

As you can see, readability is much better with a well-known JavaScript notation than Hashicorp’s HCL language. HCL is a domain-specific language that lacks the flexibility of a general purpose language. Hence, more intelligence can be added to infrastructure as code using a general purpose language.

Pulumi brings along other software development lifecycle benefits. Features such as test driven development, continuous integration, versioned releases, package management, etc.

Now application software and infrastructure software can both live side by side. A git repo can host both of these written in the same programming language.

Pulumi concepts

Terraform has concepts like provider, module, state etc. Similarly, Pulumi has reimagined and simplified concepts as follows:

Stacks:

Stacks are collections of related infrastructure resources that share the same environment variables. Stacks are similar to environments (dev, stage, prod), and each stack has a different set of environment variables. These variables are defined exclusively in “Pulumi.dev.yaml”, “Pulumi.stage.yaml”, etc.

Projects:

A Project is a logically isolated grouping of infrastructure resources encompassing stacks. Projects can be mapped to teams, initiatives or types of infrastructure (containers, serverless, networking, etc.). There is room for interpreting what type of grouping is suitable for an organization.

Backends:

Backends store state-of-execution of Pulumi in a persistent storage system. Backends can be of several types:

1. Service backends: Pulumi cloud (SAAS) or self-hosted Pulumi. Service backends offer the advantage of having snapshots of infrastructure at regular intervals. This aids in rollback and recovering from fatal errors during provisioning.

2. Self-managed object storage (AWS S3, GCP GCS, Azure blobs, etc.).

Providers

Pulumi can provision infrastructure and services on several platforms. At present, it has providers for AWS, GCP, Azure, Kubernetes, Cloudflare and many more.

Summary

There might not be significant parity when you look into technicalities of language semantics between HCL and a general purpose language. When there is a choice between a well-versed mature general purpose programming language and a proprietary brand-new domain-specific declarative language to achieve the same end result — why choose the lesser known? Both HCL and a general purpose language do the job well enough. But given the established ecosystems around general purpose languages and the talent that is available in the market, it makes sense to use a framework like Pulumi over Terraform.

Appendix

Comparison table

Coming soon

Part 2 of this article will explore advanced Pulumi features that can enable autonomous teams to provision and maintain their own infrastructure along with their application. We will also explore a codebase that can provision EKS clusters on AWS — EKS clusters that are more than “hello world” and have some hardening and production readiness sprinkled in. Stay tuned!

To stay connected, follow us on the DoiT Engineering Blog, DoiT Linkedin Channel and DoiT Twitter Channel. To explore career opportunities, visit https://careers.doit.com.