January 4, 2024

AWS CDK vs. Pulumi: Developer's Cut

This is a developer's cut comparing AWS CDK and Pulumi. It's a raw document of observations I've collected over time working with both CDK and Pulumi.

Disclaimer: Some of the issues I've noted in this post are possibly no longer issues. I wrote these down in the last several months as I was working with CDK. It is also possible that some of the cons I've mentioned are likely addressable by future enhancements to CDK. However, the likelihood of that happening or not is not something I want to speculate.

Pros

• Nice view of all the IAM roles that will be created and the reason for them.
• CloudFormation stacks show up as deployments in the AWS console so you can see all the resources, events etc. Kind of like Azure’s ARM deployments.
• The Stack construct is actually nice. Having a concrete type for a stack and creating dependencies between them is useful to visualize inter-stack dependencies.

Cons

• No way to control resource replacement order, i.e. create-before-delete vs. delete-before-create. (Keep reading this post to learn more about this.)
• No way to separate context values from individual stacks. That is, you must pass all context values required across all of your stacks.
• If you are using the command-line to pass context values to the deploy and synth commands, you will always need to pass all context values, otherwise you'll end up causing unnecessary replacements due to changing/missing context values.
  • To be fair though, you can likely fix this by persisting your context values in cdk.context.json and therefore avoid making mistakes like accidentally omitting some context values. So perhaps this is not so much as a con but I am not sure if there are times when using the command-line to pass the context value is better than the JSON file. I haven't tried sourcing from the file, so I couldn't tell if that makes this experience better.
• No local secret configuration which means you must onboard secrets out-of-band and cannot automate those via CDK.
• Implicitly retains resources when a CloudFormation stack is deleted. It’s also confusing which resources will be left intact on removal. Perhaps, the ones that allow you to pass a removal policy property as part of construction and not via the object.applyRemovalPolicy escape-hatch?
• CDK bootstrap needs to be run for every new account where you want to deploy. Not a big deal I suppose.
• CloudFormation stacks are disconnected from the actual state of the resource itself. This leads to weird drift issues. One example is, delete the CodeBuild source credential out-of-band and there is no way to recreate it via CDK. Even using --force doesn’t work. There is no way to "refresh" the state of resources unlike Pulumi. (Keep reading the rest of this post to learn why this is important.)
• There is no way to rename stacks after they have been created. Renaming the stack’s identifier actually seems to be abandoning the stack’s resources entirely.
• There is no concept of "projects" which would allow you to cross-reference stacks within those projects.
  • Without this you essentially need to pack all of your infrastructure into the same repository. Sure you can spread out the infra across Stacks. But that means all of your stacks will need to be synthesized and deployed every single time. Probably not a big deal but if you are passing a bunch of context params for each stack, you are likely to find yourself wanting to split the stacks into projects. (Keep reading to learn why Pulumi's projects are a useful feature.)

So those are my list of pros and cons. Let's take a look at a feature important features that I think you should strongly consider using Pulumi for.

Creation Order

Sometimes you'll find yourself in a situation where a resource needs to be deleted before a new replacement for it is created. In Pulumi, you can easily control the creation order using a resource option called deleteBeforeReplace. By default, however, Pulumi opts for create-before-replace semantics to reduce the chances of causing a downtime that results in destroying a resource before its replacement is created. As mentioned earlier, there are exceptions to this and sometimes you'll want to change the behavior.

Refreshing Stack State

In Pulumi, your stack has a state that ideally mirrors the actual state of the resources wherever it is those resources are created, i.e. AWS, Azure, GCP, DigitalOcean etc.

Sometimes you may find yourself modifying some configuration properties of a resource directly in the console of the respective cloud provider. If that happens, the configuration you've specified in your code should be updated to match the configuration as seen by the cloud provider. But you won't know that the drift has occurred if the tool you are using doesn't alert you of the difference.

Projects In Pulumi

An underrated feature of Pulumi is the ability to create projects. Projects allow you to separate your infrastructure into groups and control access via RBAC. Although the fine-grained RBAC is only available from the enterprise pricing tier onwards, it's a great reason to upgrade to the higher pricing tier if that's something your org needs.

Projects in-turn contain stacks and it's these stacks that you can create references to.